Independent, blind assessment of structure prediction methods is essential for establishing the state of the art, identifying current limitations, and guiding future developments in the field. The Continuous Automated Model EvaluatiOn (CAMEO) platform provides weekly, automated, and independent benchmarking of structure prediction servers, serving as a continuous complement to the Critical Assessment of Structure Prediction (CASP) experiments. This work presents recent advancements in CAMEO aimed at evaluating predictions of macromolecular complexes, including protein–protein interactions, nucleic acid-containing assemblies, and polymer–ligand complexes. A comprehensive set of evaluation metrics is employed to capture various aspects of structural accuracy, including global and local correctness, interface geometry, and ligand placement. In addition, CAMEO provides multiple reference baselines to facilitate systematic comparisons against state-of-the-art methods. Here, we analyze the CAMEO benchmark dataset and report on the performance of baseline predictors and initial participating servers. By delivering continuous, blind, and objective evaluations, CAMEO supports the ongoing development and refinement of next-generation structure prediction methodologies.

The protein-ligand component of the 16th Critical Assessment of Structure Prediction (CASP16) challenged participants to predict both binding poses and affinities of small molecules to protein targets, with a focus on drug-like compounds from pharmaceutical discovery projects. Thirty research groups submitted predictions for 229 protein-ligand pose targets and 140 affinity targets across five protein systems. Template-based pose-prediction methods did particularly well, with the best groups achieving mean LDDT-PLI values of 0.69 (scale of 0-1 with 1 best). For comparison, we also ran a set of automated baseline pose-prediction methods, including ones using deep neural networks. Of these, AlphaFold 3 did particularly well, with a mean LDDT-PLI of 0.8, thus outscoring the best CASP16 predictor. The CASP affinity predictions showed modest correlation with experimental data (maximum Kendall’s τ = 0.42), well below the theoretical maximum possible given experimental uncertainty. As seen in prior challenges, providing experimental structures did not improve affinity predictions in the second stage of the challenge, suggesting that the scoring functions used here are a key limiting factor. Overall, the accuracy achieved by CASP participants is similar to that observed in the prior Drug Design Data Resource (D3R) blinded prediction challenges. The present results highlight the progress and persistent challenges in computational protein-ligand modeling and provide valuable benchmarks for the field of computer-aided drug design.

The prediction of protein-ligand complexes (PLC), using both experimental and predicted structures, is an active and important area of research, underscored by the inclusion of the Protein-Ligand Interaction category in the latest round of the Critical Assessment of Protein Structure Prediction experiment CASP15. The prediction task in CASP15 consisted of predicting both the 3-dimensional structure of the receptor protein as well as the position and conformation of the ligand. This paper addresses the challenges and proposed solutions for devising automated benchmarking techniques for PLC prediction. The reliability of experimentally solved PLC as ground truth reference structures is assessed using various validation criteria. Similarity of PLC to previously released complexes are employed to judge the novelty and difficulty of a PLC as a prediction target. We show that the commonly used PDBBind time-split test-set is inappropriate for comprehensive PLC evaluation. Finally, we introduce a fully automated pipeline that predicts PLC and evaluates the accuracy of the protein structure, ligand pose, and protein-ligand interactions.

Prediction categories in the Critical Assessment of Structure Prediction (CASP) experiments change with the need to address specific problems in structure modeling. In CASP15, four new prediction categories were introduced: RNA structure, ligand-protein complexes, accuracy of oligomeric structures and their interfaces, and ensembles of alternative conformations. This paper lists technical specifications for these categories and describes their integration in the CASP data management system.

The Continuous Automated Model EvaluatiOn (CAMEO) platform complements the biennial CASP experiment by conducting fully automated blind evaluations of 3D protein prediction servers based on the weekly pre‐release of sequences of those structures, which are going to be published in the upcoming release of the Protein Data Bank (PDB). While in CASP14 significant success was observed in predicting the structures of individual protein chains with high accuracy, significant challenges remain in correctly predicting the structures of complexes. By implementing fully automated evaluation of predictions for protein-protein complexes, as well as for proteins in complex with ligands, peptides, nucleic acids, or proteins containing non-canonical amino acid residues, CAMEO will assist new developments in those challenging areas of active research.