Invitation to contribute to the community standard to promote reproducibility of constraint-based models - FROG analysis

[Moritz Beber]

Dear COBRApy community,

I am inviting you to contribute to our initiative to improve the reproducibility of constraints-based models (CBMs).

A recent study (Tiwari et al. 2021, Mol Sys Bio) has shown that about half of the published kinetic models could not be reproduced using the information provided in the manuscript.  Currently, it is not possible to assess whether constraint-based models, including Genome-Scale metabolic models (GEM), are reproducible because these models often have multiple solutions, and the numerical values are not always enumerated in the published manuscripts. Therefore, one cannot ascertain whether a publicly shared CBM is the same model used in the manuscript. To tackle the reproducibility crisis,  through a community effort, we have developed FROG analysis to generate a collection of numerically reproducible standardized reference datasets (aka FROG report) in a structured schema that can be used to assess the reproducibility of CBMs. We have developed a collection of tools to autogenerate FROG reports. BioModels will start curating CBMs using the FROG framework and MEMOTE test suite. To allow retrospective curation of previously published models, we propose that the model authors submit a miniFROG report and an autogenerated FROG report. The miniFROG report is a manually created data table in a standardized schema that lists at least a couple of results described in the manuscript and compares them against the model results from the FROG report. More details can be found at https://www.ebi.ac.uk/biomodels/curation/fbc.

We are preparing a manuscript to summarize our effort, which we aim to submit shortly after HARMONY2022. As active members of the constraint-based modeling community, we invite you to assess the FROG analysis and tools and contribute to the manuscript by submitting your constraint-based models and FROG reports to BioModels. A BioModels curator will attempt to reproduce and curate your model to increase its impact.

To contribute to the manuscript, please use the form below https://forms.gle/eiwD2vgU5dsa8s5p7

To stay connected with us and join our hands-on training sessions please fill out the form below https://forms.gle/XwrQbWguWzESgJfj6

For any questions, feel free to email me or biomode...@ebi.ac.uk with the keyword ‘FROG’ in the subject line. 

I look forward to hearing from you,

Moritz

On behalf of the FBC curation standards working group, COMBINE