Invitation to contribute to the community standard to promote reproducibility of constraint-based models - FROG analysis

[Rahuman Sheriff]

Dear Community members,

I invite you to contribute to our initiative to improve the reproducibility of constraints-based models (CBMs) in the community. In our recent study (Tiwari et al. 2021, Mol Sys Bio) we have shown that about half of the published kinetic models could not be reproduced using the information provided in the manuscript. Currently, it is not possible to assess whether constraint-based models, including Genome-Scale metabolic models (GEM), are reproducible because these models often have multiple solutions, and the numerical values are not always enumerated in the published manuscripts. Therefore, one cannot ascertain whether a publicly shared CBM is the same model used in the manuscript. To tackle the reproducibility crisis, through a community effort, we have developed FROG analysis to generate a collection of numerically reproducible standardized reference datasets (aka FROG report) in a structured schema that can be used to assess the reproducibility of CBMs. We have developed a collection of tools to autogenerate FROG reports. BioModels will start curating CBMs using the FROG framework and MEMOTE test suite. To allow retrospective curation of previously published models, we propose that the model authors submit a miniFROG report and an autogenerated FROG report. The miniFROG report is a manually created data table in a standardized schema that lists at least a couple of results described in the manuscript and compares them against the model results from the FROG report. More details can be found at https://www.ebi.ac.uk/biomodels/curation/fbc.

We are preparing a manuscript to summarize our effort, which we aim to submit shortly after HARMONY2022. We invite you to evaluate the FROG analysis and tools and contribute to the manuscript by submitting your constraint-based models and FROG reports to BioModels. To contribute to the manuscript, please use the form below 
https://forms.gle/eiwD2vgU5dsa8s5p7 

A BioModels curator will attempt to reproduce and curate submitted models to increase their impact. Currently we are in process of curating over 30 models and the numbers is increasing with more submissions.

To stay connected with us and join our hands-on training sessions please fill out the form below https://forms.gle/XwrQbWguWzESgJfj6

The training session at HARMONY2022 is scheduled on 30th April 2022 12:00 UTC to 16:00 UTC. Drop in during this time, one of us will be around to help to you generate FROG reports and submit your model to BioModels. You can connect via https://embl-org.zoom.us/j/95662411973?pwd=MDNNTDVDdVFRRWoyRG5MSzg5UDdPdz09

For any questions, feel free to email me or biomode...@ebi.ac.uk with the keyword ‘FROG’ in the subject line.

We are looking forward to hearing from you.

On behalf of the FBC curation standards working group, COMBINE,
Wth best regards

Sheriff

-----------------------------------------------------------------------
Rahuman Sheriff PhD
Project Leader - BioModels
European Bioinformatics Institute 
European Molecular Biology Laboratory (EMBL-EBI)
Welcome Trust Genome Campus
Hinxton Cambridge CB10 1SD, UK
Email: she...@ebi.ac.uk
-----------------------------------------------------------------------