February 03, 2020 tedana hackathon newsletter

[tedana-newsletter]

Ported message from our previous tinyletter.com system:

Tedana Hackathon

November 6-8, 2019

 

Tedana has been a project with geographically separated contributors since it’s beginning in Fall 2017. In November 2019, many of the core developers spent several days at NIH in Bethesda, MD working together on tedana. At the hackathon, we made significant progress in all of our prioritized areas (see below). Some of our improvements are already available for use in tedana and many more either need a final review or a manageable amount of additional work before they can be available.

 

For many of the contributors to tedana, this is their first software project with multiple remote collaborators. Having novice and experienced developers work together has led to immediate improvements to how we collaborate online, which will benefit the long-term growth of tedana. We also added many of our lessons-learned into our documentation for contributing to tedana and our developer guidelines, with additional improvements to come. We can now confidently say, if you’ve never contributed to a project like this, we have the resources to help you get started.

 

In advance of the hackathon, we divided our goals into four categories: (1) User friendliness, (2) Fixing PCA/ICA, (3) Modularization, and (4) Testing. Summaries of progress in these areas during and after the hackathon are below:

 

1. User friendliness

• Documentation

  • Reorganized, explanded, and clarified with more figures

  • Documentation sections with significant expansions include:

• What is multi-echo fMRI and why you want to use it

• Considerations for ME-fMRI: Clearer explanations on acquisition options, available multi-echo pulse sequences, links to other software that use multi-echo fMRI, and known, openly available multi-echo fMRI data sets.

• Multi-echo publications: More details on publications and histograms on key parameters used across publications.

• More contributions are welcome. Anyone can add papers to the publications spreadsheet and anyone can either write text or submit issues for what they’d like to see in the documentation. Suggestions for studies that highlight where multi-echo is particularly useful are welcome.

• Reports: There is ongoing work to make a results report for tedana, which will let users interact with their data and better understand why how specific components were classified. Many of the key features were implemented at the hackathon and this report will become part of the code once the overall design and connections to the rest of the code are finalized.

 

2. PCA/ICA

• Tedana had a known bug in the dimensionality reduction step for PCA. At the hackathon a large group examined existing and potential solutions for this issue and settled on porting the Moving Average (stationary Gaussian) process method [Li, Adali, Calhoun, Human Brain Mapping 2007] that is used in GIFT to python. This now runs by default in tedana.

 

3. Modularization

• The code that calculates metrics, such as kappa and rho, for each ICA component and then uses these metrics to accept or reject each component is hard to understand and harder to modify. Modularizing this code is necessary to permit substantive algorithmic improvements.. While this has not been merged with the main code yet, significant progress at the hackathon includes:

  • Metric calculation steps are modularized so that one can take the output of ICA plus a list of metrics to calculate, and those metrics will be calculated.

  • The component classification decision tree is modularized, decision trees can be defined in human-readable files, and what happens in each decision step will be tracked.

  • The modularized code runs, but is not yet integrated into the full tedana workflow

• While the outputs should be very similar to what is currently in tedana, because this modularization will alter a massive portion of the code base, we still need to add very robust testing and validation systems to the modularized code to make sure the modularized version is as good or better than what is currently in the code.

 

4. Testing 

• Testing has always been a priority for tedana, and the hackathon provided the opportunity to make overall improvements and fill in known gaps in testing. Work at the hackathon included:

  • When tests are run, the documentation for the revised ReadTheDocs page will be automatically generated.

  • The documentation on how to design and run tests has been dramatically expanded.

  • Speeding up continuous integration - the tests that are run before new code is allowed to be merged.

  • Unit tests were consolidated for efficiency and clarity.

  • A developer can trivially run subsets of the tests. 

 

Thank you to the many people who contributed to the tedana hackathon. Team leads and areas of contributions over multiple days are noted for each person:

 

The following people who traveled to NIH for the hackathon:

César Caballero-Gaudes (PCA/ICA), Logan Dowdle (Team lead: User friendliness), Elizabeth DuPre (Testing, Modularization, User friendliness), Ross Markello (Testing, Modularization), Stefano Moia (PCA/ICA), Taylor Salo (Team lead: Modularization), Joshua Teves (Team lead: Testing), Eneko Uruñuela (Team lead: PCA/ICA, Reports)

 

Thank you to the following people at NIH who worked at the hackathon:

Javier Gonzalez-Castillo (PCA/ICA, Reports), Jennifer Evans, Daniel Handwerker (Lead hackathon organizer, Modularization), John A. Lee (Testing, Modularization), Rick Reynolds (Testing), Maryam Vaziri-Pashkam (User friendliness), Qingbao Yu

 

The AFNI Code Convergence Hackathon was the following week. Many of the same people stayed for it, and Katherine Bottenhorn made contributions to reports there.

 

Kirstie Whitaker for significant contributions to planning the hackathon.

 

At NIMH, the Section on Functional Imaging Methods (Peter Bandettini), the Scientific and Statistical Computing Core (Bob Cox), and the NIMH Scientific Director (Susan Amara) contributed funds. Dorian Van Tassell and Elaine Raab coordinated travel logistics.

For feedback on this newsletter, you may contact the this newsletter's editor at x.