Extra BIDS validation opt-in for future guidelines and extensions

[Eric Earl]

Hello BIDS Community!

I have a discussion point that came up in developing BEP036 (Tabular Phenotypic Data Guidelines) for the BIDS specification where we would love to hear new opinions. We have talked about these at length among the BEP leads and some maintainers, but your opinions may be valuable here if you could share.

The BIDS Validator currently operates its validation from a machine-readable schema (more on that here). As such, it mainly just takes changes to the schema to perform new validation for BEPs. BEP036 is primarily 8 "guidelines" in the sense that we are recommending people curate datasets with these things in mind, and if they are curating data to this BEP's standard then they should have some additional checks from the BIDS validator.

The 2 questions here are:

Should the BIDS validator enable an "opt-in" feature to perform additional validation as desired (to not impose RECOMMENDED guidelines validation on everyone)?

AND

Should guidelines be present in BIDS version 1.* as an opt-in, but become mandatory in BIDS version 2.*?

Thanks everyone for reading, and thanks especially to those who have a chance to respond. Chris M or Ross B, please chime in if I missed any of our earlier discussion points.

Eric Earl

Scientist, NIMH Data Science & Sharing Team

Maintainer, BIDS Organization

[Thomas Nichols]

Hi folks,

On the first question, having an "opt-in" feature for the validator seems like a no-brainer. However, the problem it presents is that, won't users then expect that *all* facets of BIDS that are RECOMMENDED will be captured in the schema?  I don't have a sense of the scope of work, but if RECOMMENDED aspects are not already captured in the schema it would seem to be a large effort to sweep all that into the scheme (if it is already captured, then I guess this is really a no-brainer).

On the 2nd question, I don't think any blanket statement could be made... there is just too much variation in what is covered by SHOULD/RECOMMENDED to say that *all* of it clicks over to MUST/REQUIRED/SHALL with BIDS 2.0.

-Tom

--
We are all colleagues working together to shape brain imaging for tomorrow, please be respectful, gracious, and patient with your fellow group members.
---
You received this message because you are subscribed to the Google Groups "bids-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bids-discussi...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/bids-discussion/d10d8a16-67d7-4fff-aa36-f37202ac8866n%40googlegroups.com.

[yarikoptic]

recently 

- Chris M. had to go through **many** previously RECOMMENDED fields and either make them better "scoped" as to when to RECOMMEND, or just make OPTIONAL with an "addendum" for humans to read on when to make RECOMMENDED. See https://github.com/bids-standard/bids-specification/pull/2116/files#diff-7474529e9579a720122c31af04f94b13f162aa0fad7592a55df3368c178e98b2R85 and there in

- Robert, with blessings from many, added a new WARNING (PR 1834) to warn about metadata overloading  due to inheritance, and so that it would become an ERROR for BIDS 2.0 (fulfills bids-2-devel/issues/65) 

  - Overall, it is a a good strategy to introduce "Deprecation Warnings" which inform users about upcoming changes "straight into their faces"

So, overall, I feel that we need some mechanism for tagged and potentially variable "levels".  

I wonder if we could add syntax of having a list of levels defined with some tags associated, e.g. (fictional)

SomeMetadata:

   - level: optional

   - level: recommended

     tags: [best-practices, cobidas]

   - level: required

     tags: bids:2.0

which by default (no tags) keeps it optional, but then for tags "best-practices" or "cobidas" would make it RECOMMENDED. Moreover, we could keep `bids:` as reserved prefix to signal that change would be triggered in that version of BIDS, thus could alert user informatively that "SomeMetadata would become REQUIRED in BIDS 2.0".  Also add level `deprecated` so that with `bids:` tag would inform when to be entirely removed.

Then 

- `dataset_description.json` could gain some field like `BIDSSchemaTags` to list which ones to "enable" 

- bids-validator could support taking a list of tags to enable (or disable) if there is a desire  

WDYT?