Policy on LLM usage for bids-specification and bids-discussion
Chris Markiewicz
I don't know that we're getting specification text proposals written by LLMs, but I have started to see issue/discussion comments written by LLMs, and I think it is important that we set a policy that affirms the value of contributors' time.
I think we need to consider LLM usage in three contexts for this repository:
- Specification text drafting
- Communications (issues, discussions, pull request comments)
- Code generation
Each can be decided on independently, but I would suggest we adopt Ziglang's Strict No LLM / No AI Policy for the BIDS specification repository (https://github.com/bids-standard/bids-specification) and the bids-discussion mailing list (https://groups.google.com/g/bids-discussion):
No LLMs for issues.
No LLMs for pull requests.
No LLMs for comments on the bug tracker, including translation. English is encouraged, but not required. You are welcome to post in your native language and rely on others to have their own translation tools of choice to interpret your words.
BIDS as a standard values human legibility, and people generously contribute their time reading and commenting on the specification, BEPs, issues, comments and PRs. Posting large blocks of text that need to be evaluated for dubious claims feels to me an abuse of that generosity and risks disengagement. Large blocks of text written by humans already get relatively little engagement, despite the obvious effort the contributors put in.
The code in bids-specification is written in Python and small enough that it should be legible to any interested contributor, even if it takes more time than siccing Claude on it. I am personally willing to pair program with anybody who finds it too forbidding to get started.
This is not yet a large problem (as far as I can see), and my goal here is not to call anybody out, but to have this discussion before it becomes a problem and people start silently walking away. Ultimately, I think this will need a decision from steering.
Dr Cyril Pernet
--
We are all colleagues working together to shape brain imaging for tomorrow, please be respectful, gracious, and patient with your fellow group members.
---
You received this message because you are subscribed to the Google Groups "bids-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bids-discussi...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/bids-discussion/CAJqDEcGd58bmsRoOJyFUd2jr22pGUsov0v_Agu2O2-LJ5MWFdA%40mail.gmail.com.
Chris Gorgolewski
Robert Smith
- False attribution / responsibility / credit.
- Disproportionate expectations of few maintainers / reviewers to carefully critique content relative to many investing minimal effort in producing voluminous content.
- Attribution:
- All discussion content copy-pasted from an LLM are to be block-quoted and explicitly attributed to the generating LLM.
- LLM-generated content posted in a discussion must be accompanied with a description of why / how that content is suitable / insightful, not just dumped with expectation of others to interpret.
"Hey, what do you think of this LLM-generated answer?" isn't a valuable community contribution; the person at the receiving end could have generated such themselves. - All commits to which an LLM contributed are to be explicitly tagged as such in the commit footer.
Personally I'm going one step further and (automatically) quoting all LLM prompts in commit bodies—given that's what constitutes my "contribution" & they could be insightful when back-tracing issues in the future—but understand that's overkill for most.
- All discussion content copy-pasted from an LLM are to be block-quoted and explicitly attributed to the generating LLM.
- Disproportionate expectations:
Consider a hypothetical individual who wishes to take advantage of LLMs & goodwill of maintainers to generate maximal credit to self for minimal personal investment by passing off LLM-generated content as their own.
This individual would be in violation of either the Ziglang policy or my suggestions.
The difference being that:- If outright forbidden, there's always an underlying scepticism that others in the community are nevertheless getting away with exploiting these tools, thereby rationalising oneself doing the same;
- If there's prevalent precedent for how to use them in a community-acceptable way, conformity to those rules is implicitly encouraged.
Paola Di Maio
To view this discussion visit https://groups.google.com/d/msgid/bids-discussion/281c4049-1a08-49ab-aac9-0982cb7cda22n%40googlegroups.com.
Tibor Auer
Tibor
Dr Tibor Auer, MD, PhD, FHEA
Sent: 11 May 2026 08:04
To: bids-di...@googlegroups.com <bids-di...@googlegroups.com>
Subject: Re: [bids-discussion] Policy on LLM usage for bids-specification and bids-discussion
Chris Markiewicz
Hao-Ting Wang
LLM use made "passing all tests / the code can run" less of a reliable indicator of whether a contributor's understanding of the problem/code base. It turns the workload of digesting things back to the reviewer/maintainer.
To view this discussion visit https://groups.google.com/d/msgid/bids-discussion/CAJqDEcHPDStAqhhqXV5TmQGnqHDttDEzFmPhVt_Gsx%2BQPXC8qA%40mail.gmail.com.
Paola Di Maio
Warning: apologies for the long post.
Short summary at the top for brevity. Just sharing reflections on disclosure
Summary
Appropriate use of AI is not the same as blind reliance or rejection of technology.
BIDS is intended as a neuroscience data standard, not only a programmer ecosystem. Is it not?
AI tools and LLMs can democratize access to computational capabilities previously limited to well-funded institutions.
Transparency discussions should include funding, infrastructure, software access, training, accommodations, and institutional privilege — not only disclosure of LLM use.
Are there others in the BIDs ecosystem interested in discuss neurodegenerative disease, cognition, and brain research in plain language, not only through code and technical environments.
Over-reliance on technology is not good practice, but under-reliance may also be a problem. What should we consider “appropriate reliance”?
There is great literature , discuss concepts such as the “Over-reliance Rate” ((O_{wrong})), calculated as the frequency of accepting an incorrect AI suggestion. . Is this the case?
Isn’t BIDS supposed to be a data model for neuroscience, so researchers can leverage the massive amount of neuroscience data now being produced and made openly accessible?
(That is how I first came across BIDS some years ago)
Not only as a community of programmers concerned with demonstrating the integrity of their coding skills.
I do not intend to diminish the value of anything this community does, nor criticize it.
Some AI-generated output is impossible for humans to process effectively. But some AI output may genuinely extend human reasoning capability. In many cases — dare I say it — AI-generated code can be extremely good, at near-zero cost. Humans can learn a lot from AI
What we need to build into the process are learning and evaluation skills. Use AI to reason then come up with your own thing
Brain research has become so dependent on informatics that perhaps some direction may have been lost in the process. I may be wrong. If everyone understands the whole picture, perhaps they can explain it to others. Without AI it is impossible to capture.
Btw anyone working on BIDS also researching the causes, prevention, and treatment of neurodegenerative disease and cognitive decline — and having conversations about these topics in plain language rather than primarily in Python?
If so, please speak to me. That is what I want to discuss.
This is not about justifying over-reliance on anything.
It is about integrating new technical capabilities that are now available to everyone into development, learning, and research workflows that were previously accessible only to a small number of people entering academic institutions or participating in funded consortia.
It is about becoming more self-reliant:
being able to find, understand, query, and manipulate brain research datasets,
leveraging new learning opportunities and technical capabilities,
and not depending entirely on others for access to tools or validation of whether your code “works,” whether it is “good,” or whether you wrote it “correctly.”
We are being asked how to evaluate the quality of AI-generated code.
There are already established methods, parameters, and criteria for evaluating code, software, and research outputs.
LLMs are now universally accessible. Yes, they are sometimes used unintelligently or irresponsibly — like any technology.
At the same time, expensive scientific software required to work with complex datasets is often accessible only to faculty and students at certain institutions, frequently at great public cost. They get promoted for using technology infrastructure and dont have to disclose anything.
Meanwhile, publicly accessible LLMs can perform many useful computational functions with minimal setup.
Do we have demographic information about the BIDS developer and user community?
Do all BIDS users have to be programmers?
Have institutional access, funding, software access, and learning opportunities relating to BIDS been made transparent?
What about accessibility?
How many people in the BIDS ecosystem may be on the autistic spectrum, or might benefit from different training approaches? How many are able to access accommodations?
Transparency is not only about disclosing the use of LLMs.
It is also about publishing information about:
how much funding individuals and institutions receive,
what infrastructure and software environments they have access to,
what computational capacity they possess,
what licenses and tools they use,
and who is actually able to participate meaningfully in brain research.
How many researchers truly have access to the entire suite of technical tools and expertise required to conduct modern brain research? That could be interesting to know.
Should disclosure requirements also include:
tools,
licenses,
computational environments,
computational capacity,
training,
skills,
and the composition of the BIDS ecosystem itself?
There is a great deal to unpack if we are genuinely interested in transparency — far beyond simply discussing LLM use.
LLMs can provide analytical and technical capabilities to anyone willing to learn how to use them.
Some of these same capabilities may already exist inside proprietary software, platforms, and institutional environments that many users will never access — or may never even know exist.
Should those LLM capabilities embedded in other technology also be disclosed?
How much LLM integration already exists within brain research platforms that the community may not currently expect to be disclosed under the rules being discussed?
When someone creates a method, then uses software tools to refine it, debug it, integrate it, and produce a clean, presentable webpage that explains the method and runs the code:
are developers expected to disclose every suite, tool, and function they used?
or only a specific category of LLM products?