Contributing to XNAT

[Paul Smith]

Hi,

I'm Paul, a software developer supporting the XNAT service we run at UCL.

At the XNAT Workshop in London last year, it was mentioned that there is a goal to have XNAT be fully maintained and developed by the community once current grant funding runs out.

I would be happy to contribute to the development of XNAT, but I'm not sure where to start. I've seen there is some developer documentation (https://wiki.xnat.org/documentation/xnat-developer-documentation), but I'm still not too sure where to begin.

I'm wondering whether there are any plans to run a Developer Workshop where these topics are introduced? Or perhaps a series of webinars? I'd love to learn about the internals of XNAT and how I can start to contribute. I think covering the following topics would be super useful:

- the structure of the codebase

- how to set up a development environment

- how to run tests

- what issues are good for first time contributors

I think a good goal would be for the attendees of the workshop / webinars to help update the existing developer docs to make it easier for first-time contributors to get started.

Apologies if all this information is already available - I may well be missing something!

Thanks

Paul

[John Flavin]

Hi Paul,

TL;DR

• No plans right now for Developer Workshop or similar, but anyone else interested should let us know.
• Documentation of the code itself and the application architecture is sparse.
• I give some developer setup instructions below.

Full Answer

First let me say thank you for this question. It is exciting to see members of the community interested enough and engaged enough to try to start digging into how to develop XNAT. A community-supported XNAT is absolutely the future we want to create, and I'm grateful that you seem to want to get started on that future right now. I do too!

But you've quickly encountered one of the first hurdles to developing XNAT: it is not easy to figure out how to start. You’re definitely not the only person who encounters this problem. In fact there was another discussion board post just recently that ran up against a similar problem; when the post author looked into building a plugin to do some thing that XNAT doesn't do yet, their (entirely reasonable) response was

I got down the Rabbit hole of designing my own [plugin], and got lost in java, groovy, gradle, maven and then stopped.

That's just the first step in the process, and it was already too difficult (or too poorly documented) for a presumably smart, capable person to be able to make that commitment of time and energy. That experience will need to get better if we expect more people to be involved even deeper in XNAT's development.

Back to your specific questions. We don't currently have any plans for a Developer Workshop or any webinars.  That's not to say that we won't do those things or haven't discussed them, just that we don't have any plans as of today. We have definitely noted your interest, though, and if others also let us know they are interested that could influence our prioritization.

I'll go through your other questions at a high level. I do think a mini-workshop would be a better way to deliver this information, but I can give you a bit to go on now.

- the structure of the codebase

We don't have this documented as well as we should. As you found already, we have the XNAT Developer Documentation, but it is light on these kinds of fundamental details like the structure of the code base and the architecture of the application.

We have Understanding the Components of XNAT, but that's written for an audience of XNAT administrators, not developers. Might be helpful, might not.

The XNAT Under the Hood session from the 2016 Workshop seems likely to help with this. But I haven't seen it since I was there live and my memory of its contents is a bit hazy.

- how to set up a development environment

This is something we do have some internal documentation about that isn't available anywhere public. It doesn't have any kind of sensitive content or anything, so I can summarize it here.

You'll need git installed. I think it's likely that you already have it so I'll leave the details out.

You should get a Java IDE. Most of the core XNAT team use IntelliJ. The free version will almost certainly be enough to get you started.

Another thing you will need is the Java 8 JDK. That's the version that we build everything with. How to download it is actually far more complex than I would expect because there are versions from multiple vendors with varying coverage for different platforms. It could be as simple as using your package manager of choice, i.e. brew install openjdk@8 on a mac (but only if it has an intel chip).

If you have IntelliJ, you can install a JDK through it. Open a project (could be anything, even an empty one). From there open the project settings (⌘+; on a mac). From there open Platform Settings > SDKs on the left pane. Click the + button in the middle pane and select Download JDK…. Select version 1.8. Pick a vendor. I’m not sure if the particular selection matters or not, but you need to select one that matches your architecture. For instance, I'm on an M1 mac, so I need an aarch64 version. I go with the Amazon Corretto version, myself, but I really am not sure it matters.

Once you have a JDK you should install the build tools. First, maven. It's available via package managers. It's called "maven" but on the terminal the command is mvn which is odd but it is what it is. We also use gradle as a build tool but you won't need to install it manually because we use "wrapper scripts" in each repo that download gradle for you. You'll run, say, ./gradlew war to build the XNAT WAR file and it will work.

Also note that you will need to set the JAVA_HOME environment variable to point to the JDK you installed. There is a magical command that will figure it out for you. Put this command into your shell's configuration so it runs every time you open a terminal:

export JAVA_HOME=$(/usr/libexec/java_home -v 1.8)

(Assuming bash or zsh or similar. Adapt for different shells as needed.)

You know you have everything set up correctly when you can run mvn -version or ./gradlew --version (from within a repo directory) and it prints out the correct JDK version you installed.

- how to run tests

We have a mix of tests happening at different levels. Most of the core repositories include unit tests that run at build time, which are definitely easier and faster to use during development but give you a limited picture of the impact of a change on the whole application. You could likely run these using whatever build tool is used in the repo, either mvn test for maven or ./gradlew test for gradle, though the details may vary a bit by repo.

We also have the XNAT REST Tests, which are a suite of integration tests that run against a live XNAT. It's harder to set all this up and they are too slow to run manually as a regular part of development, but they are a very valuable part of automated testing infrastructure.

- what issues are good for first time contributors

Another good question. I don't have anything at hand. But we might want to start tagging issues that would be good for this purpose.

I think that's all for now. Again, thank you for asking this question. I think that connecting with people who want to get started developing on XNAT itself is a wonderful idea, and I would love it if we could organize something like that in the future. The information I've given here is admittedly not that much but hopefully it helps a little. 

John Flavin

Backend Team Lead

johnf...@flywheel.io

He/Him

flywheel.io

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/57b1e234-4bdf-40a9-aa02-b6b066c5f52en%40googlegroups.com.

[Martin Boswell]

John,

I'd also like to express my interest in resources for XNAT development.  Thanks for sharing this info; it's very handy summary. 

Is there any detailed documentation of the postgres schema?  Or even just brief descriptions of the purpose of each of the tables?  I've tried to pick some information out of there and quickly gotten confused.  

  - Martin

Martin S. Boswell
Cellular and Tissue Imaging Department
Molecular Biophysics and Integrated Bioimaging Division
Lawrence Berkeley National Laboratory
One Cyclotron Road, MS55R0121
Berkeley, CA   94720

[Guus Kolpa]

I'd also like to express my interest in contributing. I ran into the exact same scenarion as outlined above:

"I got down the Rabbit hole of designing my own [plugin], and got lost in java, groovy, gradle, maven and then stopped"

One of the reasons I did not pursue any further involvement is that development seems to be closed-source on purpose. While the XNAT code-base and surface-level issues ( https://radiologics.atlassian.net/jira/software/c/projects/XNAT/issues/?filter=allissues ) are open to the public, other pages such as https://radiologics.atlassian.net/projects/XNAT/versions/14885/tab/release-report-all-issues require a radiologics account, giving me much more of a distant observer / read-only perspective than that of an open invitation to join in on contributing to open-source software. 

[Paul Smith]

Hi,

Sorry for the very slow reply to this, but thanks so much John for the detailed info!

I managed to build `xnat-web` following the steps you outlined. I also opened a PR to add this info to the docs.

Martin and Guus, nice to know others are interested in contributing too! I agree with Guus that whilst we can view open issues, it would also be nice if we could submit new issues too.

Guus, what were the issues you ran into when creating the plugin? We have a bit of experience in the group of making plugins. It might be fun to work together on one and perhaps improve the docs around creating plugins.

On the topic of docs, is there anywhere we can make or suggest improvements to the wiki? 

Best wishes,

Paul

[Battiston, Christopher]

Just coming on to say I'd be possibly able to help with testing and validation of new features.  I'm not a programmer but I am very experienced in testing web apps (specifically REDCap) and would be happy to help with developing / running scripts and other documentation.  

Thanks, 

Chris 

---

From: xnat_di...@googlegroups.com <xnat_di...@googlegroups.com> on behalf of Paul Smith <paul.14...@gmail.com>
Sent: Monday, April 15, 2024 10:29
To: xnat_discussion <xnat_di...@googlegroups.com>
Subject: [External]:Re: [XNAT Discussion] Contributing to XNAT

 

You don't often get email from paul.14...@gmail.com. Learn why this is important

CAUTION: This email originated from outside of Women's College Hospital. Do not click links, open attachments or respond to requests for information unless you recognize the sender and know the content is safe.

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/6dc372ca-a6fc-4c97-9a0e-5a52a54d41d2n%40googlegroups.com.

CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.

AVIS DE CONFIDENTIALITÉ : Ce message électronique, ainsi que tout fichier qui y est joint, est réservé à l’usage exclusif du destinataire visé et peut contenir des renseignements confidentiels et privilégiés. Toute lecture, utilisation, divulgation ou distribution non autorisée est interdite. Si vous n’êtes pas le destinataire visé, veuillez en aviser l’expéditeur par retour de courriel et détruire toutes les copies du message original.

[Ford McDonald]

Thanks for the helpful information!

I'd also like to express interest in contributing to future development and attending a developer's workshop. 

Best,

Ford McDonald

[Mark Janse]

Hi all,

If a developer workshop is held, I'd love to attend as well.

My organization's main interest is the long-term maintainability of XNAT, updating the dependencies (Java 8, Tomcat 9) is something we'd like to contribute to.

Kind regards,

Mark Janse

Health-RI

Op vrijdag 19 april 2024 om 19:18:45 UTC+2 schreef Ford McDonald:

[Battiston, Christopher]

Just wanted to jump back on this and see if there's been any further discussion.  Would really love to get into this with other XNAT institutions. 

Thanks, 

Chris 

Women’s College Hospital is located on the traditional lands of the Anishinaabe, Mississauga, Haudenosaunee, Ojibway/Chippewa and Huron-Wendat nations. 

---

From: xnat_di...@googlegroups.com <xnat_di...@googlegroups.com> on behalf of Mark Janse <marku...@gmail.com>
Sent: Monday, April 22, 2024 05:41
To: xnat_discussion <xnat_di...@googlegroups.com>
Subject: Re: [External]:Re: [XNAT Discussion] Contributing to XNAT

 

You don't often get email from marku...@gmail.com. Learn why this is important

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/29bf60f4-e1b9-49b6-a906-0d2fa80712ecn%40googlegroups.com.

[DrRadiohead]

Dear All,

To add to the discussion.

If such a workshop is held please let us know too. Our organization would love to attend.

Regards

[George Kowalski Sr]

I'm on board for helping development and a workshop ( if it's held in the USA, can't get my boss to pony up for international work  ) 

G