XNAT studies sync

[Jose Villamizar]

Hello,

We are interested on a use case that involves keeping the data synchronized between two instances of XNAT on different networks between a medical facility and a research institute from a higher education institution. The studies must be copied based upon a specific time frame (24 hours), and a specific list of studies that is hosted on a GitHub repository. These studies need to be synced bidirectional to keep the information updated on both sides, and only the studies indicated on the GitHub repo file will be copied. Also, no PII or PHI information is present. The studies are de-identified before being transferred to their respective external networks.

One possibility that occurred to our team was to query the PostgreSQL database directly to isolate the file(s) that were modified within a specific time frame, and get the full path of the file(s) to perform a copy to a CIFS share, that eventually will copy the file(s) to their respective network for ingestion. Obviously for this approach to work properly, the PostgreSQL database must have the modified time of the file(s), the full path where the modified file(s) reside, and any other metadata that allow the isolation of these file(s).

Is this approach possible?
Are there any tools or plugins that might do this for us?
Any suggestions will be greatly appreciated.

Sincerely,

JV

[Jose Villamizar]

Good morning!

Our request so far does not have any replies is this approach possible?

Regards,

JV

[Rick Herrick]

> One possibility that occurred to our team was to query the PostgreSQL database directly

> to isolate the file(s) that were modified within a specific time frame, and get the full path of

> the file(s) to perform a copy to a CIFS share, that eventually will copy the file(s) to their

> respective network for ingestion. Obviously for this approach to work properly, the PostgreSQL

> database must have the modified time of the file(s), the full path where the modified file(s)

> reside, and any other metadata that allow the isolation of these file(s).

> Is this approach possible?

Sort of, yes. You can query the database to determine when resources were updated, which will tell you indirectly when the files within those resources were updated. Something like this:

SELECT r.uri

FROM xnat_abstractresource ar

         LEFT JOIN xnat_abstractresource_meta_data armd ON ar.abstractresource_info = armd.meta_data_id

         LEFT JOIN xnat_resource r ON ar.xnat_abstractresource_id = r.xnat_abstractresource_id

WHERE armd.last_modified > '2022-12-12';

That will return you a bunch of paths, something like:

# psql --username=xnat --no-align --tuples-only --file=uris.sql

/Users/username/Development/XNAT/Home/data/archive/XNAT_01/arc001/XNAT_01_01_MR_02/SCANS/1/DICOM/scan_1_catalog.xml

/Users/username/Development/XNAT/Home/data/archive/XNAT_01/arc001/XNAT_01_01_MR_02/SCANS/2/DICOM/scan_2_catalog.xml

/Users/username/Development/XNAT/Home/data/archive/XNAT_01/arc001/XNAT_01_01_MR_02/SCANS/3/DICOM/scan_3_catalog.xml

/Users/username/Development/XNAT/Home/data/archive/XNAT_01/arc001/XNAT_01_01_MR_02/SCANS/4/DICOM/scan_4_catalog.xml

/Users/username/Development/XNAT/Home/data/archive/XNAT_01/arc001/XNAT_01_01_MR_02/SCANS/5/DICOM/scan_5_catalog.xml

/Users/username/Development/XNAT/Home/data/archive/XNAT_01/arc001/XNAT_01_01_MR_02/SCANS/6/DICOM/scan_6_catalog.xml

/Users/username/Development/XNAT/Home/data/archive/XNAT_01/arc001/XNAT_01_01_MR_02/SCANS/7/DICOM/scan_7_catalog.xml

/Users/username/Development/XNAT/Home/data/archive/XNAT_01/arc001/XNAT_01_01_MR_02/SCANS/8/DICOM/scan_8_catalog.xml

/Users/username/Development/XNAT/Home/data/archive/XNAT_01/arc001/XNAT_01_01_MR_02/SCANS/9/DICOM/scan_9_catalog.xml

/Users/username/Development/XNAT/Home/data/archive/XNAT_01/arc001/XNAT_01_01_MR_02/SCANS/10/DICOM/scan_10_catalog.xml

You could then look in those catalog files (there’s a createdTime attribute for each entry) or just look directly at the files in the folder where the catalog file is located and get newer files from there.

Just to be clear, XNAT itself does not currently store the location of every file in the archive directly in the database, just the location of the resource catalog files, each of which in turn contains references to all of the files in that particular resource.

> Are there any tools or plugins that might do this for us?

Have you looked at XSync? That performs a similar task.

Rick Herrick

Senior Software Developer

rickh...@flywheel.io

------ Original Message ------

From "Jose Villamizar" <mcv...@gmail.com>

To "xnat_discussion" <xnat_di...@googlegroups.com>

Date 12/13/2022 8:07:37 AM

Subject [XNAT Discussion] Re: XNAT studies sync

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/45a97a33-fb2d-4390-8976-33e20a7e52f2n%40googlegroups.com.

[Charlie Moore]

To add onto this: I don't expect that file syncing would be the difficult part here. Having a bidirectional sync of all of the metadata that ends up in the database that can be updated at either end is what sounds prohibitively difficult to me. 

Thanks,

Charlie Moore

[Jose Villamizar]

Thank you very much for your input. We will look into the recommended suggestions.

Cheers!

JV

[Jose Villamizar]

Hi Rick,

We are trying to configure the Xsync plugin on a brand new XNAT instance running 1.8.6.1. However, when configuring the plugin on the source XNAT we are getting an error that reads: ERROR: Could not get alias token. Please check username and password. The login credentials are correct. Do you or anyone in the group has any advice? Any documentation besides this link :  https://bitbucket.org/xnatdev/xsync/src/master/

I followed these steps on the readme.md

Thank you.