How do I clear an "archive pending" flag from prearchive

[Ben Wagner]

I tried to archive a group of sessions and failed to realize they all had the same session name.  I ended up with 1 conflict status and several archive pending.  I've cleared the conflict, but I can't seem to do anything with the archive pending ones.  I've tried all the actions, but always recieve "No checked sessions can be [insert action]".

Any ideas?

Ben

[Ryan Muetzel]

Dear All,

I've had a similar problem since upgrading to v1.6.2.1 --- sessions are getting "stuck" in the prearchive with an "archive pending" status, and they can't be reset from the web interface.

I've been using:

curl -d "action=build" -u user:password myserver:8080/xnat/data/prearchive/projects/MY_PROJECT/DATE/SESSION

Is there a better way around this issue?

--Ryan

> --
> You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
> To post to this group, send email to xnat_di...@googlegroups.com.
> Visit this group at http://groups.google.com/group/xnat_discussion.
> For more options, visit https://groups.google.com/groups/opt_out.

[Sanket Gupte]

Hi,
I have been struggling with this too. Can't seem to get that 'Archive pending'  status go away.
The scans are all in the prearchive. Is there something that I can delete/update in the database ??

Thanks

[Jörg Stadler]

Hi,
I moved the files from prearchive to a tmp folder and restarted tomcat.
Afterwards, upload the tmp files with dicom browser again.


Cheers,
Joerg

[Herrick, Rick]

This is a known issue that came up in 1.6.2. We added the extra states to handle a more complex archiving process that we introduced in 1.6.2 so that XNAT can queue incoming sessions for archiving. We had groups running really large import operations that ended up with the server trying to archive literally hundreds of sessions at a time. We added in MQ processing as an intermediary to provide sequencing for that processing, but since that also introduces asynchronicity to the process we had to add those extra states. Our testing never ran across errors that left sessions suspended in those states, but there's no testing like actual users!

So we're working on a fix to allow you to reset and rebuild sessions that have gotten stuck in these (intended-to-be) transitional states. We're also going to try to add logging and changes to exception handling to gather information on how these sessions are getting stuck, but since we're not really sure what's causing the issues it's hard to say we're putting in fixes in the core archiving code! We don't want to fix things that aren't broken and thereby break them...

Rick Herrick
Sr. Programmer/Analyst
Neuroinformatics Research Group
Washington University School of Medicine
(314) 827-4250

________________________________________
From: xnat_di...@googlegroups.com [xnat_di...@googlegroups.com] on behalf of Jörg Stadler [sta...@lin-magdeburg.de]
Sent: Tuesday, September 17, 2013 12:49 AM
To: xnat_di...@googlegroups.com
Subject: Re: [XNAT Discussion] How do I clear an "archive pending" flag from prearchive

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at http://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/groups/opt_out.

________________________________

The material in this message is private and may contain Protected Healthcare Information (PHI). If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

[Jörg Stadler]

Hi Rick,
I can reproduce this "feature" if I try to archive several sessions. If
one of them need a review (e.g. the session label is already in use) the
process stops and wait for the review. After the "review and archive"
the rest of the sessions are "pending".

Cheers,
Joerg

> This is a known issue that came up in 1.6.2. We added the extra
> states to handle a more complex archiving process that we introduced
> in 1.6.2 so that XNAT can queue incoming sessions for archiving. We
> had groups running really large import operations that ended up with
> the server trying to archive literally hundreds of sessions at a
> time. We added in MQ processing as an intermediary to provide
> sequencing for that processing, but since that also introduces
> asynchronicity to the process we had to add those extra states. Our
> testing never ran across errors that left sessions suspended in those
> states, but there's no testing like actual users!
>
> So we're working on a fix to allow you to reset and rebuild sessions
> that have gotten stuck in these (intended-to-be) transitional states.
> We're also going to try to add logging and changes to exception
> handling to gather information on how these sessions are getting
> stuck, but since we're not really sure what's causing the issues it's
> hard to say we're putting in fixes in the core archiving code! We
> don't want to fix things that aren't broken and thereby break
> them...
>
> Rick Herrick Sr. Programmer/Analyst Neuroinformatics Research Group
> Washington University School of Medicine (314) 827-4250
>
> ________________________________________ From:
> xnat_di...@googlegroups.com [xnat_di...@googlegroups.com]

> on behalf of J�rg Stadler [sta...@lin-magdeburg.de] Sent: Tuesday,

--
Dr. rer. nat. Joerg Stadler
Leibniz Institute for Neurobiology (LIN)
Special Lab Non-Invasive Brain Imaging
Brenneckestr. 6, D - 39118 Magdeburg
Tel.: +49-391-6263-92171 Fax: +49-391-6263-92589
Gnupg-Key:http://wwwkeys.pgp.net search for E-Mail Adress
Key fingerprint:CA09 39ED 5019 1AB8 6C6E 616F E7A3 8B24 E460 E70C

[Simon Doran]

I have a similar problem, but with the status "Building session". XNAT crashed hard with "Out of permgen space" and now I am left with a session that I can't delete in the prearchive and without the ability to duplicate the session name when I next try to upload the data.

Simon

[dag]

Is this still the current working solution for the session building issue?

Also has anyone had any trouble uploading a ZIPPED file to 1.6.2.1.... for some reason I can upload via DICOM C-STORE but when I try and just upload a zip, it seems to be creating the directory, but not unzipping the file(s).... or maybe getting stuck somewhere else... the stack trace was not particularly readable..

[David Gutman]

Rick is there any updated code that allows us to reset things stuck in the archive building limbo or just go with a manual approach?

e.g.

curl -d "action=build" -u user:password myserver:8080/xnat/data/prearchive/projects/MY_PROJECT/DATE/SESSION

--
David A Gutman, M.D. Ph.D.
Assistant Professor of Biomedical Informatics
Senior Research Scientist, Center for Comprehensive Informatics
Emory University School of Medicine

[Herrick, Rick]

Yes, in 1.6.3 you can clear these pending and queued flags in the prearchive. I think you have to be an admin to clear some of the states, since resetting the session in the middle of, e.g., an archive operation could lead to really bad results if it’s actually archiving.

 

Rick Herrick

Sr. Programmer/Analyst

Neuroinformatics Research Group

Washington University School of Medicine

(314) 827-4250

 

[David Gutman]

Ok cool---  I just hacked a solution using the CURL post... but upgrading to 1.6.3 may be the better approach...  are the upgrade instructions online or is 1.6.3 still in testing..

[Herrick, Rick]

1.6.3 is in the latest throes of childbirth, arrival imminent, virtual father(s) nervously clutching swaddling blanket in anticipation. Upgrade instructions will be available as soon as the release drops. Look for Twitter post, Water Cooler post, and post here on the discussion group.

[Simon Doran]

Hi All,

  Resurrecting some very old discussion here ...

  I have a whole set of files that, for one reason or another, have got "stuck" over the years in one of our production systems - still running on 1.6.5, I'm afraid. Here's a recent example:

  I really want to get rid of all this stuff out of the prearchive, so that I can re-upload the affected sessions.

  Having found the location of the offending data in XNAT's prearchive directory, I tried this:

curl -X POST -u admin "https://<HOST>/XNAT/REST/services/prearchive/delete?src=/prearchive/projects/MR_BTC/20190715_152352062/20170921_090528_Achieva"

but it didn't seem to do anything at all. What I got back was an XML ResultSet that looked like this:

<ResultSet><results><columns><column>project</column><column>timestamp</column><column>lastmod</column><column>uploaded</column><column>scan_date</column><column>scan_time</column><column>subject</column><column>folderName</column><column>name</column><column>tag</column><column>status</column><column>url</column><column>autoarchive</column><column>prevent_anon</column><column>prevent_auto_commit</column><column>SOURCE</column><column>VISIT</column><column>PROTOCOL</column><column>TIMEZONE</column></columns>

<rows><row><cell>MR_BTC</cell><cell>20190715_152352062</cell><cell>2019-07-15 15:29:58.531</cell><cell>2019-07-15 15:23:52.062</cell><cell>2017-09-21 00:00:00.0</cell><cell>09:05:28</cell><cell>BAR331</cell><cell>20170921_090528_Achieva</cell><cell>20170921_090528_Achieva</cell><cell>1.2.840.113564.9.1.2843759204.47.2.5001264056</cell><cell>_ARCHIVING</cell><cell>/prearchive/projects/MR_BTC/20190715_152352062/20170921_090528_Achieva</cell><cell>AutoArchive</cell><cell>false</cell><cell>false</cell><cell></cell><cell></cell><cell></cell><cell></cell></row>

</rows></results></ResultSet>

which seems just to tell me that the data are still in an ARCHIVING state.

  What do I need to do to nuke these contents of the prearchive and start again?

  Best wishes,

Simon

[Herrick, Rick]

Try this:

 

https://wiki.xnat.org/display/XKB/Clearing+stuck+sessions+from+the+prearchive

 

Basically:

 

1. Shut down Tomcat.
2. Delete everything in the xdat_search.prearchive table: DELETE FROM xdat_search.prearchive;
3. If there are any problematic sessions in your prearchive (you don’t want the data any more, the data is corrupt and locks in the prearchive every time, whatever), delete those folders.
4. Restart Tomcat.

 

Upon restart, XNAT looks in the prearchive folder and generates entries for any sessions in there, so your valid data will get picked up and processed properly. Any problematic entries should be gone with that.

 

-- 

Rick Herrick

Sr. Programmer/Analyst

Neuroinformatics Research Group

Washington University School of Medicine

Phone: +1 (314) 273-1645

 

From: "xnat_di...@googlegroups.com" <xnat_di...@googlegroups.com> on behalf of Simon Doran <simon...@icr.ac.uk>
Reply-To: "xnat_di...@googlegroups.com" <xnat_di...@googlegroups.com>
Date: Tuesday, July 16, 2019 at 12:41 PM
To: "xnat_di...@googlegroups.com" <xnat_di...@googlegroups.com>
Subject: Re: [XNAT Discussion] How do I clear an "archive pending" flag from prearchive

 

Hi All,

 

  Resurrecting some very old discussion here ...

 

  I have a whole set of files that, for one reason or another, have got "stuck" over the years in one of our production systems - still running on 1.6.5, I'm afraid. Here's a recent example:

 

 

  I really want to get rid of all this stuff out of the prearchive, so that I can re-upload the affected sessions.

 

  Having found the location of the offending data in XNAT's prearchive directory, I tried this:

 

curl -X POST -u admin "https://<HOST>/XNAT/REST/services/prearchive/delete?src=/prearchive/projects/MR_BTC/20190715_152352062/20170921_090528_Achieva"

 

but it didn't seem to do anything at all. What I got back was an XML ResultSet that looked like this:

 

<ResultSet><results><columns><column>project</column><column>timestamp</column><column>lastmod</column><column>uploaded</column><column>scan_date</column><column>scan_time</column><column>subject</column><column>folderName</column><column>name</column><column>tag</column><column>status</column><column>url</column><column>autoarchive</column><column>prevent_anon</column><column>prevent_auto_commit</column><column>SOURCE</column><column>VISIT</column><column>PROTOCOL</column><column>TIMEZONE</column></columns>

<rows><row><cell>MR_BTC</cell><cell>20190715_152352062</cell><cell>2019-07-15 15:29:58.531</cell><cell>2019-07-15 15:23:52.062</cell><cell>2017-09-21 00:00:00.0</cell><cell>09:05:28</cell><cell>BAR331</cell><cell>20170921_090528_Achieva</cell><cell>20170921_090528_Achieva</cell><cell>1.2.840.113564.9.1.2843759204.47.2.5001264056</cell><cell>_ARCHIVING</cell><cell>/prearchive/projects/MR_BTC/20190715_152352062/20170921_090528_Achieva</cell><cell>AutoArchive</cell><cell>false</cell><cell>false</cell><cell></cell><cell></cell><cell></cell><cell></cell></row>

</rows></results></ResultSet>

 

which seems just to tell me that the data are still in an ARCHIVING state.

 

  What do I need to do to nuke these contents of the prearchive and start again?

 

  Best wishes,

 

Simon

--

You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.

Visit this group at https://groups.google.com/group/xnat_discussion.
To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/4c76dada-4c52-4b12-a076-19a507706b44%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

 

---

The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.