best practices for cache cleanup?

[Daniel Drucker]

https://wiki.xnat.org/documentation/xnat-it-operations/xnat-system-maintenance-tips mentions deleting old stuff out of the cache.

(1) Does anyone have any best practice suggestions for this? E.g., do people do a cron job with a find -mtime type cleanup?

(2) Is there a reason XNAT doesn't have functionality to do this cleanup on its own? It feels to me like having a "Clean up cache files after N days" function in settings would be useful to most people.

Daniel

[Moore, Charlie]

I can't comment on (1), but I can say that for (2), this is something that is starting to get some more traction to do inside XNAT. See https://issues.xnat.org/browse/XNAT-6772 for example. It's currently marked 1.8.2, but I must mention that it seems pretty unlikely at this time that a change like that would indeed make it into 1.8.2.

Thanks,
Charlie

---

From: xnat_di...@googlegroups.com <xnat_di...@googlegroups.com> on behalf of Daniel Drucker <ddru...@mclean.harvard.edu>
Sent: Thursday, April 15, 2021 3:48 PM
To: xnat_discussion <xnat_di...@googlegroups.com>
Subject: [XNAT Discussion] best practices for cache cleanup?

 

* External Email - Caution *

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/7a3f79e9-3db0-4a76-99f3-2d61ac4cedb8n%40googlegroups.com.

 

---

The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

[Daniel Drucker]

It makes me real nervous to just go deleting files. Does XNAT not have database entries that point to these files? If not, I guess I don't understand what the cache is.

In my mental model, either:

(a) a file in the cache has a database row somewhere that points to it - in which case by deleting the file, I've now left that row orphaned somehow - and shouldn't that database row get cleaned up too? 

or (b) there once was a row pointing to it, but XNAT cleaned up that row -- in which case, why didn't XNAT delete the file when it dropped the row?

or (c) there never had a row pointing to it, in which case I don't understand what the use of the file was in the first place

What am I missing?

[Jason Lunn]

Hi all,

Sorry to piggyback off of this chain, but we are also facing the same questions and worries, as Daniel has already outlined here, for the ICR XNATs and what to do with the cache. 

Our main concerns are about cleaning the cache (which is housing data of size on the order of the actual XNAT archive itself, it is not feasible to keep it for storage space issues) but in the process losing data that may end up in there outside of the prearchive or archive. The description given the in the link Daniel posted is very vague about what data passes through the cache, how long it stays if at all, and generally what the actual purpose is for the cache - this makes it unclear how safe it is to just 'clear out' every 30 days for example. 

From some initial diff checks of the file names and sizes in the cache vs the archive on one of our systems, there doesn't appear to be any DICOM data in the cache that isn't also safely stored in the archive which is reassuring at least.

Any more info on the cache would be greatly appreciated, thank you

BW,

Jason