Dataverse upload tmp files

[Bethany Seeger]

Hello, 

We are setting up automated clean up of a few directories, and I wanted to check in about temp files in the upload directory before we setup something up for those. 

In the following directory listing it's unclear to us is what the small tmp files represent.  I understand that the last one (the 3.9GB file) is a failed upload and we need to remove it.  

/usr/local/payara5/glassfish/domains/domain1/uploads: 

drwxr-xr-x. 14 dataverse dataverse        191 Jul 28  2023 ..
-rw-r--r--.  1 dataverse dataverse         47 Nov 11 13:05 upload_65bab1d2_1917540dd6d__6013_00008600.tmp
-rw-r--r--.  1 dataverse dataverse         22 Nov 11 13:05 upload_65bab1d2_1917540dd6d__6013_00008599.tmp
-rw-r--r--.  1 dataverse dataverse         45 Nov 11 13:05 upload_65bab1d2_1917540dd6d__6013_00008598.tmp
-rw-r--r--.  1 dataverse dataverse          4 Nov 11 13:05 upload_65bab1d2_1917540dd6d__6013_00008597.tmp
-rw-r--r--.  1 dataverse dataverse         38 Nov 11 13:05 upload_65bab1d2_1917540dd6d__6013_00008596.tmp
-rw-r--r--.  1 dataverse dataverse          5 Nov 11 13:05 upload_65bab1d2_1917540dd6d__6013_00008594.tmp
-rw-r--r--.  1 dataverse dataverse         11 Nov 11 13:05 upload_65bab1d2_1917540dd6d__6013_00008591.tmp
drwxr-xr-x.  2 dataverse dataverse       4096 Nov 11 13:05 .
-rw-r--r--.  1 dataverse dataverse 3904314225 Nov 11 13:18 upload_65bab1d2_1917540dd6d__6013_00008601.tmp

For the small files, some simply have: 

"datasetForm:fileUpload" 

in it, or,  

"datasetForm:fileUpload datasetForm:filesTable" 

or "true". 

Given that they are tmp files, do they have any significance or can we just clean them up on a regular basis? 

We are cleaning up files in the above location as well as the space that dataverse unpacks files into (in our case /usr/local/dvn/data/temp).  Are there other spots we should be aware of?   (aside from log directories). 

Thanks so much,

Bethany

[Philip Durbin]

Hi! It's safe to delete these files. We have a fix for this already merged for the next release (6.4). For details, please see https://github.com/IQSS/dataverse/pull/10867 and https://github.com/IQSS/dataverse/issues/10667

Also, we've been meaning to document temp files better: https://github.com/IQSS/dataverse/issues/2848

Thanks for the reminder! :)

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/dataverse-community/c15575a7-b963-409f-bfe5-f3ef79ef1d2bn%40googlegroups.com.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

[Bethany Seeger]

Thanks, Phil!