Is _this_ possible with RS?

[Douglas Thompson]

So, we're a university with thousands of photos. We have a limited amount of storage space that our IS folks have given us, so we have to do a yearly "purge" of "older" resources from the system in order to free up some of that space.

Older stuff (purged and otherwise) exist on a couple USB drives that we keep protected and only plug in to a computer when needed.

What we'd like to be able to do is:

1. Keep the thumbnails/previews and metadata (keyword, mostly) search capability in RS. For resources that are no longer in RS, though, we'd like some sort of "offline" indicator (or something along those lines), so that the people doing the searching will know when they've found an older photo that they need to grab off a USB drive.
2. Not just do this as a one-time thing...we'll no doubt have to purge things again next academic year (each academic year is a good sectioning point for this stuff, really).

For #2, even if it's something that can only be accomplished by manually running a script (or something), that's still a lot better than not being able to deal with it at all.

I've looked into the Archive functionality and the Offline Archive plugin, but haven't yet been able to get it all to work the way I want.

The wrinkle is that I'm not really much of a programmer, so I'm pretty much going to have to stick with out-of-the-box solutions here.

Any ideas how I can accomplish the basic goal of having a robust RS but balancing that with the fact that we have a finite amount of storage space to work with?

[Douglas Thompson]

So I just had a thought that might help our situation a tad...right now, all "originals" we upload to RS are resized down to (where necessary) 4000 pixels wide. So, for the most part, the "original" size is 4000 pixels wide by whatever the height is.

What if we were to change that to 3000 pixels wide instead of 4000 pixels wide? That's a reduction of 25% in size, so probably comes out to roughly that amount in disk space usage, right?

My question is...is there a way for me to resize the originals after-the-fact without having to re-upload all of them? I'm betting there _is_, but I don't know for sure...

(we keep the full-size originals on the USB drive mentioned above, so no worries if something larger than 3000 pixels wide is needed)

[Joseph Brooke]

Hi there,

To address your first message there is the config option $originals_separate_storage - if you enable that in your ResourceSpace config.php file and then run the script pages/tools/filestore_separation.php it will split your filestore into two folders - "original" and "resized". Original uploaded resource files will be stored in the "original" folder while the preview files generated for that resource will be stored in the "resized" folder. You could then try writing a script that regularly purges resources in the "original" folder when they reach a certain age or have an ID less than a certain value. Unfortunately there are no built in scripts to do this in ResourceSpace, ChatGPT may be able to help with writing the script. 

For your second question - if you change the size of the previews created via the Manage Sizes area - https://www.resourcespace.com/knowledge-base/systemadmin/manage_sizes - you need to recreate previews for your resources in order to generate the smaller sizes. You can do this by adding all your resources into a collection and in the Actions menu select "Edit collection previews". 

[Douglas Thompson]

I'll give the first one a try when I get a chance...does just deleting the "originals" automatically make them go offline or something?

For the second one, does regenerating the previews also alter the sizes of the originals? I'm betting it doesn't, because I only see the resized-down sizes in that System -> Image Sizes screen. If I do the first thing and then download all of the originals and do a batch-resizing (with Adobe Bridge or whatever) on those, can I just FTP upload them back to the separate "originals" filestore folder structure, or will that screw things up?

--
ResourceSpace: Open Source Digital Asset Management
http://www.resourcespace.com
---
You received this message because you are subscribed to a topic in the Google Groups "ResourceSpace" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/resourcespace/hhvmvn8nbXI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to resourcespac...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/resourcespace/380f2678-29d6-47d1-950d-9cca875311a3n%40googlegroups.com.

--

Douglas E. Thompson Manager of Web and Electronic Communications Office of University Communications Ohio Wesleyan University 61 S. Sandusky St. Delaware, OH 43015 Email: deth...@owu.edu Office Phone: 740-368-3356 Cell Phone: 614-582-6564 Website: owu.edu

[Joseph Brooke]

If you delete the original file for a resource the record will still exist and will be searchable and viewable in the ResourceSpace interface, you will see "Offline resource" in the download area for the original file but you can still download the preview sizes. 

Regenerating previews doesn't alter the size of the original file - it uses the original file to create the previews but that remains unchanged. So if you need to do any changing of preview image sizes you need to do that first before you delete any original files, otherwise when you recreate previews you will get no previews at all. 

FWIW I've updated the Knowledge Base article for the offline archive plugin with more steps on how to use it - https://www.resourcespace.com/knowledge-base/plugins/offline_archive It still might be useful for your situation. 

[Douglas Thompson]

Clarification needed...I think I have a general process worked out in my head:

1. Decide what needs to be purged (we probably don't want/need to do this to everything after initial upload (so that the "resized" stuff is created), so we'll still likely periodically need to identify stuff that we definitely don't need to keep the originals in RS.
2. Add everything (or just a few collections at a time...it really doesn't matter) to a single temporary collection that we make just for this.
3. Use the offline archive plugin in some way, so I'll know which ones to delete from the "original" folder structure (or to automatically delete them or something)
4. Done. The previews and keywords and other metadata and whatnot for everything will still be in RS, but the original size (which, really, is the one that takes up the most space) will say "offline resource" instead of having a Download button. When we see that, we'll know to grab the original from the disk.

So it's entirely down to figuring out how #3 can/should work in our situation...what did you mean by "it still might be useful for your situation"?

[Douglas Thompson]

Might it be possible to specify some sort of non-existent/null/etc. directory for $offline_archive_archivepath for the Offline Archive plugin so since the plugin basically moves the "originals" to that folder, it would effectively be deleting them?

Alternatively, maybe the solution is to create some sort of directory for $offline_archive_archivepath and (via CRON job or something) periodically delete the contents of that and then use the "Offline archiving steps" (as explained in the Knowledge Base article for the plugin that you mentioned upthread)?

Thoughts?

[Douglas Thompson]

Okay so I just tried the steps in the "Offline Archiving Steps" section on the Offline Archive plugin's page:

https://www.resourcespace.com/knowledge-base/plugins/offline_archive

And everything is working as expected up until step #7.

This entry on the Installation Check screen:

Tells me that there was no cron job set up...no scheduled tasks have run for 3853 days, which would put it back to where we _first_ started _using_ RS.

Sure enough, when I looked at the cron.d directory, there was _no_ "resourcespace" entry.

So, I made one, following the Linux steps on _this_ RS knowledgebase page: https://www.resourcespace.com/knowledge-base/systemadmin/cron

But then when I manually ran cron_copy_hitcount.php (via the "nice ionice php" thing), it spit out this:

[dethomps@tabit2 tools]$ nice ionice php cron_copy_hitcount.php
2025-04-14 11:22:56 https://tabit2.owu.edu/rsspace Starting cron process...
2025-04-14 11:22:56 https://tabit2.owu.edu/rsspace Executing job: 002_copy_hitcount.php

Sorry, an error has occurred. /var/www/html/rsspace/include/general_functions.php line 1875: file_put_contents(/var/www/html/rsspace/filestore/tmp/process_locks/copy_hitcount): Failed to open stream: Permission denied

#0  errorhandler()
#1  file_put_contents() called at [/var/www/html/rsspace/include/general_functions.php:1875]
#2  set_process_lock() called at [/var/www/html/rsspace/batch/cron_jobs/002_copy_hitcount.php:14]
#3  include(/var/www/html/rsspace/batch/cron_jobs/002_copy_hitcount.php) called at [/var/www/html/rsspace/batch/cron.php:35]
#4  include(/var/www/html/rsspace/batch/cron.php) called at [/var/www/html/rsspace/pages/tools/cron_copy_hitcount.php:5]

So it starts to work, but then fails with the "Permission denied" error. Is fixing that super-simple, or has this moved beyond my limited capabilities for this stuff?

If I can get the cron to work, then hopefully I have a pretty much workable solution to my original quandry:

I've created a "temp" folder in my home directory on the server and have added the $offline_archive_archivepath line to the RS config file that lists the full PWD for that. I then do step #2 (from the Offline Archive plugin's page) to set a photo to "Waiting to be archived" and then do the other steps. Once the cron is working, it should move the original file for the affected image out of the filestore (which I've already split into "original" and "resized") and into the new "temp" folder. I can hopefully then just delete the contents of that folder periodically and achieve the "all the thumbnails and preview sizes and metadata and whatnot for the resource remain in the system and are searchable, but the most-diskspace-taking original is no longer there" thing mentioned upthread.

Right?

[Douglas Thompson]

Just an update that I manually ran that cron_copy_hitcount.php with sudo and it worked...or at least it didn't error out with a "permission denied" message. But, the archived file was copied to the "temp" directory that I made (it was still in RS as well, only named whatever I put for the archive prefix)

So, I'm still missing something with the configuration or how I think this might work...any help would be greatly appreciated.

[Douglas Thompson]

Just replying to this to hopefully resurrect this conversation, as I finally have more time to look into this stuff. I've done nothing since my last "just an update" message, so something isn't quite working properly still...