cache cleanup

[Aidan Feldman]

Hi all-

So I am using a rack-cache Disk store in an image processing application I built, which creates new cached resources based on URL query params.  As there are a potentially infinite number of combinations and values of these params, the cache can grow without bound and leave lots of potential abandoned resources.  I would like to either:

• periodically purge expired entities, or
• specify a disk usage limit, and then start replacing the old ones.

Anyone have any thoughts about how to manage this?  Thanks!

-Aidan

[Ryan Tomayko]

Sorry for the late response!

This currently requires a manual / external process. I'd recommend
setting up a cronjob that runs something like:

find /var/cache/rack -type f -mtime +5 | xargs rm -f

That will delete any entries that have not been served for more than five days.

The disk stores are designed to allow individual files to be removed
at any time, so you shouldn't have to worry about this causing
problems at the app level.

Thanks,
Ryan

[Aidan Feldman]

Wow, that is so much simpler than the script I was writing (reading the metadata and deleting the N oldest files...)  So the metadata file doesn't get updated on access, right?  Therefore this would delete both the metadata and the blob?

--
You received this message because you are subscribed to the Google Groups "rack-cache" group.
To post to this group, send email to rack-...@googlegroups.com.
To unsubscribe from this group, send email to rack-cache+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/rack-cache?hl=en.

[Ryan Tomayko]

On Tue, May 17, 2011 at 8:04 PM, Aidan Feldman <aidan....@gmail.com> wrote:
> Wow, that is so much simpler than the script I was writing (reading the
> metadata and deleting the N oldest files...)  So the metadata file doesn't
> get updated on access, right?  Therefore this would delete both the metadata
> and the blob?

The metadata and blob's timestamps can get out of sync but if you
remove a blob, the next cache hit will detect that it's gone and treat
it as a miss, cleaning up the metadata along the way. It's not a
perfect system (you can end up with blobs or metadata files that have
aren't paired properly) but it's very simple and should work fine for
keeping disk use down.

[Aidan Feldman]

For anyone else who is interested, we ended up using tmpwatch, which allowed us to remove the least-recently accessed files.

--

[Aidan Feldman]

Sorry, I mean tmpreaper (which is similar).