Practical strategy for splitting directories?

[Walter Lee Davis]

I just had to re-import all the images in my entire database -- ~10,000 files -- and thanks to the file_url feature, I was able to do this in about 10 lines of irb shell. But now I have ~20,000 files in public/system/dragonfly/production/2010/03/02. That's kinda scary, even though they're mostly fairly small. I have had some advice that I should split these up until there's only ~1,000 per folder. But how can I do that?

Thanks in advance,

Walter

[Walter Lee Davis]

For posterity, here's where I've ended up in Dragonfly:

class Image < ActiveRecord::Base
belongs_to :attachable, :polymorphic => true
file_accessor :file do
storage_path{|f| "#{attachable_type.tableize}/#{attachable_id}/#{rand(100)}/#{f.name}"}
end
...
end

The rand(100) part is probably overkill, but this definitely gets the numbers under control.

Thanks,

Walter

[Andrew France]

On Friday, 2 March 2012 21:30:00 UTC, Walter Lee Davis wrote:

On Mar 2, 2012, at 2:28 PM, Walter Lee Davis wrote:

> I just had to re-import all the images in my entire database -- ~10,000 files -- and thanks to the file_url feature, I was able to do this in about 10 lines of irb shell. But now I have ~20,000 files in public/system/dragonfly/production/2010/03/02. That's kinda scary, even though they're mostly fairly small. I have had some advice that I should split these up until there's only ~1,000 per folder. But how can I do that?
>
> Thanks in advance,
>
> Walter

For posterity, here's where I've ended up in Dragonfly:

class Image < ActiveRecord::Base
  belongs_to :attachable, :polymorphic => true
  file_accessor :file do
    storage_path{|f| "#{attachable_type.tableize}/#{attachable_id}/#{rand(100)}/#{f.name}"}
  end
...
end

I do a similar thing that works across any model, always stays within filesystem limits, and eliminates duplicates (although be careful with deletion):

def generate_picture_path
    hash = Digest::SHA1.file(picture.path).hexdigest
    "profile_pictures/#{hash[0..2]}/#{hash[3..5]}/#{hash}"

  end