QuickGem: Faster gem loading through caching

[Magnus Holm]

You might be interested in this: http://github.com/judofyr/quickgem

It monkey-patches #require, Gem.bin_path, Gem::Dependency#to_spec with
one goal: To avoid calling Gem::Specification#_all (which loads all
specification files and is *slow* with a big gem directory).

A simlpe "heroku version" goes from 1.79s to 0.39s which still is
pretty bad, but a notable difference. "rails -v" (outside of a Rails
app) goes down to <80ms which feels very snappy. Anything that uses
Bundler is slow since Bundler uses Specification#find internally. The
initial "bundle exec" is faster though (0.72s to 0.3s).

Example:

- require "rails"
- It checks for the files "cache/lib/rails", "cache/lib/rails.rb" and
"cache/lib/rails.so"
- Those files contain Marshal-dumped Array of names, versions and
spec-file-location. Sorted by version number.
- It loads and activates the first spec in the array

- $ rails -v
- gem "rails", ">= 0" is called by the wrapper
- "cache/gem/rails" contains the same Marshal-dumped Array; handled as above.
- Gem.bin_path("rails") is called by the wrapper
- I've patched bin_path to look in Gem.loaded_specs instead of using
Specification#each

It also handles multiple gem paths independently, automatically
rebuilds the cache when you install/remove gems and loads dependencies
correctly. It's still an ugly monkey-patch that probably doesn't work
in older RubyGems version.

I think there are concepts here that could be integrated into mainline
RubyGems, but there's quite a few things which needs to be structured
differently. Any code that assumes you can iterate over all
specification in the system needs to change.

// Magnus Holm
_______________________________________________
RubyGems-Developers mailing list
http://rubyforge.org/projects/rubygems
RubyGems-...@rubyforge.org
http://rubyforge.org/mailman/listinfo/rubygems-developers