Are plugins supposed to be loaded alphabetically?
Chad Woolley
Are plugins supposed to be added to the load path in alphabetical
order by default? It seems to be in reverse-alphabetic order for
quite a while, since line #305 was added in [3122]
(http://dev.rubyonrails.org/changeset/3122) - which now lives in
loader.rb.
I know you can control the load order if you want, but I'm just
wondering what the default SHOULD be, or if it's supposed to be
undefined unless you explicitly set it...
Thanks,
-- Chad
Michael Koziarski
They *should* be loaded alphabetically. That's what makes the awesome
aaaaaaaaaa_first_thing so cool. It looks like it's trying to insert
correctly, what's going wrong?
--
Cheers
Koz
Chad Woolley
> They *should* be loaded alphabetically. That's what makes the awesome
> aaaaaaaaaa_first_thing so cool. It looks like it's trying to insert
> correctly, what's going wrong?
This line:
$LOAD_PATH.insert(application_lib_index + 1, lib_path)
...always inserts the current plugin's path immediately after the
application lib entry. Since the plugins are processed
alphabetically, this results in them being in reverse order on the
final load path.
-- Chad
James Adam
> This line:
>
> $LOAD_PATH.insert(application_lib_index + 1, lib_path)
>
> ...always inserts the current plugin's path immediately after the
> application lib entry. Since the plugins are processed
> alphabetically, this results in them being in reverse order on the
> final load path.
I believe this is actually as it was intended - more recently loaded
plugins "take precidence" over those loaded earlier. However, we could
certainly debate whether or not that is a good idea.
--
* J *
~
James Adam
I should clarify:
If config.plugins is [:a, :b, :c], then the plugins will be loaded in
that order - a, b, c - with 'c' being the "most recently loaded"
plugin.
Chad Woolley
Right - but I'm asking what the default order should be if
config.plugins is not specified. It seems like it should be
alphabetical, but it's actually reverse alphabetical.
Frederick Cheung
Although the actual running of the init file is done alphabetically
Fred
Michael Koziarski
> > config.plugins is not specified. It seems like it should be
> > alphabetical, but it's actually reverse alphabetical.
> Although the actual running of the init file is done alphabetically
It seems counter intuitive that the loading and init-running happen in
the opposite order.
--
Cheers
Koz
Chad Woolley
> It seems counter intuitive that the loading and init-running happen in
> the opposite order.
>
Yes. Should this be patched? The only danger might be breaking
applications which depend on the incorrectly reversed load path (which
has existed for 2 years now).
James Adam
> On 11/27/07, Michael Koziarski <mic...@koziarski.com> wrote:
> > It seems counter intuitive that the loading and init-running happen in
> > the opposite order.
Unless I'm mistaken, this isn't the case.
I'm not sure that everyone quite has the correct understanding of
what's going on when plugins are loaded. As Frederick says, the
init.rb files are evaluated in the default (A-Z) order. This is one of
the things that happens when a plugin is "loaded".
The other thing which happens, is that the plugin's lib directory is
added to the $LOAD_PATH, under the application's lib directory (so
that the app can still override anything). But, as each plugin is
loaded, it's lib directory is "pushed" into this position, forcing the
other plugin lib directories to move below it. So with plugins A, B
and C we get
Plugin A loading - $LOAD_PATH = [/lib, vendor/plugins/plugin_a/lib]
(approximately speaking, of course)
Plugin B loading - $LOAD_PATH = [/lib, vendor/plugins/plugin_b/lib,
vendor/plugins/plugin_a/lib]
Plugin C loading - $LOAD_PATH = [/lib, vendor/plugins/plugin_c/lib,
vendor/plugins/plugin_b/lib, vendor/plugins/plugin_a/lib]
So the plugins ARE loaded in the order you expect, and the load paths
ARE added in the same order, it's just that they are treating that
particular part of the $LOAD_PATH like a stack - pushing each
subsequent plugin lib dir on top of the other plugins.
Chad Wooley wrote:
> Right - but I'm asking what the default order should be if
> config.plugins is not specified. It seems like it should be
> alphabetical, but it's actually reverse alphabetical.
Hopefully it's now clear why the current implementation results in
paths in the $LOAD_PATH that appear to be the reverse of the order in
which the plugins are loaded (or the order in config.plugins).
However, the fact that the orders are different doesn't mean that
anything is broken. The order in which the plugins appear in the
$LOAD_PATH is deliberately the reversed order in which they are
loaded.
I'll explain why this is the case, and then in light of these reasons,
I think it would be useful to talk about *why* you feel this order
ought to be alphabetical.
I believe that the original thinking was that plugins loaded "later"
would have a higher precedence. To have the plugins appear in the
$LOAD_PATH "alphabetically" would simply require some tuning of the
line which determines which index in the existing $LOAD_PATH that we
currently want to insert at.
So the question really is, is this the behaviour that we want? Should
the plugin which was loaded first have precedence to others when it
comes to requiring files? Or should it be, as it currently is (and as
you'd expect if you were loading any ruby source file over another
one) that plugins loaded last take precedence?
Pascal
The behavior as you describe it is what I would expect, but this is
not quite what's currently implemented.
Currently, the $LOAD_PATH is initialized in reverse alpha order (by
set_load_path), then all plugins are initialized, in alpha order (by
load_plugins), so all plugins have the same $LOAD_PATH.
This can create some interesting situations where a plugin A sees only
the code of plugin B (if B defines the same classes) when A is
initializing. In fact, just last week, this exact behavior was
happening between rspec and rspec_on_rails (now fixed). This would
not happen if the implementation was using the behavior you describe.
This would also be closer to gems loading behavior.
That said, I'm not sure that's something you'd want to change as close
to 2.0, especially since there are ways to write your plugin to ensure
it loads its code, and not one from some other plugin.
Cheers,
Pascal.
--
http://blog.nanorails.com
Chad Woolley
> The other thing which happens, is that the plugin's lib directory is
> added to the $LOAD_PATH, under the application's lib directory (so
> that the app can still override anything). But, as each plugin is
> loaded, it's lib directory is "pushed" into this position, forcing the
> other plugin lib directories to move below it.
OK. We (me and Kelly Felkins) understand this problem now. It was
broken with changeset 8115, in the refactoring of the plugin code.
Specifically, the population of the load path was decoupled from the
initialization of the plugin, by removing the 'evaluate' from loader
[1], and making 'add_plugin_load_paths' a separate, earlier step in
initialization [2].
So, James' statement above is now false: 'AS each plugin is loaded,
it's lib directory is pushed...' This was the case up until the 2.0
Preview Release, but now, everything is added to the load path, and
THEN the plugins are initialized.
This breaks the assumption that a 'child' plugin can reopen a class
from another 'parent' plugin which it depends on. Since the dependent
'child' plugin is earlier on the load path, it's re-opened class is
found first, and if it tries to refer to any methods from the
dependency 'parent' plugin, it will fail, because they are not found.
This is the specific problem we are having with rspec and
rspec_on_rails recent version (which is fixed in rspec trunk now,
according to Pascal). The Spec::Matchers::Have class in
rspec_on_rails plugin is being found by the rspec plugin's
initialization process, because rspec_on_rails is BEFORE rspec on the
already-populated load path. This class in rspec_on_rails refers to
'failure_message', which is in the rspec plugin's version of
Spec::Matchers::Have, which of course hasn't been required yet, so
everything blows up.
I think this is a fairly bad regression that will likely break a lot
of plugins. We should consider fixing it before the 2.0 release. I
can try to give a more detailed failure scenario if you want.
[1] http://dev.rubyonrails.org/changeset/8115/trunk/railties/lib/rails/plugin/loader.rb
[2] http://dev.rubyonrails.org/changeset/8115/trunk/railties/lib/initializer.rb
Thanks,
-- Chad
James Adam
> of plugins. We should consider fixing it before the 2.0 release. I
> can try to give a more detailed failure scenario if you want.
You're right about the load path being added to before any init.rb
file is evaluated - I was paraphrasing, but it seems like this
separation of loading is one of the factors which is causing you
issues - apologies for the confusion.
> This breaks the assumption that a 'child' plugin can reopen a class
> from another 'parent' plugin which it depends on.
It's interesting, given that the reason it was decided to add all load
paths before loading plugins was exactly to remove dependency
problems! (see http://groups.google.com/group/rubyonrails-core/browse_frm/thread/7615e2ac0fa019d)
I sympathise with your issue, but I'm not convinced this is a
regression. It sounds like rspec_on_rails contained a file something
like rspec_on_rails/lib/spec/matchers/have.rb, which was intended to
add behaviour to the corresponding file in the rspec plugin; but
making this rely on the order of the load path is never going to be an
optimal solution. If one plugin really needs to modify code provided
by another one, shouldn't it do this by either:
* mixing in the new functionality via a module, or
* reopening the class manually, but *not* in a file which has the same
path as the file it is going to affect.
In other words, and in general Ruby as well as Rails, if we ever have
files which have the same "relative" path (such as
spec/matchers/have.rb), it's always going to be difficult to guarantee
that one will be loaded over the other. It seems like the best way to
solve this is to simply avoid it.
Chad Woolley
> It's interesting, given that the reason it was decided to add all load
> paths before loading plugins was exactly to remove dependency
> problems! (see http://groups.google.com/group/rubyonrails-core/browse_frm/thread/7615e2ac0fa019d)
>
> I sympathise with your issue, but I'm not convinced this is a
> regression. It sounds like rspec_on_rails contained a file something
> like rspec_on_rails/lib/spec/matchers/have.rb, which was intended to
> add behaviour to the corresponding file in the rspec plugin; but
> making this rely on the order of the load path is never going to be an
> optimal solution. If one plugin really needs to modify code provided
> by another one, shouldn't it do this by either:
>
> * mixing in the new functionality via a module, or
> * reopening the class manually, but *not* in a file which has the same
> path as the file it is going to affect.
>
> In other words, and in general Ruby as well as Rails, if we ever have
> files which have the same "relative" path (such as
> spec/matchers/have.rb), it's always going to be difficult to guarantee
> that one will be loaded over the other. It seems like the best way to
> solve this is to simply avoid it.
Thanks for the responses, James. Those are great points, and that
thread explains a lot. I'm totally in favor of gems-as-plugins. I
think this is a good direction.
-- Chad