incomplete role consumption?

[Daniel Hermann]

Hi,

I have two classes Test1 and Test2, both consuming the role
TestRole which uses another class TestGlue, which in turn uses
Test1 and Test2 internally.

My main program uses Test1 and Test2 and runs into a Moose error
when it calls a method in Test2 provided by TestRole:

Can't locate object method "load_role" via package "Test2" at
Test2.pm line 5.

What is going wrong here?

This is the full code:

=== TestRole.pm ===
package TestRole;
use Moose::Role;
use TestGlue ();

sub load_role {}

1;
==============

=== Test1.pm =====
package Test1;
use Moose;
with "TestRole";

sub load{ shift->load_role }

1;
==============

=== Test2.pm =====
package Test2;
use Moose;
with "TestRole";

sub load{ shift->load_role }

1;
==============

=== TestGlue.pm ===
package TestGlue;
use Moose;
use Test1 ();
use Test2 ();

1;
==============

=== main.pl =====
#!/usr/bin/env perl
use Test1 ();
use Test2 ();

Test1->new->load;
Test2->new->load;

1;
==============


The above code works if

- I do not 'use Test2' in TestGlue.pm or
- I use an eval block in main.pl such as

eval { use Test2; Test2->new->load; };

It seems to me that the import of the class Test2 in "role context"
via TestGlue.pm, triggered by 'use Test1' in main.pl, is incomplete
somehow, and the second import via 'use Test2' in main.pl does
not compensate this.

Thanks for any hints,
Daniel

--
Daniel Hermann

Operations Developer
1&1 Internet AG

[Ben Tilly]

To understand this you need to understand how Perl handles circular
dependencies.

When we execute code that loads other code (for instance use at
compile time, require at run time, or a variety of things in Moose),
Perl will mark the current package as incompletely loaded then go off
and try to load the other one. If curing that load, Perl is told to
load your package, Perl will skip that step to avoid infinite
recursion. Therefore the other package will continue loading with
some of its dependencies not completely loaded.

The result can be exactly what you are seeing. One of the packages
tries to load with the other one incompletely there.

Here are three possible fixes in roughly the order that I would try them.

1. Remove the circular dependency. (Always, always, always best if
you can manage it.)

2. Figure out what order you have to load things in order to avoid
breakage, and do that consistently. (I've been known to introduce a
new package to load just to make sure it happens correctly.)

3. Move the code that is needed by circular dependencies so that it is
present before whatever will load the circular dependency. (Add big
fat comments and unit tests around this issue. Because it will seem
like black magic and it is easy to break it.)

[Gianni Ceccarelli]

Short version: you have a dependency loop, and your code, as written,
can not work.

Longer version: main.pl loads Test1, which loads TestRole, which loads
TestGlue, which loads Test1. Boom!

On 2015-02-25 Daniel Hermann <daniel....@1und1.de> wrote:
> The above code works if
>
> - I do not 'use Test2' in TestGlue.pm or

This removes half the dependency loop, and (by accident) makes the
whole thing work. But if you change the order of the "use" in main, it
will fail for Test1

> - I use an eval block in main.pl such as
>
> eval { use Test2; Test2->new->load; };

In which case Test2 is loaded after all the compilation, and again you
have half-removed the loop. Again, this is still fragile.

> It seems to me that the import of the class Test2 in "role context"
> via TestGlue.pm, triggered by 'use Test1' in main.pl, is incomplete
> somehow, and the second import via 'use Test2' in main.pl does
> not compensate this.

Detailed explanation:

a "use Foo" statement is (roughly) equivalent to a "BEGIN { require
Foo }". "require", as documented in perlfun, uses the global %INC to
avoid loading a file more than once. Each entry in %INC is set
optimistically, before actually loading the corresponding file. So the
sequence in your case is:

1) main.pl is compiled, %INC is empty
2) "use Test1" is compiled and executed, $INC{'Test1.pm'}='Test1.pm'
3) Test1 loads Moose, $INC{'Moose.pm'}='/path/to/Moose.pm'
4) Test1 finishes compiling
5) Test1 is run (still inside the BEGIN block generated by the "use
Test1" in main)
6) "with 'TestRole'" causes a "require TestRole"
7) TestRole is loaded, $INC{'TestRole.pm'}='TestRole.pm'
8) TestRole loads Moose::Role,
$INC{'Moose/Role.pm'}='/path/to/Moose/Role.pm'
9) TestRole loads TestGlue
10) TestGlue is loaded, $INC{'TestGlue.pm'}='TestGlue.pm'
11) TestGlue uses Moose, but $INC{'Moose.pm'} is true, no loading
happens
12) TestGlue uses Test1, but $INC{'Test1.pm'} is true, no loading
happens
13) TestGlue uses Test2
14) Test2 is loaded, $INC{'Test2.pm'}='Test2.pm'
15) Test2 loads Moose, but $INC{'Moose.pm'} is true, no loading
happens
16) Test2 finishes compiling
17) Test2 is run (still inside the BEGIN block generated by the "use
Test2" in TestGlue)
18) "with 'TestRole'" causes a "require TestRole"
19) $INC{'TestRole.pm'} is true, no loading happens
20) the (incomplete!, because we're still compiling it, see point 9)
role is applied to the Test2 class
21) nothing happens, TestRole is currently an empty role!
22) Test2 finishes running
23) TestGlue finishes compiling
24) TestRole finishes compiling, and only now acquires the sub
load_role
25) TestRole is finally applied to Test1 (see step 6); Test1 acquires
the sub load_role
26) Test1 finishes running
27) main loads Test2, but $INC{'Test2.pm'} is true, nothing happens
28) main starts running
29) Test1->new->load works, because 25
30) Test2->new->load fails, because 20 and 21

>
> Operations Developer
> 1&1 Internet AG
>

--
Dakkar - <Mobilis in mobile>
GPG public key fingerprint = A071 E618 DD2C 5901 9574
6FE2 40EA 9883 7519 3F88
key id = 0x75193F88

Warning: Listening to WXRT on April Fools' Day is not recommended for
those who are slightly disoriented the first few hours after waking up.
-- Chicago Reader 4/22/83

[Daniel Hermann]

Hi Nick,

On 02/25/2015 10:35 PM, Nick Perez wrote:
> You are correct in your deduction. And this is not Moose specific.
> Circular dependencies like in your example cause undefined behavior
> in Perl. What I don't understand is why you need the TestGlue class
> or the circular dependency at all. Can you please take a step back
> and explain what it is you are trying to accomplish?
>

in my application, I have the following entities:

Test1 == Node (a host node)
Test2 == Vm (a virtual host)
TestRole == HostRole (common parts of Node and Vm, with sub load_host())
TestGlue == Cluster (a cluster of host nodes and virtual hosts)

Node and Vm both consume the HostRole, which has a 'cluster' attribute.
Therefore, HostRole uses Cluster.

On the other hand, the Cluster class has some functionality that
operates on Node and Vm objects, such as representing or manipulating
all nodes and vms of the cluster. Therefore, it uses Node and Vm.

I understand that the problems indicate that there is some fundamental
class design flaw here.

regards,

[Ben Tilly]

I would suggest that you don't need to use Node and VM in the Cluster
class. In order to use the Cluster class properly, someone will have
to pass in Node and VM objects, but it is their responsibility to make
sure that Node and VM are loaded.

That won't work smoothly if the Cluster class offers convenience
methods which generate Node and VM objects. In that case you should
put requires inside of those convenience methods. That way you
guarantee that those classes exist when you need them, and break the
circular dependencies while loading.

[Daniel Hermann]

Hi Ben,

On 02/26/2015 02:17 PM, Ben Tilly wrote:
> I would suggest that you don't need to use Node and VM in the Cluster
> class. In order to use the Cluster class properly, someone will have
> to pass in Node and VM objects, but it is their responsibility to make
> sure that Node and VM are loaded.
>
> That won't work smoothly if the Cluster class offers convenience
> methods which generate Node and VM objects. In that case you should
> put requires inside of those convenience methods. That way you
> guarantee that those classes exist when you need them, and break the
> circular dependencies while loading.

That sounds doable. The functionality in the Cluster class dealing
with Node and Vm objects is small, so additional requires inside
the relevant methods are a good compromise.

thanks,