Decoupling Alien::Base::ModuleBuild

[Graham Ollis]

This was touched on in gh#43 and I wanted to reopen discussion here.

Right now Alien::Base::ModuleBuild does three things.

 1. Existing library detection through pkg-config

 2. Install of packages through recipes (alien_*_command)

 3. Install of module via MB

Putting 1 and 2 in a separate class from 3 would yield some good potential features and I think it would make AB easier to maintain.  I also think it would be possible to maintain the current interface and keep things easy for people who just want something simple, but allow for a migration path for developers who need some extra features.  Let me describe some use cases:

 1. Optional prerequisite 

If you can take advantage of library but do not require it.  For example, a module can use Alien::Libarchive::Installer to detect the existence of a system libarchive but won't fail if it isn't there.  It actually uses Alien::Libarchive if it has already been installed by the user too.

 2. Specific prerequisite

If you install Alien::Libarchive, but it provides an old version of libarchive, you can use Alien::Libarchive::Installer to download the latest (or specific) version, build and use that to link your XS against.  You don't even need to install libarchive, since a static library is used.

 3. Other Alien prerequisites

Alien::Libarchive actually uses a number of Alien::*::Installer classes to install optional prerequisites itself.  Although it is sometimes possible to use an Alien::Foo as a prerequisite for Alien::Bar, in practice I have found this very tricky to get right.  The way it is done in Alien::Libarchive, all the prereqs are installed in the share directory (those that aren't provided by the system that is).  This has made Alien::Libarchive::XS much much much more useful on Windows, where many useful prereqs aren't provided by Strawberry Perl.

 4. Alternate implementations

Although I am not personally interested in a EUMM implementation, I think decoupling the detector/installer from MB would make an alternate implementation viable.

I also imagine that there are potentially other use cases outside the specific Alien mindset that we are focused on.  One of the rationale for Alien in the first place was to remove the detection/install/mb logic from the XS distribution so that it could be reused.  An installer class that was outside of the MB class can leverage the knowledge and hard work that an Alien author has put into something unrelated, or maybe even non perl.

mohawk asked if we should just use the Alien::Installer stuff that I had worked on, and I don't think that is appropriate because the role based approach does not lend itself well to the recipe based approach of AB.

I suggest an implementation for Alien::Base::ModuleBuild that maintains its existing interface but hands off most of the work to an Alien::Base::Installer (or pick your name).  I think one of the tricky parts is going to be maintaining state since MB has some specific limitations in that department.  Now an Alien:: developer can get his or her feet wet writing their Alien::Foo in the traditional way, but it becomes pretty easy to migrate to writing their own Alien::Base::Installer subclass once they run into some of the limitations of that interface.  Such an advanced user would still use (or could use) A::B::MB, but instead of specifying all the details to that class, they would specify the A::B::Installer subclass that they had written.  I imagine most A::B::Installer classes would have the traditional alien_build_command and alien_isolate_dynamic details, but in addition could override specific methods as needed.  I imagine most A::B::Installer subclasses would be minor tweaks.

I also think this would actually be easier to maintain than the current system where the MB guts are entangled with the detection and installer logic.  It would also be a plus for those who don't like MB.  

It my even make sense to further separate the detection and installer code as well.

The reason that I stopped using Alien::Base for Alien::Libarchive is that I ran into limitations that were difficult to work around, and the workarounds were starting to be more work than just implementing the everything myself, so that is what I did.  If the architecture for AB allowed for easy subclassing I might have stuck with AB.

Its a significant refactor of the code base, so we'd need buy in.  mohawk sounded interested in this sort of approach, I'd be interested in Joel and Zaki's voicing their thoughts on this.  I know Joel said that he wanted to keep things simple.  My argument is that we are keeping the interface simple for the newb, and making the maintenance and testing  of AB for core developers easier.

I also think that this approach addresses some of Chris' concerns.  In fact the architecture that I came up with for Alien::Libarchive was inspired in part in a discussion that we had at the DC Baltimore Perl workshop this year.

If people have alternate solutions for some of the use cases up there I'd be interested in hearing them too.

[Joel Berger]

 

I suggest an implementation for Alien::Base::ModuleBuild that maintains its existing interface but hands off most of the work to an Alien::Base::Installer (or pick your name).  I think one of the tricky parts is going to be maintaining state since MB has some specific limitations in that department.  Now an Alien:: developer can get his or her feet wet writing their Alien::Foo in the traditional way, but it becomes pretty easy to migrate to writing their own Alien::Base::Installer subclass once they run into some of the limitations of that interface.  Such an advanced user would still use (or could use) A::B::MB, but instead of specifying all the details to that class, they would specify the A::B::Installer subclass that they had written.  I imagine most A::B::Installer classes would have the traditional alien_build_command and alien_isolate_dynamic details, but in addition could override specific methods as needed.  I imagine most A::B::Installer subclasses would be minor tweaks.

I also think this would actually be easier to maintain than the current system where the MB guts are entangled with the detection and installer logic.  It would also be a plus for those who don't like MB.  

It my even make sense to further separate the detection and installer code as well.

The reason that I stopped using Alien::Base for Alien::Libarchive is that I ran into limitations that were difficult to work around, and the workarounds were starting to be more work than just implementing the everything myself, so that is what I did.  If the architecture for AB allowed for easy subclassing I might have stuck with AB.

Its a significant refactor of the code base, so we'd need buy in.  mohawk sounded interested in this sort of approach, I'd be interested in Joel and Zaki's voicing their thoughts on this.  I know Joel said that he wanted to keep things simple.  My argument is that we are keeping the interface simple for the newb, and making the maintenance and testing  of AB for core developers easier.

I also think that this approach addresses some of Chris' concerns.  In fact the architecture that I came up with for Alien::Libarchive was inspired in part in a discussion that we had at the DC Baltimore Perl workshop this year.

If people have alternate solutions for some of the use cases up there I'd be interested in hearing them too.

Do you have examples of what cannot easily be overloaded by simply subclassing AB::MB? The MB architecture is naturally subclassable. If it is simply a matter of splitting functionality into more fine-grained (single purpose) methods I could understand. 

[Graham Ollis]

Do you have examples of what cannot easily be overloaded by simply subclassing AB::MB? The MB architecture is naturally subclassable. If it is simply a matter of splitting functionality into more fine-grained (single purpose) methods I could understand. 

Well subclassing AB::MB means understanding the idiosyncrasies of  both MB and AB, and I don't think for most people it is a simple matter.  Both Mohawk and Chris expressed this back in gh#43.

I feel I already illustrated several use cases that cannot be addressed by subclassing AB::MB:

 1. Optional prerequisite  Entirely impossible with AB

You can specify an Alien::Foo as an optional prerequisite, but the user has to know to install the operating system package libfoo-dev and the cpan package Alien::Foo.  If the installer/detector is a separate class you can make Alien::Foo::Installer as a prereq and just use it if it can detect the operating system version, or an already installed Alien::Foo.

 2. Specific (version) prerequisite  Entirely impossible with AB

Alien::Foo can specify a specific version, but Foo::XS cannot.  Foo::XS is most likely to know its specific requirements.  Alien::Foo is least likely.

 3. Other Alien prerequisites Possible, but very difficult with current AB

The situation where libfoo needs libbar is not addressed by AB.  You could construct Alien::Foo and Alien::Bar and use Alien::Bar to build Alien:Foo, but in that case, if Alien::Bar is upgraded it will probably break Alien::Foo.

4. Alternate implementations Possible, but of limited usefulness

You can create a Makefile.PL / Makefile wrapper for MB but I see no point in this.

The first three are most important to me, and they are all things that I addressed successfully in Alien::Libarchive, (and the modules that use it Archive::Libarchive::XS, Archive::Libarchive::FFI and Archive::Ar::Libarchve) by separating the installer from the MB class.  I am pretty sure that this is not the only way to address these limitations, and I'm eager to hear if you have any good alternatives.  

Graham

[Zakariyya Mughal]

I think that this refactoring would be very helpful for making
Alien::Base deal with more special cases in a way that isolates the
behaviour into manageable pieces. It seems do-able if we first create a
bunch of private APIs in different classes.

As for the future, it is certainly possible that once these changes are
done, a mixture of the role-based and recipe-based approaches may arise.
I've read through your code for <https://github.com/plicease/Alien-Install>
and it seems that a DWIM-y build detection layer could be built around
those to, for example, use different tools when needed for different
stages of the install:

configuration: autoconf, cmake, etc.
detection : pkg-config, libtool, heuristics, etc.

And perhaps take things even further to say dependency checking:

dependencies : system deps (.deb, .rpm, cygwin, etc.),
other Alien:: packages (depending on version)

Essentially, I'd like to give Alien:: authors the tools to handle the
intricacies of different platforms and build-systems without having to
rewrite the same logic for every Alien:: module. Furthermore, moving
more of the logic into the superclass will let us create a unified way
to let the installers of Foo::XS and Foo::FFI choose between system or
shared libraries.

Cheers,
- Zaki Mughal

>
> --
> You received this message because you are subscribed to the Google Groups "Perl5-Alien" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to perl5-alien...@googlegroups.com.
> To post to this group, send email to perl5...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/perl5-alien/62d7726c-aa21-4922-8783-c6aa23c9a3bf%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.