Convert to rails engines

[Thomas Boerger]

Directly after merging the repositories I would like to transform the components into rails engines. There we need to do only some steps:

1) Move all files from crowbar_framework into the root of the repository
2) Create a $name.gemspec in the root folder of the repository
3) Create a engine.rb within the lib folder (need to create the lib folder as weel in the repo root)
4) Build a plain rails application in crowbar/crowbar which mounts all the engines like core, openstack and so on
5) Create a seperate repo and gem named crowbar-helper for the shared classes and add this dependency to the above gemspecs

Any objections? With that we are already able to write tests/specs for EVERY component (core, openstack, ha, ceph and hyperv).


Regards,
Thomas Boerger

[Vincent Untz]

Le jeudi 16 juillet 2015, à 17:36 +0200, Thomas Boerger a écrit :
> Directly after merging the repositories I would like to transform the components into rails engines. There we need to do only some steps:
>
> 1) Move all files from crowbar_framework into the root of the repository

Why is this first step required?

Vincent

> 2) Create a $name.gemspec in the root folder of the repository
> 3) Create a engine.rb within the lib folder (need to create the lib folder as weel in the repo root)
> 4) Build a plain rails application in crowbar/crowbar which mounts all the engines like core, openstack and so on
> 5) Create a seperate repo and gem named crowbar-helper for the shared classes and add this dependency to the above gemspecs
>
> Any objections? With that we are already able to write tests/specs for EVERY component (core, openstack, ha, ceph and hyperv).
>
>
> Regards,
> Thomas Boerger
>

> --
> You received this message because you are subscribed to the Google Groups "crowbar" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to crowbar+u...@googlegroups.com.
> To post to this group, send email to cro...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/crowbar/F7B2DE68-A4C7-476C-9222-4E624733EA46%40suse.de.
> For more options, visit https://groups.google.com/d/optout.
>

--
Les gens heureux ne sont pas pressés.

[Thomas Boerger]

> Am 16.07.2015 um 17:53 schrieb Vincent Untz <vu...@suse.com>:
>
>> Le jeudi 16 juillet 2015, à 17:36 +0200, Thomas Boerger a écrit :
>> Directly after merging the repositories I would like to transform the components into rails engines. There we need to do only some steps:
>>
>> 1) Move all files from crowbar_framework into the root of the repository
>
> Why is this first step required?

Because this is the required structure for rails engines. Like a rails applucation in the root folder of a git repo.

> Vincent
>
>> 2) Create a $name.gemspec in the root folder of the repository
>> 3) Create a engine.rb within the lib folder (need to create the lib folder as weel in the repo root)
>> 4) Build a plain rails application in crowbar/crowbar which mounts all the engines like core, openstack and so on
>> 5) Create a seperate repo and gem named crowbar-helper for the shared classes and add this dependency to the above gemspecs
>>
>> Any objections? With that we are already able to write tests/specs for EVERY component (core, openstack, ha, ceph and hyperv).
>>
>>
>> Regards,
>> Thomas Boerger
>>
>> --
>> You received this message because you are subscribed to the Google Groups "crowbar" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to crowbar+u...@googlegroups.com.
>> To post to this group, send email to cro...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/crowbar/F7B2DE68-A4C7-476C-9222-4E624733EA46%40suse.de.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> Les gens heureux ne sont pas pressés.
>
> --
> You received this message because you are subscribed to the Google Groups "crowbar" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to crowbar+u...@googlegroups.com.
> To post to this group, send email to cro...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/crowbar/20150716155319.GK3309%40vuntz.net.

[Adam Spiers]

Vincent Untz <vu...@suse.com> wrote:
> Le jeudi 16 juillet 2015, à 17:36 +0200, Thomas Boerger a écrit :
> > Directly after merging the repositories I would like to transform the components into rails engines. There we need to do only some steps:
> >
> > 1) Move all files from crowbar_framework into the root of the repository
>
> Why is this first step required?

I was wondering that too - I would prefer not to keep the chef/
subdirectory and other stuff cleanly separated from the Rails stuff if
possible.

> > 2) Create a $name.gemspec in the root folder of the repository
> > 3) Create a engine.rb within the lib folder (need to create the lib folder as weel in the repo root)
> > 4) Build a plain rails application in crowbar/crowbar which mounts all the engines like core, openstack and so on
> > 5) Create a seperate repo and gem named crowbar-helper for the shared classes and add this dependency to the above gemspecs
> >
> > Any objections? With that we are already able to write tests/specs for EVERY component (core, openstack, ha, ceph and hyperv).

I can't think of any objections so I am probably in favour, but it
would be good to understand the motivation a bit better. Why do
engines help us write tests/specs? And do they bring other benefits?

Thanks for the heads up!

[Thomas Boerger]

> Am 16.07.2015 um 18:17 schrieb Adam Spiers <asp...@suse.com>:
>
> Vincent Untz <vu...@suse.com> wrote:
>>> Le jeudi 16 juillet 2015, à 17:36 +0200, Thomas Boerger a écrit :
>>> Directly after merging the repositories I would like to transform the components into rails engines. There we need to do only some steps:
>>>
>>> 1) Move all files from crowbar_framework into the root of the repository
>>
>> Why is this first step required?
>
> I was wondering that too - I would prefer not to keep the chef/
> subdirectory and other stuff cleanly separated from the Rails stuff if
> possible.

It's just the chef repository. The rest is more or less related to the rails app so it is nothing negative.

>
>>> 2) Create a $name.gemspec in the root folder of the repository
>>> 3) Create a engine.rb within the lib folder (need to create the lib folder as weel in the repo root)
>>> 4) Build a plain rails application in crowbar/crowbar which mounts all the engines like core, openstack and so on
>>> 5) Create a seperate repo and gem named crowbar-helper for the shared classes and add this dependency to the above gemspecs
>>>
>>> Any objections? With that we are already able to write tests/specs for EVERY component (core, openstack, ha, ceph and hyperv).
>
> I can't think of any objections so I am probably in favour, but it
> would be good to understand the motivation a bit better. Why do
> engines help us write tests/specs? And do they bring other benefits?
>
> Thanks for the heads up!

The motivation is in first place standalone testing (engines got a rake task that generates a dummy rails application where the engine gets mounted). So we can run a big amount of specs already on travis before merging any pull request.

As this can result in a gem it can be packaged easier with a simple gem2rpm.

Other distros can install just the gem and the admin is responsible for further setup if there are no packages available.

Development environments are easier to setup as we clone all repos and add within the Gemfile the path to the core or openstack engine.

The only downside i realised is the syncronization of the chef cookbooks/databags with the chef server, but i think that i can integrate the engines codewise within 1 or maybe 2 days.

> --
> You received this message because you are subscribed to the Google Groups "crowbar" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to crowbar+u...@googlegroups.com.
> To post to this group, send email to cro...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/crowbar/20150716161758.GA1165%40indian.linksys.moosehall.

[Adam Spiers]

Thomas Boerger <tboe...@suse.de> wrote:
>
> > Am 16.07.2015 um 18:17 schrieb Adam Spiers <asp...@suse.com>:
> >
> > Vincent Untz <vu...@suse.com> wrote:
> >>> Le jeudi 16 juillet 2015, à 17:36 +0200, Thomas Boerger a écrit :
> >>> Directly after merging the repositories I would like to transform the components into rails engines. There we need to do only some steps:
> >>>
> >>> 1) Move all files from crowbar_framework into the root of the repository
> >>
> >> Why is this first step required?
> >
> > I was wondering that too - I would prefer not to keep the chef/
> > subdirectory and other stuff cleanly separated from the Rails stuff if
> > possible.
>
> It's just the chef repository. The rest is more or less related to the rails app so it is nothing negative.

Hmm, that doesn't sound quite right, for example bin/ and
crowbar_framework/bin/ serve very different purposes. The former is
for client-side scripts and the latter for server-side.

Is it absolutely required to do this move in order to support engines?

> >>> 2) Create a $name.gemspec in the root folder of the repository
> >>> 3) Create a engine.rb within the lib folder (need to create the lib folder as weel in the repo root)
> >>> 4) Build a plain rails application in crowbar/crowbar which mounts all the engines like core, openstack and so on
> >>> 5) Create a seperate repo and gem named crowbar-helper for the shared classes and add this dependency to the above gemspecs
> >>>
> >>> Any objections? With that we are already able to write tests/specs for EVERY component (core, openstack, ha, ceph and hyperv).
> >
> > I can't think of any objections so I am probably in favour, but it
> > would be good to understand the motivation a bit better. Why do
> > engines help us write tests/specs? And do they bring other benefits?
> >
> > Thanks for the heads up!
>
> The motivation is in first place standalone testing (engines got a rake task that generates a dummy rails application where the engine gets mounted). So we can run a big amount of specs already on travis before merging any pull request.

I'm curious, how is that different to just doing functional/unit tests
within a monolithic Rails app?

> As this can result in a gem it can be packaged easier with a simple gem2rpm.

Wouldn't that lose a bunch of the benefits we get from having custom
packages for barclamps, e.g. all the rpm macros?

> Other distros can install just the gem and the admin is responsible
> for further setup if there are no packages available.

True, that would be nice in theory, but I worry about the lack of
(un)install-time hooks.

> Development environments are easier to setup as we clone all repos
> and add within the Gemfile the path to the core or openstack engine.

Is that because the openstack engine could live in a separate
directory hierarchy to the core?

> The only downside i realised is the syncronization of the chef
> cookbooks/databags with the chef server, but i think that i can
> integrate the engines codewise within 1 or maybe 2 days.

OK thanks a lot for the info. I don't fully understand it but you are
the Rails guru so I will just assume you are right about engines being
the right direction to go :-)

Having said that, it would be really helpful to have answers to my
questions above. Thanks again!

[Thomas Boerger]

> Am 21.07.2015 um 14:21 schrieb Adam Spiers <asp...@suse.com>:
>
> Thomas Boerger <tboe...@suse.de> wrote:
>>
>>> Am 16.07.2015 um 18:17 schrieb Adam Spiers <asp...@suse.com>:
>>>
>>> Vincent Untz <vu...@suse.com> wrote:
>>>>> Le jeudi 16 juillet 2015, à 17:36 +0200, Thomas Boerger a écrit :
>>>>> Directly after merging the repositories I would like to transform the components into rails engines. There we need to do only some steps:
>>>>>
>>>>> 1) Move all files from crowbar_framework into the root of the repository
>>>>
>>>> Why is this first step required?
>>>
>>> I was wondering that too - I would prefer not to keep the chef/
>>> subdirectory and other stuff cleanly separated from the Rails stuff if
>>> possible.
>>
>> It's just the chef repository. The rest is more or less related to the rails app so it is nothing negative.
>
> Hmm, that doesn't sound quite right, for example bin/ and
> crowbar_framework/bin/ serve very different purposes. The former is
> for client-side scripts and the latter for server-side.

For the "server-side" there are anyway only rails, rake and bundle. The last one won't be shipped so there are only 2 left. The other scripts should be anyway replaced by crowbar-client.

> Is it absolutely required to do this move in order to support engines?

This is required as rails got some conventions for the structure of engines (pretty much the same as for rails apps itself, just the difference that engines are built in the form of gems)

>>>>> 2) Create a $name.gemspec in the root folder of the repository
>>>>> 3) Create a engine.rb within the lib folder (need to create the lib folder as weel in the repo root)
>>>>> 4) Build a plain rails application in crowbar/crowbar which mounts all the engines like core, openstack and so on
>>>>> 5) Create a seperate repo and gem named crowbar-helper for the shared classes and add this dependency to the above gemspecs
>>>>>
>>>>> Any objections? With that we are already able to write tests/specs for EVERY component (core, openstack, ha, ceph and hyperv).
>>>
>>> I can't think of any objections so I am probably in favour, but it
>>> would be good to understand the motivation a bit better. Why do
>>> engines help us write tests/specs? And do they bring other benefits?
>>>
>>> Thanks for the heads up!
>>
>> The motivation is in first place standalone testing (engines got a rake task that generates a dummy rails application where the engine gets mounted). So we can run a big amount of specs already on travis before merging any pull request.
>
> I'm curious, how is that different to just doing functional/unit tests
> within a monolithic Rails app?

Because the monolithic rails app is created out of multiple git repositories, do you know any good way to test them automated on travis? Not me. Rails engines are by design standalone testable, there are predefined rake tasks that generate a thin dummy rails app which mounts the engine and results in a real rails app with full support for various testing possibilities.

>> As this can result in a gem it can be packaged easier with a simple gem2rpm.
>
> Wouldn't that lose a bunch of the benefits we get from having custom
> packages for barclamps, e.g. all the rpm macros?

For this there is still the gem2rpm.yml, even if i don't believe that we need lots of macros. Maybe a single one that uploads the cookbooks to the chef server, but even that can be made better within the rails app.

>> Other distros can install just the gem and the admin is responsible
>> for further setup if there are no packages available.
>
> True, that would be nice in theory, but I worry about the lack of
> (un)install-time hooks.

It's just a sideeffect if it's distributed as a rubygem as well. What do you mean with install-hooks? It's not our thing to define what a user wants to do with the gems. We are supporting only the package way. That can be clearly stated in the gem description :).

>> Development environments are easier to setup as we clone all repos
>> and add within the Gemfile the path to the core or openstack engine.
>
> Is that because the openstack engine could live in a separate
> directory hierarchy to the core?

What do you mean? It's simply easier to include a rubygem from a specific path instead of splitting the source of a barclamp into various places like /opt/dell/barclamps/$barclamp, /opt/dell/chef and /opt/dell/crowbar_framework.

>> The only downside i realised is the syncronization of the chef
>> cookbooks/databags with the chef server, but i think that i can
>> integrate the engines codewise within 1 or maybe 2 days.
>
> OK thanks a lot for the info. I don't fully understand it but you are
> the Rails guru so I will just assume you are right about engines being
> the right direction to go :-)

I have already integrated engine based projects ;)

> Having said that, it would be really helpful to have answers to my
> questions above. Thanks again!

Done!

[Adam Spiers]

Thomas Boerger <tboe...@suse.de> wrote:
> > Am 21.07.2015 um 14:21 schrieb Adam Spiers <asp...@suse.com>:
> > Thomas Boerger <tboe...@suse.de> wrote:
> >>> Am 16.07.2015 um 18:17 schrieb Adam Spiers <asp...@suse.com>:
> >>> Vincent Untz <vu...@suse.com> wrote:
> >>>>> Le jeudi 16 juillet 2015, à 17:36 +0200, Thomas Boerger a écrit :
> >>>>> Directly after merging the repositories I would like to transform the components into rails engines. There we need to do only some steps:
> >>>>>
> >>>>> 1) Move all files from crowbar_framework into the root of the repository
> >>>>
> >>>> Why is this first step required?
> >>>
> >>> I was wondering that too - I would prefer not to keep the chef/
> >>> subdirectory and other stuff cleanly separated from the Rails stuff if
> >>> possible.
> >>
> >> It's just the chef repository. The rest is more or less related to the rails app so it is nothing negative.
> >
> > Hmm, that doesn't sound quite right, for example bin/ and
> > crowbar_framework/bin/ serve very different purposes. The former is
> > for client-side scripts and the latter for server-side.
>
> For the "server-side" there are anyway only rails, rake and bundle. The last one won't be shipped so there are only 2 left. The other scripts should be anyway replaced by crowbar-client.
>
> > Is it absolutely required to do this move in order to support engines?
>
> This is required as rails got some conventions for the structure of engines (pretty much the same as for rails apps itself, just the difference that engines are built in the form of gems)

That's the bit I don't understand - we already have the Rails app in a
subdirectory so why can't the engines be too? After all, Rails has no
awareness of where the root of the git repo is.

> >>>>> 2) Create a $name.gemspec in the root folder of the repository
> >>>>> 3) Create a engine.rb within the lib folder (need to create the lib folder as weel in the repo root)
> >>>>> 4) Build a plain rails application in crowbar/crowbar which mounts all the engines like core, openstack and so on
> >>>>> 5) Create a seperate repo and gem named crowbar-helper for the shared classes and add this dependency to the above gemspecs

As above, why do Rails / Bundler etc. care where the root folder is?
Even Travis allows you to have the Gemfile in a subdirectory, e.g.

https://github.com/crowbar/barclamp-pacemaker/blob/master/.travis.yml#L21

> >>>>> Any objections? With that we are already able to write tests/specs for EVERY component (core, openstack, ha, ceph and hyperv).
> >>>
> >>> I can't think of any objections so I am probably in favour, but it
> >>> would be good to understand the motivation a bit better. Why do
> >>> engines help us write tests/specs? And do they bring other benefits?
> >>>
> >>> Thanks for the heads up!
> >>
> >> The motivation is in first place standalone testing (engines got a rake task that generates a dummy rails application where the engine gets mounted). So we can run a big amount of specs already on travis before merging any pull request.
> >
> > I'm curious, how is that different to just doing functional/unit tests
> > within a monolithic Rails app?
>
> Because the monolithic rails app is created out of multiple git repositories, do you know any good way to test them automated on travis? Not me. Rails engines are by design standalone testable, there are predefined rake tasks that generate a thin dummy rails app which mounts the engine and results in a real rails app with full support for various testing possibilities.

OK. Presumably once the repos are merged, this will only make a
difference to crowbar-{ha,openstack,ceph}, not to crowbar-core.

> >> As this can result in a gem it can be packaged easier with a simple gem2rpm.
> >
> > Wouldn't that lose a bunch of the benefits we get from having custom
> > packages for barclamps, e.g. all the rpm macros?
>
> For this there is still the gem2rpm.yml, even if i don't believe that we need lots of macros. Maybe a single one that uploads the cookbooks to the chef server, but even that can be made better within the rails app.

Currently we have quite a few macros:

https://build.suse.de/package/view_file/Devel:Cloud:6/crowbar/macros.crowbar?expand=1

but I would welcome ideas for how to get rid of them.

> >> Other distros can install just the gem and the admin is responsible
> >> for further setup if there are no packages available.
> >
> > True, that would be nice in theory, but I worry about the lack of
> > (un)install-time hooks.
>
> It's just a sideeffect if it's distributed as a rubygem as
> well. What do you mean with install-hooks?

See macros.crowbar in the link above.

> It's not our thing to define what a user wants to do with the
> gems. We are supporting only the package way. That can be clearly
> stated in the gem description :).

I see, so you are saying we would keep the install-time hooks in the
packages. Makes sense.

> >> Development environments are easier to setup as we clone all repos
> >> and add within the Gemfile the path to the core or openstack engine.
> >
> > Is that because the openstack engine could live in a separate
> > directory hierarchy to the core?
>
> What do you mean? It's simply easier to include a rubygem from a
> specific path instead of splitting the source of a barclamp into
> various places like /opt/dell/barclamps/$barclamp, /opt/dell/chef
> and /opt/dell/crowbar_framework.

I think we are saying the same thing, and the answer to my question
is yes: the openstack engine(s) could live in any path independent of
the location of Crowbar's core.