Install software by running script

[Albert Shih]

Hi,

I would like to have your point of view on how you install some software.

Long time (in IT meaning) most of the software installation meaning run
some rpm, apt, and put a config file.

That's something very easy to do with puppet.

Currently lot of software need something, IMHO, more easy for the user, but
much more hard to put inside a puppet. For example, installing some nodejs
software means to

install nodejs
run npm

If I try do to the same it's pretty hard to find all packages install by the
last command and put them to a puppet module. And event If I does, I need to change
the puppet module on each update (and currently for some software that's
mean every week).

nodejs is just a example, you can take bundle in ruby, pip in python etc.

Of course for just pip/bundle/etc. I can do something like

exec { 'install':
command => 'pip/bundle/npm install',
creates => 'some file created by the pip/bundle/npm
}

but still it's painful because if the pip/bundle/npm failed the exec would
not be execute again, unless you put every file create by the 10000
dependencies need for every software.

The worst case for me is those software installed by some

wget 'URL' | sh

(Ok I know it's not secure...) but well...

I event find out a script I am unable to run through exec resource (I
didn't find out why).

So my question : How you manage those kind of installation (beside manually
or using a docker image).

Regards.

--
Albert SHIH
DIO bātiment 15
Observatoire de Paris
xmpp: j...@obspm.fr
Heure local/Local time:
Sat 11 Apr 2020 11:15:45 AM CEST

[Martin Alfke]

Hi Albert,

> On 11. Apr 2020, at 11:27, Albert Shih <Alber...@obspm.fr> wrote:
>
> Hi,
>
> I would like to have your point of view on how you install some software.
>
> Long time (in IT meaning) most of the software installation meaning run
> some rpm, apt, and put a config file.

That process is still considered best practice.

>
> That's something very easy to do with puppet.
>
> Currently lot of software need something, IMHO, more easy for the user, but
> much more hard to put inside a puppet. For example, installing some nodejs
> software means to
>
> install nodejs
> run npm
>

Don't do that as a Puppet developer.
Let the developers build complete software packages with all dependencies built into.
e.g. gem bundler directory must be part of the application package.

Building packages can easily be done by using fpm (https://github.com/jordansissel/fpm/wiki).
No special Debian or RPM knowledge required.

> If I try do to the same it's pretty hard to find all packages install by the
> last command and put them to a puppet module. And event If I does, I need to change
> the puppet module on each update (and currently for some software that's
> mean every week).
>
> nodejs is just a example, you can take bundle in ruby, pip in python etc.
>
> Of course for just pip/bundle/etc. I can do something like
>
> exec { 'install':
> command => 'pip/bundle/npm install',
> creates => 'some file created by the pip/bundle/npm
> }
>
> but still it's painful because if the pip/bundle/npm failed the exec would
> not be execute again, unless you put every file create by the 10000
> dependencies need for every software.
>
> The worst case for me is those software installed by some
>
> wget 'URL' | sh
>
> (Ok I know it's not secure...) but well...

The same pattern: let the wget | sudo bash command run on a dev platform or a container and build a package.


hth,
Martin

>
> I event find out a script I am unable to run through exec resource (I
> didn't find out why).
>
> So my question : How you manage those kind of installation (beside manually
> or using a docker image).
>
> Regards.
>
> --
> Albert SHIH
> DIO bātiment 15
> Observatoire de Paris
> xmpp: j...@obspm.fr
> Heure local/Local time:
> Sat 11 Apr 2020 11:15:45 AM CEST
>

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/20200411092718.GB2172%40io.chezmoi.fr.

[Dirk Heinrichs]

Am Samstag, den 11.04.2020, 12:15 +0200 schrieb Martin Alfke:

Of course for just pip/bundle/etc. I can do something like

  exec { 'install':

    command => 'pip/bundle/npm install',

    creates => 'some file created by the pip/bundle/npm

    }

but still it's painful because if the pip/bundle/npm failed the exec would

not be execute again, unless you put every file create by the 10000

dependencies need for every software.

The worst case for me is those software installed by some

  wget 'URL' | sh

(Ok I know it's not secure...) but well...

The same pattern: let the wget | sudo bash command run on a dev platform or a container and build a package.

Either this, or: Write a package provider, like the ones already available for Ruby gems or Python modules, which turns this into

package { 'foo':

...

provider => npm,

...

}

Bye...

Dirk

-- 

Dirk Heinrichs

Senior Systems Engineer, Delivery Pipeline

OpenText ™ Discovery | Recommind

Phone: +49 2226 15966 18

Email: dhei...@opentext.com

Website: www.recommind.de

Recommind GmbH, Von-Liebig-Straße 1, 53359 Rheinbach

Vertretungsberechtigte Geschäftsführer Gordon Davies, Madhu Ranganathan, Christian Waida, Registergericht Amtsgericht Bonn, Registernummer HRB 10646

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden

Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail sind nicht gestattet.

[Andy Hall]

exactly just push the devs of the package resource type to add a new provider. I mean nodejs is kinds popular now so I see no reason not to add it...

[David Schmitt]

To answer the original question: when a software vendor delivers software that does not conform to your distributions native packaging format, invest some time in the excellent FPM tool that can build native packages for your local environment. Injecting the complexity of dragging a vendor's ... err ... prioritisation decisions into your production systems is really only the last resort if you can't get it to work any other way.

On Tue, 14 Apr 2020 at 12:53, Andy Hall <andyjo...@gmail.com> wrote:

exactly just push the devs of the package resource type to add a new provider. I mean nodejs is kinds popular now so I see no reason not to add it...

If I may chime in as one of the devs of the package resource type:

My understanding is that - like ruby and python - only very few people actually use the system installed version of npm and npm packages. Instead the default workflow uses custom installs and multiple "virtual" environments stashed away in their application directories. At least for ruby and python this makes the main package providers practically unusable because they either

* interfere with system-packaged gems/packages/modules

or

* can't detect which virtual environment to use

or - if we were to magically solve enumerating virtual environments on the systems

* find multiple versions of the same package and confuse puppet even more

See the puppet_gem and puppetserver_gem for ways this has been "fixed" for ruby gems. Both of which are very application-specific workarounds for those problems that are not  generalizable.

I hope this provides enough reasons why this won't get added any time soon.

 

Cheers, David

--

You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/89c471a3-fc4e-466f-ad7d-0cb81e7d77f8%40googlegroups.com.

[Albert Shih]

Le 11/04/2020 à 12:15:12+0200, Martin Alfke a écrit

Hi everyone.

>
Thanks for the answer.

> > I would like to have your point of view on how you install some software.
> >
> > Long time (in IT meaning) most of the software installation meaning run
> > some rpm, apt, and put a config file.
>
> That process is still considered best practice.

I agree, but I'm not the guy who write the code of the software I need to
install

> > That's something very easy to do with puppet.
> >
> > Currently lot of software need something, IMHO, more easy for the user, but
> > much more hard to put inside a puppet. For example, installing some nodejs
> > software means to
> >
> > install nodejs
> > run npm
> >
>
> Don't do that as a Puppet developer.
> Let the developers build complete software packages with all dependencies built into.
> e.g. gem bundler directory must be part of the application package.
>
> Building packages can easily be done by using fpm (https://github.com/jordansissel/fpm/wiki).
> No special Debian or RPM knowledge required.

I will look for that very closely, Thanks for the tips. I didn't know this
one.

> >
> > (Ok I know it's not secure...) but well...
>
> The same pattern: let the wget | sudo bash command run on a dev platform or a container and build a package.

Ok, so if I'm correct here is the workflow :


Using some gitlab/github + fpm, I run all the script from the software
vendor, and built a package.

Using puppet install that package.

Put my config file in puppet

Each time I need a upgrade I rerun my git pull/push CI to build the
package

Am I correct ?

Regards.
--
Albert SHIH

Observatoire de Paris
xmpp: j...@obspm.fr
Heure local/Local time:

Wed 15 Apr 2020 03:47:44 PM CEST

[Albert Shih]

Le 14/04/2020 à 06:04:30+0000, Dirk Heinrichs a écrit

> Am Samstag, den 11.04.2020, 12:15 +0200 schrieb Martin Alfke:
>
>
>
>
> Either this, or: Write a package provider, like the ones already available for
> Ruby gems or Python modules, which turns this into
>
> package { 'foo':
> ...
> provider => npm,
> ...
> }

Well...no that easy, lots of those installer from the software vendor are
interactive, so you need to answer some stupid question to build the config
file. I need to do the reverse engineering to known where those answer end
up.

But thanks.

Regards.
--
Albert SHIH

Observatoire de Paris
xmpp: j...@obspm.fr
Heure local/Local time:

Wed 15 Apr 2020 03:52:31 PM CEST

[Martin Alfke]

Hi Albert,

You are rebuilding with increased version numbers.

>
> Am I correct ?

That description is correct.

hth,
Martin