How to add extra lines in the existing files?

[Sans]

Dear all,

As in the title, how do I add some extra lines in a existing file and
check every time for its presence? I just don't want to copy over the
existing file for some good reason (system files etc.).

Any idea how should I do that? cheers!!

[Martin Alfke]

Hi,

this is not directly built into puppet.

But: puppet offers you the flexibility to define an exec which does what
you like.

The simple text pattern site shows you an example:
http://projects.puppetlabs.com/projects/1/wiki/Simple_Text_Patterns

[Nico -telmich- Schottelius]

Martin Alfke [Thu, Jun 23, 2011 at 02:51:07PM +0200]:

> On 06/23/2011 12:55 PM, Sans wrote:

> > [...] how do I add some extra lines in a existing file [...]

>
> this is not directly built into puppet.

Is there any specific reason why this is not built into puppet?

We've put __addifnosuchline [0] into cdist [1] as one of
the most basic features, because it's often needed if you do not
care about the existing contents, but just need to add a line.

Cheers,

Nico

[0] http://www.nico.schottelius.org/software/cdist/man/cdist-type__addifnosuchline.html
[1] http://www.nico.schottelius.org/software/cdist/

--
PGP key: 7ED9 F7D3 6B10 81D7 0EC5 5C09 D7DC C8E4 3187 7DF0

[Kinzel, David]

>> this is not directly built into puppet.
>
>Is there any specific reason why this is not built into puppet?
>
>We've put __addifnosuchline [0] into cdist [1] as one of
>the most basic features, because it's often needed if you do not
>care about the existing contents, but just need to add a line.
>

I second this. This would greatly cut down on things we do with exec
patterns. And no, I don't want to use augeas as I typically see people
fighting with it to work in all their environments.

>Cheers,
>
>Nico
>

This email communication and any files transmitted with it may contain confidential and or proprietary information and is provided for the use of the intended recipient only. Any review, retransmission or dissemination of this information by anyone other than the intended recipient is prohibited. If you receive this email in error, please contact the sender and delete this communication and any copies immediately. Thank you.
http://www.encana.com

[Martin Alfke]

On 06/23/2011 03:37 PM, Kinzel, David wrote:
>>> this is not directly built into puppet.
>>
>> Is there any specific reason why this is not built into puppet?
>>
>> We've put __addifnosuchline [0] into cdist [1] as one of
>> the most basic features, because it's often needed if you do not
>> care about the existing contents, but just need to add a line.
>>
>
> I second this. This would greatly cut down on things we do with exec
> patterns. And no, I don't want to use augeas as I typically see people
> fighting with it to work in all their environments.

I know that other config management tools also use things like
"appendifnosuchline".
Personally I do not like this feature. (I do not like augeas either.)

I like having a complete configuration file in my vcs.
In case of review I see all configuration parameters which belong to the
specific configuration file.
This is quite useful since we do reviews on a monthly basis
In case of application upgrades I review the complete config file. I do
not need to check for single lines and whether the syntax has changed.

On the other hand puppet offers you a flexible way to build up features
by yourself:
http://projects.puppetlabs.com/projects/1/wiki/Development_Practical_Types
(This shows how to build up a reusable type for append_if_no_such_line.)

my 5pence.

kind regards,

Martin

[Sans]

I [found and] started looking in the "Augeas" module - but looks like
there are some advantages as well. S, what's the best options (/
suggestions) we have? Basically I wanna add lines in the file like "/
etc/sysctl.conf", "/etc/my.cnf" etc. Cheers!!

[Nico -telmich- Schottelius]

Nigel,

Nigel Kersten [Thu, Jun 23, 2011 at 07:37:30AM -0700]:
> [...]
>
> It's simple enough at first, [...]
>

that's exactly the use case for __addifnosuchline in cdist.
The type let's you do what you want to do right now, fast and
simple. That's what a feature I've always missed in puppet.

> [...] I do think the "append if no such line" semantics aren't really maintainable over
> time.

If at any time in the future, I begin to add more and more lines
to the same file, I definitely should refacter and create a type
that handles managing this important file.

For me it's mostly a "take the right tool for the right job" selection here.

Cheers,

Nico

[Nigel Kersten]

On Thu, Jun 23, 2011 at 7:46 AM, Nico -telmich- Schottelius <nico-pup...@schottelius.org> wrote:

Nigel,

Nigel Kersten [Thu, Jun 23, 2011 at 07:37:30AM -0700]:
> [...]
>
> It's simple enough at first, [...]
>

that's exactly the use case for __addifnosuchline in cdist.
The type let's you do what you want to do right now, fast and
simple. That's what a feature I've always missed in puppet.

So it is really really simple to construct this, and the wiki link on Simple Text Patterns posted earlier shows how easy it is.

> [...] I do think the "append if no such line" semantics aren't really maintainable over
> time.

If at any time in the future, I begin to add more and more lines
to the same file, I definitely should refacter and create a type
that handles managing this important file.

For me it's mostly a "take the right tool for the right job" selection here.

I understand, and I'd be really happy to see someone take those simple text patterns and create a module on the Forge for this collection of functionality so people can get it easily.

I am hesitant for us to put something like this in the core distribution though, simply because we know it's a maintenance problem over time.

I'd be much more keen to see us have a good key/value type and provider that is easier to maintain and achieves the same result as "append if no such line".

 

[Kinzel, David]

Apoologies for this top posting. Blame Outlook.

I don't disagree at all. And for your specific example of sshd_config I
precisely do have a proper pattern for editing text files to set values
as needed (or not). This works extremely well and I am satisfied with
the results.

I just think having some built-in functions for text patterns would be
lovely. There are many cases where something like addifnosuchline would
be perfectly valid. Just because it won't be the solution in all cases
doesn't mean it should be dismissed.

But perhaps I should shut up and code these sometime.. :>

________________________________

From: puppet...@googlegroups.com
[mailto:puppet...@googlegroups.com] On Behalf Of Nigel Kersten
Sent: Thursday, June 23, 2011 8:38 AM
To: puppet...@googlegroups.com
Subject: Re: [Puppet Users] How to add extra lines in the
existing files?

On Thu, Jun 23, 2011 at 6:37 AM, Kinzel, David
<David....@encana.com> wrote:

>> this is not directly built into puppet.
>
>Is there any specific reason why this is not built into
puppet?
>
>We've put __addifnosuchline [0] into cdist [1] as one
of
>the most basic features, because it's often needed if
you do not
>care about the existing contents, but just need to add
a line.
>


I second this. This would greatly cut down on things we
do with exec
patterns. And no, I don't want to use augeas as I
typically see people
fighting with it to work in all their environments.

The problem with appendifnosuchline functionality is that it's
often easy at first, but then becomes difficult to maintain.

Say you want to set:

Subsystem sftp /usr/libexec/sftp-server

in your sshd_config.

It's simple enough at first, but what happens when you want to
change the actual setting? You're no longer able to do this within the
semantics of "append if no such line".

You need to know how the relevant service behaves when you have
two values for a setting in a config file. Does the first one win? the
last? Does it complain and not let you do this?

Or you end up changing the condition to check for to include a
regular expression or do partial line matching, which starts to bring
you closer to the Augeas model anyway.

This is why people like being able to use Augeas. You get to
express this in a more natural manner, along the lines of "set the sftp
subsystem to this value".

It is a little difficult to get going with Augeas, it requires a
shift in mindset, and the syntax is a little obscure, but I feel that
the best option is often to create a defined type that expresses what
you're trying to do in a simple manner, and that defined type wraps an
internal Augeas resource.

If you're set on not using Augeas, then do the same defined type
trick with sed/grep/awk/whatever as the internal resource, but whatever
you do, I do think the "append if no such line" semantics aren't really
maintainable over time.

--
You received this message because you are subscribed to the
Google Groups "Puppet Users" group.
To post to this group, send email to
puppet...@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

[Nigel Kersten]

On Thu, Jun 23, 2011 at 8:00 AM, Kinzel, David <David....@encana.com> wrote:

I just think having some built-in functions for text patterns would be
lovely. There are many cases where something like addifnosuchline would
be perfectly valid. Just because it won't be the solution in all cases
doesn't mean it should be dismissed.

But perhaps I should shut up and code these sometime.. :>

I'm happy to give help and advice if you do, but if no-one from the community steps up, I'll convert those simple text patterns to a Forge Module and a GitHub repo and we can move to a properly collaborative effort.

[Nico -telmich- Schottelius]

Nigel Kersten [Thu, Jun 23, 2011 at 07:51:23AM -0700]:
> nico-pup...@schottelius.org> wrote:
> [...]

> I am hesitant for us to put something like this in the core distribution
> though, simply because we know it's a maintenance problem over time.

Fair enough.

Though I imagine not having this feature will lose puppet some people.

> I'd be much more keen to see us have a good key/value type and provider that
> is easier to maintain and achieves the same result as "append if no such
> line".

I agree, this is yet another good type to have [0].
But it solves a somehow a different job.

Nico

[0] http://www.nico.schottelius.org/software/cdist/man/cdist-type__key_value.html

[Nigel Kersten]

On Thu, Jun 23, 2011 at 8:06 AM, Nico -telmich- Schottelius
<nico-pup...@schottelius.org> wrote:
> Nigel Kersten [Thu, Jun 23, 2011 at 07:51:23AM -0700]:
>> nico-pup...@schottelius.org> wrote:
>> [...]
>> I am hesitant for us to put something like this in the core distribution
>> though, simply because we know it's a maintenance problem over time.
>
> Fair enough.
>
> Though I imagine not having this feature will lose puppet some people.

Then lets get a simple text patterns module together for people to use.

>> I'd be much more keen to see us have a good key/value type and provider that
>> is easier to maintain and achieves the same result as "append if no such
>> line".
>
> I agree, this is yet another good type to have [0].
> But it solves a somehow a different job.

Is there any difference between "append if no such line" and a
key/value type that doesn't require a value to be provided?

[Aaron Grewell]

There are at least three ways to do this:
1) Use an Augeas resource to edit the file. For system files this can be a good solution since Augeas generally has lenses for them.
2) Use an Exec resource and something like sed or cat.  Not my preferred approach but sometimes the easiest, especially if you already scripted the task in bash at some point.  The tricky bit is getting the 'unless' portion working right so it doesn't run every time and just keep adding lines.
3) The 'concat' third-party module allows building a file from arbitrary chunks of text, which can include a both server-side and client-side pieces.  I just started using this one but so far it's very cool.  Even if you don't end up using it this time around it's worth a look in case you need it later. 
https://github.com/ripienaar/puppet-concat

[Nigel Kersten]

On Thu, Jun 23, 2011 at 6:37 AM, Kinzel, David <David....@encana.com> wrote:

>> this is not directly built into puppet.
>
>Is there any specific reason why this is not built into puppet?
>
>We've put __addifnosuchline [0] into cdist [1] as one of
>the most basic features, because it's often needed if you do not
>care about the existing contents, but just need to add a line.
>

I second this. This would greatly cut down on things we do with exec
patterns. And no, I don't want to use augeas as I typically see people
fighting with it to work in all their environments.

The problem with appendifnosuchline functionality is that it's often easy at first, but then becomes difficult to maintain.

Say you want to set:

Subsystem sftp /usr/libexec/sftp-server

in your sshd_config.

It's simple enough at first, but what happens when you want to change the actual setting? You're no longer able to do this within the semantics of "append if no such line".

You need to know how the relevant service behaves when you have two values for a setting in a config file. Does the first one win? the last? Does it complain and not let you do this?

Or you end up changing the condition to check for to include a regular expression or do partial line matching, which starts to bring you closer to the Augeas model anyway.

This is why people like being able to use Augeas. You get to express this in a more natural manner, along the lines of "set the sftp subsystem to this value".

It is a little difficult to get going with Augeas, it requires a shift in mindset, and the syntax is a little obscure, but I feel that the best option is often to create a defined type that expresses what you're trying to do in a simple manner, and that defined type wraps an internal Augeas resource.

If you're set on not using Augeas, then do the same defined type trick with sed/grep/awk/whatever as the internal resource, but whatever you do, I do think the "append if no such line" semantics aren't really maintainable over time.

[Peter Meier]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>>> I am hesitant for us to put something like this in the core distribution
>>> though, simply because we know it's a maintenance problem over time.
>>
>> Fair enough.
>>
>> Though I imagine not having this feature will lose puppet some people.
>
> Then lets get a simple text patterns module together for people to use.

But be aware that names like appendifnosuchline do not fit the mental
model that puppet tries to propagate, but are fine for "script oriented"
configuration management systems such as cfengine or cdist.

Something like:

line{'SftpSubsystem ....':
file => '/tmp/test',
ensure => (present|absent)
}

fits much more the mental model of puppet. Usually such a statement is a
quick but dirty way to do things and I try to avoid them as much as
possible. Therefore, I completely understand and support the concerns of
Nigel to not include such a feature in the core.

But yes it would be good to have a solid module in the forge where
people can be pointed at.

~pete
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4DaPMACgkQbwltcAfKi3/eegCgkG3wg+FSkHlMx5Ze6um8jw1M
7woAn1sGv7Jc5vMaSmguIE7Yl6NQYx8z
=F+9W
-----END PGP SIGNATURE-----

[Nigel Kersten]

On Thu, Jun 23, 2011 at 9:25 AM, Peter Meier <peter...@immerda.ch> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>>>> I am hesitant for us to put something like this in the core distribution
>>>> though, simply because we know it's a maintenance problem over time.
>>>
>>> Fair enough.
>>>
>>> Though I imagine not having this feature will lose puppet some people.
>>
>> Then lets get a simple text patterns module together for people to use.
>
> But be aware that names like appendifnosuchline do not fit the mental
> model that puppet tries to propagate, but are fine for "script oriented"
> configuration management systems such as cfengine or cdist.
>
> Something like:
>
> line{'SftpSubsystem ....':
>  file => '/tmp/test',
>  ensure => (present|absent)
> }
>
> fits much more the mental model of puppet. Usually such a statement is a
> quick but dirty way to do things and I try to avoid them as much as
> possible. Therefore, I completely understand and support the concerns of
> Nigel to not include such a feature in the core.

Yep. In a previous job we put together a type called "FileLine" which
I think fits with The Puppet Way™ better.