Speculative Syntax: Varargs & Matchers

[Igor Galić]

Hi folks,

over the last couple of days I've been dabbling with the design
for a new module for Apache httpd: we use our own package & module
currently, which allows for multi-instance configurations, but are
trying to shift towards "standard" Ubuntu 2.4 package (I created a
ppa with a current version: https://launchpad.net/~apache-helpdesk/+archive/httpd-ppa )

I've been hitting a snag in the design, because httpd and many of
its modules are complex beasts, and I don't want to try and foresee every
single directive someone might want to use in templates / (defined) type / class
but rather have some form of matcher. Since most of the directives follow
simple patterns, I'd like to handle those, and reject everything else -
rather than handling everything explicitly as is the case now.

For a defined type, or a class, we could introduce the special variables
`$_` and `$*` (I'm not happy with those names. They are short, that's nice
but otherwise, they are too Perlesque, and not very speaking)

define httpd::mod (
$ensure = 'enabled',
$instance = 'default',
$* # all others
) {
validate_re($ensure, 'enabled|disabled')

Match {
/_enabled$/ => { validate_bool($_) }
/_path$/ => { validate_absolute_path($_) }
/s$/ => { validate_array($_) }
/_size$/ => { validate_number($_) }
default => { fail("${name} We really don't know what to make of this directive") }
}
}

alternatively, we could match the front part:

define httpd::directory (
$ensure = 'present',
$instance = 'default',
$*
) {
# …
Match {
/(proxy|ssl|..)_/ => { httpd::mod { $1: ensure => $ensure, instance => $instance, $* }
}
}

This is kind of the basic idea, and it's lacking a good way to transform those
matches into actual variables we can access, but I hope you get the basic idea.


The main reason I wish this was supported syntax, is that a $catch_all_other_settings
hash generally translates poorly through all the layers of
yaml -> ruby -> puppet dsl -> actual config. The chances for very specific failures
are very high simply because of the amount of layers, each of which can introduce
their own implementation leak.

Highly welcome your feedback!



-- o/~ i
Igor Galić

Tel: +43 (0) 664 886 22 883
Mail: i.g...@brainsware.org
URL: http://brainsware.org/
GPG: 8716 7A9F 989B ABD5 100F 4008 F266 55D6 2998 1641

[Andy Parker]

On Wed, Jun 11, 2014 at 6:27 AM, Igor Galić <i.g...@brainsware.org> wrote:

Hi folks,

over the last couple of days I've been dabbling with the design
for a new module for Apache httpd: we use our own package & module
currently, which allows for multi-instance configurations, but are
trying to shift towards "standard" Ubuntu 2.4 package (I created a
ppa with a current version: https://launchpad.net/~apache-helpdesk/+archive/httpd-ppa )

I've been hitting a snag in the design, because httpd and many of
its modules are complex beasts, and I don't want to try and foresee every
single directive someone might want to use in templates / (defined) type / class
but rather have some form of matcher. Since most of the directives follow
simple patterns, I'd like to handle those, and reject everything else -
rather than handling everything explicitly as is the case now.

Yes, I can see this problem. The explosion of parameters. I think the the style of creating a structure that gets passed around is viable. Ut even has a name in OO, "Parameter Object" (see http://c2.com/cgi/wiki?ParameterObject). So it seems that the problem is that the structure of that object can't be expressed very well and so you end up with all sorts of checks being done.

 

For a defined type, or a class, we could introduce the special variables
`$_` and `$*` (I'm not happy with those names. They are short, that's nice
but otherwise, they are too Perlesque, and not very speaking)

    define httpd::mod (
      $ensure   = 'enabled',
      $instance = 'default',
      $*  # all others
    ) {
      validate_re($ensure, 'enabled|disabled')

      Match {
        /_enabled$/ => { validate_bool($_) }
        /_path$/    => { validate_absolute_path($_) }
        /s$/        => { validate_array($_) }
        /_size$/    => { validate_number($_) }
        default     => { fail("${name} We really don't know what to make of this directive") }
      }
    }

I'm not saying that we should add this syntax, but one that would fit better with the lambdas is:

  define http:mod(

    $ensure = 'enabled',

    $instance = 'default',

    *$args

  ) { .... }

Which would slurp all unknown parameters into the hash $args.

At this point then you have the question of how to express what structure $args can have. You could use the new type system to describe the hash (http://puppet-on-the-edge.blogspot.com/2013/12/the-type-hierarchy-and-literals.html) by doing something like:

  $args =~ Struct[{ Pattern['_enabled$'] => Bool, Pattern['_path$'] => Pattern['....'], Pattern['s$'] => Array, Pattern['_size$'] => Number }]

Note: I haven't actually tried that out yet, so take the syntax with a grain of salt.

However, at this point you now have the question of which is better, a bunch of individual paramters, or a single structure that can be passed around. If these are slurped parameters, then you need a way of splatting them back into another resource if you want to pass them along. On the other hand, if they are just and object, then it is just passing around that single object.

 

alternatively, we could match the front part:

    define httpd::directory (
       $ensure   = 'present',
       $instance = 'default',
       $*
    ) {
      # …
      Match {
        /(proxy|ssl|..)_/ => { httpd::mod { $1: ensure => $ensure, instance => $instance, $* }
      }
    }

This is kind of the basic idea, and it's lacking a good way to transform those
matches into actual variables we can access, but I hope you get the basic idea.


The main reason I wish this was supported syntax, is that a $catch_all_other_settings
hash generally translates poorly through all the layers of
yaml -> ruby -> puppet dsl -> actual config. The chances for very specific failures
are very high simply because of the amount of layers, each of which can introduce
their own implementation leak.

Highly welcome your feedback!



-- o/~ i
Igor Galić

Tel: +43 (0) 664 886 22 883
Mail: i.g...@brainsware.org
URL: http://brainsware.org/
GPG: 8716 7A9F 989B ABD5 100F  4008 F266 55D6 2998 1641

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/306231088.39368.1402493220564.JavaMail.zimbra%40brainsware.org.
For more options, visit https://groups.google.com/d/optout.

--

Andrew Parker

an...@puppetlabs.com

Freenode: zaphod42

Twitter: @aparker42

Software Developer

Join us at PuppetConf 2014, September 22-24 in San Francisco

Register by May 30th to take advantage of the Early Adopter discount —save $349!

[Henrik Lindberg]

Having what is essentially a "call by name" support varargs can be made
by sticking any excess attribute assignments into a hash and assigning
the constructed hash when the resource is instantiated.

In Puppet 4.0 (and in 3.7 with --parser future) you will have the
ability to also type the argument. There is a Struct type that can
validate ha hash with keys where data has different type. It does not
(yet at least) support defining keys with regexps. That could be added.
Later in the 4.x series we plan to introduce the ability to create
complex types and give them names (makes it easier to use complex types).

Also in 3.7 future/4.0 is the ability to use varargs in lambdas using a
splat '*' notation. Currently this implementation does not allow defines
and classes to use varargs. If we do, the syntax will be:

define something ($x, *$y) {}

and if you also add in typing, you get

define something (String $x, Struct[{ a=>Integer, b=>String}] *$y) { }

I can imagine extending the Struct type to allow keys to be regular
expressions, or the Enum type (list of given keys).

Yet more options when we allow creation of arbitrary types (including
non resource types), is to simply pass around an instance of a Struct
(essentially a hash with defined keys / types).

Regards
- henrik


--

Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/

[Igor Galić]

Henry, Andy,

that all sounds very promising, and exciting!
I'm very much looking forward for that.

[Andy Parker]

On Thu, Jun 12, 2014 at 4:14 AM, Igor Galić <i.g...@brainsware.org> wrote:

Henry, Andy,

that all sounds very promising, and exciting!
I'm very much looking forward for that.

I don't want to get your hopes up and then dash them, but beyond what is already implemented in the type system we have no plans to implement any of this. My example turns out to not be correct. As Henrik said, the Struct type only handles explicit string values as its keys.

Was there any specific thing that we said could be done that you are interested in?

 

-- o/~ i
Igor Galić

Tel: +43 (0) 664 886 22 883
Mail: i.g...@brainsware.org
URL: http://brainsware.org/
GPG: 8716 7A9F 989B ABD5 100F  4008 F266 55D6 2998 1641

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/568369049.40513.1402571681739.JavaMail.zimbra%40brainsware.org.

For more options, visit https://groups.google.com/d/optout.

[Igor Galić]

perhaps a better way to summarize this issue is this:

define httpd::mod (
$ensure = 'enabled',
$instance = 'default',

$module = $title,
$package = undef,
) {

}

This definition of httpd::mod's interface is static.
it's a static map (Hash). If we pass any additional parameters
it'll blow up, and rightfully so.

What I wanted with this:

>
> define httpd::mod (
> $ensure = 'enabled',
> $instance = 'default',

> $module = $title,
> $package = undef,

> $* # all others
> ) {

(or whatever the syntax) was a dynamic interface. A way to
programmatically define that map.



(came up in a discussion with daff/antaflos today, so I thought
I'd like to clarify my now seemingly more clear thoughts on that ;)


~bye, o/~
-- i

[Henrik Lindberg]

On 2014-04-07 22:29, Igor Galić wrote:
> perhaps a better way to summarize this issue is this:
>
> define httpd::mod (
> $ensure = 'enabled',
> $instance = 'default',
> $module = $title,
> $package = undef,
> ) {
>
> }
>
> This definition of httpd::mod's interface is static.
> it's a static map (Hash). If we pass any additional parameters
> it'll blow up, and rightfully so.
>
> What I wanted with this:
>
>>
>> define httpd::mod (
>> $ensure = 'enabled',
>> $instance = 'default',
>> $module = $title,
>> $package = undef,
>> $* # all others
>> ) {
>
> (or whatever the syntax) was a dynamic interface. A way to
> programmatically define that map.
>

The issue here is that we cannot have a user defined type that has a
different set of parameters for different instances (which would be the
case here since different instantiations can create instances with very
different sets of parameters). The only way to do this is to define the
last parameter as a Hash, and pass the additional parameters as such.

You can use the type system to check the validity of the content of the
hash (when using the future parser).