Jira (PUP-523) Make it possible to declare classes, defines and functions (and more) as private to a module

[Henrik Lindberg (JIRA)]

Henrik Lindberg updated an issue

Puppet / PUP-523 Make it possible to declare classes, defines and functions (and more) as private to a module Change By: Henrik Lindberg Summary: Make it possible to declare classes, defines and functions  (and more)  as private to a module Add Comment

This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc)

[Henrik Lindberg (JIRA)]

Henrik Lindberg updated an issue

Puppet / PUP-523 Make it possible to declare classes, defines and functions (and more) as private to a module Change By: Henrik Lindberg

In order to support Modularity it is important that module authors can define what they consider to be API and what is an internal implementation  concern. In order to support this, the keyword 'private' has already been added to the lexer as a reserved word, but it currently has no functionality. Something that is declared private in a module can only be used from within that module. Something that is declared private in an environment is not visible to modules in that environment. We want to make the following things private: * functions * type aliases and defiitions * resources * classes * variables in public classes (they are by definition private in private classes) A number if things needs to be addressed in order for this to work: * The 4.x loaders must support that modules contain a 'private' loader that contains things only visible to it. * The 4.x loaders must be capable of loading ** Definitions ** Clases ** 3.x functions (not absolutely required as there is no way to mark them as private, but there are other benefits of doing this) * The grammar must be changed to support the private keyword in front of function, type alias, type definition, class, and define. * The respective loader/creator of the above language elements need to place them in the private loader for the module (or the environment) * Puppet Strings need to skip producing documentation; alternatively link them into a separate section that is optionally rendered. * The language specification must be updated * To support private variables in a public class, scope must know about private variables and hide them from external access. ** When doing so, it must support EPP and ERB templates and access from within the same module. * The 4.x Ruby function API must be updated to allow 4.x functions in ruby to be declared as private.

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module The comments above can be ignored as this ticket has be repurposed as an Epic to add support for module privacy.

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg updated an issue

Puppet / PUP-523

Make it possible to declare classes, defines and functions (and more) as private to a module

Change By: Henrik Lindberg Epic Name: 5.x Module Privacy

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module In PDOC-152, Trevor Vaughan brings up that it is meaningful to differentiate between private and protected classes as it is meaningful to be able to set parameters via APL for those that are protected.

Add Comment

This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)

[Trevor Vaughan (JIRA)]

Trevor Vaughan commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

Henrik Lindberg I was thinking more about this and I don't know that I see the point of a class where I can't set the parameters via data binding. If I don't want someone to mess about with the variables in my class, I just don't make them parameters. Am I missing something?

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

Yes, maybe You can set parameters in data files inside the module (hiera version 4 and 5). There could be a distinction made for private such that those data values are not allowed to be overridden by data in the environment or global layers. This would make sense if private means "private to the implementation" and therefore "private to the module author". You could still, in the same module, override those by giving value explicitly. No other module would be able to do either one of those. We could naturally take a more relaxed approach allowing values to be bound in any layer even if the class is private - thereby being closer to the notion of protected. You may want to use class parameters because you want hierarchical data resolution using APL at the same time as you do not want anyone to mess with the values.

Add Comment

[Eric Sorenson (JIRA)]

Eric Sorenson commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

(Non-Trevor users, I mean)

Add Comment

[Eric Sorenson (JIRA)]

Eric Sorenson commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

FWIW the only part of this proposal I have heard users ask for is marking classes private, so they do not show up for assignment in the console classifier and are not enumerated with the environment_classes endpoint.

Add Comment

[Trevor Vaughan (JIRA)]

Trevor Vaughan commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

Eric Sorenson and Henrik Lindberg I see your points there and I definitely get the idea behind not having truly private classes show up in the console. Why would I want to see something that I can't affect?! To that end, it would seem like you would want the following: private Does not show in the console Cannot be assigned anything via data binding Raises an error on direct inclusion outside of the module space protected Does not show in the console for inclusion Allows for different data binding scope settings data_scope = local or backend_name or global ? Defaults to global If data_scope allows for the ENC, will show in the console for parameter assignment (but not individual inclusion) The main reason that I'm looking for this is ease of maintenance. I really want to have rich classes but I don't want to have to repeat all of my parameters everywhere, that way lies fragile code and maintenance hell. init.pp will expose the most commonly used options and, should you want to do something more interesting, you can go prod individual parts of the sub-classes. I probably am one of the few that have mentioned this at this point but I also don't know of many other 100k+ LOC publicly published Puppet infrastructures.

Add Comment

[Lindsey Smith (JIRA)]

Lindsey Smith updated an issue

Puppet / PUP-523

Make it possible to declare classes, defines and functions (and more) as private to a module

Change By: Lindsey Smith Team/s: Puppet Developer Experience Agent

Add Comment

[Sean McDonald (JIRA)]

Sean McDonald updated an issue

Puppet / PUP-523 Make it possible to declare classes, defines and functions (and more) as private to a module

Change By: Sean McDonald Labels: triaged

Add Comment

[Moses Mendoza (JIRA)]

Moses Mendoza updated an issue

Puppet / PUP-523 Make it possible to declare classes, defines and functions (and more) as private to a module

Change By: Moses Mendoza Labels: triaged

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg updated an issue

Puppet / PUP-523 Make it possible to declare classes, defines and functions (and more) as private to a module

Change By: Henrik Lindberg Fix Version/s: PUP 5.0.0

Add Comment

[Melissa Casburn (JIRA)]

Melissa Casburn commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module We continue to have performance issues with private classes showing in the Node Classifier and overloading GUI controls – any update on when this particular aspect might be corrected?

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

There are related ideas for hiera about making it illegal to override key bindings in a module for "private keys" (for example for private classes) by making it possible to declare a key to be final.

Add Comment

[Trevor Vaughan (JIRA)]

Trevor Vaughan commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

Henrik Lindberg As long as it's per individual key, that would be great. Any thoughts on defaults lookups in hiera for defined types? There's really no reason that it can't be done since everything is exposed at compile time.

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-523

Make it possible to declare classes, defines and functions (and more) as private to a module

Change By: Geoff Nichols Team/s: Agent Platform Core

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module Trevor Vaughan It would be per key - yes. APL for defined types has issues if introduced: it means that that every non explicitly set parameter for every resource would need two lookups (one with resource type + title, and one with resource type) this means many thousands of additional lookups which may be a big performance problem parameter binding by APL would win over resource type defaults (probably a good thing though), but resource type defaults are more versatile as they apply per scope, now if APL wins it is not a good thing. To work as a replacement of resource type defaults hiera would need to be aware of scope (more complex, and possibly a performance problem). would need to involve the title of the resource in the key to make it unique, this creates potential issues when authoring the data as titles do not have the same constraints as does parameter names.

Add Comment

[Trevor Vaughan (JIRA)]

Trevor Vaughan commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

Henrik Lindberg Just had this thought... how about a Const Data Type that marks a parameter as immutable? It doesn't solve all of the proposed issues but it seems reasonable in terms of the data typing system and understandability.

Add Comment

This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

Trevor Vaughan Do you mean that it would prevent a resource override? IIRC that is the only way (except hackery in Ruby) that would alter already set values in resources (It is already forbidden to alter a class' parameters).

Add Comment

[Trevor Vaughan (JIRA)]

Trevor Vaughan commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

Henrik Lindberg I was thinking for defined types as well. It would prevent anything from ever modifying that parameter. So ASL would ignore it, resource collectors would ignore it, etc...

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

Ah, so you mean not even giving it in a resource instantiation expression? Example where "CONSTANT" is an imaginary marker declaring $y to be a constant. define foo($x, CONSTANT $y = 10) { ... } foo { 'a foo': x => 'hello', # ok y => 20, # error, y is a constant }

Add Comment

[Trevor Vaughan (JIRA)]

Trevor Vaughan commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

Exactly, yes. Also: <| Foo |> { y => 20

} # error, y is a constant

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

The only thing you can do at the moment is to assign to a variable in a class as those are immutable.

Add Comment

[Trevor Vaughan (JIRA)]

Trevor Vaughan commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

Henrik Lindberg Completely get how to do it right now but it gets confusing for end users/maintainers since you have to use inheritance to get the behavior that this would provide.

Add Comment

[Ewoud Kohl van Wijngaarden (JIRA)]

Ewoud Kohl van Wijngaarden commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

From a Foreman perspective I'll second what Eric Sorenson said: FWIW the only part of this proposal I have heard users ask for is marking classes private, so they do not show up for assignment in the console classifier and are not enumerated with the environment_classes endpoint.

Currently we provide an import regex filter, but obviously this is far from optimal. Module authors are in the best position to identify which classes are private and if the HTTP API would expose this information, then we could filter them out. When we think about backwards compatibility then there are 2 ways (that I know of) to make a class "private". Puppet Strings' @api private documentation marker puppetlabs-stdlib's assert_private() function The function has the downside that you need stdlib to be present and the Puppetserver shouldn't code to specific modules. IMHO this isn't a realistic option. I don't know if Puppetserver currently parses Puppet Strings. If it doesn't, then that's a downside that. It could have an additional benefit of also exposing the parameter documentation. An ENC interface could use this to provide meaningful documentation to end users.

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

The @api_private in String/Yard simply indicates that the API is not public - it is still API and you can generate docs for the private sections - the default is to skip those though. And Puppet Server does not use Puppet Strings in any way. Adding real support for private would do something like the assert_private() function, but be built into the language as a keyword. Then it is a matter of making use of that information for other purposes like filtering them out.

Add Comment

[Ben Ford (JIRA)]

Ben Ford commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

What about adding a list of private class names to metadata.json instead? That wouldn't require any keyword changes or extra functions and it would be forwards/backwards compatible.

Add Comment

[Trevor Vaughan (JIRA)]

Trevor Vaughan commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

Ben Ford That seems like a tie from data to logic to me (and likely to get confusing).   Why not just make assert_private() a native function and have Strings just pick up the fact that the class has that call in it?

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

We already have the private keyword reserved, syntax is easily modified to support this. But it is more than just taking on the `assert_private()` function since the private keyword can be applied to not only an entire class but also to individual parameters.

Add Comment

[Ewoud Kohl van Wijngaarden (JIRA)]

Ewoud Kohl van Wijngaarden commented on PUP-523

Re: Make it possible to declare classes, defines and functions (and more) as private to a module

Perhaps it'd be a good idea to split the effort. Clearly there's already an existing workflow around private classes, so that could be formalized. Having private parameters could be considered out of scope. This would allow the ecosystem to standardize and improve.

Add Comment

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-523

Make it possible to declare classes, defines and functions (and more) as private to a module

Change By: Josh Cooper Team/s: Platform Core Froyo

Add Comment

This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)

[Maggie Dreyer (Jira)]

Maggie Dreyer updated an issue

Puppet / PUP-523 Make it possible to declare classes, defines and functions (and more) as private to a module

Change By: Maggie Dreyer Team/s: Froyo

Add Comment