RFC: deprecating the 'resource_types' endpoint in Puppet Server

[Chris Price]

Hello,

We're looking into the possibility of deprecating and removing the `
resource_types` endpoint from Puppet Server.

The current implementation can return information about a lot of different
things in Puppet, but it's very expensive (in terms of CPU and memory
usage), and has some unexpected side effects that can cause subsequent
requests to other endpoints (including catalog compilation! see
https://tickets.puppetlabs.com/browse/SERVER-1200 ) to misbehave.

The main use case that we're aware of for the endpoint is to get a list of
all of the classes (and their associated parameters) for an environment.
We've just finished building a new HTTP endpoint called
'environment_classes' (which will ship in the upcoming releases of OSS and
PE Puppet Server) that will provide that data in a cleaner, less expensive
fashion.  You can see a sneak preview of the documentation for this new
endpoint, including its wire formats, here:

https://github.com/puppetlabs/puppet-server/blob/dc58bdd94246e5b68bf1adff2d38bf574ca22662/documentation/puppet-api/v3/environment_classes.md

Once that endpoint is available we'd like to deprecate and eventually
remove the resource_types endpoint.  However, we'd first like to make sure
that there aren't other important use cases that users are relying on it
for.  (If there are, we'll probably try to address those by adding
additional new HTTP endpoints until we're at feature parity for the things
that people rely on resource_types for.)

So... this is a solicitation for input.  Do you currently rely on the
resource_types endpoint for critical parts of your workflow?  What kinds of
things are you using it for?  What would you be missing if it were to be
removed?

We've got a placeholder ticket in our issue tracking system for this
deprecation / removal:

https://tickets.puppetlabs.com/browse/SERVER-1120

It doesn't have a ton of detail yet, but if you're interested in tracking
this, you can watch that ticket as that is where we'll end up tracking
things as this progresses.

Thanks in advance for your feedback!

[Erik Dalén]

The API looks good to me.

Will this be available as a command line tool (standalone one, not contacting a master)? That would be useful for scripts and local development where you might not want to upload the code to a puppetmaster in order to find the classes.

If it is using the indirector it should be pretty trivial to create a puppet face for it.

Also, will you create a similar one for defined types? Functions?

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAMx1QfLUqXuDSk_Zpqf%3DGvmqjprYmzfwa%3DnUdrbxO3LhOLT1bg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Chris Price]

On Fri, Mar 11, 2016 at 3:54 AM, Erik Dalén <erik.gus...@gmail.com> wrote:

The API looks good to me.

Will this be available as a command line tool (standalone one, not contacting a master)? That would be useful for scripts and local development where you might not want to upload the code to a puppetmaster in order to find the classes.

I don't think we have immediate plans for that, but I agree that it sounds like it could be useful, and probably pretty easy.  Might be worth creating a Jira ticket for.

Henrik may have thought about this more than I have.

 

If it is using the indirector it should be pretty trivial to create a puppet face for it.

Also, will you create a similar one for defined types? Functions?

For defined types, we are trying to determine whether there were use cases that people needed that were being satisfied by the resource_types endpoint.  If so, we'll need to do that before we get rid of 'resource_types'.  If not, and for functions, I would imagine that we will eventually end up creating them, but perhaps not until after we get rid of resource_types.  If you think those would be particularly useful, then we should capture them in Jira as well.  I hadn't thought of the idea about functions before, that's pretty interesting!

Because removing resource_types is something that would need to happen on a major version boundary, right now the main goal is just to figure out what we need to do to be ready for that whenever we start lining up dates for future major version bumps.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAAAzDLfieuo5bv3s3u%3D1Gmk%3DzuwYkUSfQYFruiqgYz%3DtvSvypw%40mail.gmail.com.

[Henrik Lindberg]

On 11/03/16 15:06, Chris Price wrote:
>
>
> On Fri, Mar 11, 2016 at 3:54 AM, Erik Dalén <erik.gus...@gmail.com
> <mailto:erik.gus...@gmail.com>> wrote:
>
> The API looks good to me.
>
> Will this be available as a command line tool (standalone one, not
> contacting a master)? That would be useful for scripts and local
> development where you might not want to upload the code to a
> puppetmaster in order to find the classes.
>
>
> I don't think we have immediate plans for that, but I agree that it
> sounds like it could be useful, and probably pretty easy. Might be
> worth creating a Jira ticket for.
>
> Henrik may have thought about this more than I have.
>

No, I have not thought about it. But have now :-)

We have several kinds of definitions that are of interest (Puppet 4.5.0):

- Resource Types
- Functions (ruby and puppet)
- Data Types (ruby and puppet)
- Aliased Data Types (typically puppet)

To be able to do something useful, somone needs to figure out the
requirements, what they want to get out of this - as opposed to say
generating documentation for the above and reading the documentation.

Is this a "quick list" of what is available?

>
> If it is using the indirector it should be pretty trivial to create
> a puppet face for it.
>
> Also, will you create a similar one for defined types? Functions?
>

Functions and data types have no indirector related functionality.

- henrik

> <mailto:puppet-dev+...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-dev/CAMx1QfLUqXuDSk_Zpqf%3DGvmqjprYmzfwa%3DnUdrbxO3LhOLT1bg%40mail.gmail.com

> <https://groups.google.com/d/msgid/puppet-dev/CAMx1QfLUqXuDSk_Zpqf%3DGvmqjprYmzfwa%3DnUdrbxO3LhOLT1bg%40mail.gmail.com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to puppet-dev+...@googlegroups.com

> <mailto:puppet-dev+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-dev/CAAAzDLfieuo5bv3s3u%3D1Gmk%3DzuwYkUSfQYFruiqgYz%3DtvSvypw%40mail.gmail.com
> <https://groups.google.com/d/msgid/puppet-dev/CAAAzDLfieuo5bv3s3u%3D1Gmk%3DzuwYkUSfQYFruiqgYz%3DtvSvypw%40mail.gmail.com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-dev+...@googlegroups.com

> <mailto:puppet-dev+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-dev/CAMx1QfJ6gCeMV35XSw5pBbCM%3DH6tf48Sw6fnOgjAT6kTgo-Zug%40mail.gmail.com
> <https://groups.google.com/d/msgid/puppet-dev/CAMx1QfJ6gCeMV35XSw5pBbCM%3DH6tf48Sw6fnOgjAT6kTgo-Zug%40mail.gmail.com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout.


--

Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/

[Chris Price]

On Wed, Mar 9, 2016 at 12:45 PM, Chris Price <ch...@puppetlabs.com> wrote:

Hello,

We're looking into the possibility of deprecating and removing the `
resource_types` endpoint from Puppet Server.

Bump... haven't heard any objections to deprecating and removing the resource_types endpoint.  I'll plan on getting some more concrete tickets about this into our issue tracker next week unless anyone raises any last-minute objections between now and then.  They'll be linked to SERVER-1200 if anyone wants to follow along at home.  Thanks!

[Trevor Vaughan]

I scraped through my code and don't seem to be using it anywhere so +1.

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAMx1QfLs_SyqWe8FKdimQXmZUwLGXfcxf30d%2Bq5Sf1S7rktKzA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699

-- This account not approved for unencrypted proprietary information --