Announce: Puppet Server 0.2.0

[Nate Wolfe]

We are thrilled to announce the preview release of Puppet Server, our newest open source project.

Puppet Server is a next-generation alternative to our current Puppet master, which builds on the

successful Clojure technology stack underlying projects like PuppetDB.

Packages are available in the Puppet Labs package repositories, so you can try it out today as a

drop-in replacement for the existing Puppet master.

As the version number 0.2.0 should imply, Puppet Server is not production ready (yet), but please

do try it out in your favorite sandbox. Additionally, the API will not be considered stable until

Puppet Server reaches 1.0.0.

Install Puppet Server from packages:

https://github.com/puppetlabs/puppet-server/blob/b6a76e/documentation/install_from_packages.markdown

Submit issues to:

https://tickets.puppetlabs.com/browse/SERVER

Source:

https://github.com/puppetlabs/puppet-server

[Gabriel Filion]

On 23/09/14 12:11 PM, Nate Wolfe wrote:
> We are thrilled to announce the preview release of Puppet Server, our
> newest open source project.
> Puppet Server is a next-generation alternative to our current Puppet
> master, which builds on the
> successful Clojure technology stack underlying projects like PuppetDB.

so... is it the long term goal to phase out the ruby-based puppet master
when the clojure-based one is mature enough?

--
Gabriel Filion

[Matt Zagrabelny]

Hopefully someone closer to the situation (and with more authority)
will respond, but "yes" that is what I was told at a Puppet training
in March.

-m

[Trevor Vaughan]

Hi Gabriel,

From the recent PuppetConf talk on Puppet Server, the goal appears to be to move this way. However, many things need to be worked out. Not the least of which is how 'puppet apply' and msterless is going to work.

The speed improvements and smaller resource usage appear quite impressive, but we'll see how things go as we move forward.

(I'm sure I'll get corrected if I'm getting the details wrong anywhere, but the Puppet Labs folks are pretty busy today ;-)

Thanks,

Trevor

--
Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvau...@onyxpoint.com

-- This account not approved for unencrypted proprietary information --

[Pete Brown]

I got to test it in one of my dev environments while I was at
Puppetconf and it was pretty awesome.
Looking forward to having the time to add it as an option to my puppet
management module.
I am intending on running it in my dev environment to help out with testing.

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/CANs%2BFoVXKd%2B9VE7H2L9RJLBZOuCeDTKBmpgD%3DYFhSASve4k-jA%40mail.gmail.com.
>
> For more options, visit https://groups.google.com/d/optout.



--
Pete Brown
Director and Primary Systems Engineer
Abstract IT Pty Ltd.

[Andy Parker]

The language will remain in ruby for a while yet (the puppet-server uses JRuby to run the puppet ruby code), and even after the interpreter gets reimplemented in a faster language, ruby will be available for extensions. It would simply be too much of a change to drop ruby and drop all of the custom code and extensions that everyone has worked so hard on.

What *might* be happening very soon is dropping rack and webrick support and moving entirely to the puppet-server. It provides a much simpler setup, better performance, and more controlled environment for us to target.

"Puppet apply" will be sticking around. The exact way in which it will work isn't completely clear yet. There are several possibilities from requiring the jvm on all nodes that run puppet apply to changing apply to use a "compile server" to actually run manifests or possibly reimplementing the interpreter in a non-jvm language and using JNI to make it available on the puppet-server. My currently leaning is to implement the language in a jvm language and requiring that a masterless setup has the jvm on the node.

 

-m

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAOLfK3W6zzYtDFEVi_BE85WHbVZ-pf9qppANDOudG8SogdXSUA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Andrew Parker

an...@puppetlabs.com

Freenode: zaphod42

Twitter: @aparker42

Software Developer

Join us at PuppetConf 2014, September 20-24 in San Francisco - www.puppetconf.com 

[Deepak Giridharagopal]

On Thu, Sep 25, 2014 at 1:32 PM, Andy Parker <an...@puppetlabs.com> wrote:

"Puppet apply" will be sticking around. The exact way in which it will work isn't completely clear yet. There are several possibilities from requiring the jvm on all nodes that run puppet apply to changing apply to use a "compile server" to actually run manifests or possibly reimplementing the interpreter in a non-jvm language and using JNI to make it available on the puppet-server. My currently leaning is to implement the language in a jvm language and requiring that a masterless setup has the jvm on the node.

Heh...I'm leaning more towards the native code route, but this is definitely something we should work out on the puppet-dev list before we actually proceed down a path. Regardless, in the meantime "apply" will work how it always has.

deepak

--

Deepak Giridharagopal / Puppet Labs

[Trevor Vaughan]

+1 for a Native Code drop in. That would make me crazy happy actually.

-1 for sticking Java everywhere. That would make some of my users hate me with  passion.

Trevor

--

You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAOjOXY0h6dK3DMYqPjPhYXbh8ta3%2B5cAbvaOU7%2BE0BsCV4FqPQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

[Rob Reynolds]

On Fri, Sep 26, 2014 at 7:56 AM, Trevor Vaughan <tvau...@onyxpoint.com> wrote:

+1 for a Native Code drop in. That would make me crazy happy actually.

-1 for sticking Java everywhere. That would make some of my users hate me with  passion.

Java just on server side. Native is moving towards C++.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CANs%2BFoUQPSaNqYEGMywXfYFZKV2XLbmm%3DHw4v6j0YUf9SzuigQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Rob Reynolds

Developer, Puppet Labs

[Niels Abspoel]

For the Archlinux users, an aur package is available:

https://aur.archlinux.org/packages/puppetserver/

Op dinsdag 23 september 2014 18:12:29 UTC+2 schreef Nate Wolfe:

[Trevor Vaughan]

Works for me.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAMJiBK7%3D7BL2FR3r9c0DStOWpz2E8AG__%2BMq7humd14G0%3DOqEQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Felix Frank]

On 09/26/2014 09:45 PM, Rob Reynolds wrote:
>
> Java just on server side. Native is moving towards C++.

No wait, what, really?

[Andy Parker]

Felix, if your "what?" is about Java (the language), that was a mistake. JVM on the server side, generally written in Clojure, is the direction things are heading. C++ on the client side. Ruby is still sticking around in order to support extensions. That said, some things we'll need to discuss and figure out what the best way to move forward is. Luke wrote up a good post about some of these changes at http://puppetlabs.com/blog/evolving-puppet-for-next-ten-years

As I said in my original response about apply, that is something that we still need to figure out and something that will be coming up on puppet-dev in the future. There is a lot to consider and discuss in that decision because of the number of people and systems affected.

 

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/5425CCD6.8050703%40Alumni.TU-Berlin.de.

For more options, visit https://groups.google.com/d/optout.

--

Andrew Parker

an...@puppetlabs.com

Freenode: zaphod42

Twitter: @aparker42

Software Developer

Join us at PuppetConf 2014, September 20-24 in San Francisco - www.puppetconf.com 

[Rob Reynolds]

On Fri, Sep 26, 2014 at 4:02 PM, Andy Parker <an...@puppetlabs.com> wrote:

On Fri, Sep 26, 2014 at 1:30 PM, Felix Frank <Felix...@alumni.tu-berlin.de> wrote:

On 09/26/2014 09:45 PM, Rob Reynolds wrote:
>
> Java just on server side. Native is moving towards C++.

No wait, what, really?

Felix, if your "what?" is about Java (the language), that was a mistake. JVM on the server side, generally written in Clojure, is the direction things are heading. C++ on the client side. Ruby is still sticking around in order to support extensions. That said, some things we'll need to discuss and figure out what the best way to move forward is. Luke wrote up a good post about some of these changes at http://puppetlabs.com/blog/evolving-puppet-for-next-ten-years

As I said in my original response about apply, that is something that we still need to figure out and something that will be coming up on puppet-dev in the future. There is a lot to consider and discuss in that decision because of the number of people and systems affected.

Yes, apologies about that. I meant JVM, generally written in Clojure. I was using language from the original statement from Trevor and should have been more clear.

 

 

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/5425CCD6.8050703%40Alumni.TU-Berlin.de.

For more options, visit https://groups.google.com/d/optout.

--

Andrew Parker

an...@puppetlabs.com

Freenode: zaphod42

Twitter: @aparker42

Software Developer

Join us at PuppetConf 2014, September 20-24 in San Francisco - www.puppetconf.com 

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CANhgQXt7E%3Dw%2Bw4fhmvargeLdtDsdMvJ7CLFikKECBTiidZ-nYg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Rob Reynolds

Developer, Puppet Labs

[Felix Frank]

On 09/26/2014 11:12 PM, Rob Reynolds wrote:

Felix, if your "what?" is about Java (the language), that was a mistake. JVM on the server side, generally written in Clojure, is the direction things are heading. C++ on the client side. Ruby is still sticking around in order to support extensions. That said, some things we'll need to discuss and figure out what the best way to move forward is. Luke wrote up a good post about some of these changes at http://puppetlabs.com/blog/evolving-puppet-for-next-ten-years

As I said in my original response about apply, that is something that we still need to figure out and something that will be coming up on puppet-dev in the future. There is a lot to consider and discuss in that decision because of the number of people and systems affected.

Yes, apologies about that. I meant JVM, generally written in Clojure. I was using language from the original statement from Trevor and should have been more clear.

Thanks guys, I should have been more specific in my wild exclamation :-)

I'm good with JVM, but C++ feels like a backward step in many regards. My judgment here may be clouded by reading too many blogpost of them naysayers.

[Jason Antman]

FWIW,

(1) at my current shop, there's an immense hatred of everything JVM. That's going to be a hard transition. Not to mention Puppet is the only place we run Ruby, so it's nice and easy to let puppet do whatever it wants with Ruby. Not so much for installing JVMs that may break production (improperly configured and installed, I'll grant) applications.

(2) I've gotta say, I'll really miss dropping log statements directly in the puppet source when something seems wonky (and not having to compile something).

--

You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/5425F9F9.4010501%40Alumni.TU-Berlin.de.

[Ken Barber]

> (1) at my current shop, there's an immense hatred of everything JVM. That's going to be a hard transition. Not to mention Puppet is the only place we
> run Ruby, so it's nice and easy to let puppet do whatever it wants with Ruby. Not so much for installing JVMs that may break production (improperly
> configured and installed, I'll grant) applications.

And rightly so - its had a bad history, but I must argue that largely
my hatred of JVM in the past wasn't the JVM per se, it was the
applications written for it. But I would gladly blame the JVM most
times. Also - most of the hatred I see in the industry is a lack of
understanding around the JVM. For me, I'm an old Perl programmer and
certainly making the transition over the last ~17 years was one I
fought against, more because of my own stubbornness I guess. But once
I started to actually study and learn about the tooling for JVM and
accepts its place in the application stack instead of just hating it,
my attitude began to change.

For example, I could never have understood memory usage in PuppetDB if
it was written in Ruby - never is probably too strong - but its hard
in Ruby to do this ... I have tried and it kind of sucks. But hey,
with clojure/jvm, I can use YourKit which gives me an almost
ludicrously simple way of seeing the memory flow. Point in case, we
used to use the urlencoded way of doing POST submissions for commands,
but when I analyzed command submission in Yourkit (live service mind
you) I quickly realized we had 2 objects, the encoded one and the
unencoded. Just think about that for a second - 2 copies of a very
large catalog in memory ... very wasteful :-). So yeah, we stopped
encoding, it wasn't needed anyway and halved our memory consumption
for command submissions and removed that processing need completely -
again thanks to JVM tooling. This work took at best a day or two,
including the patch I believe.

Same again for queries, we switched to streaming for this same reason
... versus loading up the answer and serving it all in one go ... we
now open a cursor on the db, and as answers come back we stream it via
HTTP. The Java core libraries and Clojure in particular are actually
very very good at doing streaming ... and on our platform streaming
becomes critical to reducing memory usage.

For me, I would only see the Erlang runtime coming close to this as a
serious contender (and perhaps the .Net framework/CLR might have
something here, but this isn't my area of expertise), and while the
tooling there for Erlang is pretty awesome, its not as evolved as the
JVM stuff. Don't get me wrong, I love Erlang too :-).

> (2) I've gotta say, I'll really miss dropping log statements directly in the
> puppet source when something seems wonky (and not having to compile
> something).

Our answer to this for Clojure is usually a combination of NREPL and
(log/spy <original item you want to see>) from the
clojure.tools.logging library or #spy/d statements from the spy scope
library. Works great, and can wrap just about any variable as a nice
piece of magic to drop "debug" statements.

The nice thing here that we didn't have in Ruby is that NREPL allows
changes to a running service. So no need to stop/start the service to
see your debug lines.

I do this quite often for PuppetDB while developing, that is I have a
running PuppetDB instance and the PDB source code open in Emacs (with
the cider plugin for nrepl support already bootstrapped of course) ...
I modify the code ... save it ... hit Ctrl-C Ctrl-K ... and I see the
debug lines start to appear in the log. Its a far more rapid workflow
(to be clear: Emacs is only my choice, I believe there is NREPL
support in vim, eclipse, intellij and various other editors as well).

Oh yeah, and this can be done on real running systems also it doesn't
just have to be a dev workflow, you just need to have the NREPL port
exposed in your PDB config.ini:
https://docs.puppetlabs.com/puppetdb/2.2/configure.html#repl-settings.

ken.

[Ramin K]

More information along these lines, highlighting ease of use and tools
for users to see their catalogs, will go along way towards soothing us
touchy sysadmins. The performance gains while nice don't have the appeal
of better troubleshooting. I'm happy to learn yet another stack, but I'd
like to be sure I'm getting some thing more than the status quo.

The information around tuning Passenger/Puppet explicitly provided by
Puppet Labs was mostly crap. It would be extremely useful for everyone
if there were 4-8 pages of serious and indepth docs specifically about
running puppet_server on the JVM. If that doesn't happen, you'll be
fighting the supposed poor performance of every un-tuned puppet_server
installation for years.

Ramin

[Kylo Ginsberg]

On Fri, Sep 26, 2014 at 4:42 PM, Felix Frank <Felix...@alumni.tu-berlin.de> wrote:

On 09/26/2014 11:12 PM, Rob Reynolds wrote:

Felix, if your "what?" is about Java (the language), that was a mistake. JVM on the server side, generally written in Clojure, is the direction things are heading. C++ on the client side. Ruby is still sticking around in order to support extensions. That said, some things we'll need to discuss and figure out what the best way to move forward is. Luke wrote up a good post about some of these changes at http://puppetlabs.com/blog/evolving-puppet-for-next-ten-years

I'm good with JVM, but C++ feels like a backward step in many regards. My judgment here may be clouded by reading too many blogpost of them naysayers.

C++ should be a forward step from a performance/footprint perspective. I'm guessing your backward step is from an ease of devel/debug perspective for core? But say more so the concern is clear.

Keep in mind that this would just be C++ for the client *core*. Puppet very much needs to continue to support its existing Ruby API for extensions (e.g. type and providers, custom facts). And yes this implies that the API needs to be *defined* better than it is today (which is undoubtedly going to take some collective rolling up of sleeves).

And further, I'd really like to see non-Ruby scripting languages enabled to participate as first-class citizens for the extension points - this (coupled with better definition of core APIs) would really make the on-ramp for new puppet users much lower friction.

Kylo

-- 

Kylo Ginsberg | ky...@puppetlabs.com | irc: kylo | twitter: @kylog

[Ken Barber]

> More information along these lines, highlighting ease of use and
> tools for users to see their catalogs, will go along way towards soothing us
> touchy sysadmins.

Totally understand, I was a very touchy admin myself before working at
Puppet Labs and when the tools let you down it can be frustrating.
Like chair-throwing frustrating sometimes :-). This is why I switched
to be a full time dev for PL, I wanted to make the experience better
in my own small way.

> The performance gains while nice don't have the appeal of
> better troubleshooting. I'm happy to learn yet another stack, but I'd like
> to be sure I'm getting some thing more than the status quo.

Absolutely, and this is where I think we wouldn't have it any other
way. I can think of at least a few things I hope to gain now the
Puppet platform is changing, one of them at least being that I'd like
to start considering fallback caches and queues on the master for
PuppetDB delivery (ie. a change to the puppedb-terminus application -
this was a little harder and less clean in a Ruby/Passenger world), so
that when a remote PDB instance is down the message is not lost, but
queued up for delivery later.

Not to mention the opening up of monitoring facilities via JMX and
such, which should expose a fair amount of metrics just on its own,
but allows us to poke holes into our application expose to users what
things are doing for monitoring/alerting purposes. In PDB we have a
great HTTP/JMX bridge for those who don't want to use JMX itself also
(a consideration for admins btw - as pure JMX isn't always desirable),
hopefully we'll port that over to our other services in time (we'll
make it a trapperkeeper plugin most probably - so all our applications
get it) but for now JMX is there and able to be used right now.

We hope to hear more about where people want to go with this tooling
as well, so please let us know where we are going wrong. Its very easy
to create an insular application that doesn't expose enough, but
another thing to create a great tooled eco-system.

> The information around tuning Passenger/Puppet explicitly provided
> by Puppet Labs was mostly crap.

Indeed, it was a bit of a black art because of this. It wasn't until
later that Passenger even added the ability to reasonably introspect
what was going on in Passenger.

> It would be extremely useful for everyone if
> there were 4-8 pages of serious and indepth docs specifically about running
> puppet_server on the JVM. If that doesn't happen, you'll be fighting the
> supposed poor performance of every un-tuned puppet_server installation for
> years.

Sounds like something ticket-worthy to mention. We already have some
of this for PuppetDB, a lot of it is similar for this platform as
well. I'm pretty sure this will become a hot topic, so I doubt it will
be left alone. I expect the new puppet-server to incur more traffic
than PuppetDB for example, so they'll probably see issues we have not.

Not only that, as you see issues you should be bringing them up in
these forums - and then we can discuss and feed that back into the
docs as we go (as you know most of our docs are user-contributable as
well, to make it easier). We can do as much benchmarking as we like in
the lab, but the real world is the only true learning ground. I've
been following Puppet for gosh, 7-8 years or something now, in the
forums and such - and one thing I've always enjoyed is how well we
taught each other.

At the very least I can tell you the web server is Jetty, and will
have tuning similar to PuppetDB. It has all the predominant tuning
items one would expect, like # of threads, but hopefully the reduction
in moving parts should actually help. Not only that, I've recently
patched the trapperkeeper plugin for Jetty so it exposes its JMX
monitoring capabilities:

https://github.com/puppetlabs/trapperkeeper-webserver-jetty9/commit/cb727a4731bc7e2df7151c93a1b9f91461823a91

Which I really wanted for PuppetDB, but the beauty of trapperkeeper is
that we all get it now. This means you get all kinds of introspection
into what the web server is doing, much like you would with Apache,
but in a potentially more curated/monitoring prepared way (I'm not
trying to upset an Apache fans here, I love Apache as well, but we
couldn't easily embed it :-).

ken.

[Ken Barber]

> And further, I'd really like to see non-Ruby scripting languages enabled to
> participate as first-class citizens for the extension points - this (coupled
> with better definition of core APIs) would really make the on-ramp for new
> puppet users much lower friction.

Python support would be lovely.

ken.

[Ramin K]

On 9/29/14 2:09 AM, Ken Barber wrote:
>
>> The information around tuning Passenger/Puppet explicitly provided
>> by Puppet Labs was mostly crap.
>
> Indeed, it was a bit of a black art because of this. It wasn't until
> later that Passenger even added the ability to reasonably introspect
> what was going on in Passenger.
>
>> It would be extremely useful for everyone if
>> there were 4-8 pages of serious and indepth docs specifically about running
>> puppet_server on the JVM. If that doesn't happen, you'll be fighting the
>> supposed poor performance of every un-tuned puppet_server installation for
>> years.
>
> Sounds like something ticket-worthy to mention. We already have some
> of this for PuppetDB, a lot of it is similar for this platform as
> well. I'm pretty sure this will become a hot topic, so I doubt it will
> be left alone. I expect the new puppet-server to incur more traffic
> than PuppetDB for example, so they'll probably see issues we have not.

I can take a shot at it. Where's the best place to put it? Or if you
want to file it, I'll update it. My list at the moment looks like
following if anyone else wants to chime in.

Tuning
- puppet_server specific tuning for JVM 7/8/whatever.
- Call out the top five terrible "go fast options" that are commonly
found in blog posts.
- Discussion of puppet_server bottlenecks. CPU. It's always CPU.

Operating
- The effects of of restarting, hard and graceful
- Discussion/support of log levels, where to log, etc.
- Haproxy in front, ssl termination, and other HA or throughput
enhancing techniques.

Monitoring
- status routes (is the puppet_server up and mostly working)
- dependency routes (can puppet_server hit puppet_db, etc)

Statistics
- some explanation of jmx
- interesting keys. I'm not familiar, but I imagine it's like a MIB.
"We have an snmp MIB." "Great which keys are the ones we should
definitely monitor?" "No idea." "Okay if I create a new load balancer
where does that show up?" "No idea."

Ramin

[Trevor Vaughan]

Just to add on things that I'd like to see:

Tuning

- When it's not CPU, it's RAM. What to do when you have massive catalogs (maybe not an issue?).

Operating

- Proper log rotation and maintenance

Security

- What do I need to let through?

- What do I need to lock down?

- How do I easily split off my CA from my Master?

- What to do in locations that don't allow JMX (how to disable it)

- What this setting set does:
  certificate-authority: {
    certificate-status: {
        client-whitelist: []
    }
  }

Right now, these are just things that I need to figure out when I have time. If someone knows the answers, that would make things much easier.

Thanks,

Trevor

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/5429C6B8.2040106%40badapple.net.

For more options, visit https://groups.google.com/d/optout.

[Felix Frank]

On 09/29/2014 05:19 AM, Kylo Ginsberg wrote:

I'm good with JVM, but C++ feels like a backward step in many regards. My judgment here may be clouded by reading too many blogpost of them naysayers.

C++ should be a forward step from a performance/footprint perspective. I'm guessing your backward step is from an ease of devel/debug perspective for core? But say more so the concern is clear.

Keep in mind that this would just be C++ for the client *core*. Puppet very much needs to continue to support its existing Ruby API for extensions (e.g. type and providers, custom facts). And yes this implies that the API needs to be *defined* better than it is today (which is undoubtedly going to take some collective rolling up of sleeves).

Hi,

well I did read some arguments for C++ being quite flawed from a language design perspective. I found those very compelling, but sadly cannot find the link.

That being said, I'm sure it can't hurt to get some practice with it again. Made my peace with the prospect and by now even kind of looking forward to seeing what you guys whip up.

Cheers,
Felix

[jcbollinger]

On Tuesday, September 30, 2014 8:29:42 AM UTC-5, Felix.Frank wrote:

On 09/29/2014 05:19 AM, Kylo Ginsberg wrote:

I'm good with JVM, but C++ feels like a backward step in many regards. My judgment here may be clouded by reading too many blogpost of them naysayers.

C++ should be a forward step from a performance/footprint perspective. I'm guessing your backward step is from an ease of devel/debug perspective for core? But say more so the concern is clear.

Keep in mind that this would just be C++ for the client *core*. Puppet very much needs to continue to support its existing Ruby API for extensions (e.g. type and providers, custom facts). And yes this implies that the API needs to be *defined* better than it is today (which is undoubtedly going to take some collective rolling up of sleeves).

Hi,

well I did read some arguments for C++ being quite flawed from a language design perspective. I found those very compelling, but sadly cannot find the link.

I'm fine with C (which I use a lot), and I love Java, JVM and all.  I'm satisfied with a wide variety of scripting languages.  Although it's not chic, I even like Fortran for certain uses.  But I hate C++.

As for a link, the C++ Frequently Questioned Answers helped me crystallize some of my previously less-focused distaste for C++.

John

[Jeffrey Watts]

On Wed, Oct 1, 2014 at 8:13 AM, jcbollinger <John.Bo...@stjude.org> wrote:

I'm fine with C (which I use a lot), and I love Java, JVM and all.  I'm satisfied with a wide variety of scripting languages.  Although it's not chic, I even like Fortran for certain uses.  But I hate C++.

As for a link, the C++ Frequently Questioned Answers helped me crystallize some of my previously less-focused distaste for C++.

Whenever I hear programmers argue about the merits and failings of different programming languages I remember this old joke:

"At the beginning of the week, we sealed ten BSD programmers into a computer room with a single distribution of BSD Unix.  Upon opening the room after seven days, we found all ten programmers dead, clutching each others' throats, and thirteen new flavors of BSD."

There's a reason there are dozens (hundreds?) of general purpose programming languages - people can't agree on which is the best tool for a job.  Using C++ as the underlying language won't be the end of the world.  :)

Jeffrey.

[Felix Frank]

On 10/01/2014 03:35 PM, Jeffrey Watts wrote:

On Wed, Oct 1, 2014 at 8:13 AM, jcbollinger <John.Bo...@stjude.org> wrote:
As for a link, the C++ Frequently Questioned Answers helped me crystallize some of my previously less-focused distaste for C++.

There's a reason there are dozens (hundreds?) of general purpose programming languages - people can't agree on which is the best tool for a job.  Using C++ as the underlying language won't be the end of the world.  :)

No, it won't, and let's not go to war about it.

Still, as long as the dev team is in the process of choosing the way forward, let the record show that there is two community voices (thanks for that link John, that was exactly what I had skimmed back in the day) that recommend considering alternatives.

Cheers,
Felix

[jcbollinger]

On Wednesday, October 1, 2014 9:34:25 AM UTC-5, Felix.Frank wrote:

On 10/01/2014 03:35 PM, Jeffrey Watts wrote:

On Wed, Oct 1, 2014 at 8:13 AM, jcbollinger <John.Bo...@stjude.org> wrote:
As for a link, the C++ Frequently Questioned Answers helped me crystallize some of my previously less-focused distaste for C++.

There's a reason there are dozens (hundreds?) of general purpose programming languages - people can't agree on which is the best tool for a job.  Using C++ as the underlying language won't be the end of the world.  :)

No, it won't, and let's not go to war about it.

Agreed, and I have no intention of warring.  I hesitated even to post my previous message -- I declined to do following Felix's first post to the thread -- but as long as Felix is raising the flag, I decided should add my voice.

It will not be the end of the world if C++ is used in Puppet.  It would certainly bring advantages along with its disadvantages, and it's PL that has to weigh those.  I just hope they will indeed give careful consideration to the question, for I do not think it is in any way a clear or easy decision.

John

[jcbollinger]

On Thursday, September 25, 2014 3:33:00 PM UTC-5, Andy Parker wrote:
 

The language will remain in ruby for a while yet (the puppet-server uses JRuby to run the puppet ruby code), and even after the interpreter gets reimplemented in a faster language, ruby will be available for extensions. It would simply be too much of a change to drop ruby and drop all of the custom code and extensions that everyone has worked so hard on.

I feel compelled to point out that if the "faster language" happens to be C++, then you will need a plug-in interface in some different language (presumably Ruby, but straight C would be more typical).  C++ APIs are sensitive to compiler and compile options used, and they provide essentially no compile-time encapsulation of API classes' private members, so it would not be wise to suppose that you will ever be able to directly support unwrapped, third-party, C++ plugins.  Especially with the current fast pace of Puppet development.

 

What *might* be happening very soon is dropping rack and webrick support and moving entirely to the puppet-server. It provides a much simpler setup, better performance, and more controlled environment for us to target.

"Puppet apply" will be sticking around. The exact way in which it will work isn't completely clear yet. There are several possibilities from requiring the jvm on all nodes that run puppet apply to changing apply to use a "compile server" to actually run manifests or possibly reimplementing the interpreter in a non-jvm language and using JNI to make it available on the puppet-server. My currently leaning is to implement the language in a jvm language and requiring that a masterless setup has the jvm on the node.

Although I have no problem with the JVM in general, I am not at all keen to have JVM installation as a pre-requisite for bootstrapping Puppet on nodes.  I am supposing that the plan is to not require that for the agent, but I don't think it's wise for "puppet apply", either.  Moreover, there are machines and organizations for which any client-side dependency on the JVM would be a complete show-stopper, and that would be a big concern for me if I were making this decision.

Furthermore, it's unrealistic to suppose that users of masterless Puppet would accept a solution involving an external "compile server" as a bona fide substitute for the current implementation of "puppet apply".  That's barely different from running the agent.  The whole point of "puppet apply" is that you don't rely on any other machine to apply your configuration.

You could try to maintain separate, parallel catalog builder implementations, but I promise you that that would get old (and expensive) very quickly.

So what catalog builder alternatives are left?  Let's see... you need a light(ish)-weight language and runtime environment with few detractors, yet you also want to be able to run in the JVM.  For community acceptance, you need to support plugins implemented in Ruby.  How about using ...*drumroll*... Ruby?  I hear PL has one or two Ruby programmers already ;-)  Is JRuby so different from (C)Ruby as to preclude writing code that works on both?


John

[Felix Frank]

On 10/03/2014 03:50 PM, jcbollinger wrote:

I feel compelled to point out that if the "faster language" happens to be C++, then you will need a plug-in interface in some different language (presumably Ruby, but straight C would be more typical).  C++ APIs are sensitive to compiler and compile options used, and they provide essentially no compile-time encapsulation of API classes' private members, so it would not be wise to suppose that you will ever be able to directly support unwrapped, third-party, C++ plugins.  Especially with the current fast pace of Puppet development.

I somehow just realized yesterday that CFacter is a thing already. So I guess we're already somewhere down this road.

Just cloned the repo - doesn't seem to build on Debian sid with Ruby 2.1.0. Will likely take some fiddling.

I may rummage around in the code to see if I can get a feeling for what's on the horizon.

Cheers,
Felix