Newbie got cart before horse - how to make Puppet files match reality

[Karen Ellrick]

I apologize in advance if this question is too general - I'm happy to share specifics as needed. I'll attach a couple key files as a start on that.

I used the online tool PuPHPet to create a CentOS box that was close to meeting my needs, but because of that, my Puppet files are apparently more complex than they need to be (all examples I have seen in tutorials are far simpler). Then, because I was completely new to all this, I made changes to my actual server (by SSHing into my VM and rummaging around) while having absolutely no idea how to reflect those changes in the Puppet files.

Since then I have procrastinated on climbing this learning curve, but recently I upgraded Vagrant on the host, and now "vagrant global-status" says, "There are no active Vagrant environments on this computer! Or, you haven't destroyed and recreated Vagrant environments that were started with an older version of Vagrant." Hmph! I know I will lose my changes if I destroy and recreate, so I really need to figure out how to clean up my Puppet files.

The directory structure looks like this:

│  Vagrantfile
│ 
├─.vagrant
│  └─machines
│      └─default
│          └─virtualbox
│                  action_provision
│                  action_set_name
│                  id
│                  private_key
│                  synced_folders
│                 
└─puphpet
    │  config.yaml
    │ 
    ├─files
    │  ├─dot
    │  │      .bash_aliases
    │  │      .vimrc
    │  │     
    │  ├─exec-always
    │  │      empty
    │  │     
    │  └─exec-once
    │          empty
    │         
    ├─puppet
    │      hiera.yaml
    │      manifest.pp
    │      Puppetfile
    │     
    └─shell
            execute-files.sh
            initial-setup.sh
            install_phalcon-devtools.sh
            install_phalcon.sh
            librarian-puppet-vagrant.sh
            os-detect.sh
            self-promotion.txt
            update-puppet.sh

puphpet/puppet/manifest.pp alone is 920 lines! I'm pretty sure most of that, as well as many of the other files, is unneeded code (I keep seeing things like "if $::osfamily == 'debian'...", for example). But the syntax is new to me, so I'm afraid to start slashing chunks of it.

A key piece of work I remember doing was on the nginx configuration, so I'll throw it out as a concrete example. A tree of the VM's /etc/nginx/ is as follows, with sizes and dates (as you can probably guess, I made the box in February last year and then fiddled with it off and on over the next couple months):

├── [       4096 Mar  6  2014]  conf.d
│   ├── [        888 Feb 13  2014]  default.conf
│   ├── [        427 Feb 11  2014]  example_ssl.conf
│   ├── [        415 Feb 13  2014]  proxy.conf
│   └── [       1469 Apr 26  2014]  vhost_autogen.conf
├── [       4096 Feb 13  2014]  conf.mail.d
├── [        963 Mar  6  2014]  fastcgi_params
├── [       4096 Mar  6  2014]  includes.d
│   └── [       1127 Mar 25  2014]  fastcgi_common
├── [       2837 Feb 11  2014]  koi-utf
├── [       2223 Feb 11  2014]  koi-win
├── [       3463 Feb 11  2014]  mime.types
├── [        565 Apr 19  2014]  nginx.conf
├── [        596 Feb 11  2014]  scgi_params
├── [        623 Feb 11  2014]  uwsgi_params
└── [       3610 Feb 11  2014]  win-utf

Most of the work I remember doing is in vhost_autogen.conf, but the string "autogen" doesn't appear in manifest.pp, so I don't know what code created that file. Perhaps I'm hopelessly ignorant, but can someone point me in the right direction? It's too bad that it isn't possible to create a Puppet manifest from an existing server... ;-)

[jcbollinger]

On Sunday, January 25, 2015 at 4:52:08 PM UTC-6, Karen Ellrick wrote:

I apologize in advance if this question is too general - I'm happy to share specifics as needed. I'll attach a couple key files as a start on that.

It's not a major problem that you don't know off the bat what details are important.  It is potentially a problem, however, that your question is more about PuPHPet and perhaps Vagrant than about Puppet.  There are users of both those tools who frequent this forum, so you may be in luck, but I hope you will understand that your question is somewhat tangential to the main topic of this group.

 

I used the online tool PuPHPet to create a CentOS box that was close to meeting my needs, but because of that, my Puppet files are apparently more complex than they need to be (all examples I have seen in tutorials are far simpler).

Yes and no.  The code generated by PuPHPet is more complex than would normally be found in a single Puppet manifest file, but that appears largely to be because it comprises code that would normally be divided among multiple separate files. Furthermore, most of those separate files would probably come from various downloadable third-party modules.  So you're right in thinking that you would not normally write Puppet code anything like that complex, especially when you're just getting started with Puppet.

 

Then, because I was completely new to all this, I made changes to my actual server (by SSHing into my VM and rummaging around) while having absolutely no idea how to reflect those changes in the Puppet files.

Not being a PuPHPet initiate myself, I'm uncertain whether that's an important issue.  In particular, I am uncertain whether Puppet has any role in managing a PuPHPet-generated server configuration after it is initially created, or in initializing new VMs.  If it does, however, then then I suggest taking a clue from the (maddeningly sparse) documentation:

I have completely rewritten PuPHPet to take advantage of a built-in configuration tool for Puppet called Hiera. Simply look inside your downloaded folder and open puppet/hieradata/common.yaml. This is the magical file that controls everything!

That appears to be referring to a data file for Puppet's 'Hiera' component, which would make sense.  We can certainly help you in a generic sense with Hiera data, but it remains to be seen how well we can help you match up the customizations you want with the specific Hiera keys and appropriate structures for data associated with those keys.  The latter are not a function of Puppet itself.

 

To the extent that various configuration files were created and are managed by Puppet, I would expect to see corresponding ERB template files, but I don't.  I also don't see a wide variety of other files associated with Puppet classes and defined types that are referenced by your 'manifest.pp', but not defined there.  It may be that these are downloaded to new VMs as part of the process of applying their initial configuration.

 

puphpet/puppet/manifest.pp alone is 920 lines! I'm pretty sure most of that, as well as many of the other files, is unneeded code (I keep seeing things like "if $::osfamily == 'debian'...", for example). But the syntax is new to me, so I'm afraid to start slashing chunks of it.

You are right that the manifest is more general than you need it to be, but it appears to be a generated file.  It is not intended to be maintained manually.  Indeed, I suppose that the idea is for you to use PuPHPet to generate an altogether new one if needed.  In any event, it doesn't look like trimming it back would help you solve your immediate problem.  (Nevertheless, Puppet's own docs, covering its manifest file language and many other things, are far more comprehensive than PuPHPet's.  The basics of the language are not hard to learn, if you're interested.)

 

A key piece of work I remember doing was on the nginx configuration, so I'll throw it out as a concrete example. A tree of the VM's /etc/nginx/ is as follows, with sizes and dates (as you can probably guess, I made the box in February last year and then fiddled with it off and on over the next couple months):

├── [       4096 Mar  6  2014]  conf.d
│   ├── [        888 Feb 13  2014]  default.conf
│   ├── [        427 Feb 11  2014]  example_ssl.conf
│   ├── [        415 Feb 13  2014]  proxy.conf
│   └── [       1469 Apr 26  2014]  vhost_autogen.conf
├── [       4096 Feb 13  2014]  conf.mail.d
├── [        963 Mar  6  2014]  fastcgi_params
├── [       4096 Mar  6  2014]  includes.d
│   └── [       1127 Mar 25  2014]  fastcgi_common
├── [       2837 Feb 11  2014]  koi-utf
├── [       2223 Feb 11  2014]  koi-win
├── [       3463 Feb 11  2014]  mime.types
├── [        565 Apr 19  2014]  nginx.conf
├── [        596 Feb 11  2014]  scgi_params
├── [        623 Feb 11  2014]  uwsgi_params
└── [       3610 Feb 11  2014]  win-utf

Most of the work I remember doing is in vhost_autogen.conf, but the string "autogen" doesn't appear in manifest.pp, so I don't know what code created that file. Perhaps I'm hopelessly ignorant, but can someone point me in the right direction? It's too bad that it isn't possible to create a Puppet manifest from an existing server... ;-)

As I already noted, one of the things that appears to be causing confusion is that you're not looking at all the relevant files.  There is a veritable host of relevant Puppet manifest files that do not appear in your listing, plus the key Hiera data file that PuPHPet's online help references.  The default location for such files would be either /etc/puppet or ~someuser/.puppet, depending on how Puppet is being run.  Perhaps you could 'find' the 'hieradata' directory, the 'common.yaml' file, or a 'puppet.conf' file.

I would certainly back up your VM -- at least the parts containing configuration you don't want to lose.  The task before you is to figure out how to record your customizations in the form of Hiera data.  Moreover, you need to be aware that although you can probably extract the needed data and present it to Puppet, the config files it generates when you have successfully done so are likely to be equivalent rather than identical.  I'm afraid that will complicate the verification process.

Good luck, and feel free to follow up with other questions you may have.


John

[Karen Ellrick]

John,

Thanks for a thorough and helpful reply.

In particular, I am uncertain whether Puppet has any role in managing a PuPHPet-generated server configuration after it is initially created, or in initializing new VMs.

My understanding (which could be flawed, of course) is that PuPHPet is merely an online tool designed to quickly build a functioning set of Puppet files tailored for PHP developers, based on a finite set of selections (OS, server packages, etc.). It's a time saver for getting started - think "wizard". After that, the developer would customize the resulting Puppet code as needed - PuPHPet's interface is not even close to being detailed enough to allow every contingency (if it was, it would take a week to go through the wizard to get anything done!). I'm pretty sure it is not intended to be used over and over for minor changes to the same box.

(docs) "I have completely rewritten PuPHPet to take advantage of a built-in configuration tool for Puppet called Hiera. Simply look inside your downloaded folder and open puppet/hieradata/common.yaml. This is the magical file that controls everything!"

Based on the fact that my downloaded folder doesn't have a hieradata directory and my hiera.yaml file has only six lines of code, I think that Juan did that rewrite after the one and only time I used PuPHPet, which was almost a year ago. Here is the entire contents of hiera.yaml:

---
:backends: yaml
:yaml:
    :datadir: '/vagrant/puphpet'
:hierarchy:
    - config
:logger: console

Based on my cursory look at the Hiera docs, that looks like something Juan was just starting to use, not yet pivotal to his setup. I don't have any urge to use Hiera unless there is a huge benefit - it's enough for me to learn one tool.
 

You are right that the manifest is more general than you need it to be, but it appears to be a generated file.  It is not intended to be maintained manually.  Indeed, I suppose that the idea is for you to use PuPHPet to generate an altogether new one if needed.

As I said before, I think PuPHPet is just a "wizard" - yes, the manifest is generated, but manually maintaining it from then on is the normal workflow, I assume. In any case, I have no choice - PuPHPet cannot give me everything I need. If I should start over with a simpler, manually written Puppet setup, that's fine.
 

In any event, it doesn't look like trimming it back would help you solve your immediate problem.

True, but it would make the file easier to look through. And my server would have less bloat, also - earlier today I looked through manifest.pp and noticed modules it installs that I don't think I need (or want) on my server at all, so I started my cleanup by commenting out those sections. (As far as I remember, PuPHPet's interface didn't ask me if I wanted each of those things - it just assumed.) Once I get as far as I can with reflecting my desired server changes in Puppet and bravely recreate the server, if it still does what I need, I'll remove the commented code.
 

  The basics of the language are not hard to learn, if you're interested.

Yup, that's the idea of posting here. I do want to learn, but I'm the impatient type - I tend to start learning a new tool by doing something I actually need to do, not just experimenting with a tutorial.
 

As I already noted, one of the things that appears to be causing confusion is that you're not looking at all the relevant files.  There is a veritable host of relevant Puppet manifest files that do not appear in your listing, plus the key Hiera data file that PuPHPet's online help references.  The default location for such files would be either /etc/puppet or ~someuser/.puppet, depending on how Puppet is being run.  Perhaps you could 'find' the 'hieradata' directory, the 'common.yaml' file, or a 'puppet.conf' file.

Oh! I didn't realize that there was more Puppet stuff on the VM itself! Sorry - I was just looking in the files in the ZIP file PuPHPet generated (which is on the host). It looks like the stuff you're looking for (at least some of it) is in /etc/puppet/modules/puphpet/ - here is the tree:

puphpet/
├── files
│   └── xdebug_cli_alias.erb
├── LICENSE
├── manifests
│   ├── adminer.pp
│   ├── apache
│   │   ├── modpagespeed.pp
│   │   └── modspdy.pp
│   ├── ini.pp
│   ├── mariadb.pp
│   ├── nginx.pp
│   ├── params.pp
│   └── xdebug.pp
├── README.md
└── templates
    ├── apache
    │   └── mod
    │       └── spdy
    │           ├── php-wrapper.erb
    │           └── spdy_conf.erb
    └── nginx
        └── default_conf.erb

Apparently these files get created by Vagrant after the server is created and fired up for the first time. There's clearly more to how Puppet runs than I realized. Anyway, the /etc/puppet/modules directory has other modules, too - at first glance it looks like there is a directory there for every section of the original manifest.pp:

apt/         concat/  iptables/     mysql/  php/      rabbitmq/  supervisord/
beanstalkd/  drush/   mailcatcher/  nginx/  puphpet/  staging/   vcsrepo/
composer/    git/     mongodb/      ntp/    puppi/    stdlib/    yum/

And yes, there is also a /etc/puppet/puppet.conf file. But there is no common.yaml, and although Hiera itself is installed on the server, there is no hieradata.

The task before you is to figure out how to record your customizations in the form of Hiera data.

Does it need to be Hiera, or is it optionally Puppet itself?
 

Moreover, you need to be aware that although you can probably extract the needed data and present it to Puppet, the config files it generates when you have successfully done so are likely to be equivalent rather than identical.  I'm afraid that will complicate the verification process.

As long as my stuff runs, I'm happy. What I have done so far is easily testable. I'm new to nginx also, so I may not have made the cleanest config files in the world, anyway. After this, I'll keep the horse in front of the cart, making changes in Puppet and rebuilding, rather than messing with the server directly. It's a new paradigm, but I can learn.

Karen

[Karen Ellrick]

Addendum: I got a chance to talk to someone at a PHP user's group gathering with my computer in front of me, and he explained a lot of fundamentals I was missing. I am better informed now about the different roles of Vagrant and Puppet, and what PuPHPet was doing for me. My main configuration file for PuPHPet was not manifest.pp at all, but config.yaml - once I saw what was in there, things starting making more sense. And I discovered that PuPHPet has grown a lot since I used it a year ago - it still can't do everything I need, but I now understand why you saw it as having a more active ongoing role. Since it can accept my current config.yaml as a starting point so that I don't have to go through all the steps again, I'm going to have it make me another server (so that I get the newest, best version of the PuPHPet-generated code) and work from there. I may still end up with a simplified Puppet setup (sans PuPHPet) long term, but since I'm such a newbie, it's nice to be able to reference example code that a Puppet expert wrote, even if it's more complex than it needs to be.

Thanks again for your help.

[jcbollinger]

On Thursday, January 29, 2015 at 6:24:14 PM UTC-6, Karen Ellrick wrote:

Addendum: I got a chance to talk to someone at a PHP user's group gathering with my computer in front of me, and he explained a lot of fundamentals I was missing. I am better informed now about the different roles of Vagrant and Puppet, and what PuPHPet was doing for me. My main configuration file for PuPHPet was not manifest.pp at all, but config.yaml - once I saw what was in there, things starting making more sense. And I discovered that PuPHPet has grown a lot since I used it a year ago - it still can't do everything I need, but I now understand why you saw it as having a more active ongoing role. Since it can accept my current config.yaml as a starting point so that I don't have to go through all the steps again, I'm going to have it make me another server (so that I get the newest, best version of the PuPHPet-generated code) and work from there. I may still end up with a simplified Puppet setup (sans PuPHPet) long term, but since I'm such a newbie, it's nice to be able to reference example code that a Puppet expert wrote, even if it's more complex than it needs to be.

Thanks again for your help.

Thanks for the update, and I'm glad you're moving forward.  I don't feel like I actually rendered much real assistance, but you're welcome.  Feel free to come back around if you have more questions; maybe we'll be able to render better assistance.


John

[Juan Treminio]

Hi y'all!

I'm guessing it may have been a good idea to join this group a while ago.

jcbollinger - you're right in that PuPHPet has changed drastically since your original archive was created.

Previously, everything was throwing into a single manifest.pp - even for things you may not have chosen via the GUI.

However, everything is also controlled via the puphpet/puppet/config.yaml file using simple 0/1 flags, so anything you don't want isn't installed, but can be installed later by flipping the int.

My documentation is quite sparse, I'll admit, and I keep putting off updating it, so many things are unfortunately out of date. PRs are very welcome and highly appreciated, though!

In the latest puphpet archives you'll notice that the manifest.pp file was split out into several different files under a nodes directory. This makes it quite a bit easier to see what's going on after the archive is generated.

Apologies for any troubles you encountered - I'm trying my hardest to create a tool that's as simple to use as possible, but time constraints keep me from dedicating everything that I want to it :(

Hope your current VM is running smoothly, though!

Thanks,

Juan

[Karen Ellrick]

Hi, Juan! It's great to hear from you here. As you can see, John felt that my question was more of a PuPHPet question than a Puppet one. After a little more work on my own I then posted on PuPHPet Issues (https://github.com/puphpet/puphpet/issues/1352), and you told me that it was a Puppet question. I felt a little like a ping pong ball... :( So I'm glad the loop is closing a bit.

jcbollinger - you're right in that PuPHPet has changed drastically since your original archive was created.

It's not John, but me, that had the old archive. After creating a new one from the old config.yaml, PHP didn't work, requiring two changes (one you told me about, and the other someone else named Kevin who had the same problem and figured it out - https://github.com/puphpet/puphpet/issues/1364).
 

Hope your current VM is running smoothly, though!

I'm still stuck at the overall question I tried to ask here and on github, which is how to modify either the PuPHPet config.yaml or the Puppet files to handle things beyond PuPHPet's GUI. Naturally it would be good to include as much of it in config.yaml as possible so that it won't be lost if I go back to the PuPHPet GUI later, but so far I have only figured out a few other places to put particular things - see https://github.com/puphpet/puphpet/issues/1352#issuecomment-72766107 for a list of mods I have done so far, which are scattered in four places. I have not gotten any farther - I still don't know how to include nginx config changes and installation of things that don't have a yum repo, like LaTeX and specialty files I'll need. Help from Juan, John, or anyone else is appreciated.

[jcbollinger]

On Saturday, February 21, 2015 at 8:49:05 PM UTC-6, Karen Ellrick wrote:

Hi, Juan! It's great to hear from you here. As you can see, John felt that my question was more of a PuPHPet question than a Puppet one. After a little more work on my own I then posted on PuPHPet Issues (https://github.com/puphpet/puphpet/issues/1352), and you told me that it was a Puppet question. I felt a little like a ping pong ball... :( So I'm glad the loop is closing a bit.

Here's the thing: PuPHPet is fundamentally a code generator.  Based on your inputs, it generates Puppet DSL code and corresponding data.  By all accounts, it does a decent job of it, in the sense that the generated code and data do what they are supposed to do. From what I've seen, however, PuPHPet is like most code generators in generating output that is not well suited for subsequent hand editing.

It's understandable if the PuPHPet project does not support hand editing its output, but in that case, if PuPHPet doesn't get you all the way to where you need to go then that leaves you in a bit of a no-man's land.  I can talk to you about Puppet DSL in general, or about details of code that you wrote and mostly understand, but the analysis required to tell you specifically what to do to your machine-generated code would take more time and effort than I am prepared to donate.  As PuPHPet developer, Juan is in a better position to give such advice, or at least to tell you about how the generated code and data are organized, which would help the rest of us advise you about what changes to make.  It is in that sense that yours is a PuPHPet issue.

 

I'm still stuck at the overall question I tried to ask here and on github, which is how to modify either the PuPHPet config.yaml or the Puppet files to handle things beyond PuPHPet's GUI. Naturally it would be good to include as much of it in config.yaml as possible so that it won't be lost if I go back to the PuPHPet GUI later, but so far I have only figured out a few other places to put particular things - see https://github.com/puphpet/puphpet/issues/1352#issuecomment-72766107 for a list of mods I have done so far, which are scattered in four places. I have not gotten any farther - I still don't know how to include nginx config changes and installation of things that don't have a yum repo, like LaTeX and specialty files I'll need. Help from Juan, John, or anyone else is appreciated.

With regard to LaTeX specifically, there are packages in CentOS's standard repositories, so you shouldn't need to configure a custom repository for it.  If you in fact do need to manage software that's not available pre-packaged from any public repository, then there are multiple options.  A reasonably good one is to set up your own local repository (it's pretty easy).  How best to fit any of that into your existing code, and especially how to do it so the customizations can be carried forward automatically, is beyond what I'm prepared to advise you about.


John

[Karen Ellrick]

It's understandable if the PuPHPet project does not support hand editing its output...

If that was the case, Juan wouldn't have told me in https://github.com/puphpet/puphpet/issues/1352#issuecomment-72649784 that I need to work with Puppet to do what I want. He seems to be saying that the generated files can indeed be edited.

As PuPHPet developer, Juan is in a better position to give such advice, or at least to tell you about how the generated code and data are organized, which would help the rest of us advise you about what changes to make.

When I asked Juan for help understanding the structure of the generated code so that I'd know where to customize (https://github.com/puphpet/puphpet/issues/1352#issuecomment-72766107), there was no response. He apparently doesn't have time to get into specifics.
 

With regard to LaTeX specifically, there are packages in CentOS's standard repositories, so you shouldn't need to configure a custom repository for it.

The flavor I need is upTeX (https://www.ctan.org/pkg/uptex), which is a bit non-standard, but I see that there are more repos for texlive stuff than there were the last time I installed it, so I'll continue to investigate that. I thought I was clever when I found a Puppet module for texlive (https://github.com/andschwa/puppet-latex), but I apparently don't understand how to use it. In Puppetfile I put

mod 'andschwa/puppet-latex', :git => 'https://github.com/andschwa/puppet-latex.git'

but all I get is the module downloaded and unpacked, not installed. I also tried putting the contents of that github repo in puphpet/puppet/modules, but that didn't help.
 

If you in fact do need to manage software that's not available pre-packaged from any public repository...

The one I have been asking about all along is not really "software" per se, but nginx configuration file content. So far no one has suggested anything (even a link for more info) about how one gets that kind of thing into Puppet. I'm using a PHP framework called Phalcon that needs some special web server settings (http://docs.phalconphp.com/en/latest/reference/nginx.html#configuration-by-host) - I have it working on a VM, but I don't know how to get Puppet to create it the next time.

[jcbollinger]

I hear your frustration, Karen.  I'm sorry this is proving to be such a hassle for you.

On Wednesday, February 25, 2015 at 4:35:11 PM UTC-6, Karen Ellrick wrote:

It's understandable if the PuPHPet project does not support hand editing its output...

If that was the case, Juan wouldn't have told me in https://github.com/puphpet/puphpet/issues/1352#issuecomment-72649784 that I need to work with Puppet to do what I want. He seems to be saying that the generated files can indeed be edited.

I'm trying not to bad-mouth Juan, but I read that github thread already, and it sounds to me as if he's brushing you off.  Yes, in principle you can edit the Puppet code and data that PuPHPet outputs to achieve any result you want.  By that same principle, you can also write Puppet code from scratch to do what you want.  Saying so doesn't get you any closer.  I am honestly uncertain whether starting from the PuPHPet output is any advantage to you, given that it doesn't provide everything you want, and you are not already well versed in Puppet.

 

As PuPHPet developer, Juan is in a better position to give such advice, or at least to tell you about how the generated code and data are organized, which would help the rest of us advise you about what changes to make.

When I asked Juan for help understanding the structure of the generated code so that I'd know where to customize (https://github.com/puphpet/puphpet/issues/1352#issuecomment-72766107), there was no response. He apparently doesn't have time to get into specifics.

I'm sorry.  I don't have the much more time that I would need to analyze the code before I could spend time telling you what Juan already knows.  Perhaps this thread will yet be noticed by someone who has already gone through that, or by someone who has the time and inclination to do all that analysis.  If PuPHPet has some kind of organized user community then that might be a richer environment for finding such a person.

Otherwise, all I can think to recommend is to either (a) learn Puppet well enough to take the lead yourself on sorting it out (we can help in more narrowly-scoped ways), or (b) hire a consultant.  If you're willing to do the former, then I've always found Puppet's online docs to be quite good.  There are good books about Puppet out there, too, though Puppet moves fast enough that those fall out of date unfortunately swiftly.

 

The one I have been asking about all along is not really "software" per se, but nginx configuration file content. So far no one has suggested anything (even a link for more info) about how one gets that kind of thing into Puppet. I'm using a PHP framework called Phalcon that needs some special web server settings (http://docs.phalconphp.com/en/latest/reference/nginx.html#configuration-by-host) - I have it working on a VM, but I don't know how to get Puppet to create it the next time.

Usually, files will be managed in Puppet via 'file' resources.  Very likely, your generated Puppet code has that wrapped in a class; a typical name for a class for that purpose would be "nginx::config".  In a hand-turned Puppet manifest set you would expect to find a class of that name in file <confdir>/modules/nginx/manifests/config.pp.  Chances are good that such a File resource relies on an ERB template to define the file content; possibly you would find that in <confdir>/modules/nginx/templates/nginx.conf.erb.  Class and template likely one or both rely on data recorded in one or more YAML files somewhere central among the Puppet files.

I cannot say more without undertaking the analysis I described earlier, and you will have recognized that even what I did say is variously vague, uncertain, or both.  PuPHPet is in no way constrained to generate Puppet code that adheres to the same norms as hand-turned code, and in any case, most of the key bits involve details that I cannot hope to guess from here.  All this is multiplied by each thing you need to customize, and I am uncertain what it will take to carry any of it forward to future VMs.


John

[Juan Treminio]

On Wednesday, February 25, 2015 at 4:35:11 PM UTC-6, Karen Ellrick wrote:

It's understandable if the PuPHPet project does not support hand editing its output...

If that was the case, Juan wouldn't have told me in https://github.com/puphpet/puphpet/issues/1352#issuecomment-72649784 that I need to work with Puppet to do what I want. He seems to be saying that the generated files can indeed be edited.

Hi Karen, I've responded to you on puphpet's issue tracker.

 

As PuPHPet developer, Juan is in a better position to give such advice, or at least to tell you about how the generated code and data are organized, which would help the rest of us advise you about what changes to make.

When I asked Juan for help understanding the structure of the generated code so that I'd know where to customize (https://github.com/puphpet/puphpet/issues/1352#issuecomment-72766107), there was no response. He apparently doesn't have time to get into specifics.
 

I hope I addressed your questions. Note that everything I said is available within the gui itself, but it's probably my fault that you missed it - I am not a designer and UX isn't my forte. Most information is kind of haphazardly thrown in, and might be easy to miss.

 

With regard to LaTeX specifically, there are packages in CentOS's standard repositories, so you shouldn't need to configure a custom repository for it.

The flavor I need is upTeX (https://www.ctan.org/pkg/uptex), which is a bit non-standard, but I see that there are more repos for texlive stuff than there were the last time I installed it, so I'll continue to investigate that. I thought I was clever when I found a Puppet module for texlive (https://github.com/andschwa/puppet-latex), but I apparently don't understand how to use it. In Puppetfile I put

mod 'andschwa/puppet-latex', :git => 'https://github.com/andschwa/puppet-latex.git'

but all I get is the module downloaded and unpacked, not installed. I also tried putting the contents of that github repo in puphpet/puppet/modules, but that didn't help.
 

If you in fact do need to manage software that's not available pre-packaged from any public repository...

The one I have been asking about all along is not really "software" per se, but nginx configuration file content. So far no one has suggested anything (even a link for more info) about how one gets that kind of thing into Puppet. I'm using a PHP framework called Phalcon that needs some special web server settings (http://docs.phalconphp.com/en/latest/reference/nginx.html#configuration-by-host) - I have it working on a VM, but I don't know how to get Puppet to create it the next time.

If Phalcon needs special settings, you can either throw them into a bash file within the `puphpet/files/exec*` folders that will run on pre-defined `Vagrant` events (note that the Nginx puppet module I've chosen to use with PuPHPet *will* overwrite any non-standard configuration it did not add itself on every `vagrant provision`), or you can create Puppet code yourself to do what you want.

 Juan

[Juan Treminio]

On Thursday, February 26, 2015 at 10:03:35 AM UTC-6, jcbollinger wrote:

I hear your frustration, Karen.  I'm sorry this is proving to be such a hassle for you.

On Wednesday, February 25, 2015 at 4:35:11 PM UTC-6, Karen Ellrick wrote:

It's understandable if the PuPHPet project does not support hand editing its output...

If that was the case, Juan wouldn't have told me in https://github.com/puphpet/puphpet/issues/1352#issuecomment-72649784 that I need to work with Puppet to do what I want. He seems to be saying that the generated files can indeed be edited.

I'm trying not to bad-mouth Juan, but I read that github thread already, and it sounds to me as if he's brushing you off.  Yes, in principle you can edit the Puppet code and data that PuPHPet outputs to achieve any result you want.  By that same principle, you can also write Puppet code from scratch to do what you want.  Saying so doesn't get you any closer.  I am honestly uncertain whether starting from the PuPHPet output is any advantage to you, given that it doesn't provide everything you want, and you are not already well versed in Puppet.

Unfortunately this is a criticism I receive often. Maybe it's deserved. In my defense, however, I'd like to point out that when I first started puphpet I honestly thought only 10 people would ever find it useful. I had absolutely no idea it would ever gain as much popularity as it has seen. It's caught me completely off-guard and has turned me from a lucky-go-happy developer working on his small, personal project that just happens to be publicly-accessible, into the sole developer of a tool that thousands of developers are now using and depending on.

I don't *know* Puppet. I learned it as I went along. That's why the guys in Freenode's #puppet are supportive of me when I ask Puppet-specific questions, but generally look down on PuPHPet itself - because my Puppet code is amateur at worst, mediocre at best.

It has now reached a point where I am afraid of making changes to puphpet for fear of breaking something that I am unaware of. For PHP I am pretty good with writing unit tests and more or less follow TDD practices, but for Puppet I have no idea how to write unit tests. I know it would involve learning Ruby which I absolutely have no time (or much of an inclination, really) to do.

So the end result is that I'm stuck in a continuous rut where I will ignore puphpet for weeks, even months at a time, as my issue tracker grows steadily. Then I'll hit a day when my daytime work is done, my contracting work is caught up, my family is happy, and I miss working on it, and I will go through and work on ~50 issues in a 2-day span. Invariably I'll end up implementing some new feature and introducing 50 new bugs and the process begins again.

You'll notice in puphpet's commit history that the vast majority of commits have been from me. There's the odd PR here and there, of course, but mostly it's just been me chugging along on a project that I've enjoyed working on, but detracts from my day to day life.

So when you say I'm blowing someone off, I don't see it that way. I will provide support for my project. If someone wants to run something non-standard in their VM, I will offer encouragement and maybe some slight guidance, but that is all I am prepared to offer. I have enough on my plate to have to try and worry about things that aren't even in the project.

Karen, I understand my documentation is subpar. It's been on my to-do list as long as puphpet has existed. However, I do expect that when developers run into a brick wall that they tighten their belts and dive into the code, learn what's going on and implement their own solutions. It's completely open source, after all. I started puphpet having known not a thing about puppet a month previous (and it shows). The "hard" work has been done, now it's just trying to make sense of my madness :)

Thanks,

Juan

[Karen Ellrick]

One short question for Juan: How does PuPHPet install/run Puppet? I decided to try learning Puppet by doing a simple server without PuPHPet, but I keep running into instructions to install a Windows-based Puppet Agent service and/or a Ruby gem. As far as I can tell, PuPHPet did its magic successfully without any Windows "installation" and without even Ruby on the host. How did it do that?

[Juan Treminio]

Evening Karen,

I have Vagrant run shell scripts to setup the environment as required. They start here:

https://github.com/puphpet/puphpet/blob/master/archive/puphpet/vagrant/Vagrantfile-local#L197

The shell scripts are located here: https://github.com/puphpet/puphpet/tree/master/archive/puphpet/shell

[Karen Ellrick]

Wow, thanks for the quick reply. I looked through the shell scripts, but by the very fact that they are shell scripts, it's clear that they are only for running on the server (the VM). Does your code do anything on the host computer? The Puppet folks seem to think that I have to install the .msi file for a Puppet "agent" on my Windows PC, but your bundle manages to do everything without that step. I only seem to need VirtualBox and Vagrant, and you do the rest. See https://groups.google.com/forum/#!topic/puppet-users/h6cdvjgQnXg for my question and their answer.

[Juan Treminio]

> Does your code do anything on the host computer?

No, everything that happens, happens inside the VM. You only need Vagrant and your virtualizer of choice (Virtualbox, Vmware, Parallels).