"consume" files generated by other Packer Build processes

[qubitrenegade]

Hello,

I asked a similar question back in January, there wasn't any solution at that time, however, I'm hoping with the release of 1.0 there might be a way to support our workflow...

The short version is, we have a build pipeline in one datacenter that publishes OVAs to an Artifactory instance.  Then at various datacenters throughout the country, another user pulls that OVA down, manually registers it, and then can use terraform to spin up instances from that template.  Unfortunately, we can't just use the vsphere post processor from the packer that builds the OVA, as that build pipeline doesn't have reach into all the various data centers where the ova needs to be registered.

I don't know how else to describe it but, I'd like to run packer Post Processors without running any builders.

I tried this (I replaced the vsphere post-processor with the shasum for the purposes of testing, I figured if I can shasum I can upload):

{

    "post-processors": [

    {

      "type": "artifice",

      "files": "CentOS_7.4.1708-20171010.ova"

    },

    {

      "type": "checksum",

      "checksum_types": "sha256"

    }

  ]
}

But as you might imagine, this fails with an error that you must create at least one builder... well, drat.

so my next attempt was the null resource, but that requires the ability to login to a VM via ssh...  dealing with username and pass isn't something I really want to deal with... so I skipped onto the File resource, which looks promising! but...

{

  "builders": [

    {

      "type": "file",

      "target": "CentOS_7.4.1708-20171010.ova"

    }

  ],

  "post-processors": [

    {

      "type": "artifice",

      "files": "CentOS_7.4.1708-20171010.ova"

    },

    {

      "type": "checksum",

      "checksum_types": "sha256"

    }

  ]

}

This completely blanked my file!  Doh!  LOL, you can imagine my surprise!

I also tried:

export IMAGE=`cat CentOS_7.4.1708-20171010.ova

But this causes bash to have fits...

If there was just some way to say:

{

  "builders": [

    {

      "type": "file",
      "content": "./CentOS_7.4.1708-20171010.ova",

      "target": "CentOS_7.4.1708-20171010.ova"

    }

  ],

  "post-processors": [

    {

      "type": "artifice",

      "files": "CentOS_7.4.1708-20171010.ova"

    },

    {

      "type": "checksum",

      "checksum_types": "sha256"

    }

  ]

}

This would solve my problem.

However, this puts literally the string "./CentOS_7.4.1708-20171010.ova" in my file.

Am I barking up the wrong tree here?  I really don't want to go build a tool just to do the uploads especially since Packer already has the desired functionality... but I can't for the life of me figure out how to get packer to consume an existing artifact...

any thoughts?

Thanks,

Jon A

[qubitrenegade]

Correct link to previous post

[Rickard von Essen]

The situation is the same. It's not possible and it steps away a bit from the purpose of Packer.

Regarding the null builder it's main purpose is to speed up development of provisioners and their script, since you can quickly iterate over them without the need of launching and destroying instances. The purpose of the file builder is similar but for exercising a post-processor.

If you want this it's easy to fork the file builder to "import" builder, which needs to take file list or glob etc of the files to include and also set the correct builder id (mitchellh.vmware-esx or mitchellh.vmware).

// Rickard

On 10 October 2017 at 23:03, qubitrenegade <qubitr...@gmail.com> wrote:

Correct link to previous post

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/9fdc942a-8ca2-4fb1-9136-94ee30bd4275%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[qubitrenegade]

Hi Rickard,

Thanks for the reply.  I've not done any Go and it all looks greek to me, but I guess it's worth a shot.

For registering, are you talking about setting the "BuilderId" here: https://github.com/hashicorp/packer/blob/master/builder/file/builder.go#L18  to mitchellh.vmware?  And not something like "packer.import"?

Also, maybe I'm just confused, but are you suggesting that Packer should _NOT_ be used as a part of a CI/CD pipeline?  If so, is there another tool you can recommend?  As I've not found a tool that even compares to Packer...

Essentially what we have is one build pipeline that spits out artifacts to an artifact repository.  then we have another pipeline that consumes that artifact, validates the image for that environment, then someone has to manually come along and publish that artifact to their "production" environment.  At least in my mind, it seems like a lot of wasted compute cycles to have to reprovision the OS every time someone wants to consume the artifact.

In other CI pipelines, we have say a C program.  Dev checks out code, makes changes, commits it.  That commit kicks off a build which compiles the code and publishes it to an internal bintray.  From there other dev teams consume the binary from bintray in their jenkins pipelines, and when the change is decided it's acceptable it's pushed to "production".  It would be silly if every team had to recompile that code, just to test it with their application.  Especially considering compiling C programs can sometimes take hours.

From my perspective, Packer seems like the perfect tool to enable this CI/CD pattern that we're using... in my mind, if I have 10 teams consuming a packer artifact, it's a complete waste of time to force them all to rebuild a base image.

Again, if I'm barking up the wrong tree here and Packer isn't supposed to be used as part of CI/CD please set me straight... but we'd really hate to stop using it because Packer really is such a great tool that solves all of our problems save this one minor disconnect.

Thanks,

Jon A

On Tuesday, October 10, 2017 at 11:25:33 PM UTC-5, Rickard von Essen wrote:

The situation is the same. It's not possible and it steps away a bit from the purpose of Packer.

Regarding the null builder it's main purpose is to speed up development of provisioners and their script, since you can quickly iterate over them without the need of launching and destroying instances. The purpose of the file builder is similar but for exercising a post-processor.

If you want this it's easy to fork the file builder to "import" builder, which needs to take file list or glob etc of the files to include and also set the correct builder id (mitchellh.vmware-esx or mitchellh.vmware).

// Rickard

On 10 October 2017 at 23:03, qubitrenegade <qubitr...@gmail.com> wrote:

Correct link to previous post

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.

To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool...@googlegroups.com.

[Rickard von Essen]

For registering, are you talking about setting the "BuilderId" here: https://github.com/hashicorp/packer/blob/master/builder/file/builder.go#L18  to mitchellh.vmware?  And not something like "packer.import"?

Yes, the vsphere post-processor checks that the source artifact is one of mitchellh.vmware or mitchellh.vmware-vmx, see https://github.com/hashicorp/packer/blob/master/post-processor/vsphere/post-processor.go#L97 

Also, maybe I'm just confused, but are you suggesting that Packer should _NOT_ be used as a part of a CI/CD pipeline?  If so, is there another tool you can recommend?  As I've not found a tool that even compares to Packer...

Packer is a excellent _part_ of a CD pipeline. But it solves exactly one thing, creating repeatable (VM) images on different platforms from code. If you need to do some image lifecycle management or more advanced testing that is better handled in another pipeline step with other tools such as govomi/gcloud/aws-cli/serverspec etc.

In your case I think using govomi will nicely handle uploading of images to different DC's.

To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/7ce1a259-14c6-41e9-8c03-ce079e1eb479%40googlegroups.com.