Template validation error.

113 views
Skip to first unread message

TheTube

unread,
May 13, 2020, 9:24:39 PM5/13/20
to Packer


If I validate packer says:

Errors validating build 'vmware-iso'. 2 error(s) occurred:

* access_token must be set if vagrant_cloud_url has not been overriden
* Failed to verify authentication token: 401 Unauthorized

I don´t know what it means ¨access_token must be set.¨
this is the  cloud_token which is filled from the env variabele
VAGRANT_CLOUD_TOKEN

Any ideaś how i can do this better ?


create-cloudbox-redacted.sh
ubuntu-20.04-amd64-redacted.json

TheTube

unread,
May 14, 2020, 4:07:14 AM5/14/20
to Packer


Op donderdag 14 mei 2020 03:24:39 UTC+2 schreef TheTube:
=========================================================

While cycling trough the answers of this group maybe I am missing some brackets ....  

"post-processors": [
    {
      "type": "shell-local",
      "script": "create-cloudbox.sh"    
    },
    {
      "output": "{{ user `build_directory` }}/{{user `box_basename`}}.{{.Provider}}.box",
      "type": "vagrant"
    },
    {
      "type": "vagrant-cloud",
      "box_tag": "username/nameofmybox",
      "access_token": "{{user `cloud_token`}}",
      "version": "{{user `version`}}"
    }
  ],


 

TheTube

unread,
May 14, 2020, 5:38:18 AM5/14/20
to Packer


Op donderdag 14 mei 2020 10:07:14 UTC+2 schreef TheTube:
======================================================================================================
HERE IS THE POST WITH THE BRACKETS. 
 
========================================================================================================







 

TheTube

unread,
May 14, 2020, 5:39:56 AM5/14/20
to Packer


Op donderdag 14 mei 2020 11:38:18 UTC+2 schreef TheTube:
So it seems that I don have to precreate the box through the webinterface or have to use the API ....





 

TheTube

unread,
May 14, 2020, 5:46:37 AM5/14/20
to Packer


Op donderdag 14 mei 2020 11:39:56 UTC+2 schreef TheTube:
"post-processors": [
    [
    {
      "output": "{{ user `build_directory` }}/{{user `box_basename`}}.{{.Provider}}.box",
      "type": "vagrant"
    },
    {
      "type": "vagrant-cloud",
      "box_tag": "myusername/test13",
      "access_token": "{{user `cloud_token`}}",
      "version": "{{user `version`}}"
    }
    ]
  ],

Still the same error:

TheTube

unread,
May 14, 2020, 6:11:48 AM5/14/20
to Packer


Op donderdag 14 mei 2020 11:46:37 UTC+2 schreef TheTube:

TheTube

unread,
May 14, 2020, 6:14:10 AM5/14/20
to Packer
So if I give accesstoken a value. Letś say foo, it still gives me the same error message: * access_token must be set if vagrant_cloud_url has not been overriden


Op donderdag 14 mei 2020 12:11:48 UTC+2 schreef TheTube:

TheTube

unread,
May 14, 2020, 6:27:51 AM5/14/20
to Packer


Op donderdag 14 mei 2020 12:14:10 UTC+2 schreef TheTube:
I exported access_token=foo
           
             echo $access_token
            foo
            
sudo packer build -only=vmware-iso ubuntu-20.04-amd64.json
 
 * access_token must be set if vagrant_cloud_url has not been overriden
* Failed to verify authentication token: 401 Unauthorized

           *Pulling my hair out *





 




 

Megan Marsh

unread,
May 14, 2020, 2:52:15 PM5/14/20
to packe...@googlegroups.com
The first example you should should have worked. Are you sure the env var VAGRANT_CLOUD_TOKEN is set?

TheTube

unread,
May 14, 2020, 3:26:47 PM5/14/20
to Packer


Op donderdag 14 mei 2020 20:52:15 UTC+2 schreef Megan Marsh:
The first example you should should have worked. Are you sure the env var VAGRANT_CLOUD_TOKEN is set?

Yes. I can echo it.
 

TheTube

unread,
May 14, 2020, 3:37:16 PM5/14/20
to Packer


Op donderdag 14 mei 2020 21:26:47 UTC+2 schreef TheTube:


Op donderdag 14 mei 2020 20:52:15 UTC+2 schreef Megan Marsh:
The first example you should should have worked. Are you sure the env var VAGRANT_CLOUD_TOKEN is set?

Yes. I can echo it. I went back to the first example and it doesn give me the error any more .. *confused*  I need to protect myself with a verioncontrol system I guess. Its building. Fingers crossed ...
 

TheTube

unread,
May 14, 2020, 4:28:29 PM5/14/20
to Packer
I ran on the first example of on ubuntu-20.04.json with a reference to create-cloudbox.sh :
And this is the output from the build I just made.

* Post-processor failed: Unknown artifact type, requires box from vagrant post-processor or vagrant builder: mitchellh.vmware
Build 'vmware-iso' errored: 1 error(s) occurred:

* Post-processor failed: Unknown artifact type, requires box from vagrant post-processor or vagrant builder: mitchellh.vmware
2020/05/14 22:19:29 machine readable: error-count []string{"1"}

==> Some builds didn't complete successfully and had errors:
2020/05/14 22:19:29 machine readable: vmware-iso,error []string{"1 error(s) occurred:\n\n* Post-processor failed: Unknown artifact type, requires box from vagrant post-processor or vagrant builder: mitchellh.vmware"}

* Post-processor failed: Unknown artifact type, requires box from vagrant post-processor or vagrant builder: mitchellh.vmware
==> Builds finished but no artifacts were created.
==> Some builds didn't complete successfully and had errors:
--> vmware-iso: 1 error(s) occurred:

* Post-processor failed: Unknown artifact type, requires box from vagrant post-processor or vagrant builder: mitchellh.vmware

==> Builds finished but no artifacts were created.
2020/05/14 22:19:29 [INFO] (telemetry) Finalizing.
2020/05/14 22:19:30 waiting for all plugin processes to complete...
2020/05/14 22:19:30 [ERR] Error decoding response stream 8: EOF
2020/05/14 22:19:30 [ERR] Error decoding response stream 7: EOF
2020/05/14 22:19:30 /usr/local/bin/packer: plugin process exited
2020/05/14 22:19:30 /usr/local/bin/packer: plugin process exited
2020/05/14 22:19:30 /usr/local/bin/packer: plugin process exited
2020/05/14 22:19:30 /usr/local/bin/packer: plugin process exited


Op donderdag 14 mei 2020 21:37:16 UTC+2 schreef TheTube:

Megan Marsh

unread,
May 14, 2020, 4:40:32 PM5/14/20
to packe...@googlegroups.com
This is where that bracket issue you found on another thread comes into play.

Instead of

  "post-processors": [
    {
      "type": "shell-local",
      "script": "create-cloudbox.sh"    
    },
    {
      "output": "{{ user `build_directory` }}/{{user `box_basename`}}.{{.Provider}}.box",
      "type": "vagrant"
    },
    {
      "type": "vagrant-cloud",
      "box_tag": "username/nameofmybox",
      "access_token": "{{user `cloud_token`}}",
      "version": "{{user `version`}}"
    }
  ],

You need

  "post-processors": [[
    {
      "type": "shell-local",
      "script": "create-cloudbox.sh"    
    },
    {
      "output": "{{ user `build_directory` }}/{{user `box_basename`}}.{{.Provider}}.box",
      "type": "vagrant"
    },
    {
      "type": "vagrant-cloud",
      "box_tag": "username/nameofmybox",
      "access_token": "{{user `cloud_token`}}",
      "version": "{{user `version`}}"
    }
  ]],

Any post-processors only wrapped in the first set of brackets will get the build artifact as its input. 

Any post-processors inside the second set of brackets will run in order, with the output artifact from one post-processor becoming the input artifact to the next. 

Why? 

The double-bracketing doesn't make intuitive sense in a single-flow context like yours, but it allows users to create multiple output artifacts using different postprocessor flows from the same build artifact.

So, for example, you could take the same vmware artifact and both create a vagrant box that you upload to vagrant cloud in one flow like you're doing here, but then, and in another flow upload it to vsphere, and the postprocessors wouldn't get in each other's ways. 

For a little more detail on sequencing, you can see the docs at https://www.packer.io/docs/templates/post-processors/#post-processor-definition

Hope this helps!

TheTube

unread,
May 14, 2020, 6:56:13 PM5/14/20
to Packer


Op donderdag 14 mei 2020 22:40:32 UTC+2 schreef Megan Marsh:
Executing local shell command [/bin/sh -c PACKER_BUILDER_TYPE='vmware-iso' PACKER_BUILD_NAME='vmware-iso'  ./create-cloudbox.sh]
==> vmware-iso (shell-local):   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
==> vmware-iso (shell-local):                                  Dload  Upload   Total   Spent    Left  Speed
==> vmware-iso (shell-local): 100   643    0   402  100   241    300    179  0:00:01  0:00:01 --:--:--   479
    vmware-iso (shell-local): {"tag":"cool-base-nl/testmenow16","username":"myusername","name":"mybox","private":false,"downloads":0,"created_at":"2020-05-15T00:44:24.812+02:00","updated_at":"2020-05-15T00:44:24.812+02:00","short_description":"My dev box11","description_markdown":"Testbox11 to include Ansible scripts","description_html":"<p>Testbox11 to include Ansible scripts</p>\n","current_version":null,"versions":[]}
2020/05/15 00:44:25 [INFO] (telemetry) ending shell-local
==> vmware-iso: Running post-processor: vagrant
2020/05/15 00:44:25 Flagging to keep original artifact from post-processor 'shell-local'
2020/05/15 00:44:25 [INFO] (telemetry) Starting post-processor vagrant
2020/05/15 00:44:25 [INFO] (telemetry) ending vagrant
Build 'vmware-iso' errored: 1 error(s) occurred:

* Post-processor failed: Unknown artifact type, can't build box: 

* Post-processor failed: Unknown artifact type, can't build box: 
2020/05/15 00:44:25 machine readable: error-count []string{"1"}

==> Some builds didn't complete successfully and had errors:
==> Some builds didn't complete successfully and had errors:
2020/05/15 00:44:25 machine readable: vmware-iso,error []string{"1 error(s) occurred:\n\n* Post-processor failed: Unknown artifact type, can't build box: "}
--> vmware-iso: 1 error(s) occurred:


* Post-processor failed: Unknown artifact type, can't build box: 
==> Builds finished but no artifacts were created.
* Post-processor failed: Unknown artifact type, can't build box: 

==> Builds finished but no artifacts were created.
2020/05/15 00:44:25 [INFO] (telemetry) Finalizing.
2020/05/15 00:44:26 waiting for all plugin processes to complete...
2020/05/15 00:44:26 [ERR] Error decoding response stream 12: EOF
2020/05/15 00:44:26 [ERR] Error decoding response stream 7: EOF
2020/05/15 00:44:26 [ERR] Error decoding response stream 8: EOF
2020/05/15 00:44:26 [ERR] Error decoding response stream 9: EOF
2020/05/15 00:44:26 [ERR] Error decoding response stream 10: EOF
2020/05/15 00:44:26 [ERR] Error decoding response stream 11: EOF
2020/05/15 00:44:26 /usr/local/bin/packer: plugin process exited
2020/05/15 00:44:26 /usr/local/bin/packer: plugin process exited
2020/05/15 00:44:26 /usr/local/bin/packer: plugin process exited
2020/05/15 00:44:26 /usr/local/bin/packer: plugin process exited
2020/05/15 00:44:26 /usr/local/bin/packer: plugin process exited

"post-processors": [[
    {
      "type": "shell-local",
      "script": "./create-cloudbox.sh"
    },
    {
      "output": "{{ user `build_directory` }}/{{user `box_basename`}}.{{.Provider}}.box",
      "type": "vagrant"
    },
    {
      "type": "vagrant-cloud",
      "box_tag": "myusername/mybox",
      "access_token": "{{user `cloud_token`}}",
      "version": "{{user `version`}}"
    }
  ]],
You need 

  "post-processors": [[
    {
      "type": "shell-local",
      "script": "create-cloudbox.sh"    
    },
    {
      "output": "{{ user `build_directory` }}/{{user `box_basename`}}.{{.Provider}}.box",
      "type": "vagrant"
    },
    {
      "type": "vagrant-cloud",
      "box_tag": "username/nameofmybox",
      "access_token": "{{user `cloud_token`}}",
      "version": "{{user `version`}}"
    }
  ]],

Mybe first the vagrant and than the output ? 

"type": "vagrant",
 "output": "{{ user `build_directory` }}/{{user `box_basename`}}.{{.Provider}}.box"

will give it a try.



 

TheTube

unread,
May 14, 2020, 9:17:47 PM5/14/20
to Packer


Op vrijdag 15 mei 2020 00:56:13 UTC+2 schreef TheTube:

"post-processors": [ { "type": "shell-local", "inline": ["echo Doing stuff..."] }, [ { "type": "vagrant", "include": ["image.iso"], "vagrantfile_template": "vagrantfile.tpl", "output": "proxycore_{{.Provider}}.box" }, { "type": "vagrant-cloud", "box_tag": "hashicorp/precise64", "access_token": "{{user `cloud_token`}}", "version": "{{user `version`}}" } ] ] }




if I do the same I get:

 access_token must be set if vagrant_cloud_url has not been overriden
* Failed to verify authentication token: 401 Unauthorized


ok simple
{ "post-processors": ["compress"] }



detailed:
{ "post-processors": [ { "type": "compress", "format": "tar.gz" } ] }

sequenced
{ "post-processors": [[
"compress",
{ "type": "upload", "endpoint": "http://example.com" }

]] }







TheTube

unread,
May 14, 2020, 10:34:34 PM5/14/20
to Packer


Op donderdag 14 mei 2020 03:24:39 UTC+2 schreef TheTube:

TheTube

unread,
May 14, 2020, 10:38:50 PM5/14/20
to Packer


Op vrijdag 15 mei 2020 03:17:47 UTC+2 schreef TheTube:
Thi is from 

TheTube

unread,
May 15, 2020, 10:16:36 AM5/15/20
to Packer


Op vrijdag 15 mei 2020 04:38:50 UTC+2 schreef TheTube:
 
{
  "variables": {
    "cloud_token": "{{ env `VAGRANT_CLOUD_TOKEN` }}",
    "version": "1.0.{{timestamp}}"
  },
  "post-processors": [
    {
      "type": "shell-local",
      "inline": ["echo Doing stuff..."]
    },
    [
      {
        "type": "vagrant",
        "include": ["image.iso"],
        "vagrantfile_template": "vagrantfile.tpl",
        "output": "proxycore_{{.Provider}}.box"
      },
      {
        "type": "vagrant-cloud",
        "box_tag": "hashicorp/precise64",
        "access_token": "{{user `cloud_token`}}",
        "version": "{{user `version`}}"
      }
    ]
  ]
}

Letś break it down.

The first post-processor is self explenatory.

 {
      "type": "shell-local",
      "inline": ["echo Doing stuff..."]
    },

The second one:
 {
        "type": "vagrant",
        "include": ["image.iso"],
        "vagrantfile_template": "vagrantfile.tpl",
        "output": "proxycore_{{.Provider}}.box"
      },

 I don´t want to include an iso file so I leave this inlude line   "include": ["image.iso"],  out of the json file 
Handy option but not for now.    "vagrantfile_template": "vagrantfile.tpl",
output etc will be "output": "{{ user `build_directory` }}/{{user `box_basename`}}.{{.Provider}}.box"
The last processor will be the same.

Still errors. 
2020/05/15 12:21:10 [INFO] (telemetry) ending vagrant
2020/05/15 12:21:10 [INFO] (telemetry) Starting post-processor vagrant-cloud
==> vmware-iso: Running post-processor: vagrant-cloud
2020/05/15 12:21:10 packer-post-processor-vagrant-cloud plugin: Post-Processor Vagrant Cloud API GET: https://vagrantcloud.com/api/v1/box/cool-base-nl/testmenow20
==> vmware-iso (vagrant-cloud): Verifying box is accessible: cool-base-nl/testmenow20
2020/05/15 12:21:11 packer-post-processor-vagrant-cloud plugin: Post-Processor Vagrant Cloud API Response:
2020/05/15 12:21:11 packer-post-processor-vagrant-cloud plugin: 
2020/05/15 12:21:11 packer-post-processor-vagrant-cloud plugin: &{Status:404 Not Found StatusCode:404 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Cache-Control:[no-cache] Connection:[keep-alive] Content-Type:[application/json; charset=utf-8] Date:[Fri, 15 May 2020 10:21:10 GMT] Referrer-Policy:[strict-origin-when-cross-origin] Server:[Cowboy] Set-Cookie:[_atlas_session_data=V3NrYzlTbkhoWmhKL21JRHdycHFybkdNUmh3Uk54UkxmS1BjZHo4VkZLK3pqVy9sdVFKLzk1UlU0WVRrd21DOUE1STVoNDErMlBnZXRiYlErOG9qNkc3ZkFYeHZCNDc4NkNwUHp6RFh6Q1U9LS1oQlNSWXRXQnJSaG5rS0RwS2JiZ1pnPT0%3D--0e6aadc88af3860b329e7d9aa35537bab8582341; path=/; expires=Sun, 14 Jun 2020 10:21:11 GMT; secure; HttpOnly] Strict-Transport-Security:[max-age=31536000; includeSubDomains; preload] Via:[1.1 vegur] X-Content-Type-Options:[nosniff] X-Download-Options:[noopen] X-Frame-Options:[SAMEORIGIN] X-Permitted-Cross-Domain-Policies:[none] X-Request-Id:[d74d1111-aecc-44c4-a831-dd142fd7d5d2] X-Runtime:[0.215624] X-Vagrantcloud-Rate-Limit:[99/100] X-Xss-Protection:[1; mode=block]] Body:0xc000488a40 ContentLength:-1 TransferEncoding:[chunked] Close:false Uncompressed:false Trailer:map[] Request:0xc0000e0400 TLS:0xc0007640b0}
2020/05/15 12:21:11 [INFO] (telemetry) ending vagrant-cloud
2020/05/15 12:21:11 Deleting original artifact for build 'vmware-iso'

* Post-processor failed: Script exited with non-zero exit status: 127.Allowed exit codes are: [0]
* Post-processor failed: Error retrieving box: Resource not found!
Build 'vmware-iso' errored: 2 error(s) occurred:

* Post-processor failed: Script exited with non-zero exit status: 127.Allowed exit codes are: [0]
* Post-processor failed: Error retrieving box: Resource not found!
2020/05/15 12:21:11 machine readable: error-count []string{"1"}

==> Some builds didn't complete successfully and had errors:
2020/05/15 12:21:11 machine readable: vmware-iso,error []string{"2 error(s) occurred:\n\n* Post-processor failed: Script exited with non-zero exit status: 127.Allowed exit codes are: [0]\n* Post-processor failed: Error retrieving box: Resource not found!"}

* Post-processor failed: Script exited with non-zero exit status: 127.Allowed exit codes are: [0]
* Post-processor failed: Error retrieving box: Resource not found!
==> Builds finished but no artifacts were created.
==> Some builds didn't complete successfully and had errors:
--> vmware-iso: 2 error(s) occurred:

* Post-processor failed: Script exited with non-zero exit status: 127.Allowed exit codes are: [0]
* Post-processor failed: Error retrieving box: Resource not found!

==> Builds finished but no artifacts were created.
2020/05/15 12:21:11 [INFO] (telemetry) Finalizing.
2020/05/15 12:21:12 waiting for all plugin processes to complete...
2020/05/15 12:21:12 /usr/local/bin/packer: plugin process exited
2020/05/15 12:21:12 /usr/local/bin/packer: plugin process exited
2020/05/15 12:21:12 [ERR] Error decoding response stream 7: EOF
2020/05/15 12:21:12 /usr/local/bin/packer: plugin process exited
2020/05/15 12:21:12 [ERR] Error decoding response stream 8: EOF
2020/05/15 12:21:12 /usr/local/bin/packer: plugin process exited
2020/05/15 12:21:12 /usr/local/bin/packer: plugin process exited

Unfortunately the example leads to an error. Why do people publish this ?

packer-post-processor-vagrant-cloud plugin: Post-Processor Vagrant Cloud API GET: https://vagrantcloud.com/api/v1/box/cool-base-nl/testmenow20
2020/05/15 12:21:11 packer-post-processor-vagrant-cloud plugin: Post-Processor Vagrant Cloud API Response:
2020/05/15 12:21:11 packer-post-processor-vagrant-cloud plugin: 
2020/05/15 12:21:11 packer-post-processor-vagrant-cloud plugin: &{Status:404 Not Found StatusCode:404

So I need to create the box. Like I do on the web. I create manually a box. Then I build with packer the box and it uploads to the cloud.
Now I use the API for creating the box.    
Let´s listen again to Megan:

Any post-processors only wrapped in the first set of brackets will get the build artifact as its input. 

Any post-processors inside the second set of brackets will run in order, with the output artifact from one post-processor becoming the input artifact to the next. 

Why? 

The double-bracketing doesn't make intuitive sense in a single-flow context like yours, but it allows users to create multiple output artifacts using different postprocessor flows from the same build artifact.

So, for example, you could take the same vmware artifact and both create a vagrant box that you upload to vagrant cloud in one flow like you're doing here, but then, and in another flow upload it to vsphere, and the postprocessors wouldn't get in each other's ways. 

For a little more detail on sequencing, you can see the docs at https://www.packer.io/docs/templates/post-processors/#post-processor-definition

  "post-processors": [I read from left to right. I count from left to right so Iám now in the first set of brackets. So i get the build artifact as input. Well I must be wrong.
   This must be the second set of brackets. because I get the build artifact as input.  [

    {
      "type": "shell-local",
      "script": "create-cloudbox.sh"    
    },
    {
      "output": "{{ user `build_directory` }}/{{user `box_basename`}}.{{.Provider}}.box",
      "type": "vagrant"
    },
    {
      "type": "vagrant-cloud",
      "box_tag": "username/nameofmybox",
      "access_token": "{{user `cloud_token`}}",
      "version": "{{user `version`}}"
    }
  ]],
===============================================================================================
So.

  "post-processors": [Second set of brackets. Any pp here will run in order, with the output artifact from one post-processor becoming the input artifact to the next. Here is nothing. 

 [ post-processors only wrapped in this first set of brackets will get the build artifact as its input. 
********** My shell-local ¨post-provider¨ isn´t used as a post provider. In that sense that is modifing the build artifact. It is only creating a placeholde for a box.
But the vagrant-cloud preprocessor doesn´t modify a box either. It just copies the box over.  However I had good results with manually via the webintetface creating
the placeholder for the box. And then run packer afterwards.
¨¨

dragon788

unread,
May 16, 2020, 12:09:47 PM5/16/20
to Packer
I think the root of your problem is not understanding what the steps do when you are copying and pasting another person's template. You should build your own once with a minimal set of steps to understand how it works and prove out your local environment before you go grabbing complex templates that you don't understand.

From what I can see you grabbed a template that is setup to publish to Vagrant Cloud, this requires an account there and I believe requires a subscription if you aren't hosting the boxes on your own infrastructure. If you have an account there you should be able to generate a token in your account settings and pass it on as a `-var` or put it in a JSON file and pass that as a `-var-file`.

If you don't need this functionality you should delete the line under variables referencing Vagrant Cloud and also any section in post processors that contains a reference to that variable.

TheTube

unread,
May 16, 2020, 3:03:04 PM5/16/20
to Packer
It works. The root of the problem is my linucx instllation which sends me in some cases to a sort of environment. As an example sudo su gives me (base) root@.... while sudo bash gives me root@ ... running the scripts with sudo leads to other results than running them with a n elevated prompt. if I use sudo bash I can follow the advise of Megan very well. The only thing I'll have on forehand is creating a placekholder for the box then everuyting works like a charm ! Tank you Megan for helping me out !!!
 


Op zaterdag 16 mei 2020 18:09:47 UTC+2 schreef dragon788:
Reply all
Reply to author
Forward
0 new messages