cant' make openstack image using packer

[Jihyun Choi]

I'm trying to build a centos image on openstack with packer. For some reason, the build is terminated in the middle of the script and I can't figure out problems.

Additionally, I can't find out any logs in glance.

This is my packer script and error log.

packer script

{
  "builders": [
    {
      "availability_zone": "nova",
      "domain_id": "xxx",
      "flavor": "m1.tiny",
      "identity_endpoint": "http://xxx:5000/v3",
      "image_name": "centos",
      "image_visibility": "private",
      "image_members": "myname",
      "networks": "xx-xxx-xxx",
      "password": "mypassword",
      "region": "RegionOne",
      "source_image": "17987fc7-e5af-487f-ae74-754ade318824",
      "ssh_keypair_name": "mykeypairname",
      "ssh_private_key_file": "/root/.ssh/id_rsa",
      "ssh_username": "mysshusername",
      "tenant_name": "admin",
      "type": "openstack",
      "username": "mynuserame"
    }
  ],
  "provisioners": [
    {
      "script": "setup-centos.sh",
      "type": "shell"
    }
  ]
}

Error Log

...
2018/07/27 13:01:31 packer: 2018/07/27 13:01:31 Waiting for image creation status: SAVING (25%)
2018/07/27 13:01:33 packer: 2018/07/27 13:01:33 Waiting for image creation status: SAVING (25%)
2018/07/27 13:01:35 packer: 2018/07/27 13:01:35 Waiting for image creation status: SAVING (25%)
2018/07/27 13:01:37 packer: 2018/07/27 13:01:37 Waiting for image creation status: SAVING (25%)
2018/07/27 13:01:39 packer: 2018/07/27 13:01:39 Waiting for image creation status: SAVING (25%)
2018/07/27 13:01:41 packer: 2018/07/27 13:01:41 Waiting for image creation status: SAVING (25%)
2018/07/27 13:01:43 ui error: ==> openstack: Error waiting for image: Resource not found
==> openstack: Error waiting for image: Resource not found
2018/07/27 13:01:43 ui: ==> openstack: Terminating the source server: 1034619b-4dc9-45d1-b160-20290e0c4c08 ...
==> openstack: Terminating the source server: 1034619b-4dc9-45d1-b160-20290e0c4c08 ...
2018/07/27 13:01:43 packer: 2018/07/27 13:01:43 Waiting for state to become: [DELETED]
2018/07/27 13:01:44 packer: 2018/07/27 13:01:44 Waiting for state to become: [DELETED] currently SHUTOFF (0%)
2018/07/27 13:01:46 packer: 2018/07/27 13:01:46 [INFO] 404 on ServerStateRefresh, returning DELETED
2018/07/27 13:01:46 [INFO] (telemetry) ending openstack
2018/07/27 13:01:46 [INFO] (telemetry) found error: Error waiting for image: Resource not found
2018/07/27 13:01:46 ui error: Build 'openstack' errored: Error waiting for image: Resource not found
2018/07/27 13:01:46 Builds completed. Waiting on interrupt barrier...
2018/07/27 13:01:46 machine readable: error-count []string{"1"}
2018/07/27 13:01:46 ui error:
==> Some builds didn't complete successfully and had errors:
2018/07/27 13:01:46 machine readable: openstack,error []string{"Error waiting for image: Resource not found"}
2018/07/27 13:01:46 ui error: --> openstack: Error waiting for image: Resource not found
2018/07/27 13:01:46 ui:
==> Builds finished but no artifacts were created.
2018/07/27 13:01:46 [INFO] (telemetry) Finalizing.
Build 'openstack' errored: Error waiting for image: Resource not found


==> Some builds didn't complete successfully and had errors:
--> openstack: Error waiting for image: Resource not found


==> Builds finished but no artifacts were created.
2018/07/27 13:01:47 waiting for all plugin processes to complete...
2018/07/27 13:01:47 /root/pack/packer: plugin process exited
2018/07/27 13:01:47 /root/pack/packer: plugin process exited

Thanks in advance.

[Rickard von Essen]

Could you rebuild with: PACKER_LOG=1 packer build template.json

and attach the full log. Also which version of OpenStack are you using?

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/40c5530a-b52a-4ad8-9857-b082601088e9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jihyun Choi]

My openstack version is ocata.

As the log size is too big, I uploaded the full log in my gist! Thanks!

https://gist.github.com/jihyun00/1f6fbcada9da2a4ce33e7831ad5143b3#file-openstacklog

2018년 7월 27일 금요일 오후 5시 20분 15초 UTC+9, Rickard von Essen 님의 말:

[Rickard von Essen]

Ok, first download the latest release 1.2.5 from https://www.packer.io/downloads.html

Then rerun and gist a new log. 

To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/159c3e24-6f59-4061-931f-79aa89b89ca7%40googlegroups.com.

[Jihyun Choi]

Here is the log which rerun with packer version 1.2.5.

https://gist.github.com/jihyun00/1f6fbcada9da2a4ce33e7831ad5143b3

2018년 7월 27일 금요일 오후 6시 35분 42초 UTC+9, Rickard von Essen 님의 말:

[Rickard von Essen]

This looks like there might be some problem with your OS installation. It would be good if you open an issue with the admins of it. 

What happens is that:

1) Compute V2 API: image create is successfully called and a image Id is returned.

2) Packer waits for this image to be available (i.e ready for use) by calling the Compute V2 API - image get on the specific image Id.

During this OS returns a non 200/500/404 status code which it shouldn't. 

It would be good to get some more info about what is logged by OpenStack.

// Rickard

To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/0aa3d6f7-f335-41a6-b95e-78147dc40b45%40googlegroups.com.

[Jihyun Choi]

I think you missed a log.

Could you click this link again? https://gist.github.com/jihyun00/1f6fbcada9da2a4ce33e7831ad5143b3#file-openstacklog-L232

As you see, it returned 404.

I also add openstack logs.

[nova-compute] [instance: 93075e5d-cfde-4f2e-b68d-711a3f84ee0f] Successfully reverted task state from image_uploading on failure for instance.

[haproxy] 172.30.249.208:43236 [31/Jul/2018:17:06:20.273] nova_api nova_api/oskdv08gm 0/0/0/204/204 204 334 - - ---- 372/0/0/0/0 0/0 "DELETE /v2.1/f92551f3a953416a9135b141462ad448/servers/93075e5d-cfde-4f2e-b68d-711a3f84ee0f HTTP/1.1"

[nova-compute] [instance: 93075e5d-cfde-4f2e-b68d-711a3f84ee0f] Deleting instance files /var/lib/nova/instances/93075e5d-cfde-4f2e-b68d-711a3f84ee0f_del

[nova-api] HTTP exception thrown: Image not found.

[haproxy] 172.30.249.215:45689 [31/Jul/2018:17:06:20.617] neutron_server neutron_server/oskdv09gm 2/0/0/71/73 200 4031 - - ---- 379/1/0/1/0 0/0 "GET /v2.0/security-groups.json?id=8e6803cc-235f-458c-9f82-a7adc3f10076 HTTP/1.1"

[haproxy] 172.30.249.213:29009 [31/Jul/2018:17:06:20.252] glance_api glance_api/oskdv07gm 0/0/0/18/18 404 269 - - ---- 374/1/0/1/0 0/0 "GET /v2/images/58645091-75d3-4ad8-a900-ae9bf325ca54 HTTP/1.1"

[nova-compute] [instance: 93075e5d-cfde-4f2e-b68d-711a3f84ee0f] Deletion of /var/lib/nova/instances/93075e5d-cfde-4f2e-b68d-711a3f84ee0f_del complete

[haproxy] 172.30.249.208:43238 [31/Jul/2018:17:06:20.479] nova_api nova_api/oskdv09gm 0/0/0/236/236 200 2122 - - ---- 378/0/0/0/0 0/0 "GET /v2.1/f92551f3a953416a9135b141462ad448/servers/93075e5d-cfde-4f2e-b68d-711a3f84ee0f HTTP/1.1"

[haproxy] 172.30.249.208:43234 [31/Jul/2018:17:06:20.243] nova_api nova_api/oskdv07gm 0/0/0/29/29 404 411 - - ---- 373/0/0/0/0 0/0 "GET /v2.1/f92551f3a953416a9135b141462ad448/images/58645091-75d3-4ad8-a900-ae9bf325ca54 HTTP/1.1"

[nova-compute] Successfully unplugged vif VIFBridge(active=False,address=fa:16:3e:2a:f2:e6,bridge_name='qbr860df470-bd',has_traffic_filtering=True,id=860df470-bde0-4787-a5a4-6a8d84fdad69,network=Network(7798e01b-c7a5-4a8f-a879-97879165bcac),plugin='ovs',port_profile=VIFPortProfileBase,preserve_on_delete=False,vif_name='tap860df470-bd')

[nova-novncproxy] 172.30.249.15: ignoring empty handshake

[nova-compute] [instance: 93075e5d-cfde-4f2e-b68d-711a3f84ee0f] Terminating instance

[haproxy] 172.30.249.215:45689 [31/Jul/2018:17:06:20.558] neutron_server neutron_server/oskdv08gm 0/0/0/59/59 200 1252 - - ---- 377/1/0/1/0 0/0 "GET /v2.0/ports.json?device_id=93075e5d-cfde-4f2e-b68d-711a3f84ee0f HTTP/1.1"

[nova-compute] [instance: 93075e5d-cfde-4f2e-b68d-711a3f84ee0f] Took 0.41 seconds to destroy the instance on the hypervisor.

[nova-api] 172.30.249.208,172.30.249.214 "GET /v2.1/f92551f3a953416a9135b141462ad448/images/58645091-75d3-4ad8-a900-ae9bf325ca54 HTTP/1.1" status: 404 len: 411 time: 0.0287371

[neutron-server] 172.30.249.211,172.30.249.214 "GET /v2.0/ports.json?device_id=93075e5d-cfde-4f2e-b68d-711a3f84ee0f HTTP/1.1" status: 200 len: 1252 time: 0.0554459

[nova-api] 172.30.249.208,172.30.249.214 "DELETE /v2.1/f92551f3a953416a9135b141462ad448/servers/93075e5d-cfde-4f2e-b68d-711a3f84ee0f HTTP/1.1" status: 204 len: 334 time: 0.2039509

[nova-compute] [instance: 93075e5d-cfde-4f2e-b68d-711a3f84ee0f] Instance destroyed successfully.

[neutron-openvswitch-agent] port_unbound(): net_uuid None not managed by VLAN manager

[neutron-openvswitch-agent] Security group member updated [u'8e6803cc-235f-458c-9f82-a7adc3f10076']

2018년 7월 31일 화요일 오전 3시 24분 54초 UTC+9, Rickard von Essen 님의 말:

[Rickard von Essen]

But the 404 is after the failure, during the clean up, and it's expected. This is the interesting part of the log:

2018/07/31 17:06:09 ui: ==> openstack: Creating the image: centos
==> openstack: Creating the image: centos
2018/07/31 17:06:10 ui:     openstack: Image: 58645091-75d3-4ad8-a900-ae9bf325ca54
    openstack: Image: 58645091-75d3-4ad8-a900-ae9bf325ca54
2018/07/31 17:06:10 ui: ==> openstack: Waiting for image centos (image id: 58645091-75d3-4ad8-a900-ae9bf325ca54) to become ready...
==> openstack: Waiting for image centos (image id: 58645091-75d3-4ad8-a900-ae9bf325ca54) to become ready...
2018/07/31 17:06:10 packer: 2018/07/31 17:06:10 Waiting for image creation status: SAVING (25%)
2018/07/31 17:06:12 packer: 2018/07/31 17:06:12 Waiting for image creation status: SAVING (25%)
2018/07/31 17:06:14 packer: 2018/07/31 17:06:14 Waiting for image creation status: SAVING (25%)
2018/07/31 17:06:16 packer: 2018/07/31 17:06:16 Waiting for image creation status: SAVING (25%)
2018/07/31 17:06:18 packer: 2018/07/31 17:06:18 Waiting for image creation status: SAVING (25%)
2018/07/31 17:06:20 ui error: ==> openstack: Error waiting for image: Resource not found
2018/07/31 17:06:20 ui: ==> openstack: Terminating the source server: 93075e5d-cfde-4f2e-b68d-711a3f84ee0f ...

==> openstack: Error waiting for image: Resource not found

==> openstack: Terminating the source server: 93075e5d-cfde-4f2e-b68d-711a3f84ee0f ...
2018/07/31 17:06:20 packer: 2018/07/31 17:06:20 Waiting for state to become: [DELETED]
2018/07/31 17:06:20 packer: 2018/07/31 17:06:20 Waiting for state to become: [DELETED] currently SHUTOFF (0%)
2018/07/31 17:06:22 packer: 2018/07/31 17:06:22 [INFO] 404 on ServerStateRefresh, returning DELETED
2018/07/31 17:06:22 [INFO] (telemetry) ending openstack

The failure is on:
2018/07/31 17:06:20 ui error: ==> openstack: Error waiting for image: Resource not found

To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/34cc5b95-1e28-46d2-8775-34a9fdbe3cad%40googlegroups.com.

[Rickard von Essen]

This log line seems to indicate that an OpenStack node failed during the image save:

[nova-compute] [instance: 93075e5d-cfde-4f2e-b68d-711a3f84ee0f] Successfully reverted task state from image_uploading on failure for instance.

[haproxy] 172.30.249.208:43236

[Jihyun Choi]

I rebuild image and I found another logs in glance-api service.

2018-07-31 19:02:30.809 24 INFO eventlet.wsgi.server [req-b5d9fd83-3e3f-4383-bac8-abaea016463b b650b3fa22e0e3ddd54c4819e1f75afa844437ce141770b18d035ddfd0337179 922abe0e2e40459c8ae3632bb2d93ece - 5d48fc988eaa440ebe29aca0934db245 5d48fc988eaa440ebe29aca0934db245] xxx.xx.xxx.xxx,xxx.xx.xxx.xxx - - [31/Jul/2018 19:02:30] "PATCH /v2/images/487137b5-ed0d-47cc-85ca-256317abde59 HTTP/1.1" 403 271 0.026601

2018-07-31 19:02:32.364 22 INFO eventlet.wsgi.server [req-ec063fb3-0441-4698-bafa-80dd2d495e43 b650b3fa22e0e3ddd54c4819e1f75afa844437ce141770b18d035ddfd0337179 922abe0e2e40459c8ae3632bb2d93ece - 5d48fc988eaa440ebe29aca0934db245 5d48fc988eaa440ebe29aca0934db245] xxx.xx.xxx.xxx,xxx.xx.xxx.xxx - - [31/Jul/2018 19:02:32] "DELETE /v2/images/487137b5-ed0d-47cc-85ca-256317abde59 HTTP/1.1" 204 189 0.169432

2018-07-31 19:02:34.144 27 INFO eventlet.wsgi.server [req-57585196-47b1-4af7-940b-5067a86e9227 b650b3fa22e0e3ddd54c4819e1f75afa844437ce141770b18d035ddfd0337179 922abe0e2e40459c8ae3632bb2d93ece - 5d48fc988eaa440ebe29aca0934db245 5d48fc988eaa440ebe29aca0934db245] xxx.xx.xxx.xxx,xxx.xx.xxx.xxx - - [31/Jul/2018 19:02:34] "GET /v2/images/487137b5-ed0d-47cc-85ca-256317abde59 HTTP/1.1" 404 269 0.016231

After this log, packer stopped build and returned resource not found error.

2018년 7월 31일 화요일 오후 6시 4분 51초 UTC+9, Rickard von Essen 님의 말:

[Rickard von Essen]

Can you manually create an image from an instance? All indications points at there is some problem with the OpenStack system.

To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/b0a316a3-599c-4b9a-9df8-16ef39b418d9%40googlegroups.com.

[Jihyun Choi]

I found solution! 

After I set show_multiple_locations = True in glance_api, packer can make openstack image wel!!

Thanks for helping me, Rickard!!

2018년 8월 1일 수요일 오전 1시 53분 47초 UTC+9, Rickard von Essen 님의 말:

[Rickard von Essen]

Great to hear that you found a solution. 

To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/53b58d2b-8b40-4b8c-99be-8c0ea5275be2%40googlegroups.com.