Hi
I'm new bosh user and i'm using micro bosh to install CF on OpenStack
so as described
here i'm trying to deploy bosh vm but i got an error :
-----------------------------------------
console output
-----------------------------------------
$ bosh micro deploy ~/bosh-workspace/stemcells/latest-micro-bosh-stemcell-openstack.tgz
Verifying stemcell...
File exists and readable OK
Using cached manifest...
Stemcell properties OK
Stemcell info
-------------
Name: micro-bosh-stemcell
Version: 870
Deploy Micro BOSH
unpacking stemcell (00:00:40)
uploading stemcell (00:56:39)
creating VM from d9a595bb-5db6-4b7c-83e2-c89e5ffb8e91 (00:01:38)
Waiting for the agent |oooooo | 3/11 01:06:36 ETA: 02:29:39log writing failed. can't be called from trap context
Unable to connect to Bosh agent. Check logs for more details
-----------------------------------------
(cf manifest under)
NB: the stemcell upload -image seems to be 1,2Go size - is very long : 56minutes my case.
(micro bosh client) logs :
microbosh-openstack/bosh_micro_deploy.log
(...)
HTTPClient::BadResponseError: connect to ssl proxy failed with status 503 Service Unavailable>
D, [2013-08-06T17:45:15.579426 #28122] [0x43a8f6e] DEBUG -- : Waiting for agent to be ready: #<Bosh::Agent::Error: Agent HTTP 404>
D, [2013-08-06T17:45:16.666938 #28122] [0x43a8f6e] DEBUG -- : Waiting for agent to be ready: #<Bosh::Agent::Error: Agent HTTP 404>
D, [2013-08-06T17:45:17.758981 #28122] [0x43a8f6e] DEBUG -- : Waiting for agent to be ready: #<Bosh::Agent::Error: Agent HTTP 404>
(...repeat 404 until the end...)
(bosh target vm) logs :
cf. under(...)current.txt
-----------------------------------------
Q-1: how could i diagnose and fix this error ? maybe it's a network or disk issue?
Q-2: does exists a way to validate my deployment manifest file without restarting all the process and wait the stemcell upload again ? (even if it seems a stupid question)
Q-3: how should i reproduce what deploy expect to get while waiting the agent ?
for information, here is the way i raz & restart a micro bosh deploy (Q-4 : does exist a better way?) :
bosh micro status
(then (A) or (B))
((A) bosh target vm exists )
bosh micro delete
cd ~/bosh-workspace/deployments && rm bosh-deployments.yml bosh_registry.log microbosh-openstack/bosh_micro_deploy.log
((B) bosh target vm hasn't been created; this avoid stem cell already exists error)
cd ~/bosh-workspace/deployments && rm bosh-deployments.yml bosh_registry.log microbosh-openstack/bosh_micro_deploy.log
bosh micro deployment microbosh-openstack
bosh micro deploy ~/bosh-workspace/stemcells/latest-micro-bosh-stemcell-openstack.tgz
(wait one hour and pray .. ^^ )
in advance thanks
Regards
Brice
PS: sorry for the flood; my company restrict the ggdrive access so i can't attach file right here :"(
-----------------------------------------
manifest (ntp ips, url & creds have been replaced)
-----------------------------------------
~/bosh-workspace/deployments$ cat microbosh-openstack/micro_bosh.yml
---
name: microbosh-openstack
logging:
level: DEBUG
network:
type: dynamic
# vip: 10.192.36.23 # Optional
resources:
persistent_disk: 16384
cloud_properties:
instance_type: custom.m1.small
cloud:
plugin: openstack
properties:
openstack:
username: myuser
api_key: MyApiKey
tenant: myTenant
# region: <region> # Optional
default_security_groups: ["default", "microbosh"]
default_key_name: microbosh
private_key: ~/.ssh/microbosh/id_microbosh
apply_spec:
properties:
director:
max_threads: 3
hm:
resurrector_enabled: true
ntp:
- ww.xx.yy.zz
- ww.xx.yy.zzz
-----------------------------------------
(target vm) /var/vcap/bosh/log/current
-----------------------------------------
2013-08-07_09:00:04.10341 #[1379] INFO: Starting agent 1.5.0.pre.870...
2013-08-07_09:00:04.10367 #[1379] INFO: Configuring agent...
2013-08-07_09:00:04.10584 #[1379] INFO: Configuring instance
2013-08-07_09:00:04.12333 #[1379] INFO: OpenStack user data: "{\"registry\":{\"endpoint\":\"http://admin:admin@localhost:25889\"},\"server\":{\"name\":\"vm-c6b01d4c-5fba-4680-ab63-011e3f055027\"}}"
2013-08-07_09:00:04.12444 #[1379] INFO: failed to load infrastructure settings: Error requesting endpoint http://admin:admin@localhost:25889/instances/vm-c6b01d4c-5fba-4680-ab63-011e3f055027/settings: #<Errno::ECONNREFUSED:
Connection refused - connect(2) (
http://localhost:25889)>
2013-08-07_09:00:04.12496 /var/vcap/bosh/lib/ruby/gems/1.9.1/gems/bosh_agent-1.5.0.pre.870/lib/bosh_agent/settings.rb:52:in `rescue in load_from_cache': could neither load infrastructure settings nor cached settings from: /var/vcap/bosh/settings.json (Bosh::Agent::LoadSettingsError)
2013-08-07_09:00:04.12504 from /var/vcap/bosh/lib/ruby/gems/1.9.1/gems/bosh_agent-1.5.0.pre.870/lib/bosh_agent/settings.rb:47:in `load_from_cache'
2013-08-07_09:00:04.12506 from /var/vcap/bosh/lib/ruby/gems/1.9.1/gems/bosh_agent-1.5.0.pre.870/lib/bosh_agent/settings.rb:29:in `rescue in load'
2013-08-07_09:00:04.12508 from /var/vcap/bosh/lib/ruby/gems/1.9.1/gems/bosh_agent-1.5.0.pre.870/lib/bosh_agent/settings.rb:25:in `load'
2013-08-07_09:00:04.12510 from /var/vcap/bosh/lib/ruby/gems/1.9.1/gems/bosh_agent-1.5.0.pre.870/lib/bosh_agent/settings.rb:14:in `load'
2013-08-07_09:00:04.12512 from /var/vcap/bosh/lib/ruby/gems/1.9.1/gems/bosh_agent-1.5.0.pre.870/lib/bosh_agent/bootstrap.rb:61:in `load_settings'
2013-08-07_09:00:04.12514 from /var/vcap/bosh/lib/ruby/gems/1.9.1/gems/bosh_agent-1.5.0.pre.870/lib/bosh_agent/bootstrap.rb:35:in `configure'
2013-08-07_09:00:04.12515 from /var/vcap/bosh/lib/ruby/gems/1.9.1/gems/bosh_agent-1.5.0.pre.870/lib/bosh_agent.rb:106:in `start'
2013-08-07_09:00:04.12517 from /var/vcap/bosh/lib/ruby/gems/1.9.1/gems/bosh_agent-1.5.0.pre.870/lib/bosh_agent.rb:85:in `run'
2013-08-07_09:00:04.12519 from /var/vcap/bosh/lib/ruby/gems/1.9.1/gems/bosh_agent-1.5.0.pre.870/bin/bosh_agent:102:in `<top (required)>'
2013-08-07_09:00:04.12521 from /var/vcap/bosh/bin/bosh_agent:23:in `load'
2013-08-07_09:00:04.12523 from /var/vcap/bosh/bin/bosh_agent:23:in `<main>'
(...repeat)
2013-08-07_09:00:29.85468 #[1595] INFO: Starting agent 1.5.0.pre.870...
2013-08-07_09:00:29.85493 #[1595] INFO: Configuring agent...
2013-08-07_09:00:29.85715 #[1595] INFO: Configuring instance
2013-08-07_09:00:29.87382 #[1595] INFO: OpenStack user data: "{\"registry\":{\"endpoint\":\"http://admin:admin@localhost:25889\"},\"server\":{\"name\":\"vm-c6b01d4c-5fba-4680-ab63-011e3f055027\"}}"
2013-08-07_09:00:29.91755 #[1595] INFO: loaded new infrastructure settings: {"vm"=>{"name"=>"vm-c6b01d4c-5fba-4680-ab63-011e3f055027"}, "agent_id"=>"bm-53c86613-17e4-4410-9b1d-511292c9a839", "networks"=>{"bosh"=>{"cloud_properties"=>{}, "netmask"=>nil, "gateway"=>nil, "ip"=>nil, "dns"=>nil, "type"=>"dynamic", "default"=>["dns", "gateway"]}, "vip"=>{"ip"=>"10.192.36.23", "type"=>"vip", "cloud_properties"=>{}}}, "disks"=>{"system"=>"/dev/sda", "persistent"=>{}, "ephemeral"=>"/dev/sdb"}, "env"=>{"bosh"=>{"password"=>nil}}, "ntp"=>[], "blobstore"=>{"provider"=>"local", "options"=>{"blobstore_path"=>"/var/vcap/micro_bosh/data/cache"}}, "mbus"=>"
https://vcap:b00t...@0.0.0.0:6868"}
2013-08-07_09:00:29.91757 #[1595] INFO: Loaded settings: #<Bosh::Agent::Settings:0x000000017e4798 @settings={"vm"=>{"name"=>"vm-c6b01d4c-5fba-4680-ab63-011e3f055027"}, "agent_id"=>"bm-53c86613-17e4-4410-9b1d-511292c9a839", "networks"=>{"bosh"=>{"cloud_properties"=>{}, "netmask"=>nil, "gateway"=>nil, "ip"=>nil, "dns"=>nil, "type"=>"dynamic", "default"=>["dns", "gateway"]}, "vip"=>{"ip"=>"10.192.36.23", "type"=>"vip", "cloud_properties"=>{}}}, "disks"=>{"system"=>"/dev/sda", "persistent"=>{}, "ephemeral"=>"/dev/sdb"}, "env"=>{"bosh"=>{"password"=>nil}}, "ntp"=>[], "blobstore"=>{"provider"=>"local", "options"=>{"blobstore_path"=>"/var/vcap/micro_bosh/data/cache"}}, "mbus"=>"
https://vcap:b00t...@0.0.0.0:6868"}, @cache_file="/var/vcap/bosh/settings.json">
2013-08-07_09:00:29.92795 #[1595] WARN: no ntp-servers configured
2013-08-07_09:00:29.95687 #[1595] INFO: Found unformatted drive
2013-08-07_09:00:29.95689 #[1595] INFO: Partition /dev/vdb
2013-08-07_09:00:29.99912 Checking that no-one is using this disk right now ...
2013-08-07_09:00:30.00192 OK
2013-08-07_09:00:30.00250
2013-08-07_09:00:30.00250 sfdisk: ERROR: sector 0 does not have an msdos signature
2013-08-07_09:00:30.00254 /dev/vdb: unrecognized partition table type
2013-08-07_09:00:30.00262 No partitions found
2013-08-07_09:00:30.00279 Warning: no primary partition is marked bootable (active)
2013-08-07_09:00:30.00280 This does not matter for LILO, but the DOS MBR will not boot this disk.
2013-08-07_09:00:30.12010 If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
2013-08-07_09:00:30.12011 to zero the first 512 bytes: dd if=/dev/zero of=/dev/foo7 bs=512 count=1
2013-08-07_09:00:30.12011 (See fdisk(8).)
2013-08-07_09:00:30.12786 #[1595] INFO: Create swap and data partitions
2013-08-07_09:00:30.18795 mke2fs 1.41.11 (14-Mar-2010)
2013-08-07_09:00:32.56035 #[1595] INFO: Swapon partition /dev/vdb1
2013-08-07_09:00:32.68719 #[1595] INFO: Mount data partition /dev/vdb2 to /var/vcap/data
2013-08-07_09:00:32.90676 mke2fs 1.41.11 (14-Mar-2010)
2013-08-07_09:00:33.18994 chmod: cannot access `/dev/sr0': No such file or directory
2013-08-07_09:00:33.19721 chown: cannot access `/dev/sr0': No such file or directory
2013-08-07_09:00:33.34306 #[1595] INFO: Monit: /var/vcap/bosh/etc/monitrc:8: Warning: include files not found '/var/vcap/monit/job/*.monitrc'
2013-08-07_09:00:33.34396 #[1595] INFO: Monit: monit: generated unique Monit id ff535845c2dbe437f283265bd5423859 and stored to '/root/.
monit.id'
2013-08-07_09:00:33.34452 #[1595] INFO: Monit: Starting monit daemon with http interface at [
127.0.0.1:2822]
2013-08-07_09:00:34.26492 #[1595] INFO: Monit Service Connection Refused: retrying
2013-08-07_09:00:34.40108 #[1595] INFO: Starting up https agent
2013-08-07_09:00:35.62216 #[1595] INFO: Message processors: {"migrate_disk"=>Bosh::Agent::Message::MigrateDisk, "list_disk"=>Bosh::Agent::Message::ListDisk, "mount_disk"=>Bosh::Agent::Message::MountDisk, "unmount_disk"=>Bosh::Agent::Message::UnmountDisk, "state"=>Bosh::Agent::Message::State, "drain"=>Bosh::Agent::Message::Drain, "apply"=>Bosh::Agent::Message::Apply, "compile_package"=>Bosh::Agent::Message::CompilePackage, "fetch_logs"=>Bosh::Agent::Message::FetchLogs, "ssh"=>Bosh::Agent::Message::Ssh, "ping"=>Bosh::Agent::Message::Ping, "noop"=>Bosh::Agent::Message::Noop, "start"=>Bosh::Agent::Message::Start, "stop"=>Bosh::Agent::Message::Stop, "prepare_network_change"=>Bosh::Agent::Message::PrepareNetworkChange, "release_apply_spec"=>Bosh::Agent::Message::ReleaseApplySpec}
2013-08-07_09:00:36.09081 >> Thin web server (v1.5.1 codename Straight Razor)
2013-08-07_09:00:36.09157 >> Maximum connections set to 1024
2013-08-07_09:00:36.09161 >> Listening on
0.0.0.0:6868, CTRL+C to stop
2013-08-07_09:00:36.23581 10.193.118.30 - - [07/Aug/2013 09:00:36] "POST HTTP/1.1" 401 - 0.0002
2013-08-07_09:00:36.24907 10.193.118.30 - vcap [07/Aug/2013 09:00:36] "POST HTTP/1.1" 404 507 0.0009
2013-08-07_09:00:37.36632 10.193.118.30 - - [07/Aug/2013 09:00:37] "POST HTTP/1.1" 401 - 0.0002
2013-08-07_09:00:37.37926 10.193.118.30 - vcap [07/Aug/2013 09:00:37] "POST HTTP/1.1" 404 507 0.0006
2013-08-07_09:00:38.49966 10.193.118.30 - - [07/Aug/2013 09:00:38] "POST HTTP/1.1" 401 - 0.0001
2013-08-07_09:00:38.51157 10.193.118.30 - vcap [07/Aug/2013 09:00:38] "POST HTTP/1.1" 404 507 0.0006
(repeat ..)