Error 400007: `ccdb_postgres/0' is not running after update

[ramonm...@gmail.com]

finally after some network difficulties
all vm's are starting

and now i get the following error
and i really don't know where to look

this is a small part ofmy template.yml

Jobs:
- name: ccdb_postgres
  template: postgres
  instances: 1
  resource_pool: infrastructure
  networks:
  - name: default
    default: [dns, gateway]
  - name: static
    static_ips:
    - 172.24.4.231
  properties:
    db: ccdb

Properties:
  ccdb:
    address: 172.24.4.231
    port: 5524
    pool_size: 10
    roles:
    - tag: admin
      name: ccadmin
      password: aaaBauWauZZb2
    databases:
    - tag: cc
      name: cloudfoundry

also i have the following security group just in case

open
        ALLOW 1:65535 from 0.0.0.0/0


but i keep getting the error

Binding instance VMs
  dea/0 (00:00:01)                                                                                 
  cloud_controller/0 (00:00:01)                                                                    
  vcap_redis/0 (00:00:01)                                                                          
  stager/0 (00:00:01)                                                                              
  services_redis/0 (00:00:01)                                                                      
  ccdb_postgres/0 (00:00:01)                                                                       
  router/0 (00:00:01)                                                                              
  nats/0 (00:00:01)                                                                                
  health_manager/0 (00:00:01)                                                                      
  mysql_node/0 (00:00:01)                                                                          
  mysql_gateway/0 (00:00:01)                                                                       
Done                    11/11 00:00:01                                                             

Preparing configuration
  binding configuration (00:00:00)                                                                 
Done                    1/1 00:00:00                                                               

Updating job nats
  nats/0 (canary) (00:00:21)                                                                       
Done                    1/1 00:00:21                                                               

Updating job ccdb_postgres
  ccdb_postgres/0 (canary) (00:01:39)                                                              
Done                    1/1 00:01:39                                                               

Error 400007: `ccdb_postgres/0' is not running after update

[Martin Englund]

You need to log on to the VM and check the logs for clues to why it
didn't start. You can get the IP by running:
bosh vms

log on, and then check the logs in:
/var/vcap/sys/log

/M

--
cheers,
/Martin
--
Martin Englund, Staff Engineer, Cloud Foundry, VMware Inc.
"The question is not if you are paranoid, it is if you are paranoid enough."

[ramonm...@gmail.com]

thanks for that info :)

Removing stale pidfile...
kernel.shmmax = 284934144
ERROR: storage directory doesn't exist
Please add persistent disk to this job
Removing stale pidfile...
kernel.shmmax = 284934144
ERROR: storage directory doesn't exist

but the root and Ephemeral Disk are 20GB



Op vrijdag 5 oktober 2012 16:13:48 UTC+2 schreef ramonm...@gmail.com het volgende:

[Martin Englund]

Ah, you don't have a persistent disk mounted (on /var/vcap/store). Add:
persistent_disk: <disk size in MB>
to the ccdb_postgres job.

You are going to need persistent disks for more jobs, so make sure you
follow the sample deployment manifest:
https://github.com/cloudfoundry/oss-docs/blob/master/bosh/samples/cloudfoundry.yml

/M

[ramonm...@gmail.com]

persistent disks is something for vsphere right?
because i'm using openstack

Op maandag 8 oktober 2012 16:05:33 UTC+2 schreef Martin Englund het volgende:

[Martin Englund]

Nope! A persistent disk is a disk that is bound to a specific job (and
index) which gets reattached if the vm is recreated (hence persistent)

The cloud specific stuff is specified in the cloud_properties section.
The persistent disk is part of the job, not part of the
resource_pools.

/M

[ramonm...@gmail.com]

ah okay i understand
but what do i need to provide in the resource pool for the openstack cpi
is there a list of properties specific for the bosh/openstack cpi?

Op vrijdag 5 oktober 2012 16:13:48 UTC+2 schreef ramonm...@gmail.com het volgende:

finally after some network difficulties

[Martin Englund]

Nope, unfortunately not - you'll have to use the source:
https://github.com/piston/openstack-bosh-cpi/blob/master/lib/cloud/openstack/cloud.rb

You'll have to ask Ferdy for OpenStack specific stuff - I've just been
working on vSphere & AWS...

/M

[Ferran Rodenas]

Just add a persistent_disk option at the jobs section:

Jobs:
- name: ccdb_postgres
  template: postgres
  instances: 1
  resource_pool: infrastructure

  persistent_disk: 2048

  networks:
  - name: default
    default: [dns, gateway]
  - name: static
    static_ips:
    - 172.24.4.231
  properties:
    db: ccdb

- Ferdy

2012/10/8 Martin Englund <meng...@vmware.com>

[ramonm...@gmail.com]

ahhhh i see
so you need a flavor with a Ephemeral Disk like say 2G

and then the persistent_disk: 2048
the persistent is disk is the ephermal disk

thanks

Op maandag 8 oktober 2012 19:42:41 UTC+2 schreef ferdy het volgende:

[Ferran Rodenas]

No, the ephemeral disk is always attached to the vm when is started (it's part of the vm image: /dev/vda is the root device and /dev/vdb is the ephemeral (free) space you configured at the OpenStack flavor spec), but it will be destroyed whenever the vm is stopped. The persistent disk is optional (only required by some packages), and it attaches an OpenStack volume to the vm (usually /dev/vdc). 

So when you specify a persistent_disk option, the CPI will create an OpenStack volume and attach it to the vm. If for some reason the vm is stopped (i.e. redeploy a new release version), the director will start a new vm and attach to it the old persistent disk, so you won't lost any data.

An example could be a release that contains mysql: the mysql instance is deployed in a vm, all the binaries and configuration data are stored in the ephemeral disk, but the DB data can be configured to be stored in the persistent disk. If you redeploy the release (i.e. changing some mysql conf files), director will destroy the vm and create a new one, then it'll deploy the mysql server with the new configuration on the new vm (in the ephemeral disk), and it will attach the old persistent disk, so any data you had previously it won't be destroyed.

Hope it helps.

- Ferdy

2012/10/9 <ramonm...@gmail.com>

[ramonm...@gmail.com]

oka thanks after some trial and error i know how it works now with the persistent disk part

but still can't get ccdb_postgress running
/var/vcap/sys/log/monit/postgres_ctl.err.log show

pg_ctl: could not start server
Examine the log output.

but there are no more log files in this directory


Op dinsdag 9 oktober 2012 15:04:09 UTC+2 schreef ferdy het volgende:

[Peter Le]

I ran into the same issue with mysql.

Error 400007: `mysql/0' is not running after update

I fixed it by rebooting the mysql VM.

Peter

[Ferran Rodenas]

Check /var/vcap/store/postgres/pg_log/startup.log to see if there's any error.

- Ferdy

2012/10/10 <ramonm...@gmail.com>

[ramonm...@gmail.com]

i' have some seriously strange problems
the postgresql can't listen on the  vip adres it was assigned to 172.24.4.227  (default devstack gives away)

i can ping the ip adres
nslookup results in
Server:        10.0.0.1
Address:    10.0.0.1#53

** server can't find 227.4.24.172.in-addr.arpa.: NXDOMAIN

and there is another strange problem
on every vm where there are 2 networks assigned
the default one and the static

- name: ccdb_postgres
  template: postgres
  instances: 1
  resource_pool: infrastructure
  persistent_disk: 2048
  networks:
  - name: default
    default: [dns, gateway]

  - name: elastic
    static_ips:
    - 172.24.4.227

i can't seem to ping/resolve anything on the WWW
but i can ping all internal vm's

so i thought there was something with my network
but on the vms that where created with bosh had only one network

- name: stager
  template: stager

  instances: 1
  resource_pool: infrastructure
  networks:
  - name: default

can ping to WWW and works fine..
so i think BOSH is doing something but not on the vm it self as it seems
because on the vm the resolv.conf / route -n etc etc are all the same

Op woensdag 10 oktober 2012 21:02:04 UTC+2 schreef ferdy het volgende:

[ramonm...@gmail.com]

finaly i solved my problem :)

i needed to change the jobs/postgres/templates/postgresql.conf.erb
listen_addresses = '<%= db.address %>'
to
listen_addresses = '*'

and for jobs/postgres/templates/postgres_ctl.erb
HOST="<%= db.address %>"
to
#HOST="*"



Op donderdag 11 oktober 2012 10:32:08 UTC+2 schreef ramonm...@gmail.com het volgende: