Issues while deploying Cloud Foundry on OpenStack via BOSH

[Guido Davide Dall'Olio]

Hi guys,

I'm pretty new here and I hope to write in the right place.
First of all, I'm a Cloud Foundry user and currently I'm modifying it. I'm using a single machine installation, obtained via cf_nise_installer.
Now the big jump, install Cloud Foundry on OpenStack with BOSH
I want to share with you my configuration and settings in order to understand better why I get that error. Sorry for the long post.

Server machine:
- 16 CPUs, 64GB ram, 500GB hdd.
- server reachable via external ip

-network interfaces: -br100: ip 10.11.12.1 netmask: 255.255.255.0 Bcast 10.11.12.255

                             -eth0: access to internet

                             -virbr0: ip 192.168.122.1 netmask: 255.255.255.0 Bcast 192.168.122.255

Installed OpenStack grizzly/stable via Devstack (I know it's not really installed, but since I do not unstack it, it will work).
Here attached there's my Devstack localrc file.
I removed API rate limit here (http://docs.openstack.org/grizzly/openstack-compute/admin/content/configuring-compute-API.html) and set variable API_RATE_LIMIT to false.
I can create Volumes bigger than 30GB, as I set VOLUME_BACKING_FILE_SIZE accordingly. Horizon is working on port 80 perfectly, reachable from external network.
Floating range ips: FLOATING_RANGE=192.168.124.224/27 , internal instance ips FIXED_RANGE=10.11.12.0/24

I can spawn instances, ping them, ssh them on internal ip (10.11.12.x) and on floating ip range(192.168.x.x) from server (instances are not visible from external net).
So OpenStack should be ok.

I installed microbosh following these steps: http://ryangrenz.com/blog/2013/06/11/installing-cf-release-onto-openstack-with-bosh/
If I check bosh status, this is the output:

Config

             /home/guidodavide/.bosh_config

Director

  Name       firstbosh

  URL        https://192.168.124.225:25555

  Version    1.5.0.pre.1193 (release:31de3413 bosh:31de3413)

  User       admin

  UUID       randomUUID

  CPI        openstack

  dns        enabled (domain_name: microbosh)

  compiled_package_cache disabled

  snapshots  disabled

Deployment

  Manifest   /home/guidodavide/cf-open.yml

Release

  dev        FirstCF/145.1-dev

  final      cf/145

As you can see BOSH version is 1.5.0.pre.1193
On openstack all Security Groups are set correctly for Cloud Foundry.
Bosh create release WORKED
Bosh upload release WORKED (using Cloud Foundry release candidate branch)

Bosh upload stemcell WORKED (I'm using bosh-stemcell-latest-openstack-kvm-ubuntu.tgz)

Bosh deployment WORKED
But whenever I try to "BOSH DEPLOY" I get some errors.
I'm pretty sure I made some mistakes on the deployment file. I'm using the one provided by Cloud Foundry documentation. I set UUID and ip accordingly (but probably network configuration is wrong)
I need help with that deployment file.

When I try to execute the deploy, this is what happens:
- machine becomes unreachable as the switch's port is submerged by many DHCP requests and it shuts down.
- when I get back to machine BOSH failed deployment with "waiting for agent to get task" or a timeout error.

Attached you have DEBUG log of BOSH deployment task.
I'm pretty sure I there are some errors in BOSH deployment file, I tried to understand how it works, but I couldn't find any help on internet.
I think I'm really close to a working CF on OpenStack, but I don't get how that deployment file has to be set to work on OpenStack.

Any help is really appreciated

bosh-status: http://pastebin.com/xBfTisme
devstack localrc: http://pastebin.com/iJYuUybe

bosh task deploy debug and error: https://drive.google.com/file/d/0B8RAaCyPs8aVVHhYNEl1VDg1UWM

[Dr Nic Williams]

If you can also pastebin your manifests then I can read them on my iPhone over breakfast. The drive doc wants me to download something. Else I'll try to look later in the day.

On Fri, Nov 1, 2013 at 7:53 AM, Guido Davide Dall'Olio <guidodavid...@gmail.com> wrote:

BOSH deployment file
https://drive.google.com/file/d/0B8RAaCyPs8aVaUdQOWcyVk5HR3c

To unsubscribe from this group and stop receiving emails from it, send an email to bosh-users+...@cloudfoundry.org.

[Guido Davide Dall'Olio]

BOSH deployment file:
http://pastebin.com/dLdYpJtJ

[Dr Nic Williams]

Thanks, also the task log too. Sorry didn't notice you'd google drive'd that also.

On Fri, Nov 1, 2013 at 7:59 AM, Guido Davide Dall'Olio <guidodavid...@gmail.com> wrote:

BOSH deployment file:
http://pastebin.com/dLdYpJtJ

To unsubscribe from this group and stop receiving emails from it, send an email to bosh-users+...@cloudfoundry.org.

[Ferran Rodenas]

I recommend you to also take a look at the cloudfoundry.org docs, the OpenStack guides should be up to date.

If the machine cannot support too many network calls, one thing you can try is to reduce the number of concurrent Bosh calls. Look at the apply spec properties section at the Deploying microBOSH guide, you can set the number of concurrent threads Micro BOSH director will use to perform some actions at the director.max_threads property.

- Ferdy

2013/11/1 Dr Nic Williams <drnicw...@gmail.com>

[Guido Davide Dall'Olio]

I installed devstack with neutron/quantum. Now I'm going to try manual configuration, as with nova-network doesn't work.

2013/11/1 Ferran Rodenas <frod...@gmail.com>

[ramon makkelie]

well we use openstack with nova flat-networking just fine.

i have seen someone who had a timeout issue with creating the vm on openstack before
but i could not remember if he had fixed it or not.

as i can see in your logs a few vms are starting and are reachable so i think its a limit configuration issue in devstack
devstack is not the best option to use for trying the full features of openstack
i also had a lot of problems with it

here is a good vm sandbox guide
https://github.com/dguitarbite/OpenStack-Grizzly-VM-SandBox-Guide


Op zaterdag 2 november 2013 18:57:40 UTC+1 schreef Guido Davide Dall'Olio:

[Jeremy Budnack]

I'm not sure if this helps, but I've had trouble with DHCP storms on nova-network, where /var/log/messages is full of DHCPREQESTs, DHCPACKs every few seconds from the same vm's over and over.  Usually, I'd suspect a competing DHCP server on your network, but that wasn't the culprit in this case.  What ended up stopping it was killing all dnsmasq processes before restarting nova-network.

What happens if you do the same before restarting quantum/neutron?