bosh deployed VMs get assigned the same IP address

[chinc...@gmail.com]

Hi,

I am trying to deploy redis & postgres on two different nodes using two deployment descriptors. I have copy-pasted the networks & resource_pools sections into both the deployment descriptors. When I deploy redis it gets IP: 10.10.128.7 & when I deploy postgres next it uses the same 10.10.128.7 & mucks up the deployment. I am assuming that mentioning type: manual causes bosh to manage the IPs. What am I doing wrong ?

postgres.yml
---
networks:
- name: default
  type: manual
  subnets:
  - range: 10.10.128.0/24
    reserved:
    - 10.10.128.20 - 10.10.128.254
    static:
    - 10.10.128.2 - 10.10.128.5
    gateway: 10.10.128.1
    cloud_properties:
      name: VM Network
..
...
jobs:
- name: pg
  templates:

  - name: postgres
    release: latest

  instances: 1
  resource_pool: common
  networks:
  - name: default
    default: [dns, gateway]

===
redis.yml
---
networks:
- name: default
  type: manual
  subnets:
  - range: 10.10.128.0/24
    reserved:
    - 10.10.128.20 - 10.10.128.254
    static:
    - 10.10.128.2 - 10.10.128.5
    gateway: 10.10.128.1
    cloud_properties:
      name: VM Network
..
...
jobs:
- name: redis
  templates:

  - name: redis
    release: latest

  instances: 1
  resource_pool: common
  networks:
  - name: default
    default: [dns, gateway]

Thanks,
-C

[Dr Nic Williams]

The bosh deployments do not know that other deployments (or external orchestrators) are trying to use the same IPs. Your deployments need to be explicitly non-intersecting/overlapping in their IP ranges so different IPs are allocated.

Yep, would be cool if bosh knew it had already allocated IPs to other VMs in different deployments.

To unsubscribe from this group and stop receiving emails from it, send an email to bosh-users+...@cloudfoundry.org.

[Chinchu Sup]

Thanks Dr Nic, that explains it.

[dmi...@pivotallabs.com]

Do you guys run into this a lot -- sharing same subnet for multiple deployments? 

Recently I've seen a few people mention this which begs a question: should there be some kind of shared network definitions within a single BOSH Director so that multiple deployments are aware of overlapping IPs. If so what if multiple BOSH Directors were given same subnet -- is that a scenario you had to deal with? 

-dk

[Chinchu Sup]

Yes. I was planning on having each of us in my team do a bosh deploy (with the same deployment descriptor differing only in the name) from their desktops (with the microbosh hosting our release) to get our own VMs with our product + hadoop/hbase + other dependencies installed (& upgraded) seamlessly. I also felt that we could have our customers install our product similarly.

I originally wanted to use dynamic addresses, but that didn't work (Ferran mentioned that the vsphere CPI didn't probably support it). Then I tried this & it failed too..

I haven't gone as far as using multiple BOSH directors.. but if I do then I expect them to share the subnet nicely :-)

[James Masson]

Both CF London and CloudCredo run in to this a lot.

We often are tenants on address space that isn't under our direct
control. eg. 4+ BOSH directors sharing a /21 network.

This is often the case in corporate environments, where network
allocation is not as straightforward as you might like.

BOSH-Director keeping track of allocated IPs across deployments would be
a useful feature.

regards

James M

> mentioning *type: manual* causes bosh to manage the IPs.

> What am I doing wrong ?
>
> postgres.yml
> ---
> networks:
> - name: default
> type: manual
> subnets:

> - range: 10.10.128.0/24 <http://10.10.128.0/24>

> - range: 10.10.128.0/24 <http://10.10.128.0/24>

> reserved:
> - 10.10.128.20 - 10.10.128.254
> static:
> - 10.10.128.2 - 10.10.128.5
> gateway: 10.10.128.1
> cloud_properties:
> name: VM Network
> ..
> ...
> jobs:
> - name: redis
> templates:
>
> - name: redis
> release: latest
>
> instances: 1
> resource_pool: common
> networks:
> - name: default
> default: [dns, gateway]
>
> Thanks,
> -C
>
> To unsubscribe from this group and stop receiving emails
> from it, send an email to bosh-users+...@cloudfoundry.org

> <javascript:>.

>
>
>
> To unsubscribe from this group and stop receiving emails from it, send
> an email to bosh-users+...@cloudfoundry.org

> <mailto:bosh-users+...@cloudfoundry.org>.

[Alan Morán]

I agree, It would be nice if bosh knew what exist already on the network, the only problem here is that there might exist nodes that were not deployed by bosh, or at least not by the same bosh. This makes it difficult to keep track of the available and unavailable IPs. 

Fortunately with openstack this is not a problem if you have DHCP but I can see how it can get pretty annoying in vsphere.

Anyone found a clean way of tracking this when dynamic networks are not an option?

- - - -

Alan Moran

Altoros — Cloud Foundry deployment, training and integration
http://bonzofenix.com/ 

To unsubscribe from this group and stop receiving emails from it, send an email to bosh-users+...@cloudfoundry.org.

[gwenn.e...@rakops.com]

Yes I agree too, can be nice if bosh can know other deployment IP use, actually I need to restrict all ip use everytime, not really straightforward.

Gwenn

<mailto:bosh-users+unsub...@cloudfoundry.org>.

[Johannes Hiemer]

I think this a nearly impossible task. Getting around in large IaaS environments lately, I saw a high number of segmented network infrastructures. It would take unnecessary resources and time for bosh to keep track of available IPs. I still see it as your duty to be aware of the segments the ip allocations.