Received: by 10.50.161.167 with SMTP id xt7mr6074239igb.0.1346720807347; Mon, 03 Sep 2012 18:06:47 -0700 (PDT) X-BeenThere: bosh-...@cloudfoundry.org Received: by 10.50.178.73 with SMTP id cw9ls758448igc.4.canary; Mon, 03 Sep 2012 18:06:46 -0700 (PDT) Received: by 10.50.47.167 with SMTP id e7mr5138518ign.1.1346720806666; Mon, 03 Sep 2012 18:06:46 -0700 (PDT) Received: by 10.50.47.167 with SMTP id e7mr5138517ign.1.1346720806646; Mon, 03 Sep 2012 18:06:46 -0700 (PDT) Return-Path: Received: from mail-iy0-f169.google.com (mail-iy0-f169.google.com [209.85.210.169]) by mx.google.com with ESMTPS id eh1si17814647icb.30.2012.09.03.18.06.46 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 03 Sep 2012 18:06:46 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.210.169 is neither permitted nor denied by best guess record for domain of vspi...@rbcon.com) client-ip=209.85.210.169; Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.210.169 is neither permitted nor denied by best guess record for domain of vspi...@rbcon.com) smtp.mail=vspi...@rbcon.com Received: by iagk10 with SMTP id k10so9531835iag.0 for ; Mon, 03 Sep 2012 18:06:46 -0700 (PDT) d=google.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :x-gm-message-state; bh=dASbfjjoowlj9T/YTP6JpkdnAgtmYuTCQHVvw/E4+4A=; b=AoGfU7zQnW7dmIrzaHTDqmHXN5RGu8hFBPin4y6ZLblQZ2hIMsVpcOWkFuvNeXFYMV pBiVGRFLfsOEScIG4aWQcEVHLxGupQseUg0m8eiW6Np0FdgEkidTfQ2otk+kCAwkMZbP G+C0Ztfae4SbAYKip10NM3qWAm1vlKMslwT7x9pTOzYIvE08mMHJbemcdsBgSDkjDBut wTrrJhhozzfevC/l5ZmaWYe9hPNV5fJMxs4LJ1ET5WJgLRgp7t1bFL76nSkCuOeviyOB 00/0SNlVhSyEdBsOtAzIZlBFHfe5yjEqgy6Vqk1pqktuYz6QgMPl6G1ZECezfNIX3xNi Rp/A== MIME-Version: 1.0 Received: by 10.50.169.39 with SMTP id ab7mr4795729igc.8.1346720805848; Mon, 03 Sep 2012 18:06:45 -0700 (PDT) Sender: vspi...@rbcon.com Received: by 10.50.242.74 with HTTP; Mon, 3 Sep 2012 18:06:45 -0700 (PDT) In-Reply-To: References: <8a4d6f5a-ba2e-46e1-865a-3c0c9bfc5...@cloudfoundry.org> <5044639E.8020...@active.by> <9a498dae-65ea-44c4-bce4-4d40aa2bf...@cloudfoundry.org> <8a8f0daf-a853-41d5-9d5c-2ea4f1e24...@cloudfoundry.org> Date: Mon, 3 Sep 2012 18:06:45 -0700 Message-ID: Subject: Re: [bosh-dev] Tutorial on how to add a new CPI? From: Vadim Spivak To: bosh-...@cloudfoundry.org Content-Type: multipart/alternative; boundary=e89a8f3ba84b3e872404c8d5dffe X-Gm-Message-State: ALoCoQkFkRd7hFL952PYR99P/MnP5SaA12zVvN8KEBNbiEADOXBcquqplCCMCZMxqFxtdsYvzo41 --e89a8f3ba84b3e872404c8d5dffe Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Yes, you have to customize the agent to understand this new CPI and in your case use a registry. The stemcell has the agent baked in, hence any change to the agent will require a new stemcell. -Vadim On Mon, Sep 3, 2012 at 2:59 PM, Mike wrote: > Once the template is uploaded it becomes ready to be deployed to the ESX > using its API. I parse the configuration of storage and network and make > the appropriate calls to the API (my tool to deploy). > > But I guess I need to add the registry to the VM? and theres where I need > to create my stemcells? > > El lunes, 3 de septiembre de 2012 23:54:52 UTC+2, Vadim Spivak escribi=F3= : >> >> I think the answer is yes, since today's stemcell builder only works for >> vSphere (ESX) working as described earlier and AWS (AMI/AKI/pv-grub), as >> well as OpenStack (KVM). >> >> Can you describe step 3 in detail? All of this complexity is due to how >> to set the configuration of the VM in a way that's mutable and secure. >> >> -Vadim >> >> On Mon, Sep 3, 2012 at 2:52 PM, Mike wrote: >> >>> It actually works like AWS/OpenStack. My CPI connects to an API: >>> >>> 1) uploads the stemcell (OVF and VMDK) would be sufficient >>> 2) returns the id of this template >>> 3) creates a VM out of the stemcell and sets the configuration >>> 4) VM boots >>> >>> My question was if I need to create a special stemcell if my >>> infrastructure is on ESXi. >>> >>> Thanks for your help. >>> >>> El lunes, 3 de septiembre de 2012 23:47:26 UTC+2, Vadim Spivak escribi= =F3: >>>> >>>> It depends on what your CPI will do, if it follows the contract of the >>>> current vSphere CPI in how it provides an ISO with the specified meta = data >>>> then the answer is no. However, if you're planning on doing something = else >>>> then you'll have to write code to implement the initial bootstrap. >>>> >>>> The vSphere workflow is as follows: >>>> >>>> 1) BOSH calls vSphereCPI.create_vm >>>> 2) vSphereCPI creates a new VM (A) >>>> 3) vSphereCPI creates an ISO with a single JSON file containing the >>>> network configuration, passwords, NTP address, and other bootstrap >>>> configuration >>>> 4) vSphereCPI attaches this ISO to VM (A) >>>> 3) VM (A) boots and tries to mount this ISO to configure itself >>>> >>>> on AWS/OpenStack it works slightly different >>>> >>>> 1) BOSH calls OpenStackCPI.create_vm >>>> 2) OpenStackCPI creates a new VM (A) >>>> 3) OpenStackCPI adds an entry to the OpenStack Registry with the >>>> instance ID and any required bootstrap configuration >>>> 4) VM (a) boots, uses DHCP, and queries the user meta data API for the >>>> registry end point >>>> 5) Using the registry end point it queries the registry to fetch >>>> additional bootstrap configuration (passwords, NTP, etc) >>>> >>>> Hope these examples help. >>>> >>>> -Vadim >>>> >>>> >>>> On Mon, Sep 3, 2012 at 2:30 PM, Mike wrote: >>>> >>>>> Hi all, >>>>> >>>>> These guidelines look exactly what I was looking for. Thank you. >>>>> >>>>> One question thou, if I can deploy on ESX do I need to implement the >>>>> stemcell builder? Can I reuse the one for vsphere? >>>>> >>>>> Thanks. >>>>> >>>>> Mike. >>>>> >>>>> El lunes, 3 de septiembre de 2012 20:45:45 UTC+2, Vadim Spivak >>>>> escribi=F3: >>>>>> >>>>>> It depends on the CPI, for vSphere there is no registry since we use >>>>>> an ISO for passing data to the VM. For AWS and OpenStack that doesn'= t exist >>>>>> so we use a registry in conjunction with the instance meta-data API >>>>>> provided by the IaaS. >>>>>> >>>>>> The best way to add a new CPI is to follow the example of the >>>>>> previous contribution, e.g. OpenStack CPI. >>>>>> >>>>>> https://github.com/**cloudfoundr****y/bosh/commit/**99a767415eafdbf6= * >>>>>> ***e8184f60f143b8**55366cdb90(agent, needed for bootstrap= ping/configuring the VM when it starts) >>>>>> https://github.com/**cloudfoundr****y/bosh/commit/**05073e7cd4188506= * >>>>>> ***cebcb99efc6d81**86f3498918 (stemcell builder, used to = automate the creation of VM stemcells - >>>>>> templates) >>>>>> https://github.com/**cloudfoundr****y/bosh/commit/**6dcee09c2453a3f5= * >>>>>> ***15503898c5ecba**9c8807c11a(deployer, used for self hos= ting BOSH) >>>>>> >>>>>> The CPI itself is checked in here: >>>>>> >>>>>> https://github.com/piston/**open****stack-bosh-cpi >>>>>> >>>>>> And the only references of it are as follows: >>>>>> >>>>>> https://github.com/**cloudfoundr****y/bosh/blob/master/**director/** >>>>>> Ge**mfile#L11 >>>>>> https://github.com/**cloudfoundr****y/bosh/blob/master/**deployer/** >>>>>> bo**sh_deployer.**gemspec#L22 >>>>>> >>>>>> We're going to need to revamp the plugin system for the CPI so >>>>>> they're provided at runtime instead of requiring them all to be list= ed as >>>>>> dependencies. We'll have to see what's the best way of doing this an= d still >>>>>> play nice with Bundler. >>>>>> >>>>>> -Vadim >>>>>> >>>>>> On Mon, Sep 3, 2012 at 11:35 AM, Mike wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I cannot disclose which API, but it is an internal tool. >>>>>>> >>>>>>> The CPI itself works since I can run it from the 'irb' console. I >>>>>>> also developed a registry, I can see in the log: >>>>>>> >>>>>>> I, [2012-09-03T18:30:17.411160 #17797] [0x100c818] INFO -- : >>>>>>> my_registry is ready on port 25888 >>>>>>> I, [2012-09-03T18:30:32.363929 #17797] [0x100c818] INFO -- : >>>>>>> Loading yaml from /tmp/sc-20120903-17797-**1jpsh3q**** >>>>>>> /apply_spec.yml >>>>>>> I, [2012-09-03T18:30:32.366022 #17797] [0x100c818] INFO -- : >>>>>>> Loading yaml from /tmp/sc-20120903-17797-**1jpsh3q****/stemcell.MF >>>>>>> >>>>>>> But nothing is outputted from the CPI nor a .bosh_error is created. >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> Mike. >>>>>>> >>>>>>> El lunes, 3 de septiembre de 2012 10:00:34 UTC+2, Ruslan K escribi= =F3: >>>>>>>> >>>>>>>> Hello! >>>>>>>> >>>>>>>> See this thread: >>>>>>>> https://groups.google.com/a/**cl******oudfoundry.org/forum/?** >>>>>>>> fromgrou******ps=3D#!topic/bosh-dev/**vqu_**uqdb8W****o >>>>>>>> >>>>>>>> What interface do you want to add? >>>>>>>> >>>>>>>> On 09/02/2012 02:44 PM, impres...@gmail.com wrote: >>>>>>>> > Hi all, >>>>>>>> > >>>>>>>> > I'm thinking in adding a new CPI to BOSH. But after reading the >>>>>>>> > documentation I have some doubts. >>>>>>>> > >>>>>>>> > Do I have to implement anything aside the cloud.rb itself and th= e >>>>>>>> files >>>>>>>> > needed by the CPI? I mean things like the registry. >>>>>>>> > >>>>>>>> > Once I implement the CPI, what configuration is needed in BOSH t= o >>>>>>>> not >>>>>>>> > see that 'Could not find Cloud Provider Plugin: ' in .bosh_error= ? >>>>>>>> > >>>>>>>> > Is there any tutorial I can have a look at? >>>>>>>> > >>>>>>>> > Thanks very much! >>>>>>>> > >>>>>>>> >>>>>>> >>>>>> >>>> >> --e89a8f3ba84b3e872404c8d5dffe Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Yes, you have to customize the agent to understand this new CPI and in your= case use a registry. The stemcell has the agent baked in, hence any change= to the agent will require a new stemcell.

-Vadim

On Mon, Sep 3, 2012 at 2:59 PM, Mike <impresenta...@gmail.com> wrote:
Once the template is uploaded it becomes ready to be deployed to the ESX us= ing its API. I parse the configuration of storage and network and make the = appropriate calls to the API (my tool to deploy).

But I = guess I need to add the registry to the VM? and theres where I need to crea= te my stemcells?

El lunes, 3 de septiembre de 2012 23:54:52 UTC+2, Vadim Spiva= k escribi=F3:
I t= hink the answer is yes, since today's stemcell builder only works for v= Sphere (ESX) working as described earlier and AWS (AMI/AKI/pv-grub), as wel= l as OpenStack (KVM).

Can you describe step 3 in detail? All of this complexity is= due to how to set the configuration of the VM in a way that's mutable = and secure.

-Vadim

On Mon, Sep 3, 2012 at 2:52 PM, Mike <impres...@gmail.com> wrote:
It actually works like AWS/OpenStack. My CPI= connects to an API:

1) uploads the stemcell (OVF and VM= DK) would be sufficient
2) returns the id of this template
3) creates a VM out of th= e stemcell and sets the configuration
4) VM boots

<= /div>
My question was if I need to create a special stemcell if my infr= astructure is on ESXi.

Thanks for your help.

El lunes, 3 de septiembre = de 2012 23:47:26 UTC+2, Vadim Spivak escribi=F3:
It depends on what your CPI will do, if it follows the contract of the= current vSphere CPI in how it provides an ISO with the specified meta data= then the answer is no. However, if you're planning on doing something = else then you'll have to write code to implement the initial bootstrap.=

The vSphere workflow is as follows:

1) BOSH calls vSphereCPI.create_vm
2) vSphereCPI creates a new V= M (A)
3) vSphereCPI creates an ISO with a single JSON file contai= ning the network configuration, passwords, NTP address, and other bootstrap= configuration
4) vSphereCPI attaches this ISO to VM (A)
3) VM (A) boots an= d tries to mount this ISO to configure itself

on A= WS/OpenStack it works slightly different

1) BOSH c= alls OpenStackCPI.create_vm
2)=A0OpenStackCPI creates a new VM (A)
3)=A0OpenStackCPI add= s an entry to the OpenStack Registry with the instance ID and any required = bootstrap configuration
4) VM (a) boots, uses DHCP, and queries t= he user meta data API for the registry end point
5) Using the registry end point it queries the registry to fetch addit= ional bootstrap configuration (passwords, NTP, etc)

Hope these examples help.

-Vadim


On Mon, Sep 3, 2012 at 2:30 PM, Mike <impres...@gma= il.com> wrote:
Hi all,

These guidelines look exactly what I was looking= for. Thank you.

One question thou, if I can deplo= y on ESX do I need to implement the stemcell builder? Can I reuse the one f= or vsphere?

Thanks.

Mike.

El lunes,= 3 de septiembre de 2012 20:45:45 UTC+2, Vadim Spivak escribi=F3:
It depends on the CPI, for vSphere there is no registry since we use a= n ISO for passing data to the VM. For AWS and OpenStack that doesn't ex= ist so we use a registry in=A0conjunction=A0with the instance meta-data API= provided by the IaaS.=A0

The best way to add a new CPI is to follow the example of th= e previous contribution, e.g. OpenStack CPI.=A0

https://github.com/cloudfoundr= y/bosh/commit/99a767415eafdbf6e8184f60f1= 43b855366cdb90 (agent, needed for bootstrapping/configuring the = VM when it starts)
https://github.com/clou= dfoundry/bosh/commit/05073e7cd4188506ceb= cb99efc6d8186f3498918 =A0(stemcell builder, used to automate the= creation of VM stemcells - templates)
https://github.com/clou= dfoundry/bosh/commit/6dcee09c2453a3f5155= 03898c5ecba9c8807c11a (deployer, used for self hosting BOSH)

The CPI itself is checked in here:

=
https://github.com/piston/openstack-bosh-cpi<= /div>

And the only references of it are as follows:=A0


We're going to need to revamp the plugin system for= the CPI so they're provided at runtime instead of requiring them all t= o be listed as dependencies. We'll have to see what's the best way = of doing this and still play nice with Bundler.

-Vadim

On Mon, Sep 3, 2012 at 11:35 AM, Mike <imp= res...@gmail.com> wrote:
Hi,

I cannot disclose whi= ch API, but it is an internal tool.

The CPI itself= works since I can run it from the 'irb' console. I also developed = a registry, I can see in the log:

I, [2012-09-03T18:30:17.411160 #17797] [0x100c818]= =A0INFO -- : my_registry is ready on port 25888
I, [2012-09-03T1= 8:30:32.363929 #17797] [0x100c818] =A0INFO -- : Loading yaml from /tmp/sc-2= 0120903-17797-1jpsh3q/apply_spec.yml
I, [2012-09-03T18:30:32.366022 #17797] [0x100c818] =A0INFO -- : Loadin= g yaml from /tmp/sc-20120903-17797-1jpsh3q/stemcell.MF=

But nothing is outputted from the CPI nor a .bosh_erro= r is created.

Thanks.

Mike.

El lunes, 3 de septiembre de 2012 10:00:34 UTC+2, Ruslan K escrib= i=F3:
On 09/02/2012 02:44 PM, impres...@gmail.com wrote:
> Hi all,
>
> I'm thinking in adding a new CPI to BOSH. But after reading th= e
> documentation I have some doubts.
>
> Do I have to implement anything aside the cloud.rb itself and the = files
> needed by the CPI? I mean things like the registry.
>
> Once I implement the CPI, what configuration is needed in BOSH to = not
> see that 'Could not find Cloud Provider Plugin: ' in .bosh= _error?
>
> Is there any tutorial I can have a look at?
>
> Thanks very much!
>




--e89a8f3ba84b3e872404c8d5dffe--