Service broker configuration storage

[Bas de Bakker]

Hi,

I'm implementing a V2 service broker and I need to store configuration information on provisioned instances for binding and deprovisioning purposes. Because multiple instances of the broker will be running, this information needs to be in some type of database, not on local storage. Is there a standard or recommended way to do this?

In particular, I'm wondering whether it is possible to use the Postgres instance that is already running for the cloud controller for this purpose. Is that reasonable?

Regards,
Bas

[James Bayer]

bas,

it's up to the implementer of the Service Broker on what type of persistence to use and there is no default or recommended one.

i would recommend against using an existing CF database process like the Cloud Controller DB. in sharing database processes, there may be issues with schemas colliding, a user of one database unknowingly affecting another database, running out of database connections, and similar by not having these isolated. another thing that matters is whether the service broker will be private or not. if it's a SaaS service, then you can use a public SaaS service for your datastore. the example service uses Github [1]. if it's a private service that may not have external connectivity, then i'd use whatever persistence story you are comfortable with, which is likely a relational database like mysql, which has a nice bosh packaging that you can reuse [2]. the s3 compatible riakcs bosh package [3] could also potentially be used as a single node mode, but in order to use multi-node it uses the cf router to register a route.

[1] https://github.com/cloudfoundry-samples/github-service-broker-ruby 

[2] https://github.com/cloudfoundry/cf-mysql-release

[3] https://github.com/cloudfoundry-incubator/cf-riak-cs-release

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

--

Thank you,

James Bayer

[Cornelia Davis]

One thing I will add is that you might be able to use the store of the service you are brokering (if it is a persistence service) and even be clever about the way you store things.  For example, the MySql broker creates the new DB and embeds the id (and other metadata) into the DB name (or other fields) that is created: https://github.com/cloudfoundry/cf-mysql-broker/blob/master/app/models/service_instance.rb#L32. So for the mysql broker, the place it "maintains" that state is in the provisioned resources themselves.

[Maaz Khan]

One way to achieve this will be to have DB (sqlite/h2/postgres) running on a seperate VM deployed via BOSH and then all your Broker persistent data configuration is pointing to that DB instead of local persistent storage. We can always use CC postgres but that adds a dependcy on CF release and ideally broker should be completely independent of any CF related stuff except implementing the 5 REST points.

 

Maaz

[Shannon Coen]

You might also consider whether you definitely need to maintain state in the broker. The API requests contain identifiers for service instance and service plan, as well as ids for org and space.

If your backend is simple enough, the plan guid can be sufficient for the broker to know what backend to contact. If you have multiple backends for a given plan, and the broker balances placement of instances across backends, then maintaining state may be necessary. Another way to avoid keeping state in the broker is to use a message bus system between the broker and an agent on your backends, such that the broker simply puts a request out on the bus and the agents are responsible for (1) responding that they have capacity for the instance and taking the provision request (first to respond takes it), and (2) identifying that they are responsible for the instance for subsequent operations.