Cloudfoundry requirements and scalabality

[Amir Montazer]

Hi Guys,

I have 2 questions:

1. what is the minimum required AWS instance(s) to run CloudFoundry on AWS?

2. how many small CF app instances (php apps with 256mb memory limit) can be run on a m1.large AWS instance or any other AWS instance that you know of?

[Mark Rushakoff]

Hi Amir,

Here's a snippet of one of our config files for our development instances, which are intentionally pretty small:

```

resource_pools:
- name: common
  network: cf1
  size: 7
  stemcell:
    name: bosh-stemcell
    version: latest
  cloud_properties:
    instance_type: m1.small
    availability_zone: us-east-1a
- name: dea
  network: cf1
  size: 2
  stemcell:
    name: bosh-stemcell
    version: latest
  cloud_properties:
    instance_type: m1.large
    availability_zone: us-east-1a
- name: router
  network: cf1
  size: 1
  stemcell:
    name: bosh-stemcell
    version: latest
  cloud_properties:
    instance_type: m1.small
    availability_zone: us-east-1a
    elbs:
    - cfrouter

jobs:
# ====================== BEGIN CF JOBS =========================

- name: syslog_aggregator
  template: syslog_aggregator
  instances: 1
  resource_pool: common
  persistent_disk: 100000
  networks:
  - name: cf1
    static_ips:
    - 10.10.16.10

- name: nats
  template: nats
  instances: 1
  resource_pool: common
  networks:
  - name: cf1
    static_ips:
    - 10.10.16.11

- name: uaa
  template: uaa
  instances: 1
  resource_pool: common
  networks:
  - name: cf1
    static_ips:
    - 10.10.16.12

- name: login
  template: login
  instances: 1
  resource_pool: common
  networks:
  - name: cf1

- name: cloud_controller
  template: cloud_controller_ng
  instances: 1
  resource_pool: common
  networks:
  - name: cf1
    static_ips:
    - 10.10.16.13
  properties:
    ccdb: ccdb

- name: router
  template: gorouter
  instances: 1
  resource_pool: router
  networks:
  - name: cf1
    default:
    - dns
    - gateway
    static_ips:
    - 10.10.16.15

- name:  health_manager
  template: health_manager_next
  instances: 1
  resource_pool: common
  networks:
  - name: cf1

- name: dea_next
  template: dea_next
  instances: 2
  resource_pool: dea
  update:
    max_in_flight: 8
  networks:
  - name: cf1
    default:
    - dns
    - gateway

- name: collector
  template: collector
  instances: 1
  resource_pool: common
  networks:
  - name: cf1

```

You could get by with one DEA if you wanted, so bare minimum you're looking at something like 8 m1.smalls and whatever you want for your DEA (probably shouldn't go smaller than an m1.large though).

The m1.large has about 7.5GB of memory IIRC, so subtract a little for OS overhead - maybe 6.5 GB / 256 MB leaves you with something in the range of 26 apps at 256MB apiece.  

With 8 on-demand m1.smalls and 1 on-demand m1.large, the AWS cost estimator says that's $43.92 + $175.88 / month, or $219.80 per month.  You can save some money there if you switch to a reserved instance.

I've likely missed some other machines in this calculation, but I hope that helps as an estimate!

Mark

[tlabeeuw]

If you really wanted you could go further and co-locate some of the jobs one one AWS instance. This is smaller but obviously introduces more single points of failure.

[Dr Nic Williams]

Amir, I can currently squeeze CF down into 4 VMs with bosh-cloudfoundry. I think 2 is technically possible today. One is the goal (a la Micro CF).

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

[Amir Montazer]

Hi Mark,

Thanks for the response, two questions,

please correct me if I'm wrong because I'm a beginner: we need 8 m1.smalls to run the CF itself, and then we can have multiple DEAs which we can run  multiple apps on each,

1. does each app runs a linux or they share the linux that is running on the DEA?

2. can a php app using a LAMP stack be run using 256 MB of ram?

3. is there a limit for DEAs on this infrastructure? I mean if i need to extend my DEA to two m2.large intances, do i need to allocate more resources to the CF itself?

thanks.

[Amir Montazer]

Hi Nic,

Thanks for the reply, How can it be possible?, can you provide me any resources on that? I didn't find the CF documentation expressive enough.

[James Bayer]

Inline replies

On Sun, Sep 15, 2013 at 4:51 AM, Amir Montazer <ammon...@gmail.com> wrote:

Hi Mark,

Thanks for the response, two questions,

please correct me if I'm wrong because I'm a beginner: we need 8 m1.smalls to run the CF itself, and then we can have multiple DEAs which we can run  multiple apps on each,

1. does each app runs a linux or they share the linux that is running on the DEA?

The standard CF deployment has a DEA job that is a Linux virtual machine. Typically multiple application instances that run on each DEA are each placed in their own isolated Linux Container, but they share the Linux host OS. The Linux Container functionality is provided by Warden and uses Linux cgroups among other things.

 

2. can a php app using a LAMP stack be run using 256 MB of ram?

It's difficult to answer a question like this as each app can be different. I recommend you over-provision memory for the app, run it, then use "cf stats" to see how much memory is actually used for that application.

 

3. is there a limit for DEAs on this infrastructure? I mean if i need to extend my DEA to two m2.large intances, do i need to allocate more resources to the CF itself?

The DEA config file [1] needs to be adjusted to take advantage of additional memory. The template [2] will either use the spec file defaults [3] or use the override value set in the deployment manifest [4].

[1] https://github.com/cloudfoundry/dea_ng/blob/master/config/dea.yml#L13

[2] https://github.com/cloudfoundry/cf-release/blob/master/jobs/dea_next/templates/dea.yml.erb#L46

[3] https://github.com/cloudfoundry/cf-release/blob/master/jobs/dea_next/spec#L57

[4] search for "memory_mb" here http://docs.cloudfoundry.com/docs/running/deploying-cf/openstack/install_cf_openstack.html for an example manifest override in OpenStack

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

--

Thank you,

James Bayer

[Amir Montazer]

Thanks James, seems that I need to test everything, can you give me place to start? I mean how can I test it? I have one more question, how is the storage handled? I read in the docs that if an instance dies, it's storage will be lost, so how can I handle the people's static files without the need to change their app source? 

[James Bayer]

Inline

On Sun, Sep 15, 2013 at 10:51 PM, Amir Montazer <ammon...@gmail.com> wrote:

Thanks James, seems that I need to test everything, can you give me place to start? I mean how can I test it?

We use the vcap-yeti test suite to completely test a CF instance.

https://github.com/cloudfoundry/vcap-yeti

 

I have one more question, how is the storage handled? I read in the docs that if an instance dies, it's storage will be lost, so how can I handle the people's static files without the need to change their app source? 

Whatever files that were present in the application when it was last "pushed" to the system will be the ones that it uses when the app restarts. Cloud Foundry does not support persistent disks for applications, only for services that are configured to use a persistent disk in BOSH. If your app needs file persistence, then we suggest a persistent service, either in or out of Cloud Foundry. Eg. a blob store like AWS s3 or OpenStack Swift, database storage or NoSQL storage such as MongoDB GridFS. 

[Amir Montazer]

Ok let's say we want to use S3, does this integration need change in application source codes? because we want to offer a cloud hosting service using CF. so we need to serve regular php sites.

[James Bayer]

CF does not come with a built-in persistent disk service for applications. The project team is not currently planning on working on a persistent disk service as a core part of Cloud Foundry in the short-term.

We have considered some designs where a persistent disk could be used for either a single instance of an application or possibly several instances. Apps that require persistent file systems are typically not build for high scalability. Cloud Foundry caters to highly horizontally scalable apps first.

Some people have extended CF and used services like http://ceph.com/ to address persistent disk for apps.

[Dr Nic Williams]

Hey Amir, here is a template for colocating job templates down into 4 VMs https://github.com/cloudfoundry-community/bosh-cloudfoundry/blob/master/templates/v141/aws/medium/deployment_file.yml.erb#L88-L147

Nic

--

Dr Nic Williams

Stark & Wayne LLC - consultancy for Cloud Foundry users

http://drnicwilliams.com

http://starkandwayne.com

cell +1 (415) 860-2185

twitter @drnic