deploy NG-service failure 413 Request Entity Too Large

87 views
Skip to first unread message

Apple

unread,
Jul 17, 2013, 5:00:31 AM7/17/13
to bosh-...@cloudfoundry.org
Hi, all
   I deploy CloudFoundry-NG using bosh on OpenStack, and use cf-release-132 as cloudfoundry components, and deploy services from cf-services-release@release candidate branch. 
   When I set one service in job, say mysql_node (with mysql_gateway), or postgresql_node(with postgresql_gateway) , it is ok.
   But, when I set these two services in one deployment or deploy the second service after one deployed, it will be wrong. The error log of $bosh deploy is :
Error 100: Expected([200, 202]) <=> Actual(413 Request Entity Too Large)
  request => {:chunk_size=>1048576, :connect_timeout=>60, :headers=>{"User-Agent"=>"fog/1.9.0", "Content-Type"=>"applicaon", "X-Auth-Token"=>"fb6b70bb211647f887b6b0e004ba22a2", "Host"=>"9.115.210.221:8774", "Content-Length"=>108}, :idempote", :middlewares=>[Excon::Middleware::Instrumentor, Excon::Middleware::Expects, Excon::Middleware::Mock], :mock=>false, :etry_limit=>4, :ssl_ca_file=>"/var/vcap/packages/director/gem_home/gems/excon-0.18.3/data/cacert.pem", :ssl_verify_peer=115.210.221", :path=>"/v2/44f47d9befd84f938617113bf940f027/os-volumes", :port=>"8774", :query=>"ignore_awful_caching1374 :password=>nil, :family=>0, :body=>"{\"volume\":{\"display_name\":\"volume-32bbdf8a-c182-45e0-9a4e-6e42189868a8\",\"dis", :expects=>[200, 202], :method=>"POST", :retries_remaining=>4}
  response => {:body=>"{\"overLimit\": {\"message\": \"The body of your request was too large for this server.\", \"codeh"=>"98", "Content-Type"=>"application/json; charset=UTF-8", "X-Compute-Request-Id"=>"req-c50937c1-e894-4947-8ae3-a5b564:22:04 GMT"}, :status=>413, :remote_ip=>"9.115.210.221"}


and, my manifest files are:
---
name: CloudFoundry-Services
director_uuid: cdd61f55-b60e-4ca2-a596-cee17cf7868a 

releases:
- name: bosh-release
  version: 0.2-dev

compilation:
  workers: 6
  network: default
  cloud_properties:
    instance_type: m1.microbosh

update:
  canaries: 1
  canary_watch_time: 3000-90000
  update_watch_time: 3000-90000
  max_in_flight: 1 #2
  max_errors: 1

networks:
- name: floating
  type: vip
  cloud_properties:
    security_groups:
      - default
   
- name: default
  type: dynamic
  dns:
  - 9.115.210.221
  cloud_properties:
    security_groups:
      - default

cloud:
  plugin: openstack
  properties:
  openstack:
      username: us...@cn.ibm.com
      api_key: password
      tenant: CloudFoundry-NG
      default_key_name: CloudOE
      default_security_groups: ["default"]
      private_key: /root/cloudoe.pem


resource_pools:

- name: infrastructure
  network: default 
  size: 4
  stemcell:
    name: bosh-stemcell
    version: 1.5.0.pre2
  cloud_properties:
      instance_type: m1.microbosh
  
  # config the sudo vm password default password: c1oudc0w
  env:
    bosh:
      password: $6$4EEnqzFdW$Ni8zZ6kWasyzllm4ZKdmULWqP0baiVVUSaSB5dgOuGg/asznmkoGRefiLs.ZtnqaL7INArzlNPnpeWTPBkH6E0
      #generated using mkpasswd -m sha-512,also configured in micro_bosh.yml $vi ../microbosh/micro_bosh.yml passw0rd

jobs:

- name: mysql_node
  template: mysql_node
  instances: 1
  resource_pool: infrastructure
  persistent_disk: 12058
  properties:
    plan: "100"
  networks:
  - name: default
    default: [dns, gateway]
  - name: floating
    static_ips:
    - 9.115.210.248

- name: mysql_gateway 
  template: mysql_gateway
  instances: 1
  resource_pool: infrastructure
  networks:
  - name: default
    default: [dns, gateway]
  - name: floating
    static_ips:
    - 9.115.210.213
  properties:
    uaa_client_id: "vmc"
    uaa_endpoint: http://uaa.cloudoe.bluezone
    uaa_client_auth_credentials:
      username: us...@cn.ibm.com
      password: pass
  
- name: postgresql_node
  template: postgresql_node
  instances: 1
  resource_pool: infrastructure
  persistent_disk: 12058
  properties:
    plan: "100" 
  networks:
  - name: default
    default: [dns, gateway]
  - name: floating
    static_ips:
    - 9.115.210.122

- name: postgresql_gateway
  template: postgresql_gateway
  instances: 1
  resource_pool: infrastructure
  networks:
  - name: default
    default: [dns, gateway]
  - name: floating
    static_ips:
    - 9.115.210.125
  properties:
    uaa_client_id: "vmc"
    uaa_endpoint: http://uaa.cloudoe.bluezone
    uaa_client_auth_credentials:
      username: us...@cn.ibm.com
      password: pass

properties:
  domain: cloudoe.bluezone
  system_domain: cloudoe.bluezone
  system_domain_organization: cdl-eti
  app_domains:
    - cloudoe.bluezone

  networks:
    apps: default
    management: default

  nats:
    user: nats
    password: time4fun
    address: 9.115.210.184
    port: 4222

  service_plans:
    mysql:
      "100":
        description: "build in mysql 5.5"
        free: true
        job_management:
          high_water: 900
          low_water: 100
        configuration:
          capacity: 500
          max_db_size: 10
          key_buffer: 512
          innodb_buffer_pool_size: 512
          max_allowed_packet: 16
          thread_cache_size: 128
          query_cache_size: 128
          max_long_query: 3
          max_long_tx: 30
          max_clients: 10
          max_connections: 1000
          table_open_cache: 2000
          innodb_tables_per_database: 50
          connection_pool_size:
            min: 5
            max: 10
          backup:
            enable: true
          lifecycle:
            enable: true
            serialization: enable
            snapshot:
              quota: 1
          warden:
            enable: false
    postgresql:
      "100":
        description: "Shared server, postgresql 9.1"
        free: true
        job_management:
          high_water: 900
          low_water: 100
        configuration:
          capacity: 500
          max_db_size: 10
          key_buffer: 512
          innodb_buffer_pool_size: 512
          max_allowed_packet: 16
          thread_cache_size: 128
          query_cache_size: 128
          max_long_query: 3
          max_long_tx: 30
          max_clients: 10
          max_connections: 1000
          table_open_cache: 2000
          innodb_tables_per_database: 50
          connection_pool_size:
            min: 5
            max: 10
          backup:
            enable: true
          lifecycle:
            enable: true
            serialization: enable
            snapshot:
              quota: 1
      #  description: "Shared server, shared VM, 1MB memory, 10MB storage, 10 connections"
      #  free: true
      #  job_management:
      #    high_water: 900
      #    low_water: 100
      #  configuration:
      #    capacity: 500
      #    max_db_size: 10
      #    max_long_query: 3
      #    max_long_tx: 10
      #    max_clients: 10
      #    max_db_conns: 100 #max_connections: 1000
      #    shared_buffers: 320
      #    effective_cache_size: 1142
          warden:
            enable: false

  mysql_gateway:
    token: cloudoemysqltoken
    default_plan: "100"
    supported_versions: ["5.5"]
    version_aliases:
      current: "5.5"
    cc_api_version: v2
  mysql_node:
    password: passw0rd
    plan: "100"
    supported_versions: ["5.5"]
    default_version: "5.5"
    max_tmp: 1024

  postgresql_gateway:
    token: cloudoepostgresqltoken
    default_plan: "100" #supported_plan: "100"
    supported_versions: ["9.1"]
    version_aliases:
      current: "9.1"
    cc_api_version: v2
  postgresql_node:
    supported_versions: ["9.1"]
    default_version: "9.1"
    password: passw0rd   
    plan: "100"
    max_tmp: 1024

  nfs_server:
    address: 9.115.210.117
    network: 9.115.210.0/24
   
  ccdb_ng:
    address: 9.115.210.186
    db_scheme: postgres
    port: 5524
    pool_size: 10
    roles:
    - tag: admin
      name: ccadmin
      password: time4fun
    databases:
    - tag: cc
      name: cf
      citext: true

  ccdb:
    address: 9.115.210.186 #the same sa ccdb_ng
    db_scheme: postgres
    port: 5524
    pool_size: 10
    roles:
      - tag: admin
        name: ccadmin
        password: time4fun
    databases:
      - tag: cc
        name: cf
        citext: true
   
  uaadb:
    db_scheme: postgresql 
    address: 9.115.210.188
    port: 5524
    roles:
      - tag: admin
        name: root
        password: time4fun
    databases:
      - tag: uaa
        name: uaa
  
  cc_api_version: v2
  
  cc: &cc
    srv_api_uri: http://api.cloudoe.bluezone
    external_host: ccng
    logging_level: debug
    bulk_api_password: password
    uaa_resource_id: cloud_controller
    staging_upload_user: jkfgjkfg
    staging_upload_password: pass
    bootstrap_admin_email: us...@cn.ibm.com
    db_encryption_key: pass

  ccng: *cc

  syslog_aggregator:
    address: 9.115.210.183
    port: 54321

  uaa:
    catalina_opts: -Xmx768m -XX:MaxPermSize=256m
    resource_id: account_manager
    no_ssl: true
    cc:
      token_secret: asdfasdfasdfadfs
      client_secret: asdfasdfasdffth
    admin:
      client_secret: erthth45ydbgaerg
    batch:
      username: sdfgjwetrysdgf
      password: time4fun
    client:
      autoapprove:
        - cf
        - vmc
        - my
        - portal
        - micro
        - support-signon
        - login
    clients:
      login:
        override: true
        scope: openid
        authorities: oauth.login
        secret: MJ99J9sdf9ji
        authorized-grant-types: authorization_code,client_credentials,refresh_token
        redirect-uri: http://login.cloudoe.bluezone
      portal:
        override: true
        scope: openid,cloud_controller.read,cloud_controller.write,password.write
        authorities: scim.write,scim.read,cloud_controller.read,cloud_controller.write,password.write,uaa.admin,uaa.resource,cloud_controller.admin
        secret: portalsecret
        authorized-grant-types: authorization_code,client_credentials
        access-token-validity: 1209600
        refresh-token-validity: 1209600
      billing:
        override: true
        scope: uaa.none
        authorities: cloud_controller.read,cloud_controller.write,cloud_controller.admin,uaa.resource,scim.read,scim.write
        secret: ijdfg9890ewrt234
        authorized-grant-types: client_credentials
        access-token-validity: 1209600
        refresh-token-validity: 1209600
      app-direct:
        override: true
        authorities: app_direct_invoice.write
        secret: 9ujsdfg93wer5
        authorized-grant-types: authorization_code,client_credentials,password,refresh_token,implicit
        access-token-validity: 1209600
        refresh-token-validity: 1209600
        redirect-uri: http://console.cloudoe.bluezone
      support-services:
        scope: scim.write,scim.read,openid,cloud_controller.read,cloud_controller.write
        secret: ssosecretsso
        authorized-grant-types: authorization_code,client_credentials
        redirect-uri: http://support-signon.cloudoe.bluezone
        authorities: portal.users.read
        access-token-validity: 1209600
        refresh-token-validity: 1209600
      vmc:
        override: true
        authorized-grant-types: password,implicit
        authorities: uaa.none
        scope: cloud_controller.read,cloud_controller.write,openid,password.write,cloud_controller.admin,scim.read,scim.write
      cf:
        override: true
        authorized-grant-types: implicit,password,refresh_token
        authorities: uaa.none
        scope: cloud_controller.read,cloud_controller.write,openid,password.write,cloud_controller.admin,scim.read,scim.write
      servicesmgmt:
        secret: IOASJD93hjakdf
        scope: openid,cloud_controller.read,cloud_controller.write
        authorities: uaa.resource,oauth.service,clients.read,clients.write,clients.secret
        authorized-grant-types: authorization_code,client_credentials,password,implicit
        override: true
        autoapprove: true

    scim:
      users:
        - us...@cn.ibm.com|pass|scim.write,scim.read,openid,cloud_controller.admin
        - dash-user|dash-password|openid,dashboard.user




I made some changes with nova config, in /etc/nova/
[filter:ratelimit]
paste.filter_factory = nova.api.openstack.compute.limits:RateLimitingMiddleware.factory
limits =(POST, "*", .*, 1000, MINUTE);(POST, "*/servers", ^/servers, 1000, MINUTE);(PUT, "*", .*, 1000, MINUTE);(GET, "*changes-since*", .*changes-since.*, 1000, MINUTE);(DELETE, "*", .*, 1000,MINUTE)


but it not works, or should I add add a sizelimit instead of ratelimit? and how?
[filter:sizelimit]
paste
.filter_factory = nova.api.sizelimit:RequestBodySizeLimiter.factory

some suggestions and solutions? Thanks in advance~


Ferran Rodenas

unread,
Jul 17, 2013, 5:40:11 AM7/17/13
to bosh-...@cloudfoundry.org
Check the quotas for your project. Seems you reached the max number of allowed volumes.

- Ferdy


2013/7/17 Apple <jianji...@gmail.com>

jianji...@gmail.com

unread,
Jul 17, 2013, 11:23:28 PM7/17/13
to bosh-...@cloudfoundry.org
thanks for your quick rely~

在 2013年7月17日星期三UTC+8下午5时40分11秒,ferdy写道:

jianji...@gmail.com

unread,
Jul 18, 2013, 12:42:47 AM7/18/13
to bosh-...@cloudfoundry.org
Thanks, it is the problem. I changed the quotas, and now it works. 

在 2013年7月17日星期三UTC+8下午5时40分11秒,ferdy写道:
Reply all
Reply to author
Forward
0 new messages