Using Ctl without Workbench at large scale on AWS

29 views
Skip to first unread message

Anthony Shortland

unread,
Jun 13, 2011, 1:10:17 AM6/13/11
to ControlTier Accounting
We're working on a project to deploy a Rundeck/ControlTier solution to Amazon Web Services (AWS). What has become very clear is that the Workbench and Jackrabbit components of the ControlTier server cannot directly support large-scale cloud operations (100s-1000s of nodes).

Fortunately, Ctl (the ControlTier client) has long had a "headless" mode which avoids the need to deploy a server for operational use. The general idea is that instead of dynamically acquiring modules and resources from Workbench at deploy time, Ctl receives them as version controlled artifacts from a package repository at installation time.

Here's what we did to dust-off this approach (which has been around for 3-4 years) and update it to work with S3 (could also be applied to any web-based repository, by the way; e.g. Apache mod_dav).

Resource model development process

In the AWS example, we're looking to be able to create any number of identically configured instances based on a common resource model.

With this in mind, we can use the DukesBank sample application to demonstrate developing such a resource model by using a modified version of the sample environment XML (a development environment in this case) included in the Elements DukesBankProjectBuilder module to create a "node/role" definition that can be assigned to any number of AWS instances:

Anthonys-MacBook-Pro-2:environments anthony$ cat development.xml 
<?xml version="1.0"?>

<!--
<!DOCTYPE environment PUBLIC "-//ControlTier Software Inc.//DTD Project Document 1.0//EN" "environment.dtd">
-->

<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<!-- Dukes Bank demonstration application environment definition document                -->
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->

<Environment name="development">
  <Token name="JBOSS_HOME" value="${user.home}/demo/elements/development/dukesbank/jboss-4.0.3SP1"/>
  <Token name="JBOSS_SERVER" value="default"/>
  <Token name="JBOSS_VERSION" value="4.0.3SP1"/>
  <Token name="JBOSS_PORTS" value="ports-01"/>
  <Token name="JBOSS_JAVA_OPTS" value="-Djava.naming.provider.url=jnp://localhost:1199"/>
  <Token name="HSQLDB_HOME" value="${user.home}/demo/elements/development/dukesbank/hsqldb"/>
  <Token name="HSQLDB_VERSION" value="1_8_0_10"/>
  <Token name="HSQLDB_DMP_VERSION" value="00000000.0"/>
  <Token name="RDB_PORT" value="1701"/>
  <Token name="RDB_CONNECTION" value="jdbc:hsqldb:hsql://localhost:1701"/>

  <Node name="appserver">
    <Token name="HOSTNAME" value="localhost"/>

    <JBossServer name="dukesbank">
      <Packages>
        <JBossZip name="jboss-4.0.3SP1.zip"/>
      </Packages>
      <Settings>
        <JBossPortConfig name="dukesbank"/>
        <JBossJavaOpts name="dukesbank"/>
      </Settings>
      <Dependencies>
      <HsqldbRdb name="dukesbank"/>
      </Dependencies>
    </JBossServer>

    <HsqldbRdb name="dukesbank">
      <Packages>
        <HsqldbZip name="hsqldb_1_8_0_10.zip"/>
        <HsqldbRdbDmp name="db1-00000000.0.zip"/>
      </Packages>
      <Settings>
        <RdbPort name="dukesbank"/>
        <RdbConnection name="dukesbank"/>
      </Settings>
      <Dependencies>
        <HsqldbRdbSchema name="dukesbank"/>
      </Dependencies>
    </HsqldbRdb>

    <HsqldbRdbSchema name="dukesbank"/>
  </Node>
</Environment>

Note that the environment XML defines what is essentially a "meta-node" ("appserver" in this case) and the attendant service, package and setting resources necessary to support deploying the correct version of the application to it. 

A complete example would include all distinct node/role definitions necessary to describe the full logical topology (service model) of the environment without specifying any of the physical topology (node model) information (this comes later when the resource model is mapped to particular AWS instances during the provisioning process).

Given the environment XML as a starting point, then, its a straightforward process to generate and load the resource model to a ControlTier server running locally:

  • Created a "development" project in Workbench to represent the target environment.
  • Registered my Mac with the project and setup standard ProjectBuilder resources, as usual:

Anthonys-MacBook-Pro-2:tmp anthony$ ctl-project -p development -a create
Project structure created: /Users/anthony/ctier/ctier.services.dtosolutions.com/ctl/projects/development
Invoking external setup script: /Users/anthony/ctier/ctier.services.dtosolutions.com/pkgs/ctl-3.6.0/bin/commander-ctl-project.xml
Beginning client setup ...
Trying to override old definition of task document-property
CTL project setup procedure completed.
Beginning node registration ...
Registering macosx[Node] with ctl.base: /Users/anthony/ctier/ctier.services.dtosolutions.com/ctl, ctl.home: /Users/anthony/ctier/ctier.services.dtosolutions.com/pkgs/ctl-3.6.0 ...
Registered macosx[Node] node to the resource model at http://ctier.services.dtosolutions.com:8080/itnav
Completed client side project setup. Node macosx registered in project: "development".

Anthonys-MacBook-Pro-2:src anthony$ cat builders.macosx.xml 
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE project PUBLIC "-//ControlTier Software Inc.//DTD Project Document 1.0//EN" "project.dtd">

<project>
  <deployment type="ProjectBuilder" name="elements-3-6-support" description="3.6 support branch Elements library project builder" basedir="${user.home}/src/controltier-3-6-support/elements" installRoot="${user.home}/target/controltier-3-6-support/elements" startuprank="">
    <referrers replace="false">
      <resource name="macosx" type="Node"/>
    </referrers>
  </deployment>
  <deployment type="ProjectBuilder" name="atg-3-6-support" description="3.6 support branch ATG Solution Library project builder" basedir="${user.home}/src/controltier-3-6-support/atg" installRoot="${user.home}/target/controltier-3-6-support/atg" startuprank="">
    <referrers replace="false">
      <resource name="macosx" type="Node"/>
    </referrers>
  </deployment>
  <deployment type="ProjectBuilder" name="core-3-6-support" description="3.6 support branch core modules project builder" basedir="${user.home}/src/controltier-3-6-support/core" installRoot="${user.home}/target/controltier-3-6-support/core" startuprank="">
    <referrers replace="false">
      <resource name="macosx" type="Node"/>
    </referrers>
  </deployment>
  <deployment type="ProjectBuilder" name="development" description="Development project builder" basedir="${user.home}/src/development" installRoot="${user.home}/target/development" startuprank="">
    <referrers replace="false">
      <resource name="macosx" type="Node"/>
    </referrers>
  </deployment>
</project>
Anthonys-MacBook-Pro-2:src anthony$ ctl -p development -m ProjectBuilder -c load-resources -- -filename builders.macosx.xml 
Loading "/Users/anthony/src/builders.macosx.xml" ...
1 file(s) have been successfully validated.
Processing /Users/anthony/src/builders.macosx.xml to /Users/anthony/ctier/ctier.services.dtosolutions.com/ctl/var/tmp/projectxml-308128351.xml
Loading stylesheet /Users/anthony/ctier/ctier.services.dtosolutions.com/ctl/modules/ProjectBuilder/lib/load-resources/projectxml/project.xsl
Mapping XML to properties ...
[timer: 0.582 sec]
Batching new object creation requests ...
[timer: 3.273 sec]
Executing batched new object creation for 4 objects ...
[timer: 5.057 sec]
Batching object attribute update requests ...
[timer: 7.691 sec]
Batching resource and referrer updates ...
[timer: 8.005 sec]
Executing 8 batched object updates ...
[timer: 9.608 sec]
Adding document transforms to deployments ...
[timer: 9.825 sec]
Anthonys-MacBook-Pro-2:src anthony$ ctl-project -p development -a install 
"resource-install" command running for resource: development[ProjectBuilder]
"resource-install" command running for resource: elements-3-6-support[ProjectBuilder]
"resource-install" command running for resource: atg-3-6-support[ProjectBuilder]
"resource-install" command running for resource: core-3-6-support[ProjectBuilder]

  • Built and loaded the Elements modules to the project (since the DukesBank sample application requires them, of course):

Anthonys-MacBook-Pro-2:src anthony$ ctl -p development -t ProjectBuilder -r elements-3-6-support -c build-library -- -upload -
Building library from modules in opts.basedir: /Users/anthony/src/controltier-3-6-support/elements/modules
setting property seed.build.name to elements-3-6-support-seed
processing modules in directory: /Users/anthony/src/controltier-3-6-support/elements/modules
.
.
.
Building jar: /Users/anthony/target/controltier-3-6-support/elements/elements-3-6-support-seed.jar
Uploading module jar: "/Users/anthony/target/controltier-3-6-support/elements/elements-3-6-support-seed.jar" ...
Imported types defined in library: /Users/anthony/target/controltier-3-6-support/elements/elements-3-6-support-seed.jar

  • Used the example resource XML included in the DukesBankProjectBuilder module to setup an Dukesbank-specific project builder (includes the resource templates):

Anthonys-MacBook-Pro-2:resources anthony$ cat development.xml 
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE project PUBLIC "-//ControlTier Software Inc.//DTD Project Document 1.0//EN" "project.dtd">

<!-- Sample DukesBankProjectBuilder configured for using the GenerateAndLoadEnvironment command: -->

<project>
  <deployment type="DukesBankProjectBuilder" name="development" description="Sample project builder for the Duke's Bank development environment " basedir="${modules.dir}/DukesBankProjectBuilder/examples" installRoot="${user.home}/target" startuprank="">
    <referrers replace="false">
      <resource name="macosx" type="Node"/>
    </referrers>
  </deployment>
</project>
Anthonys-MacBook-Pro-2:resources anthony$ ctl -p development -m ProjectBuilder -c load-resources -- -filename development.xml 
Loading "/Users/anthony/src/development/resources/development.xml" ...
1 file(s) have been successfully validated.
Processing /Users/anthony/src/development/resources/development.xml to /Users/anthony/ctier/ctier.services.dtosolutions.com/ctl/var/tmp/projectxml-140190914.xml
Loading stylesheet /Users/anthony/ctier/ctier.services.dtosolutions.com/ctl/projects/development/modules/ProjectBuilder/lib/load-resources/projectxml/project.xsl
Mapping XML to properties ...
[timer: 0.679 sec]
Batching new object creation requests ...
[timer: 2.892 sec]
Executing batched new object creation for 1 objects ...
[timer: 6.475 sec]
Batching object attribute update requests ...
[timer: 7.235 sec]
Batching resource and referrer updates ...
[timer: 7.503 sec]
Executing 2 batched object updates ...
[timer: 11.482 sec]
Adding document transforms to deployments ...
[timer: 11.855 sec]
Anthonys-MacBook-Pro-2:resources anthony$ ctl-project -p development -a install 
"resource-install" command running for resource: elements-3-6-support[ProjectBuilder]
"resource-install" command running for resource: development[ProjectBuilder]
"resource-install" command running for resource: atg-3-6-support[ProjectBuilder]
"resource-install" command running for resource: core-3-6-support[ProjectBuilder]
"resource-install" command running for resource: development[DukesBankProjectBuilder]

  • Note that I hacked the DukesBankProjectBuilder resource templates to ensure that the various package resources' repository URLs remain unresolved until deployment time (using a reference to the framework.pkgRepo.uri property as opposed to relying on the "relative repository path" convention) since this will facilitate their being picked up from an S3 bucket on AWS:

Anthonys-MacBook-Pro-2:DukesBankProjectBuilder anthony$ pwd
/Users/anthony/src/controltier-3-6-support/elements/modules/DukesBankProjectBuilder
Anthonys-MacBook-Pro-2:DukesBankProjectBuilder anthony$ svn diff templates/
Index: templates/types/HsqldbZip.xml
===================================================================
--- templates/types/HsqldbZip.xml       (revision 2257)
+++ templates/types/HsqldbZip.xml       (working copy)
@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE project PUBLIC "-//ControlTier Software Inc.//DTD Project Document 1.0//EN" "project.dtd">
 <project>
-  <package type="HsqldbZip" name="@PACKAGE@" version="@HSQLDB_VERSION@" release="" buildtime="" arch="noarch" filename="hsqldb_@HSQLDB_VERSION@.zip" filetype="zip" base="hsqldb" installroot="${entity.attribute.hsqldbRdbHome}" repoUrl="/zip/zips/hsqldb_@HSQLDB_VERSION@.zip" releasetag="" installrank="10" restart="false" vendor="The hsqldb Development Group" description="&quot;@ENVIRONMENT@&quot; environment Hypersonic SQL database engine"/>
+  <package type="HsqldbZip" name="@PACKAGE@" version="@HSQLDB_VERSION@" release="" buildtime="" arch="noarch" filename="hsqldb_@HSQLDB_VERSION@.zip" filetype="zip" base="hsqldb" installroot="${entity.attribute.hsqldbRdbHome}" repoUrl="$${framework.pkgRepo.uri}/zip/zips/hsqldb_@HSQLDB_VERSION@.zip" releasetag="" installrank="10" restart="false" vendor="The hsqldb Development Group" description="&quot;@ENVIRONMENT@&quot; environment Hypersonic SQL database engine"/>
 </project>
Index: templates/types/JBossZip.xml
===================================================================
--- templates/types/JBossZip.xml        (revision 2257)
+++ templates/types/JBossZip.xml        (working copy)
@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE project PUBLIC "-//ControlTier Software Inc.//DTD Project Document 1.0//EN" "project.dtd">
 <project>
-  <package type="JBossZip" name="@PACKAGE@" version="@JBOSS_VERSION@" release="" buildtime="" arch="noarch" filename="jboss-@JBOSS_VERSION@.zip" filetype="zip" base="jboss-@JBOSS_VERSION@" installroot="${entity.attribute.jboss_install_root}" repoUrl="/zip/zips/jboss-@JBOSS_VERSION@.zip" releasetag="" installrank="30" restart="false" vendor="Red Hat Middleware, LLC." description="&quot;@ENVIRONMENT@&quot; environment JBoss Enterprise Application Platform package"/>
+  <package type="JBossZip" name="@PACKAGE@" version="@JBOSS_VERSION@" release="" buildtime="" arch="noarch" filename="jboss-@JBOSS_VERSION@.zip" filetype="zip" base="jboss-@JBOSS_VERSION@" installroot="${entity.attribute.jboss_install_root}" repoUrl="$${framework.pkgRepo.uri}/zip/zips/jboss-@JBOSS_VERSION@.zip" releasetag="" installrank="30" restart="false" vendor="Red Hat Middleware, LLC." description="&quot;@ENVIRONMENT@&quot; environment JBoss Enterprise Application Platform package"/>
 </project>
Index: templates/types/HsqldbRdbDmp.xml
===================================================================
--- templates/types/HsqldbRdbDmp.xml    (revision 2257)
+++ templates/types/HsqldbRdbDmp.xml    (working copy)
@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE project PUBLIC "-//ControlTier Software Inc.//DTD Project Document 1.0//EN" "project.dtd">
 <project>
-  <package type="HsqldbRdbDmp" name="@PACKAGE@" version="@HSQLDB_DMP_VERSION@" release="" buildtime="" arch="noarch" filename="db1-@HSQLDB_DMP_VERSION@.zip" filetype="zip" base="db1" installroot="${entity.attribute.hsqldbRdbHome}" repoUrl="/HsqldbRdbDmp/zips/db1-@HSQLDB_DMP_VERSION@.zip" releasetag="" installrank="20" restart="false" vendor="ControlTier Software, Inc." description="&quot;@ENVIRONMENT@&quot; environment initial Hypersonic SQL database schema"/>
+  <package type="HsqldbRdbDmp" name="@PACKAGE@" version="@HSQLDB_DMP_VERSION@" release="" buildtime="" arch="noarch" filename="db1-@HSQLDB_DMP_VERSION@.zip" filetype="zip" base="db1" installroot="${entity.attribute.hsqldbRdbHome}" repoUrl="$${framework.pkgRepo.uri}/HsqldbRdbDmp/zips/db1-@HSQLDB_DMP_VERSION@.zip" releasetag="" installrank="20" restart="false" vendor="ControlTier Software, Inc." description="&quot;@ENVIRONMENT@&quot; environment initial Hypersonic SQL database schema"/>
 </project>

  • Used the environment XML presented above to generate and load the resource model:

Anthonys-MacBook-Pro-2:environments anthony$ ctl -p development -t DukesBankProjectBuilder -r development -c GenerateAndLoadEnvironment -- -environmentfile development.xml
Start: "generate and load an environment " commands: generate-environment,load-resources
begin workflow command (1/2) -> "generate-environment -basedir /Users/anthony/ctier/ctier.services.dtosolutions.com/ctl/projects/development/modules/DukesBankProjectBuilder/examples -targetdir /Users/anthony/target -environmentfile development.xml" ...
Processing "/Users/anthony/src/development/environments/development.xml" ...
Transforming "/Users/anthony/src/development/environments/development.xml" ...
Processing /Users/anthony/src/development/environments/development.xml to /Users/anthony/ctier/ctier.services.dtosolutions.com/ctl/var/tmp/environment-826729741.xml
Loading stylesheet /Users/anthony/ctier/ctier.services.dtosolutions.com/ctl/projects/development/modules/ProjectBuilder/lib/generate-environment/environment.xsl
[timer: 0.314 sec]
Mapping XML to properties ...
[timer: 0.383 sec]
Generating environment "development" ...
Deleting directory /Users/anthony/target/development
Created dir: /Users/anthony/target/development
Created dir: /Users/anthony/target/development/tokenfiles
Created dir: /Users/anthony/target/development/resources
Created dir: /Users/anthony/target/development/dependencies
Generating nodes of type "Node" ...
Generating node "appserver" ...
Generating nodes of type "Node" ...
Generating HsqldbRdb deployment "dukesbank" ...
Generating settings of type "RdbConnection" ...
Copying 1 file to /Users/anthony/target/development/resources
Generating settings of type "RdbPort" ...
Copying 1 file to /Users/anthony/target/development/resources
Generating packages of type "HsqldbRdbDmp" ...
Copying 1 file to /Users/anthony/target/development/resources
Generating packages of type "HsqldbZip" ...
Copying 1 file to /Users/anthony/target/development/resources
Generating dependencies of type "HsqldbRdbSchema" ...
Copying 1 file to /Users/anthony/target/development/resources
Generating nodes of type "Node" ...
Generating HsqldbRdbSchema deployment "dukesbank" ...
Copying 1 file to /Users/anthony/target/development/resources
Generating nodes of type "Node" ...
Generating JBossServer deployment "dukesbank" ...
Generating settings of type "JBossJavaOpts" ...
Copying 1 file to /Users/anthony/target/development/resources
Generating settings of type "JBossPortConfig" ...
Copying 1 file to /Users/anthony/target/development/resources
Generating packages of type "JBossZip" ...
Copying 1 file to /Users/anthony/target/development/resources
Generating dependencies of type "HsqldbRdb" ...
Copying 1 file to /Users/anthony/target/development/resources
Copying 1 file to /Users/anthony/target/development/resources
Moving 1 file to /Users/anthony/target/development/resources
Moving 1 file to /Users/anthony/target/development/resources
Moving 1 file to /Users/anthony/target/development/resources
[timer: 0.952 sec]
end workflow command (1/2) -> "generate-environment -basedir /Users/anthony/ctier/ctier.services.dtosolutions.com/ctl/projects/development/modules/DukesBankProjectBuilder/examples -targetdir /Users/anthony/target -environmentfile development.xml"
begin workflow command (2/2) -> "load-resources -basedir /Users/anthony/ctier/ctier.services.dtosolutions.com/ctl/projects/development/modules/DukesBankProjectBuilder/examples -targetdir /Users/anthony/target -filename /Users/anthony/target/development/resources.xml" ...
Loading "/Users/anthony/target/development/resources.xml" ...
1 file(s) have been successfully validated.
Processing /Users/anthony/target/development/resources.xml to /Users/anthony/ctier/ctier.services.dtosolutions.com/ctl/var/tmp/projectxml-827147423.xml
Loading stylesheet /Users/anthony/ctier/ctier.services.dtosolutions.com/ctl/projects/development/modules/ProjectBuilder/lib/load-resources/projectxml/project.xsl
Mapping XML to properties ...
[timer: 2.545 sec]
Batching new object creation requests ...
[timer: 4.970 sec]
No new objects to be created ...
[timer: 4.972 sec]
Batching object attribute update requests ...
[timer: 11.175 sec]
Batching resource and referrer updates ...
[timer: 11.253 sec]
Executing 23 batched object updates ...
[timer: 15.662 sec]
Adding document transforms to deployments ...
[timer: 15.782 sec]
end workflow command (2/2) -> "load-resources -basedir /Users/anthony/ctier/ctier.services.dtosolutions.com/ctl/projects/development/modules/DukesBankProjectBuilder/examples -targetdir /Users/anthony/target -filename /Users/anthony/target/development/resources.xml"
[command.timer.development.ProjectBuilder.GenerateAndLoadEnvironment: 16.942 sec]
Workflow completed. execution time: 16.942 sec

At this juncture we've got the target resource model setup in the development Workbench project of the local ControlTier server in a general form that would ensure that any AWS instance whose Ctl framework.node.name was set to "appserver" would be able to deploy the assigned resources:

Anthonys-MacBook-Pro-2:environments anthony$ cat ~/target/development/resources.xml 
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE project PUBLIC "-//ControlTier Software Inc.//DTD Project Document 1.0//EN" "project.dtd">

<project>

  <node type="Node" name="appserver" description="&quot;development&quot; environment server" hostname="localhost" tags="development">
    <resources replace="false">
      <resource type="HsqldbRdb" name="dukesbank"/>
      <resource type="HsqldbRdbSchema" name="dukesbank"/>
      <resource type="JBossServer" name="dukesbank"/>
    </resources>
  </node>

  <setting type="JBossJavaOpts" name="dukesbank" description="&quot;development&quot; environment server JBoss Java options" settingValue="-Djava.naming.provider.url=jnp://localhost:1199"/>

  <setting type="JBossPortConfig" name="dukesbank" description="&quot;development&quot; environment JBoss port configuration set" settingValue="ports-01"/>

  <setting type="RdbConnection" name="dukesbank" description="&quot;development&quot; environment database connection string" settingValue="jdbc:hsqldb:hsql://localhost:1701"/>

  <setting type="RdbPort" name="dukesbank" description="&quot;development&quot; environment database port" settingValue="1701"/>
  <package type="HsqldbRdbDmp" name="db1-00000000.0.zip" version="00000000.0" release="" buildtime="" arch="noarch" filename="db1-00000000.0.zip" filetype="zip" base="db1" installroot="${entity.attribute.hsqldbRdbHome}" repoUrl="$${framework.pkgRepo.uri}/HsqldbRdbDmp/zips/db1-00000000.0.zip" releasetag="" installrank="20" restart="false" vendor="ControlTier Software, Inc." description="&quot;development&quot; environment initial Hypersonic SQL database schema"/>
  <package type="HsqldbZip" name="hsqldb_1_8_0_10.zip" version="1_8_0_10" release="" buildtime="" arch="noarch" filename="hsqldb_1_8_0_10.zip" filetype="zip" base="hsqldb" installroot="${entity.attribute.hsqldbRdbHome}" repoUrl="$${framework.pkgRepo.uri}/zip/zips/hsqldb_1_8_0_10.zip" releasetag="" installrank="10" restart="false" vendor="The hsqldb Development Group" description="&quot;development&quot; environment Hypersonic SQL database engine"/>
  <package type="JBossZip" name="jboss-4.0.3SP1.zip" version="4.0.3SP1" release="" buildtime="" arch="noarch" filename="jboss-4.0.3SP1.zip" filetype="zip" base="jboss-4.0.3SP1" installroot="${entity.attribute.jboss_install_root}" repoUrl="$${framework.pkgRepo.uri}/zip/zips/jboss-4.0.3SP1.zip" releasetag="" installrank="30" restart="false" vendor="Red Hat Middleware, LLC." description="&quot;development&quot; environment JBoss Enterprise Application Platform package"/>

  <deployment type="HsqldbRdb" name="dukesbank" description="&quot;development&quot; environment stand-alone Hypersonic SQL database instance" basedir="${user.home}/demo/elements/development/dukesbank/hsqldb" installRoot="${user.home}/demo/elements/development/dukesbank/hsqldb" startuprank="1">
    <resources replace="true">
      <resource type="RdbConnection" name="dukesbank"/>
      <resource type="RdbPort" name="dukesbank"/>
      <resource type="HsqldbRdbDmp" name="db1-00000000.0.zip"/>
      <resource type="HsqldbZip" name="hsqldb_1_8_0_10.zip"/>
      <resource type="HsqldbRdbSchema" name="dukesbank"/>
    </resources>
  </deployment>

  <deployment type="HsqldbRdbSchema" name="dukesbank" description="&quot;development&quot; environment stand-alone Hypersonic SQL database schema" basedir="" installRoot="">
    <resources replace="true">
      <!-- Resources automatically inserted here: -->
    </resources>
  </deployment>

  <deployment type="JBossServer" name="dukesbank" description="&quot;development&quot; environment JBoss server" basedir="${user.home}/demo/elements/development/dukesbank/jboss-4.0.3SP1/server/default" installRoot="${user.home}/demo/elements/development/dukesbank/jboss-4.0.3SP1" startuprank="2">
    <resources replace="true">
      <resource type="JBossJavaOpts" name="dukesbank"/>
      <resource type="JBossPortConfig" name="dukesbank"/>
      <resource type="JBossZip" name="jboss-4.0.3SP1.zip"/>
      <resource type="HsqldbRdb" name="dukesbank"/>
    </resources>
  </deployment>
</project>

Whether you maintain environment XML, or resource XML directly as your primary source artifact you can see that all this is essentially identical to the process you've always used to develop and maintain a ControlTier resource model, with few key considerations:

  • Node resources are specified with generalized names that represent their role (as opposed to their physical location in the network).
  • Package resources include explicit references to the "framework.pkgRepo.uri" property to facilitate dynamic resolution of their location at deploy time.
  • Service/package resource relationships are statically described with no thought to managing them dynamically at deploy-time (since Workbench will not be available to drive Change-Dependencies).
  • Care must be taken in setting up configuration elements that handle "plumbing" the services together since these are dynamic in a cloud environment. This is typically achieved by using "well-known" host names, managed by a dynamic DNS service, that represent VIPs, queues, or load-balanced listening endpoints and such like.

The big take-away here is that the resource model is completely statically specified at development time so that every resource model change devolves to modifying and committing a new version of the environment (or resource) XML source artifact(s) and regenerating and loading the resource model.

The next step is to be able to build and release these resources and the modules that support them for use on AWS.

Resource model build process

The fundamental switch in mindset here is to consider Workbench as only a development tool used to build a set of version controlled and packaged resource model artifacts (modules and resources) for release to operational environments (I know ... talk about eating your own dog food!).

Firstly, modules are built as libraries packaged as "seed" Jars and Ctl extensions using the ProjectBuilder build-library command. A typical project may rely on several such module libraries: the ControlTier AWS tasks and modules, core modules, the community's Elements modules, and any site-specific modules.

They are all built in the same fashion from a module source directory:

  • Build the AWS extension ((checked out from the Moduleforge Sourceforge project):

Anthonys-MacBook-Pro-2:AwsS3Util anthony$ ctl -p development -t ProjectBuilder -r aws-3-6-support -c build-library -- -archive extension
Building library from modules in opts.basedir: /Users/anthony/src/controltier-3-6-support/aws/modules
setting property seed.build.name to aws-3-6-support-seed
processing modules in directory: /Users/anthony/src/controltier-3-6-support/aws/modules
.
.
.
Creating extension archive ...
processing modules in directory: /Users/anthony/src/controltier-3-6-support/aws
Copying 3 files to /Users/anthony/target/controltier-3-6-support/aws/extension/aws-3-6-support/modules
Deleting: /Users/anthony/target/controltier-3-6-support/aws/aws-3-6-support-extension-1.jar
Building jar: /Users/anthony/target/controltier-3-6-support/aws/aws-3-6-support-extension-1.jar
Extension archive created. You can install the new extension like so: ctl-extension -f aws-3-6-support-extension-1.jar

  • Built the core module library:

Anthonys-MacBook-Pro-2:dukesbank anthony$ ctl -p development -t ProjectBuilder -r core-3-6-support -c build-library -- -archive extension
Building library from modules in opts.basedir: /Users/anthony/src/controltier-3-6-support/core/modules
setting property seed.build.name to core-3-6-support-seed
processing modules in directory: /Users/anthony/src/controltier-3-6-support/core/modules
.
.
.
processing modules in directory: /Users/anthony/src/controltier-3-6-support/core
Copying 28 files to /Users/anthony/target/controltier-3-6-support/core/extension/core-3-6-support/modules
Copying 1 file to /Users/anthony/target/controltier-3-6-support/core/extension/core-3-6-support/bins
Deleting: /Users/anthony/target/controltier-3-6-support/core/core-3-6-support-extension-1.jar
Building jar: /Users/anthony/target/controltier-3-6-support/core/core-3-6-support-extension-1.jar
Extension archive created. You can install the new extension like so: ctl-extension -f core-3-6-support-extension-1.jar

  • Built the Elements module library:

Anthonys-MacBook-Pro-2:dukesbank anthony$ ctl -p development -t ProjectBuilder -r elements-3-6-support -c build-library -- -archive extension
Building library from modules in opts.basedir: /Users/anthony/src/controltier-3-6-support/elements/modules
setting property seed.build.name to elements-3-6-support-seed
processing modules in directory: /Users/anthony/src/controltier-3-6-support/elements/modules
.
.
.
Building jar: /Users/anthony/target/controltier-3-6-support/elements/elements-3-6-support-seed.jar
Creating extension archive ...
processing modules in directory: /Users/anthony/src/controltier-3-6-support/elements
Copying 100 files to /Users/anthony/target/controltier-3-6-support/elements/extension/elements-3-6-support/modules
Copying 1 file to /Users/anthony/target/controltier-3-6-support/elements/extension/elements-3-6-support/bins
Deleting: /Users/anthony/target/controltier-3-6-support/elements/elements-3-6-support-extension-1.jar
Building jar: /Users/anthony/target/controltier-3-6-support/elements/elements-3-6-support-extension-1.jar
Extension archive created. You can install the new extension like so: ctl-extension -f elements-3-6-support-extension-1.jar

These extension Jars should be treated like any other version-controlled packaged artifact and be managed by a builder with the artifacts being posted (imported) to the package repository.

The project's resources are also archived in Jar format for distribution:

  • This is achieved using the ProjectBuilder archive-resources command for the project:

Anthonys-MacBook-Pro-2:resources anthony$ ctl -p development -t DukesBankProjectBuilder -r development -c archive-resources -- -type '^Node$|^Service$|^Package$'
Cleaning out previous target directory: /Users/anthony/target/archive-resources/development
Deleting directory /Users/anthony/target/archive-resources/development
Querying repository for resources matching specified type and name pattern ...
|
|--(Node) appserver
|
|--(JBossServer) dukesbank
|
|--(HsqldbRdb) dukesbank
|
|--(HsqldbRdbDmp) dukesbank
|
|--(HsqldbRdbDmp) db1-00000000.0.zip
|
|--(Node) centos55
|
|--(Node) macosx
|
|--(JBossZip) jboss-4.0.3SP1.zip
|
|--(HsqldbZip) hsqldb_1_8_0_10.zip
|
|--(HsqldbZip) dukesbank
|
|--(JBossZip) dukesbank
|
|--(HsqldbRdbSchema) dukesbank
.
.
.
Archiving data in /Users/anthony/target/archive-resources/development
Archival process completed. Archive file: /Users/anthony/src/development/resources/development-resources.jar

  • The resulting package contains the selected resources' properties files:

Anthonys-MacBook-Pro-2:resources anthony$ jar tf development-resources.jar 
META-INF/MANIFEST.MF
resources/HsqldbRdb/dukesbank/var/resource.properties
resources/HsqldbRdbDmp/db1-00000000.0.zip/var/resource.properties
resources/HsqldbRdbDmp/dukesbank/var/resource.properties
resources/HsqldbRdbSchema/dukesbank/var/resource.properties
resources/HsqldbZip/dukesbank/var/resource.properties
resources/HsqldbZip/hsqldb_1_8_0_10.zip/var/resource.properties
resources/JBossServer/dukesbank/var/resource.properties
resources/JBossZip/dukesbank/var/resource.properties
resources/JBossZip/jboss-4.0.3SP1.zip/var/resource.properties
resources/Node/appserver/var/resource.properties
resources/Node/centos55/var/resource.properties
resources/Node/macosx/var/resource.properties

These resource Jars should also be treated like any other version-controlled packaged artifact and be managed by a builder with the artifacts being posted (imported) to the package repository.

Although I didn't describe setting up the full build process in detail, these artifacts (plus the application packages themselves) are all that need to be promoted to S3 to prepare for deployment. 

Resource model package artifact promotion process

Not easy to get a direct text listing of an Amazon bucket, so here's what it looks like on a file system!:

Anthonys-MacBook-Pro-2:dukesbank anthony$ find . -type f
./pkgs/HsqldbRdbDmp/zips/db1-00000000.0.zip
./pkgs/zip/zips/hsqldb_1_8_0_10.zip
./pkgs/zip/zips/jboss-4.0.3SP1.zip
./resources/development-resources.jar
./extensions/aws-3-6-support-extension-1.jar
./extensions/core-3-6-support-extension-1.jar
./extensions/elements-3-6-support-extension-1.jar

Note that the ControlTier AWS extension contains the S3Upload Ant task that can be used to automate adding files to an S3 bucket as part of the promotion process.

Workbench-less, S3-enabled Ctl installation and configuration

The following steps document installing a client system starting with a vanilla CentOS 5.5 EC2 instance.  

NOTE: this method requires a bunch of bug fixes and feature enhancements that I've just committed to Sourceforge (and will be released with ControlTier 3.6.1) so you'll need to be running on the latest build if you want to try this.


  • Started with creating the ControlTier user:

[ctier@ip-10-112-30-250 ~]$ cd
[ctier@ip-10-112-30-250 ~]$ pwd
/opt/ctier
[ctier@ip-10-112-30-250 ~]$ id
uid=100(ctier) gid=102(ctier) groups=10(wheel),102(ctier)

  • Setup the user's environment:

[ctier@ip-10-112-30-250 ~]$ pwd
/opt/ctier
[ctier@ip-10-112-30-250 ~]$ mkdir etc 
[ctier@ip-10-112-30-250 ~]$ cp /tmp/ctierrc etc/
[ctier@ip-10-112-30-250 ~]$ cat etc/ctierrc 
export CTIER_ROOT=/opt/ctier

export CTL_HOME=/opt/ctier/pkgs/ctl-3.6.1

#
# The CTL_BASE path you are using:
if [ -z "$CTL_BASE" ] ; then
  export CTL_BASE=/opt/ctier/ctl
fi

export PATH=$CTL_HOME/bin:$PATH

if [ -n "$BASH" -a -n "$CTL_BASE" ] ; then
    . $CTL_HOME/etc/bash_completion.sh ;
    if [ -t 0 -a -z "$CTL_CLI_TERSE" ]
    then
      CTL_CLI_TERSE=true
      export CTL_CLI_TERSE
    fi
fi
[ctier@ip-10-112-30-250 ~]$ cp /tmp/.bashrc .bashrc       
[ctier@ip-10-112-30-250 ~]$ cat .bashrc 
# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
        . /etc/bashrc
fi

# User specific aliases and functions

if [ -f /opt/ctier/etc/ctierrc ]; then
        . /opt/ctier/etc/ctierrc
else
        echo /opt/ctier/etc/ctierrc not found 1>&2
fi
. ~/.bashrc

  • Minimal install of Ctl (artifact is a "by-product" of the ControlTier build):

[ctier@ip-10-112-30-250 ~]$ mkdir -p  $CTL_HOME
[ctier@ip-10-112-30-250 ~]$ cd $CTL_HOME
[ctier@ip-10-112-30-250 ctl-3.6.1]$ unzip -q  /tmp/ctl-dispatch-3.6.1.zip
[ctier@ip-10-112-30-250 ctl-3.6.1]$ chmod +x bin/*

  • Added the AWS extension to support Ctl framework S3 integration:

[ctier@ip-10-112-30-250 ctl-3.6.1]$ mkdir -p lib/extensions/aws-3-6-support
[ctier@ip-10-112-30-250 ctl-3.6.1]$ cd lib/extensions/aws-3-6-support
[ctier@ip-10-112-30-250 aws-3-6-support]$ jar xf /tmp/aws-3-6-support-extension-1.jar

  • At this point the commander extension should be installed too since it contains tasks and types in broad use across many of the modules (e.g. the report task).

  • Setup the CTL_BASE. 

[ctier@ip-10-112-30-250 development]$ ctl-setup --log4j.logger.com.controltier.log.common=INFO --framework.ctl-project.create.buildfile=$CTL_HOME/lib/ant/controllers/ctl/projectsetupCmd.xml --resource.properties.autoupdate=false --framework.workbench.integration.enabled=false --framework.s3.integration.enabled=true --framework.s3.accessid=************ --framework.s3.secretkey=************ --framework.s3.bucket=dukesbank --framework.node.type=LinuxNode -n appserver
Using nodename as hostname: appserver

upgradeModuleLibdirs:

inquire-properties-regenerate:

generate-etc-properties:
     [copy] Copying 1 file to /opt/ctier/ctl/etc
     [copy] Copying 1 file to /opt/ctier/ctl/etc
     [copy] Copying 1 file to /opt/ctier/ctl/etc
     [copy] Copying 1 file to /opt/ctier/ctl/etc
     [copy] Copying 1 file to /opt/ctier/ctl/etc
     [copy] Copying 1 file to /opt/ctier/ctl/etc
     [copy] Copying 1 file to /opt/ctier/ctl/etc
     [copy] Copying 1 file to /opt/ctier/ctl/etc
     [copy] Copying 1 file to /opt/ctier/ctl/etc
     [copy] Copying 1 file to /opt/ctier/ctl/etc
     [copy] Copying 1 file to /opt/ctier/ctl/etc
     [copy] Copying 1 file to /opt/ctier/ctl/etc

generate-log4j-properties:
     [copy] Copying 1 file to /opt/ctier/ctl/etc
     [copy] Copying 1 file to /opt/ctier/ctl/etc

setup-directories:
    [mkdir] Created dir: /opt/ctier/ctl/var
    [mkdir] Created dir: /opt/ctier/ctl/var/tmp
    [mkdir] Created dir: /opt/ctier/ctl/var/logs
    [mkdir] Created dir: /opt/ctier/ctl/projects
    [mkdir] Created dir: /opt/ctier/ctl/modules
    [mkdir] Created dir: /opt/ctier/ctl/src
    [mkdir] Created dir: /opt/ctier/ctl/etc/extensions

installExtensionConfig:

copyExtensionConfig:
     [copy] Copying 1 file to /opt/ctier/ctl/etc/extensions/aws-3-6-support
     [echo] Updated extension configuration: aws-3-6-support

copyExtensionConfig:
     [copy] Copying 6 files to /opt/ctier/ctl/etc/extensions/commander
     [echo] Updated extension configuration: commander

modules.fromDir:

modules.fromJar:

modules.install:

framework:

-load-defaults:

check-props:

ctl-config:

modules-install:

post-setup:
     [echo] Commander extension post-setup completed.

Setup build successful.

  • Here's what each of the options does:
    • "--log4j.logger.com.controltier.log.common=INFO" turns off Log4J "CommonLog" logging to the ControlTier server.
    • "--framework.ctl-project.create.buildfile=$CTL_HOME/lib/ant/controllers/ctl/projectsetupCmd.xml" specifies the alternative to the Workbench/Commander project setup build file used in the default configuration.
    • "--resource.properties.autoupdate=false" disables the automatic updating of properties (since it relies on Workbench being available).
    • "--framework.workbench.integration.enabled=false" disables workbench integration across the Ctl framework.
    • "--framework.s3.integration.enabled=true" enables the use of S3 for package artifacts and framework related archives like extensions and resources.
    • "--framework.s3.accessid=************" specifies the S3 access identifier to use.
    • "--framework.s3.secretkey=************" specifies the key that goes with the identifier.
    • "--framework.s3.bucket=dukesbank" specifies the bucket that will contain all the artifacts and archives used by the Ctl installation.
    • "--framework.node.type=LinuxNode" the type (module) name of the node resource (should be a mediator/node hybrid module to facilitate node-level administration of services).
    • "-n appserver" the node name of the client installation (maps to the node role assigned to this instance).

  • Create the project:

[ctier@ip-10-112-30-250 ~]$ ctl-project -p development -a create
Project structure created: /opt/ctier/ctl/projects/development
Invoking external setup script: /opt/ctier/pkgs/ctl-3.6.0/lib/ant/controllers/ctl/projectsetupCmd.xml

  • (Formally) install the AWS extension (to setup its modules):

[ctier@ip-10-112-30-250 aws-3-6-support]$ ctl-extension -S -f /tmp/aws-3-6-support-extension-1.jar 

  • Get the Core extension from S3 and install:

[ctier@ip-10-112-30-250 ~]$ ctl -p development -m AwsS3Util -c get -- -file extensions/core-3-6-support-extension-1.jar -dest /tmp/core-3-6-support-extension-1.jar
copied: dukesbank:extensions/core-3-6-support-extension-1.jar
to: /tmp/core-3-6-support-extension-1.jar
[ctier@ip-10-112-30-250 aws-3-6-support]$ ctl-extension -S -f /tmp/core-3-6-support-extension-1.jar

  • Get and install the Elements extension:

[ctier@ip-10-112-30-250 ~]$ ctl -p development -m AwsS3Util -c get -- -file extensions/elements-3-6-support-extension-1.jar -dest /tmp/elements-3-6-support-extension-1.jar
copied: dukesbank:extensions/elements-3-6-support-extension-1.jar
to: /tmp/elements-3-6-support-extension-1.jar
[ctier@ip-10-112-30-250 aws-3-6-support]$ ctl-extension -S -f /tmp/elements-3-6-support-extension-1.jar 

  • Get the project resource archive:

[ctier@ip-10-112-30-250 ~]$  ctl -p development -m AwsS3Util -c get -- -file resources/development-resources.jar -dest /tmp/development-resources.jar
copied: dukesbank:resources/development-resources.jar
to: /tmp/development-resources.jar

  • Install the project's resources (currently ctl-archive only installs the Node type resource setup by project creation - ctl-archive needs to be extended to allow a full set of resources to be installed):

[ctier@ip-10-112-30-250 ~]$ cd $CTL_BASE/projects/development
[ctier@ip-10-112-30-250 development]$ jar xvf /tmp/development-resources.jar
 inflated: META-INF/MANIFEST.MF
 inflated: resources/HsqldbRdb/dukesbank/var/resource.properties
 inflated: resources/HsqldbRdbDmp/db1-00000000.0.zip/var/resource.properties
 inflated: resources/HsqldbRdbDmp/dukesbank/var/resource.properties
 inflated: resources/HsqldbRdbSchema/dukesbank/var/resource.properties
 inflated: resources/HsqldbZip/dukesbank/var/resource.properties
 inflated: resources/HsqldbZip/hsqldb_1_8_0_10.zip/var/resource.properties
 inflated: resources/JBossServer/dukesbank/var/resource.properties
 inflated: resources/JBossZip/dukesbank/var/resource.properties
 inflated: resources/JBossZip/jboss-4.0.3SP1.zip/var/resource.properties
 inflated: resources/LinuxNode/appserver/var/resource.properties
 inflated: resources/Node/centos55/var/resource.properties
 inflated: resources/Node/macosx/var/resource.properties

  • At this point the application services can be deployed and started as follows:

[ctier@ip-10-112-30-250 development]$ ctl -p development -t LinuxNode -r appserver -c Deploy     

Finally, here's a script that put's it all together that can form the basis of preparing pre-installed AMI and driving a provisioning workflow to assign a new instance the target node role:

[ctier@ip-10-112-30-250 ~]$ cat /tmp/doit 
#!/bin/bash -x -e

rm -rf $CTL_HOME
rm -rf $CTL_BASE

mkdir -p $CTL_HOME
cd $CTL_HOME
unzip -q  /tmp/ctl-dispatch-3.6.1.zip 
chmod +x bin/*
mkdir -p lib/extensions/aws-3-6-support
cd lib/extensions/aws-3-6-support
jar xf /tmp/aws-3-6-support-extension-1.jar

cd $CTL_HOME/lib/extensions
cp -rf /tmp/commander commander

export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk.x86_64
ctl-setup --log4j.logger.com.controltier.log.common=INFO --framework.ctl-project.create.buildfile=$CTL_HOME/lib/ant/controllers/ctl/projectsetupCmd.xml --resource.properties.autoupdate=false --framework.workbench.integration.enabled=false --framework.s3.integration.enabled=true --framework.s3.accessid=************* --framework.s3.secretkey=*************** --framework.s3.bucket=dukesbank --framework.pkgRepo.uri=/pkgs --framework.pkgRepo.upload-url=pkgs -n appserver

ctl-project -p development -a create

ctl-extension -S -f /tmp/aws-3-6-support-extension-1.jar 

ctl -p development -m AwsS3Util -c get -- -file extensions/core-3-6-support-extension-1.jar -dest /tmp/core-3-6-support-extension-1.jar
ctl-extension -S -f /tmp/core-3-6-support-extension-1.jar 

ctl -p development -m AwsS3Util -c get -- -file extensions/elements-3-6-support-extension-1.jar -dest /tmp/elements-3-6-support-extension-1.jar
ctl-extension -S -f /tmp/elements-3-6-support-extension-1.jar 

ctl -p development -m AwsS3Util -c get -- -file resources/development-resources.jar -dest /tmp/development-resources.jar

# ctl-archive -p development -a extract -f /tmp/development-resources.jar
cd $CTL_BASE/projects/development
jar xvf /tmp/development-resources.jar

cp /tmp/resources.xml /opt/ctier/ctl/projects/development/etc/resources.xml

... the commented execution of ctl-archive in favor of unjar'ing the resource archive and the copying of the temporary resources.xml works-around the loose-end regarding resource setup described above that still needs to be resolved).

Anthony.



Anthony Shortland

unread,
Jun 13, 2011, 4:07:29 PM6/13/11
to ControlTier Accounting
OK ... I solved the issue of wanting to install all the resources from a given resource archive by adding the "extractall" usage to ctl-archive (is committed and will be included in 3.6.1).

I envisage that one could either install all the project's resources to each node, or prepare separate resource archive files that contain just the resources (nodes, services and packages, etc) pertinent to the local node for the given project. e.g:

[ctier@ip-10-112-30-250 ctl-3.6.1]$ ctl-archive -p development -a extractall -f /tmp/development-appserver-resources.jar
 INFO [main] (DataArchive.java:196) - Extracted data for entry: resources/Node/centos55/var/resource.properties
 INFO [main] (DataArchive.java:196) - Extracted data for entry: resources/HsqldbRdbSchema/dukesbank/var/resource.properties
 INFO [main] (DataArchive.java:196) - Extracted data for entry: resources/Node/macosx/var/resource.properties
 INFO [main] (DataArchive.java:196) - Extracted data for entry: resources/HsqldbRdbDmp/db1-00000000.0.zip/var/resource.properties
 INFO [main] (DataArchive.java:196) - Extracted data for entry: resources/JBossServer/dukesbank/var/resource.properties
 INFO [main] (DataArchive.java:196) - Extracted data for entry: resources/HsqldbZip/dukesbank/var/resource.properties
 INFO [main] (DataArchive.java:196) - Extracted data for entry: resources/HsqldbRdbDmp/dukesbank/var/resource.properties
 INFO [main] (DataArchive.java:196) - Extracted data for entry: resources/HsqldbRdb/dukesbank/var/resource.properties
 INFO [main] (DataArchive.java:196) - Extracted data for entry: resources/JBossZip/dukesbank/var/resource.properties
 INFO [main] (DataArchive.java:196) - Extracted data for entry: resources/JBossZip/jboss-4.0.3SP1.zip/var/resource.properties
 INFO [main] (DataArchive.java:196) - Extracted data for entry: resources/Node/appserver/var/resource.properties
 INFO [main] (DataArchive.java:196) - Extracted data for entry: resources/HsqldbZip/hsqldb_1_8_0_10.zip/var/resource.properties

Here's the updated provisioning script:

[ctier@ip-10-112-30-250 tmp]$ cat /tmp/doit

#!/bin/bash -x -e

rm -rf $CTL_HOME
rm -rf $CTL_BASE

mkdir -p $CTL_HOME
cd $CTL_HOME
unzip -q  /tmp/ctl-dispatch-3.6.1.zip 
chmod +x bin/*
mkdir -p lib/extensions/aws-3-6-support
cd lib/extensions/aws-3-6-support
jar xf /tmp/aws-3-6-support-extension-1.jar

cd $CTL_HOME/lib/extensions
cp -rf /tmp/commander commander

export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk.x86_64
ctl-setup --log4j.logger.com.controltier.log.common=INFO --framework.ctl-project.create.buildfile=$CTL_HOME/lib/ant/controllers/ctl/projectsetupCmd.xml --resource.properties.autoupdate=false --framework.workbench.integration.enabled=false --framework.s3.integration.enabled=true --framework.s3.accessid=AKIAIQSBXJNO6BNHYKFQ --framework.s3.secretkey=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 --framework.s3.bucket=dukesbank --framework.pkgRepo.uri=pkgs --framework.pkgRepo.upload-url=pkgs -n appserver


ctl-project -p development -a create

ctl-extension -S -f /tmp/aws-3-6-support-extension-1.jar 

ctl -p development -m AwsS3Util -c get -- -file extensions/core-3-6-support-extension-1.jar -dest /tmp/core-3-6-support-extension-1.jar
ctl-extension -S -f /tmp/core-3-6-support-extension-1.jar 

ctl -p development -m AwsS3Util -c get -- -file extensions/elements-3-6-support-extension-1.jar -dest /tmp/elements-3-6-support-extension-1.jar
ctl-extension -S -f /tmp/elements-3-6-support-extension-1.jar 

ctl -p development -m AwsS3Util -c get -- -file resources/development-resources.jar -dest /tmp/development-resources.jar
ctl-archive -p development -a extractall -f /tmp/development-resources.jar

# cp /tmp/resources.xml /opt/ctier/ctl/projects/development/etc/resources.xml

The last step is to decide an approach for managing role-based resources.xml ...

Anthony.

Anthony Shortland

unread,
Jun 13, 2011, 4:34:06 PM6/13/11
to ControlTier Accounting
One last (?) tweak, here!

Added the "framework.moduleRepo.integration.enabled" property to ctl-setup (see below) to draw a fine distinction between the module repository (which can be any old web server by the way) and the optional use of the Workbench server, and the use of S3 for resource model and packages artifacts:

[ctier@ip-10-112-30-250 ~]$ more /tmp/doit
#!/bin/bash -x -e

rm -rf $CTL_HOME
rm -rf $CTL_BASE

mkdir -p $CTL_HOME
cd $CTL_HOME
unzip -q  /tmp/ctl-dispatch-3.6.1.zip 
chmod +x bin/*
mkdir -p lib/extensions/aws-3-6-support
cd lib/extensions/aws-3-6-support
jar xf /tmp/aws-3-6-support-extension-1.jar

cd $CTL_HOME/lib/extensions
cp -rf /tmp/commander commander

export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk.x86_64
ctl-setup --log4j.logger.com.controltier.log.common=INFO --framework.ctl-project.create.buildfile=$CTL_HOME/lib/ant/controllers/ctl/projectsetupCmd.xml --resource.properties.autoupdate=false --framework.workbench.integration.enabled=false --framework.moduleRepo.integration.enabled=false --framework.s3.integration.enabled=true --framework.s3.accessid=************ --framework.s3.secretkey=************ --framework.s3.bucket=dukesbank --framework.pkgRepo.uri=pkgs --framework.pkgRepo.upload-url=pkgs -n appserver

ctl-project -p development -a create

ctl-extension -S -f /tmp/aws-3-6-support-extension-1.jar 

ctl -p development -m AwsS3Util -c get -- -file extensions/core-3-6-support-extension-1.jar -dest /tmp/core-3-6-support-extension-1.jar
ctl-extension -S -f /tmp/core-3-6-support-extension-1.jar 

ctl -p development -m AwsS3Util -c get -- -file extensions/elements-3-6-support-extension-1.jar -dest /tmp/elements-3-6-support-extension-1.jar
ctl-extension -S -f /tmp/elements-3-6-support-extension-1.jar 

ctl -p development -m AwsS3Util -c get -- -file resources/development-resources.jar -dest /tmp/development-resources.jar
ctl-archive -p development -a extractall -f /tmp/development-resources.jar

# cp /tmp/resources.xml /opt/ctier/ctl/projects/development/etc/resources.xml

Anthony.

Anthony Shortland

unread,
Jun 14, 2011, 4:26:19 PM6/14/11
to ControlTier Accounting
Ah ha! With all the tweaks committed I've re-run my tests using the 3.6.1 release candidate build from http://services.dtosolutions.com:8080.

Here's an updated version of the script:

[ctier@ip-10-112-30-250 tmp]$ cat /tmp/doit
#!/bin/bash -x -e

# This to be baked into the AMI:

rm -rf $CTL_HOME
rm -rf $CTL_BASE
rm -rf $HOME/demo

mkdir -p $CTL_HOME
cd $CTL_HOME
unzip -q  /tmp/ctl-dispatch-3.6.1.zip 
chmod +x bin/*

mkdir -p $CTL_HOME/lib/extensions/aws
cd $CTL_HOME/lib/extensions/aws
jar xf /tmp/aws-extension-3.6.1.jar

mkdir -p $CTL_HOME/lib/extensions/commander
cd $CTL_HOME/lib/extensions/commander
jar xf /tmp/commander-extension-3.6.1.jar

# This as part of the bootstrap provisioning workflow:

export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk.x86_64
ctl-setup --log4j.logger.com.controltier.log.common=INFO --framework.ctl-project.create.buildfile=$CTL_HOME/lib/ant/controllers/ctl/projectsetupCmd.xml --resource.properties.autoupdate=false --framework.workbench.integration.enabled=false --framework.moduleRepo.integration.enabled=false --framework.s3.integration.enabled=true --framework.s3.accessid=************ --framework.s3.secretkey=************ --framework.s3.bucket=dukesbank --framework.pkgRepo.uri=pkgs --framework.pkgRepo.upload-url=pkgs -n appserver

ctl-project -p development -a create

ctl-extension -S -f /tmp/aws-extension-3.6.1.jar 

ctl -p development -m AwsS3Util -c get -- -file extensions/core-extension-3.6.1.jar -dest /tmp/core-extension-3.6.1.jar
ctl-extension -S -f /tmp/core-extension-3.6.1.jar 

ctl -p development -m AwsS3Util -c get -- -file extensions/elements-extension-3.6.1.jar -dest /tmp/elements-extension-3.6.1.jar
ctl-extension -S -f /tmp/elements-extension-3.6.1.jar 

ctl -p development -m AwsS3Util -c get -- -file resources/development-resources.jar -dest /tmp/development-resources.jar
ctl-archive -p development -a extractall -f /tmp/development-resources.jar

cp /tmp/resources.xml /opt/ctier/ctl/projects/development/etc/resources.xml

ctl -p development -t JBossServer -r dukesbank -c Deploy 

Anthony.

Noah Campbell

unread,
Jun 14, 2011, 5:26:38 PM6/14/11
to contr...@googlegroups.com
What!?!  No diff Anthony?  :)

-Noah

--
You received this message because you are subscribed to the Google Groups "ControlTier" group.
To post to this group, send email to contr...@googlegroups.com
To unsubscribe from this group, send email to controltier...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/controltier?hl=en
http://wiki.controltier.org

chengkai

unread,
Feb 6, 2012, 2:33:38 PM2/6/12
to contr...@googlegroups.com
Anthony,

  I am able to follow this email you send out last year to prototype our environment to be ControlTier workbench-less deployment.  Yet, I ran into one problem,  Package (PPMAppZip/PPMZip) scripts are not triggered.  Here are some information I have,


*   re.ppm.resources.xml is a trimmed down resources file (contains only one service and one package, and its related settings).
*   re-resources.jar is an archive being generated via ctl -p reTest -t ProjectBuilder -r ppm -c archive-resources -- -type ^Node$|^Service$|^Package$ command.
*   ppm-extension-1.jar is a ControlTier extension generated via ctl -p re -t ProjectBuilder -r ppm -c build-library -- -archive extension command

I then copy the re-resoufces.jar, and ppm-extension-1.jar to one of my node, and extract re-resources.jar to $CTL_BASE/projects directory, and use ctl-extension -S -f ppm-extension-1.jar to install the installation

I also ran the ctl-setup command as

ctl-setup --log4j.logger.com.controltier.log.common=INFO --framework.ctl-project.create.buildfile=$CTL_HOME/lib/ant/controllers/ctl/projectsetupCmd.xml --resource.properties.    autoupdate=false --framework.workbench.integration.enabled=false -n $COMPUTERNAME -N $COMPUTERNAME --framework.pkgRepo.uri=xxx:

I overload the get method to use scp, so I change framework.pkgRepo.uri to xxx:, and when execute this command,

ctl -p re -t PPMApp -c Deploy

Here's the portion of the log output

d:\tools\ctier\ctl\modules\PPMIISWS\commands>
end workflow command (1/1) -> "assertServiceIsDown "
end workflow command (1/4) -> "Stop "
begin workflow command (2/4) -> "Packages-Install " ...
Dispatching command 'assertPackageIsVerified ' to:  ...
end workflow command (2/4) -> "Packages-Install "
begin workflow command (3/4) -> "Configure " ...

Apparently, the Package-Install method was unable to figure how to execute ... I attached related file for you to examine:


This is a resource archive being generated by -c archive-resources command
RE-resources.jar
ppm-extension-1.jar

chengkai

unread,
Feb 8, 2012, 11:38:53 AM2/8/12
to contr...@googlegroups.com
Hi Anthony,

    Thanks for the 30 minutes skype session yesterday.  That solved the my problem and now when I kick off ctl deployment command it will try to fetch the resources.  While this is happening, the following error happened,

$ ctl -p re -t PPMApp -r re -c Deploy
begin workflow command (1/4) -> "Stop " ...
begin workflow command (1/1) -> "assertServiceIsDown " ...
Running handler command -> "stopService "
Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

d:\tools\ctier\ctl\modules\PPMIISWS\commands>
Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

d:\tools\ctier\ctl\modules\PPMIISWS\commands>
end workflow command (1/1) -> "assertServiceIsDown "
end workflow command (1/4) -> "Stop "
begin workflow command (2/4) -> "Packages-Install " ...
Start: "Install the configured package dependencies for the deployment." Beginning installation for packages:   PPM_v6.33.11765.23400.zip[PPMAppZip] ...
Dispatching command 'assertPackageIsVerified ' to: PPM_v6.33.11765.23400.zip[PPMAppZip] ...
Created dir: d:\tools\ctier\ctl\projects\re\artifacts\modules
To: d:\tools\ctier\ctl\projects\re\artifacts\modules\PPMAppZip-head.jar
Error getting http://localhost:8080/jackrabbit/repository/controltier/projects/re/artifacts/modules/PPMAppZip-head.jar to d:\tools\ctier\ctl\projects\re\artifacts\modules\PPMAppZip-head.jar
-execute has detected a build error: java.net.ConnectException: Connection refused: connect
PPMAppZip-head.jar not found in project repository.
Getting PPMAppZip-head.jar module from global repository...
To: d:\tools\ctier\ctl\projects\re\artifacts\modules\PPMAppZip-head.jar
Error getting http://localhost:8080/jackrabbit/repository/controltier/artifacts/modules/PPMAppZip-head.jar to d:\tools\ctier\ctl\projects\re\artifacts\modules\PPMAppZip-head.jar
Error handler caught BuildException: The following error occurred while executing this line:
d:\tools\ctier\ctl\modules\Deployment\commands\Packages-Install.xml:79: The following error occurred while executing this line:
d:\tools\ctier\ctl\modules\Module\commands\Install-Module.xml:77: java.net.ConnectException: Connection refused: connect
Error handler caught failed command execution: re[PPMApp]#Packages-Install. reason: The following error occurred while executing this line:
d:\tools\ctier\ctl\modules\Deployment\commands\Packages-Install.xml:113: No message
Error: The following error occurred while executing this line:
d:\tools\ctier\ctl\modules\Service\commands\Deploy.xml:27: No message

From the console error message, it looks like that although ctl-setup command disable the workbench integration from the backend ControlTier server, yet the Packages-Install method think the local node is a webdav repo and try to fetch the artifact from it!!  How do I fix this?

The ctl-setup command I ran is this:

ctl-setup --log4j.logger.com.controltier.log.common=INFO --framework.ctl-project.create.buildfile=$CTL_HOME/lib/ant/controllers/ctl/projectsetupCmd.xml --resource.properties.autoupdate=false --framework.workbench.integration.enabled=false -n $COMPUTERNAME -N $COMPUTERNAME --framework.pkgRepo.uri=http://ctier-ubuntu/builds

Also why module-install method was called if we already install extension ourself?

Thanks,
Chengkai

On Jun 12, 2011, at 10:10 PM, Anthony Shortland wrote:

Moses Lei

unread,
Feb 8, 2012, 11:58:46 AM2/8/12
to contr...@googlegroups.com
Chengkai, you'll need to override Packages-Install not to execute the module installs in this mode. You can copy the command Deployment#Packages-Install to your subtype and then comment out this command (line 79 in the handler file... may be elsewhere in type.xml):

      <controller> 
        <execute> 
          <context project="${context.project}"/>  
          <command name="Install-Module" module="Module"/>  
          <arg line="-module @{type}"/> 
        </execute> 
      </controller>  

--
Moses Lei
[ Professional Services | DTO Solutions, Inc. ]
[ mobile: +1 703.901.5969 | e-mail: ml...@dtosolutions.com | aim/gtalk: ml...@controltier.com | yahoo: moseslei | windows live (msn): ml...@dtosolutions.com ]

chengkai

unread,
Feb 8, 2012, 12:31:42 PM2/8/12
to contr...@googlegroups.com
Thanks, and I will give this a try.
Reply all
Reply to author
Forward
0 new messages