Managing multiple environments and version control
Zack Pitts
processes for multiple developers, multiple sites, and multiple development
environments. I am asking for some guidance on how to set up these pieces
for maximum efficiency. Please excuse the long descriptions and
off-topicness of this post (believe me, I wish this were a Mach-II
question!).
Current Environments:
- CF7 & CF8 on IIS 6
- All servers use FTP for file transfer
- Using Perforce for version control, but probably moving to SVN soon
- Production severs are clustered across 3 nodes that must be manually
synched
- Staging server where clients view progress also acting as development
server (so clients also see real-time errors)
- CEO cringes
Current Development Process:
- Developer checks file out of Perforce, edits in Dreamweaver, saves file,
and submits back to Perforce
- Developer logs into staging server through Remote Desktop and synchs the
server's Perforce client to the submitted changes
- Developer refreshes staging site to view change
- Developer cringes
Current Production Process:
- Developer verifies change on staging
- Developer logs into production node 1 and synchs Perforce client to the
submitted changes
- Developer logs into production node 2 and synchs Perforce client to the
submitted changes
- Developer logs into production node 3 and synchs Perforce client to the
submitted changes
- Developer thinks 'there's gotta be a better way!'
Future:
- Isolated development environments for each developer, testing server for
integration testing, and staging server for client review
- Developers submit changes to version control when ready for integration
and synchs testing server (or perhaps nightly automated builds)
- Validated changes on testing server are deployed to staging server for
client review
- Final QA'd version is deployed to all 3 nodes on cluster in one step
- Developer exclaims 'woot!'
Brutal Truth:
- We don't know how to accomplish this
- CEO wonders why he hired us
I think we're on the right track conceptually, but we're missing some of the
nuts and bolts.
Questions that arise in our minds:
- Should developers work on their localhost, or should we create separate
instances on a common development server for each developer to work from?
- Should the files on the servers (dev, test, stg, prod) be directly under
version control, or should they contain merely a particular deployed
version?
- How do we establish nightly builds on our testing server? (I've heard of
Cruise Control, is that relevant?)
- How do we create push-button deployments for staging and (especially)
production nodes? (I've heard of Ant, is that relevant?)
- What's the best IDE to use for seamless integration with our version
control? Dreamweaver? CFEclipse? Heck, HomeSite?
- Should we dump Perforce and use SVN? (We already have one project in SVN
through CVSDude, but our Flex contractors are the only ones who understand
it.)
The reason I'm reaching out to you kind folks is that I've seen that the new
Mach-II stuff is in SVN through CVSDude and creates nightly builds. Cool
stuff. So I'm hoping you will be able/willing to get us pointed in the
right direction.
Thanks,
Zack
Peter J. Farrell
nightmares as you already outlined. FTPing to a development server is
slow, error prone and just plain inefficient. Not to mention you step
on other people's work. Go local and then have a staging server where
working changes are pushed and shown to clients.
> - Should the files on the servers (dev, test, stg, prod) be directly under
> version control, or should they contain merely a particular deployed
> version?
>
store gb sized audio files that change infrequently is just plain stupid
-- I know from personal experience). For large binary files, use a
webDAV enabled server as a networked drive (CVSDude just announced a
beta of webDAV that comes with their plans).
> - How do we establish nightly builds on our testing server? (I've heard of
> Cruise Control, is that relevant?)
>
that accomplishes this, however it's more geared toward languages like
Java that need to be compiled before you deploy. I'd look into ANT
which is a tool used to create builds. I use it pretty extensively in
my projects in the past couple of months. For example, there is a
build.xml file in the Mach-II repo. It does things that we have to do
manually -- creating zips, nightlies and even ftp. There are a lot of
third-party task extension to ANT such as SVN, JSMin (minify JS),
CFCUnit and Selenium to name a few I use all the time. In short, you
can write an ANT file that has a task that will checkout a copy of the
app from SVN, does stuff to it (minify JS, run unit tests with CFCUnit
or functional UI tests with Selenium) and ultimately deploy it to staging.
> - How do we create push-button deployments for staging and (especially)
> production nodes? (I've heard of Ant, is that relevant?)
>
it to the trunk (or could be a branch) and basically double click deploy
in Eclipse. This could be automated by running a cron job on the ANT
task or the equivalent scheduled task on windows (could even be a CF
schedule task that uses cfexecute).
> - What's the best IDE to use for seamless integration with our version
> control? Dreamweaver? CFEclipse? Heck, HomeSite?
>
it's hard to beat at the price. It also ships with ANT (it even has a
perspective in Eclipse for easy GUI stuff).
> - Should we dump Perforce and use SVN? (We already have one project in SVN
> through CVSDude, but our Flex contractors are the only ones who understand
> it.)
>
go with a hosted option like CVSDude (Mach-II lives in a module in our
corporate account there). It saves you the pain of managing a system
and gives you things like Trac or Bugzilla out of the box, a nice GUI
for permissions / adding new users, 10 min hot backups, etc. The list
of benefits is outweighs the minimal cost (pennies a day per devoper).
> The reason I'm reaching out to you kind folks is that I've seen that the new
> Mach-II stuff is in SVN through CVSDude and creates nightly builds. Cool
> stuff. So I'm hoping you will be able/willing to get us pointed in the
> right direction.
>
Hatcher (it's the second edition of Jave Development with ANT put out
by Manning Press). Quite frankly, the nightly builds for Mach-II are
manual - on of the developers has to double click the ANT task in
Eclipse - mostly because I've been plain lazy to setup the cron job.
HTH,
Peter
Sean Corfield
but I wanted to just emphasize a couple of issues...
On 11/14/07, Zack Pitts <folks...@hotmail.com> wrote:
> - Should developers work on their localhost, or should we create separate
> instances on a common development server for each developer to work from?
Localhost. Everyone should have a full local development stack.
Preferably with a local copy of the DB if they are doing active schema
development, otherwise using a shared dev DB instance is OK.
> - Should the files on the servers (dev, test, stg, prod) be directly under
> version control, or should they contain merely a particular deployed
> version?
The central repo (on some well maintained, well backed up server
somewhere) should contain *everything* under version control (well,
everything that is reasonable). Developers should interact with
version control for all their localhost files.
Builds to dev/test/stage should be automated and should pull files
from the central repo. Nightly builds are good. Continuous integration
is better. Builds to prod should be automated but tightly controlled
(builds to stage should be carefully controlled).
> - How do we establish nightly builds on our testing server? (I've heard of
> Cruise Control, is that relevant?)
Cruise Control is a reasonable way to automate builds as part of a
continuous integration cycle but any scheduled script that can run ant
every night will do most of what you need.
Cruise Control is great if you have an additional server instance
where you can set it up to automatically watch the source code repo
and automatically kick off a build and test cycle whenever code is
committed. It should export all the code, deploy to a known clean
environment and automatically run your entire test suite. It has a web
dashboard showing the status of the most recent build and, if it
failed to build / test, who broke the build :)
You can also get cool toys like a wifi bunny that glows green / red
and whose ears droop when a build fails. This sort of stuff helps
people feel ownership of code quality!
> - How do we create push-button deployments for staging and (especially)
> production nodes? (I've heard of Ant, is that relevant?)
ant is your friend. ant can export files from CVS / SVN / Perforce and
can auto-deploy, run scripts, run tests - both unit tests with cfcUnit
and scripted browser tests with Selenium for example.
> - What's the best IDE to use for seamless integration with our version
> control? Dreamweaver? CFEclipse? Heck, HomeSite?
Eclipse. Hands down.
> - Should we dump Perforce and use SVN? (We already have one project in SVN
> through CVSDude, but our Flex contractors are the only ones who understand
> it.)
There's a reasonable Perforce plugin and ant can play nice with it
too. My team at Adobe (Hosted Services) ran Perforce and had fully
automated builds, test suites and continuous integration via Cruise
Control. It was a sweet setup.
That said, given full free choice, I'd pick SVN every time. It's free,
it's *much* easier to setup and the tool support is much, much better
than Perforce. You can also easily outsource your version control and
bug tracking for a low monthly cost to any number of secure hosted
services. Very good for infrastructure costs, especially if you have a
distributed workforce.
You did not mention bug tracking. Bug tracking is critically important
to high quality software and, more importantly, bug tracking needs to
be fully integrated with your source code repo system to be easy to
use.
Trac / SVN is a killer combo. Folks open tickets in Trac for bugs,
they have a roadmap, a wiki, a timeline (showing who did what when).
Then when you commit a fix, you put in a comment that says (for
example) "Fixes #123 by adding error checks to the file access" and
Trac / SVN collaborate to add the ticket number and comment to the
changeset and also the changeset number and comment to the ticket and
automatically closes the ticket. That means you can easily jump from
the ticket to the code changes and back. It's very powerful and it
really helps keep your development process under control.
Nearly all of the major open source frameworks in ColdFusion use Trac
/ SVN now. I can't recommend this highly enough! I've driven three
major releases of Fusebox on top of Trac / SVN as well as my startup's
entire development process.
--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood
Peter J. Farrell
> You did not mention bug tracking. Bug tracking is critically important
> to high quality software and, more importantly, bug tracking needs to
> be fully integrated with your source code repo system to be easy to
> use.
>
Eclipse that integrates with Trac (or other major defect management
tools like Bugzilla, etc.). It's a perspective in Eclipse with nifty
automatic popups (like when your email comes in) when the tickets you
have an interest in are changed or assigned to you (this is done through
a custom queries that you setup - so it's easy to setup up a query that
watches all active tickets or all of your co-workers, etc.). You can
open new tickets in Eclipse via a form in a tab. You can also attach
contexts to tickets (you can see this on some Mach-II tickets - with an
attachment called Mylyn.zip). The attach contexts is very nifty as it
allows you to associate files with a particular ticket. The next time
you somebody opens the ticket, they can have all the files that are
associated with the context opened for them. This makes for easy
documentation of what goes with what when fixing a defect or if a defect
gets assigned to somebody else.
A little hidden gem with Mylyn is that it sync up every 20 minutes with
your defect management tool so you can work offline (e.g. no internet
connection). Any changes to tickets are automatically sync'ed up when
you leave offline mode and become connected again. I haven't had a
chance to try this out yet, but it sounds really cool for the times I
work in while away on the weekend.
As an aside, I've found the wiki in Trac very useful for documenting
requirements for applications (whether it's design or deployment
requirements) and as a tool to instruct a developer about a typical repo
structure, file naming conventions or anything that usually gets sent as
an email memo and then immediately lost in your inbox.
.Peter
Stephen Moretti
As an aside, I've found the wiki in Trac very useful for documenting
requirements for applications (whether it's design or deployment
requirements) and as a tool to instruct a developer about a typical repo
structure, file naming conventions or anything that usually gets sent as
an email memo and then immediately lost in your inbox.
Are there any plug-ins, short-cuts, cool tricks that anyone has used to automate documentation of cfcs, database schemas etc on the wiki?
Regards
Stephen
Zack Pitts
learning curve ahead of me!
Zack