ConMan Spec
Jack Cassidy
that was developed by four engineers from three companies.
It's designed to allow heavy sharing among code modules that are similar
but not identical. This is good for teams of developers working on a
line of related products.
The concept has been proved in practice for several years at one
company. Another company (now defunct) implemented much of the spec as a
commercial software product.
Our current positions don't allow us to pursue this, so we're posting
the spec in hopes that somebody (maybe one of us) will one day be able
to lead an Open Source development project to implement ConMan.
There are five files in the spec:
- CMtheory gives some basic info about the system
- command gives a detailed spec of user commands
- reports describes info users can get from the database
- quick is a one page trifold quick reference guide for users
- ez-con describes a reduced instruction set version of ConMan
They can be found at ftp://ftp.rbsmith.com/pub/proj/conman (sign on as
anonymous).
Jack Cassidy
my-las...@mediaone.net
1) Rigorous use of change sets with composite versioning and database
transactions, applied to all metadata as well as file content.
2) Extensive web-based reporting.
3) Advanced product configuration modeling/management capabilities
(ability to define and share component views across products,
and automatically share changes across related components).
If there are any potential open source sponsors for a reimplementation of this
spec, there are a few of us who would be very interested in hearing from you.
Example sponsors might include Collab.net, SourceForge, or any large enterprise
that is currently unhappy with the either offerings of the 800lb CM gorilla or
CVS, and wants an alternative CM tool.
The original implementation of this spec was done by Content Integrity, Inc, but
they've gone belly-up. Interested parties can contact the court trustee
liquidating the assets (including software) in Chapter 7. Case Number
01-13115-JNF. I can provide the court trustee contact information if you're
really interested. There is good commercial value for the asset, but there are
encumbrances that make it less suitable for use in open-source development.
I confess I am not without bias here, I was heavily involved in the
implementation of the software at Content Integrity. The spec authors would like
to see the ideas survive and be available to everybody, whether it's through
someone selling the former Content Integrity product, or reimplementation of the
spec as an open source project. But $$ (not many!) are required to make either
of these things happen. The spec is too complex to implement haphazardly, it
requires some full-time developers.
You can reach me by email by removing the hyphens (but not underscores) from the
following email address: 'd-a-v-e_...@hotmail.com'. (Anti-spam measure)
Jonathan Leffler
> There are three things that really set this system apart:
Have you looked at Subversion (http://subversion.tigris.org/)?
How does it compare?
> 1) Rigorous use of change sets with composite versioning and database
> transactions, applied to all metadata as well as file content.
>
> 2) Extensive web-based reporting.
>
> 3) Advanced product configuration modeling/management capabilities
> (ability to define and share component views across products,
> and automatically share changes across related components).
--
Jonathan Leffler (jlef...@earthlink.net, jlef...@informix.com)
Guardian of DBD::Informix 1.00.PC1 -- see http://www.cpan.org/
#include <disclaimer.h>
no-email...@because-i-will-just-get-auto-spammed.com
>my-las...@mediaone.net wrote:
>
>> There are three things that really set this system apart:
>
>Have you looked at Subversion (http://subversion.tigris.org/)?
>How does it compare?
Karl Fogel and I spent an afternoon talking about a year ago, he may be a better
person to answer this question. Apologies in advance, here's what I believe to
be some similarities and differences:
Subversion is attempting to maintain a CVS flavor of use while remedying many of
the classic flaws of CVS. Conman was built from scratch to solve problems of
higher-order change management and product configuration management.
(It's always nice to start from scratch when you can, but by the same token,
conman isn't compatible with all those cvs tools and knowledge).
Subversion and Conman both address the following issues:
- Directories, renames, file meta-data are all versioned
- commits are atomic (you the the whole change, never a partial change, on
commit)
- Repeated merges are handled gracefully (see composite versioning below)
- costs proportional to change size, not project size
-------------
Where Conman differs from subversion is:
- Conman isn't attempting CVS protocol and model compability. Subversion does.
However, the e-zman spec that Jack posted provides CVS and RCS
style of scm models at the component level (i.e. no higher product
configuration model).
- More metadata is versioned in Conman
You can view every bit of product/subsystem/branch/release existence
names and other data as it appeared at any time in history. The only
metadata that isn't versioned is stuff that doesn't make sense, like user
permissions (rights within the conman model).
- Content Integrity's implementation of conman was three-tier, for scalability
The conman spec would technically allow any two+ tier architecture,
but scalability for enterprise projects is part of the spec (20 second
workspace updates, etc.)
- Conman features four roles of users,
. developer (or "user", akin to cvs developer),
. product code manager ("uber-developer" who determines how products are
assembled using various components, and how changes flow between related
component views between products).
. release manager (people who fork releases, work to slush and freeze them
,etc)
. administrator (backup, server configuration, etc).
- Content Integrity's implementation of conman relied on a database.
I'm not sure if the conman spec required it, but mission critical reliance on
ACID database transaction properties is important for reliable change set
implementations. If I did it again in open source, I'd use a database again.
PostgreSQL provides the necessary multi-table update capabilities.
MySQL does not (at this time).
- Conman has a full model for product configurations. Products are assembled of
subsystems, which are basically branch-on-steroid views of shared product
components. Subsystems can inherit changes from related subsystems of a
component (termed a "class" in the conman spec).
The conman "port" command stands alone in the industry, providing
a coherent way to move hundreds of changes a week from one product
to another in a controlled and well reported fashion. Change migration can
happen automatically, or with "select, reject, defer" designations for
component csets.
- Conman provides for many reports.
Basically you should be able to ask, for any component/product/branch at any
time, is change X present at that time in that place or not, and receive a
yes/no answer. Contrast thus with most SCM products that focus on lists of
deltas that differ between branches to the point of unusability.
With conman's reporting you can see exactly who changed a given line of the
file you're looking at, and changes to every other file touched by that
change set.
It's a total big-brother system. Further more, you can deactivate the change
at any time simply by invoking 'cset_remove'.
- Conman's use of hierarchical change sets lets you move component csets of a
product change or move the whole aggregated set of component changes. (An
example is real useful here, but I'll spare you for now).
- Conman allows for creation of a change without promotion.
This is very useful. Developer A can create a cset in the repository without
promoting it to any product branch. Developer B can then add that cset to
his workspace and play with it, perhaps building more csets. So you can
audit/approve/test/deploy csets without impacting any branch of development
before the csets are in a blessed state. Changes are only promoted to a
branch (such that others can see it) when a "master_change" is executed.
---------------------------------
The Conman spec was designed to let any SCM vendor tackle the problem that it
tries to solve, but it's built on the notion of hierarchical change sets, which
is somewhat difficult for some legacy vendors to achieve, particularly if
they're building change sets on delta-relative engines.
Composite versioning lets you turn changes on and off in any view at any time,
and you'll get the whole change and nothing but the change when you do so. This
is a feature of composite versioning (not unique to Content Integrity's
offering) that plays well to the conman spec.
There's a religous debate about which approach is better (composite versioning
or delta-relative). I won't debate it, either approach is technically valid
for implementing change-sets, but they have wildly different implementations.
Implementing full on-off behavior and rigorous reporting on the status of
changes is essential to implementing the conman spec. (E.g. I want to know if
the virus bugfix was present in that release on that date, not weed through 300
deltas to see if the bugfix might be there).
The conman spec (and Content Integrity or other implementation of it) is really
designed to address the highest level of product complexity. A real world
example might have the following product/component statistics:
75 Products
E.g. printers, digital cameras, J2EE applications
150 Product Configurations
E.g. printers supporting postscript, printers without postscript
200 components (thing 'project root directory with makefile')
E.g. a 'pcl' printer component, a camera CMYK component, a set of java beans
for XML editing.
20,000 different views of those 200 compoents
E.g. Camera A and B both use the CMYK component, but have different views
of that component because they have different features (10-bit vs. 16-bit
color depth, new color smoothing algorithms, etc)
200 components * 100 component views each = 20,000
For a product assembled of 120 components, it would use 120 of the 20,000
possible subsystem views of the components.
2000 change-sets a month submitted against the products
That might actually be 6000 component csets (where a change against a product
actually effects 3 component subsystems). The logical changes against the
product might be migrated to other products wholesale (all 3 components
csets in a change to to new product branches), or the logical change might
be divided at the component level (e.g. "we want the postscript driver
component change, but not the JPEG rendering change")