Suppose your project tests frantically, integrates continuously, and releases easily.
Suppose then you hire a new coder, and need another workstation.
Then you have a party. All the source requires 3rd-party libraries, of various versions, which must be matched, and these require configuration options and scripts installed here there and everywhere. The project's eldest guru must pair with the new guy, run the system over and over again, diagnose each glitch, and reconfigure it away. This situation resembles delaying integration long enough to encounter "integration hell". Instead of smooth saling you get dominos of bugs.
Is this a process smell? Does it indicate someone should add a motherhood story indicating the codebase is continuously outegratable? Would an off-the-shelf configuration manager handle it? If so, wouldn't configuring that tool be a chore each time the team changes a library?
(Cloning a dev machine's hard drive is cheating, and insecure.)
> Does it indicate someone should add a motherhood story indicating the > codebase is continuously outegratable?
"All release builds are to be done off a virgin workstation."
VMWare provides an economical way of getting a virgin workstation whenever you need one.
If you release frequently enough (like once per iteration), that rule would flush out the appropriate solutions to that problem in your context. (There's a meta-pattern here - "Prescribe the problem".)
"Eldest guru" is an antipattern. The new guy has to be able to make a product build, provided only a pointer to the source repository and a page of documentation. The dark lore of what incantations are required to reproduce a build needs to be extracted from the gurus and stored in the form of shell scripts and (lightweight) docs.
Third-party version issues: there are different kinds of dependencies involved, and thus as many different kinds of solutions. "Nice" components (dependency is on a single file, such as a JAR file) go into a "binary" directory of the version control tool; the file needs to be adorned with a version number. Components with a more complex directory structure can reduce to "nice" components if you zip up the whole thing.
"Nasty" components (such as Oracle or Sybase DBs) are a different problem. One solution is to install those on your virtual workstation, off a "clean" OS, and snapshot the workstation's hard disk. Label the snapshot file with a version number and store it with binaries. As soon as you have more than one "nasty" component, that stops being as good a solution - you get closer to "clone a developer's hard disk".
In article <S5e2c.56030$J97.48...@newssvr33.news.prodigy.com>,
"Phlip" <phlip_...@yahoo.com> wrote: > Then you have a party. All the source requires 3rd-party libraries, of > various versions, which must be matched, and these require configuration > options and scripts installed here there and everywhere.
This is why we have configuration management. You need to know how to create a new workstation when needed, just as you need to know how to replace one of your servers.
> The project's > eldest guru must pair with the new guy, run the system over and over again, > diagnose each glitch, and reconfigure it away.
Interesting that you do it by pairing, rather than having a document, which would be more usual in a non-extreme setting. But surely anyone should be able to do it... maybe you could have a different person configure a blank machine each week, to keep the knowledge live. I'd still recommend writing something down, though.
> This situation resembles > delaying integration long enough to encounter "integration hell". Instead of > smooth saling you get dominos of bugs.
You forestall integration problems by integrating frequently; you could do the same for configuration issues by regularly taking a blank machine and following the script all the way through to checking out and building the product.
> Suppose your project tests frantically, integrates continuously, and > releases easily.
> Suppose then you hire a new coder, and need another workstation.
> Then you have a party. All the source requires 3rd-party libraries, of > various versions, which must be matched, and these require configuration > options and scripts installed here there and everywhere. The project's > eldest guru must pair with the new guy, run the system over and over again, > diagnose each glitch, and reconfigure it away. This situation resembles > delaying integration long enough to encounter "integration hell". Instead of > smooth saling you get dominos of bugs.
> Is this a process smell? Does it indicate someone should add a motherhood > story indicating the codebase is continuously outegratable? Would an > off-the-shelf configuration manager handle it? If so, wouldn't configuring > that tool be a chore each time the team changes a library?
Yes, it's a process smell of the same order as assuming that a program is never going to be updated, so it isn't important to write it clearly.
I'm not going to write a motherhood story for that, just like I'm not going to write a motherhood story for coming to work in locally acceptable attire. There are certain things you do because they are "generally accepted good practice", and keeping everyone's development environment in synch with the integration system is one of them.
If the customer wants to cut corners on that, then we've got a real division of responsibility issue.
John Roth Cloning a dev machine's hard drive is cheating, and insecure.)
Laurent Bossavit wrote: > > Does it indicate someone should add a motherhood story indicating the > > codebase is continuously outegratable?
> "All release builds are to be done off a virgin workstation."
> VMWare provides an economical way of getting a virgin workstation > whenever you need one.
> If you release frequently enough (like once per iteration), that rule > would flush out the appropriate solutions to that problem in your > context. (There's a meta-pattern here - "Prescribe the problem".)
> "Eldest guru" is an antipattern. The new guy has to be able to make a > product build, provided only a pointer to the source repository and a > page of documentation. The dark lore of what incantations are required > to reproduce a build needs to be extracted from the gurus and stored in > the form of shell scripts and (lightweight) docs.
The eldest guru does not assist every danged release.
(The corollary here is that every workstation can simulate building a release.)
In my parable, the guru must assist installing the build environment onto a new workstation, the first time, before the new kid can use it. But you are calling _that_ an antipattern.
So by "all release builds are to be done off a virgin workstation" do you mean the workstation has no libraries, compilers, script languages, databases, or anything, and the top-level build script installs everything that wasn't there?
If so, all projects that release anything significant run under the Motherhood Story "each time you upgrade, look at, or think about a new tool, you must upgrade the build scripts to test automatically installing it onto a virgin release computer, or a new development workstation.
If so, the XP verbiage under-represents this. Newbie teams ain't doing it.
Some components fiercely resist automated installing. Researching how to squeeze them into a script could turn into a very long job.
Sean Case wrote: > Interesting that you do it by pairing, rather than having a document, > which would be more usual in a non-extreme setting. But surely anyone > should be able to do it... maybe you could have a different person > configure a blank machine each week, to keep the knowledge live. I'd > still recommend writing something down, though.
In my antipattern, if they didn't think to script it, they wouldn't think to document it. Would you like to be the newbie, handed a shabby and obsolete document, and told "here, install this stuff, and don't look bad or interrupt us, on your first day?"
Next, manually practicing installing breaks the rule "Three Strikes and you Automate". Anything a programmer finds themselves doing often enough to spot the pattern, they have a duty to automate that task.
> You forestall integration problems by integrating frequently; you could > do the same for configuration issues by regularly taking a blank machine > and following the script all the way through to checking out and building > the product.
In our pattern, one need only execute the script, and it does the rest - installs, configures, checks out source, rebuilds totally, tests totally, builds an installer, and tests the installer.
(A related question: Would probing the installer with its own diagnostic tools be enough to "test" it? Or should we install onto our own clean machine to test it?)
> The eldest guru does not assist every danged release.
I've consulted to one firm where the people /thought/ they could do a build without the Eldest in attendance. Then one day they actually tried that, at my prompting. Surprise...
> In my parable, the guru must assist installing the build environment onto a > new workstation, the first time, before the new kid can use it. But you are > calling _that_ an antipattern.
Indeed I am. "What we need to do a build" is part of the intellectual capital tied up in the product, just as much as the source code. It must be inspectable, reviewable, modifiable, just as much as the source code.
> So by "all release builds are to be done off a virgin workstation" do you > mean the workstation has no libraries, compilers, script languages, > databases, or anything, and the top-level build script installs everything > that wasn't there?
As close to that as you can manage. How close you can manage depends wildly on the environment, though. Installing a Linux environment brings in any number of nice things, such as CVS, gcc, etc. which only require checking a few checkboxes after inserting the CD. An equivalent Windows setup is more involved.
I'd install "big" items such as the compiler and databases manually, then ghost the disk or snapshot the virtual PC, and call the resulting image a "virgin workstation". Everything less pervasive, such as third party libraries, build tools, etc. would go into a repository of some sort.
> If so, all projects that release anything significant run under the > Motherhood Story "each time you upgrade, look at, or think about a new tool, > you must upgrade the build scripts to test automatically installing it onto > a virgin release computer, or a new development workstation.
You can probably dispense with scripting for "think about", but yes, if you bring in something crucial to reproducing the product, the decision has to be under collective ownership - reviewable, testable, etc.
It's also a reason to minimize the frills and keep the build design simple, just like the code design.
> Some components fiercely resist automated installing. Researching how to > squeeze them into a script could turn into a very long job.
I'm fiercely hoping that widespread application of the principles I have outlined turns into a market force, driving out of the market tools which resist reviewability of *any* part of the product design. :)
> Would you like to be the newbie, handed a shabby and obsolete > document, and told "here, install this stuff, and don't look bad or > interrupt us, on your first day?"
Point. :)
"Hi Dan, this here is your brand new workstation. Please take ten minutes to pull down the sources from the CVS repository 'CoolProduct', which lives on the server named Gandalf. Run the top-level make script; when you get to a green bar, raise your hand and I'll scoot over to pair with you, and introduce you to the story I'm working on. We'll see how well that twenty-minute presentation of the metaphor from this morning worked. If we've screwed up and you don't get to a green bar within twenty minutes, just holler."
> (A related question: Would probing the installer with its own diagnostic > tools be enough to "test" it? Or should we install onto our own clean > machine to test it?)
This gets hairy if the installer does things like tweak the Windows registry. In the simplest case, the installer is just a wrapper atop the equivalent of unzipping an archive to a directory; I wouldn't test much beyond "doing a build results in populating a directory, the contents of which match the following manifest".
In more complex cases, there will be user stories driving what the installer does, and the installer is a subproject in its own right; my prejudice is that we'll have to find a way to separate testing the installer from testing the main product.
Laurent Bossavit <laur...@dontspambossavit.com> wrote... > "Eldest guru" is an antipattern. The new guy has to be able to make a > product build, provided only a pointer to the source repository and a > page of documentation. The dark lore of what incantations are required > to reproduce a build needs to be extracted from the gurus and stored in > the form of shell scripts and (lightweight) docs.
For builds, yes. But for setting up and configuring a new workstation (which should be just like the existing workstations), I'd say it's a common anti-pattern to inflict this on the new guy.
I favor having an experienced member of the team help the new guy set up the development environment and get it running. They should be able to do private builds, updates, etc. from then on, with no further outside help and minimal documentation. (But in a pairing environment, help would be well neigh inescapable, I think. ;-)
I've had success with starting development machines from a clean master image. It's *much* better for an experienced developer to create the master image by building on a clean machine than to copy an existing development machine that's been in use for a number of months. The problem this solves is that it can often take *DAYS* to install all the appropriate software needed on a development workstation from CDs. People forget about this issue.
> Third-party version issues: [...]
All the developers must be on the same version of the libraries (if they're working on the same release version of the software). To upgrade, one or two developers can "experiment" with the new version until they get the bugs worked out, and then everyone else upgrades.
Internet Explorer is the product I've had the most trouble with: It's somewhere between hard and impossible to get old versions, and they're changing it all the time -- with automated downloads. On my current project, which uses extended browser functionality extensively, this has been a significant thorn in our sides for some time. (Eventually we dedicated a person to track down and fix the problems in our code, so it would be more portable across versions.)
> "Nasty" components (such as Oracle or Sybase DBs) [...]
Databases are usually upward compatible. But that doesn't mean that successful integration test with version 9 will work the same in production with version 8. It's usually not a big problem, but it *can* bite you. (I've found that people being in denial are a bigger problem than the actual technical issues, on this one.)
Laurent Bossavit wrote: > Phlip wrote: > > The eldest guru does not assist every danged release.
> I've consulted to one firm where the people /thought/ they could do a > build without the Eldest in attendance. Then one day they actually tried > that, at my prompting. Surprise...
> > [cut]
The whole exchange was simply excellent. Very refreshing.
> Suppose your project tests frantically, integrates continuously, and > releases easily.
> Suppose then you hire a new coder, and need another workstation.
> Then you have a party. All the source requires 3rd-party libraries, of > various versions, which must be matched, and these require configuration > options and scripts installed here there and everywhere. The project's > eldest guru must pair with the new guy, run the system over and over again, > diagnose each glitch, and reconfigure it away. This situation resembles > delaying integration long enough to encounter "integration hell". Instead of > smooth saling you get dominos of bugs.
> Is this a process smell? Does it indicate someone should add a motherhood > story indicating the codebase is continuously outegratable? Would an > off-the-shelf configuration manager handle it? If so, wouldn't configuring > that tool be a chore each time the team changes a library?
> (Cloning a dev machine's hard drive is cheating, and insecure.)
The way we do it is through the use of an image of the tools for the current configuration and a wiki site to take you through any steps that are needed for finishing off the configuration and where the CVS repository resides and which CVS modules to check out. I have noticed that there is need for some personal interaction with any new hire. They feel that they have to prove themselves and if the wiki site has incorrect directions then the new hire might feel like they have failed their first task (since they might assume the steps are correct).
I find it a more concerning problem that early on the new hire is taught that the senior guru is busy and not to bother him! Does anyone consider the cost of properly integrating a new hire? If you train a new hire not to "bother" anyone, then how can one complain when they do not communicate?
In article <MPG.1ab3bb4e15895fd4989...@news.noos.fr>, Laurent Bossavit wrote: > involved, and thus as many different kinds of solutions. "Nice" > components (dependency is on a single file, such as a JAR file) go into > a "binary" directory of the version control tool; the file needs to be
That bit is easy... we have lib under version control.
> "Nasty" components (such as Oracle or Sybase DBs) are a different
I've been dealing with this all week.
Our project team have just moved from the customer site to our office (yes yes, onsite customer and all that -- but you don't want to go there). Recent staff movements have made *me* the resident guru (!) and I've had to set up four new workstations.
Install ATG Dynamo, patch ATG Dynamo, install IIS, install IIS connection module, configure IIS connection module by tweaking registry, install server and client certs for IIS. Install VSS client. Configure VSS client on Novell network with no NT authentication. This stuff is *very* hard to automate. Some of these things can only be scripted on paper (especially under rotten environments like Windows).
For the frequency it is done, I think the most practical solution is the one mentioned upthread: local guru gives setup doc to newbie and says, "I'll be over in 30 minutes if (when?) it doesn't work". Note that I don't like this either.
Robert Atkins wrote: > Install ATG Dynamo, patch ATG Dynamo, install IIS, install IIS > connection module, configure IIS connection module by tweaking registry, > install server and client certs for IIS. Install VSS client. Configure > VSS client on Novell network with no NT authentication. This stuff is > *very* hard to automate. Some of these things can only be scripted on > paper (especially under rotten environments like Windows).
I worked a site with a cool solution for "virgin release machine" - but not for warming up a newbie's 'puter.
They ran Linux with a dual-boot. The Linux partitions could treat the other partitions as data, and copy complete images in. So the Linux side had a menu of install sets to chose from.
Note that the Law of Pairing requires the newbie to pair the first day, on communal machines. If they all roll over and one falls out, that will be the guru, who must now configure a new workstation for himself.
(Rob - I have to ask - have you been the target of much ribbing for all the "health food" these days that tastes like the cardboard tube from the center of a roll of paper towels?
In article <nb_3c.57802$eG1.35...@newssvr33.news.prodigy.com>, Phlip wrote: > partitions as data, and copy complete images in. So the Linux side had a > menu of install sets to chose from.
This sort of thing is much easier (possible?) when you have a Real Operating System, but often you don't get that choice.
> (Rob - I have to ask - have you been the target of much ribbing for all the > "health food" these days that tastes like the cardboard tube from the center > of a roll of paper towels?
Actually, being in Australia I wasn't aware of my more famous namesake until someone emailed me thanking me for operating on his wife's kidneys 15 years ago. And you've got no idea how odd it is to wake up to a newspaper headline telling you that you're dead...
In article <nb_3c.57802$eG1.35...@newssvr33.news.prodigy.com>,
"Phlip" <phlip_...@yahoo.com> wrote: > Note that the Law of Pairing requires the newbie to pair the first day, on > communal machines. If they all roll over and one falls out, that will be the > guru, who must now configure a new workstation for himself.
So how do people other than the guru learn how to configure new workstations?
> > Note that the Law of Pairing requires the newbie to pair the first day, on > > communal machines. If they all roll over and one falls out, that will be the > > guru, who must now configure a new workstation for himself.
> So how do people other than the guru learn how to configure new > workstations?
Note that the Law of Promiscuous Pairing requires the guru to pair when configuring a new workstation. Note this is also a development effort - tuning all the scripts.
In article <gsc-ADFBCF.23192913032...@nasal.pacific.net.au>, Sean Case wrote: > In article <nb_3c.57802$eG1.35...@newssvr33.news.prodigy.com>, > "Phlip" <phlip_...@yahoo.com> wrote: >> Note that the Law of Pairing requires the newbie to pair the first day, on >> communal machines. If they all roll over and one falls out, that will be the >> guru, who must now configure a new workstation for himself.
> So how do people other than the guru learn how to configure new > workstations?
When the Guru leaves or is incapacitated, the apprentice guru must puzzle it out from the available docs.
"Phlip" <phlip_...@yahoo.com> wrote in message <news:S5e2c.56030$J97.48219@newssvr33.news.prodigy.com>... > The project's > eldest guru must pair with the new guy, run the system over and over again, > diagnose each glitch, and reconfigure it away. This situation resembles > delaying integration long enough to encounter "integration hell". Instead of > smooth saling you get dominos of bugs.
> Suppose your project tests frantically, integrates continuously, and > releases easily.
> Suppose then you hire a new coder, and need another workstation.
> Then you have a party. All the source requires 3rd-party libraries, of > various versions, which must be matched, and these require configuration > options and scripts installed here there and everywhere. The project's > eldest guru must pair with the new guy, run the system over and over again, > diagnose each glitch, and reconfigure it away. This situation resembles > delaying integration long enough to encounter "integration hell". Instead of > smooth saling you get dominos of bugs.
> Is this a process smell?
Yes. You need to automate this as much as possible, and then painstakingly document anything that can't be automated. On Linux or Unix that would be pretty trivial -- you put all the software on a network disk, and tell the user to add a call to setup_hairy_project.sh.
> Does it indicate someone should add a motherhood > story indicating the codebase is continuously outegratable? > Would an off-the-shelf configuration manager handle it?
On UNIX, yes, practically any configuration manager could handle 99% of this problem, and you can take care of the remaining 1% by defining some symbolic links. On Windows, AFAICT, it seems the answer is no, since every file reference in windows is qualified by some drive letter that will vary across installations.
> If so, wouldn't configuring > that tool be a chore each time the team changes a library?
Again, on Unix it's no problem at all. You just change the symbolic reference to point to the new version, or else just install the new version in your configuration management system.
> > The project's > > eldest guru must pair with the new guy, run the system over and over again, > > diagnose each glitch, and reconfigure it away. This situation resembles > > delaying integration long enough to encounter "integration hell". Instead of > > smooth saling you get dominos of bugs.
|
