Developers can work in parallel and not tromp on each others' toes.
The huge majority of the hosts (e.g. 390 out of 400) are ``in production'', must remain rock-stable, and are not available as ``guinea-pigs''.

Each developer has one or two machines that they can mangle as they wish -- their own `playpen'.

There are, say, five machines that can be used for `staging'; i.e. practicing roll-outs of new stuff, from-raw-disk system builds, etc. There is at least one `staging' machine for each platform (Solaris, Linux, AIX, etc.) at the site.

In CVS terms, I would have a workspace (cvs checkout...) for production, another for staging, and one each (at least) for each developer.

Each host would be tied to a workspace. Each developer would be given one or more machines to mangle -- they would be "tied" to the developer's workspace. ("Tying" requires nothing more than setenv'ing ARK_PROFILE to point to a particular file.) Developers presumably work against the HEAD branch (cvs'ly speaking).

Then you'd have a small group of machines -- one of each type, probably -- tied (perhaps temporarily?) to the "staging" workspace. This workspace would be 'cvs update'd against a not-yet-in-production branch.

All the other machines would be tied to the production workspace. This workspace would be 'cvs update'd against a (stable) release branch.

Back to teams: I can imagine a 'sidai-plus' team which says, "Yes, we agree with how Sidai does the low-level stuff; but we need a strict development/test/staging/production approach on top of that". The prototyping machinery will let you set this up quite easily: where `glasli1' has...

   <prototype team="sidai" name="GNU" />

... you would have ...

   <prototype team="sidai-plus" name="GNU" />

... or perhaps ...

   <prototype team="sidai-plus" name="ALL" />
   <prototype team="sidai"      name="GNU" />

(The latter would work if 'sidai-plus' didn't introduce much new <code>; instead, it just <param>'d existing sidai code to make it do the right thing.)

Using ARK teams as an `air gap' between development and production

[File this under "weird things you could do with ARK"...]

Imagine having two teams, 'devel'(opment) and 'prod'(uction). 'devel' would assemble things to be deployed on production machines; however, it wouldn't even have methods for deploying and revealing! In fact, its 'install' method might simply produce RPMs (or equiv) of the bits to be deployed/revealed.

Meanwhile, 'prod' wouldn't even have methods for compiling/checking/installing! It wouldn't know how to do anything but accept the RPMs from 'devel' and splat them around in a productionly manner.

By doing it with teams in this way, you set up an 'air gap' between development and production which cannot be crossed by mistake.