Arusha Project package management

"Package management" is a big subject in Arusha land. This document gives the "Arusha lingo" for packages, and How To Think About Packages in General. It does not give down-to-earth details about package handling, because we emphatically do not mandate a specific way of doing things. In fact, many of the potential ways to use ARK involve adoption of different package managers.

Our Sidai team supplies a working multi-platform package manager. Its supplies practical detail about how that works and how to use it.

For the beginning Arusha person, a skim of this document is a good idea. Don't worry about the parts that sound hair-raising (!); you can come back to it when you're further along...

Terminology

In ARK land, what you might normally think of as a single package may really be several. For example, you might have one package for sendmail binaries, and another for a sendmail configuration. (Many teams would share the former, each team might have its own of the latter.)

A package's bits need not be "co-located". That is, the coherent bundle of bits might be scattered in various places in the file system. For example, a "cvs" package might have binaries in /our/bin, but also some dribblings in /etc. (We say that it is revealed in two (file) namespaces, one rooted at /our and another rooted at /.)

Goals for ARK package management

The overriding goal for ARK package management is to be able to control ALL software packages, across all platforms, in a consistent and straightforward (ha!) way.

(Not to put too fine a point on it...) That means we'd like to be able to type one command that installs, say, 200 packages across many dozens of machines of various flavors, in spite of hardware and software vendors' efforts to make such a thing impossible! Industrial-strength package handling, please...

That said, tasks include:

• Installing a package (including co-existing multiple versions of the same tool);

• Building it (before installing), if applicable;

• Uninstalling it;

• Deploying it on each individual machine -- as little as adding an automount-table entry, as much as a full copy on every individual host;

• Making the users able to use it, i.e. have it show up in their PATHs (we call this revealing the package);

• Verifying that the package is (and remains) in working order.

• Being able to "empower" (some?) users to do all of the above for at least some packages; i.e. we don't want sysadmin to be the bottleneck for New Cool Software getting into the system. (And we really don't want the "every user has compiled a copy in their home directory" syndrome...)

There's (lots) more -- e.g. announcing a package's availability or its retirement -- but we'll leave it at that for the moment.

How does ARK relate to other package managers?

Yes, and we will use them. For example, to manage the base bits of any Unix OS these days, you need to use the "native" package manager for that system. ARK may put a thin veneer over such use, but that's all.

(Perhaps more importantly, we acknowledge that we aren't the first to stumble into this territory. Some related systems have venerable track records, and we hope to do business with them.)

The main reason we might forego the pleasures of a "native" package manager and use ARK mechanisms instead is our preference for compiling from source, if possible.

Again, if another package manager (e.g. RPM) gives us a good handle on compiling from source like we want to, then ARK will simply be a veneer on that process.

The other reason we may not use a "native" package manager is: Never underestimate the wacky weirdness that third-party software vendors will come up for distribution of their software. Not only will their scheme have nothing to do with any package manager, as often as not, it will be something that could only be invented under the influence of major pharmaceutical products.

ARK tries to provide a framework in which ALL package management can be expressed. Even a formal way to say "This package has to be installed by hand" is better than nothing.

The basic idea

% get-and-unpack-source package--1.0
% cd package--1.0
% ./configure --prefix=/somewhere
% make all
% make check
% make install
% deploy-to-each-machine package--1.0
% reveal-to-users package--1.0
% announce-to-users package--1.0
# and later...
% verify-on-each-machine package--1.0

What's important is that it also makes sense for other package types! For example, if the package is a binary RPM, then "make install" might map to "rpm -Uvh package-1.0.i386.rpm" and all the other steps might map to "echo nothing to do".

So that's the idea: we push a package through a sequence of steps (e.g.: configure, compile, install, ...), firing relevant code to Do The Business for each.

The installing person hopes that most/all of the code to do the installation already exists in ARK land; in any case, s/he can always override the "default" behaviour.

As important as the automated installation of diverse packages is the fact that you have also created an unambiguous, executable description of how you did it!

User interface to package management

Higher-order (and first-order) packages

A package that directly provides the bits to use on a host is a first-order package (the norm). A package that produces as its output a first-order package is a second-order package; and so on.

All packages above first-order are lumped together as higher-order packages.

Higher-order packages are an incredibly useful way to go about certain sysadmin tasks.

Consider, for example, the System-V-style boot scripts (`/etc/init.d', `rc2.d' and all that...) which you find on many systems... These determine what services are set up at boot time, with various important implications, not least for security.

Ideally, you would generate and/or manage all such boot scripts for your whole site from a single specification. (Trying to do it by hand-tweaking symbolic links is a recipe for bitrot of the worst kind.)

But you can't simply clobber all boot-script directories into a single state -- write over a Solaris set with a Redhat Linux set, and you can kiss your Solaris boxes good-bye.

The Arusha Way is to have a second-order `boot-script-manager' package which, from the all-important single specification, generates boot-script packages, one per host in your system. Deploy these (normal, first-order) packages as usual, and all is perfect!