This document describes aspects of the initial setup of one or more hosts for full-blown Sidai-style use of the Arusha Project (ARK). There are three reasons you might read (some of) this document:

You are doing the `sample1' team's try-ark thing, and you need to know how to get/install the pre-requisite tools.
You've done try-ark, and are now aiming to set up ARK on one or more machines for real.
(You may be following the Sidai `ARK for real' recipe; it will refer you here for specific tasks...)
You already have ARK running, and are bringing a new machine into play; you want to `cheat' and clone an existing machine's ARK setup.

Getting/installing pre-req tools

You only have to play this game for the very first host of a given type (but you might choose to do otherwise). Because we manage all of these tools in ARK-land, cloning an existing box is enough to get these tools onto the second and all subsequent boxes.

What prerequisite tools you need

Here's what you'll need to get off the ARK ground; all are freely available. Platform-specific details for how to get them are below, for HP-UX, Linux, and Solaris.

A modern Python, with the expat parser built in:

We strongly recommend a modern Python system, version 2.1 or above. (If you just can't, see about a `not-so-modern' Python, below.)

The Python must have an Expat XML parser built in, which can be a kinda random property... Here's a check:

partain@hagrid% /usr/bin/python
Python 1.5.2 (#1, Jul  5 2001, 03:02:19)  [GCC 2.96 20000731 (Red Hat Linux 7.1 2 on linux-i386
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam

>>> import xml.parsers.expat
Traceback (innermost last):
  File "<stdin>", line 1, in ?
  File "xml/__init__.py", line 21, in ?
    __version__ = "$Revision: 1.16 $".split()[-2:][0]
AttributeError: 'string' object has no attribute 'split'
>>>

Bad!

Python 2.2.1 (#3, Aug 29 2002, 12:17:41) 
[GCC 3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import xml.parsers.expat
>>>

Lovely!

GCC (GNU Compiler Collection):

All you really have to have is enough C compiler to build GCC yourself, but for now it's probably simplest just to get a decent GCC and use it.

Perl

Optional, strictly speaking... The try-ark script uses it; the one you've got lying around will probably work...

A not-so-modern Python, with PyXML

(Only if you don't have a modern Python; above.)

(Note: instructions to check the health of a putative PyXML are with the installation instructions, below.)

If you use an older Python system (e.g. version 1.5.2), then you will need the PyXML XML-munging goodies (we've used versions 0.5.5.1 and 0.6.2+).

The recent versions of PyXML (0.8.x) may only work with Python 2; check carefully before you try one.

PyXML, however, needs grabbing/installing ``by hand'' (not hard). See the PyXML page for that.

HP-UX 11 (or 10.20, for that matter)

Went to Software Porting And Archive Centre for HP-UX, and grabbed bits as outlined below.

Gunzip all the *.depot.gz files. (Check in /usr/contrib/bin/gzip before you decide you don't have Gzip...)

Make sure you have copies of the *.depot files on the victim host's disk(s) in, say, /._disk1/tmp. (You will probably need to install as 'root', which is typically NFS-hostile.)

On the victim host, do:

cd /._disk1/tmp

# your choices here; this is a useful minimum:
swinstall -s `pwd`/gcc-2.95.2-sd-11.00.depot gcc
swinstall -s `pwd`/make-3.79-sd-11.00.depot make
swinstall -s `pwd`/sudo-1.6.2b1-sd-11.00.depot sudo

# I *think* gcc needs the GNU assembler out of binutils:
swinstall -s `pwd`/binutils-2.9.1-sd-11.00.depot binutils

# here are some others you might do while you were at it:
swinstall -s `pwd`/python-1.6-sd-11.00.depot python
swinstall -s `pwd`/binutils-2.9.1-sd-11.00.depot binutils
swinstall -s `pwd`/perl-5.6.0-sd-11.00.depot perl
swinstall -s `pwd`/flex-2.5.4a-sd-11.00.depot flex
swinstall -s `pwd`/bison-1.28-sd-11.00.depot bison

/bin/rm -rf /._disk1/tmp # tidy up, if you like

If appropriate, make Perl and Python available at a location we prefer:

cd /usr/local/bin
ln -s /opt/perl5/bin/perl .
ln -s /opt/python/bin/python .

For an old Python only: Build/install PyXML, which is a platform-independent game...

You won't get far building any X11-oid program without some X11 headers and things like /opt/imake/bin/makedepend. You will need to install the `HP C/ANSI C Developer's Bundle', for which you will need to have paid some money to HP.

You can get the bits off an `Applications' CD -- they will be `codeword-protected' -- or directly from the ITRC (or: European one).

Make your /etc/sudoers file on the box what you want (so you can use `sudo').

gzip all of your original *.depot files, if you like, for later use.

If you aren't using the /opt-bound packages swinstall'd above anymore, you can swremove them.

Linux, at least Red Hat

A Linux box is likely to have everything you need already.

A modern Red Hat should have a good Python in /usr/bin/python. Older versions (7.x and before) had a very old Python as /usr/bin/python, but a modern Python as /usr/bin/python2, so go for that.

For site-wide consistency, you may wish to put these symlinks in /usr/local/bin:

ln -s /usr/bin/perl    /usr/local/bin/perl
ln -s /usr/bin/python2 /usr/local/bin/python

For 'sudo', you will already have the tool, but will need to set up an /etc/sudoers file (with visudo) so you can use it.

Solaris

You can get everything you need from the Solaris Freeware Project.

The list of things I would get are the same as for HP-UX.

Say I grabbed gcc-2.95.3-sol26-sparc-local.gz... Copy it to the victim machine. Then, as `root': /usr/sbin/pkgadd -d gcc-2.95.3-sol26-sparc-local.

Ditto for all the rest. Things seem to install into /usr/local, mostly.

Note: I had a terrible time with the pre-packaged Pythons. They had references to all sorts of shared libraries that my newly-minted Solaris box did not have, rendering the Python DOA (dead on arrival). In the end, it was simpler to build an initial Python from scratch; just unpack the source and (as root):

PATH=/usr/ccs/bin:$PATH; export PATH
CC=/usr/local/bin/gcc ./configure && make all && make install

Installing PyXML -- platform-independent

This only applies if you have a pre-Python-2.0 Python!

You need to put PyXML into your ARK-booting Python.

If your Python is a pre-1.6 version (e.g. the 1.5.2 that comes with Red Hat Linux 7.1), you will first need to install `distutils'. Here is the source, which includes install instructions (which, actually, are just the same as what you are about to do to PyXML...).

Unpack the source somewhere:

cd /._disk1
gzip -dc < /d/open-src/PyXML-0.6.5.tar.gz | tar xf -

# make sure GCC is in PATH
PATH=$PATH:/opt/gcc/bin ; export PATH

# build it, invoking python where it really is (not symlinked from)
/opt/python/bin/python setup.py build
/opt/python/bin/python setup.py install

OK, time for a quick sanity-check; start an interactive Python and see if it behaves more-or-less as follows:

% python
Python 2.2.1 (#1, Aug 31 2002, 11:59:00) 
[GCC 3.2] on sunos5
Type "help", "copyright", "credits" or "license" for more information.

>>> import xml.dom.ext.reader.Sax

>>> import xml
>>> xml.__version__
'0.8'
>>>

If so, you're probably OK.

Setting up host-specific ARK dirs

(This is a specific task in the Sidai `ARK for real' recipe; it reflects Sidai-style blessed names; you are free to do something different. Please ask if unsure!)

NB: sidai/sysadmin-utils/add-perhost-ark-dirs.sample is a sample script that does this stuff...

Main reveal/deploy areas:

cd /._disk1
mkdir -p .-our/.-ark-deploy
# set correct owner, group, mode

# make user-visible symlink:
cd /
ln -s ./._disk1/.-our  our

/usr/local deploy area:

cd /usr/local
mkdir .-ark-deploy
# set correct owner, group, mode

/ (root) deploy area:

cd /
mkdir .-ark-deploy
# set correct owner, group, mode

Setting up site-wide ARK dirs

(This is a specific task in the Sidai `ARK for real' recipe; it reflects Sidai-style blessed names; you are free to do something different. Please ask if unsure!)

If you are doing just one host, you can just mkdir these directories in the dullest possible way; if you are planning to do multiple hosts, you need to set up appropriate automount-map entries.

Here's a script that would create the directories in the dull way; it is indicative of what is needed. (The exact details really depend on what you are doing with your site.)

For details about the roles of these directories, please see the blessed names document.

for d in /sys/.-ark-install-ALL            \
         /sys/.-ark-install-sparc-solaris8 \
         /sys/ark-builds                   \
         /sys/ark-gen-pkg                  \
         /sys/ark-state                    \
         ; do
    mkdir -p $d
done

We also typically have directories where we park open-source tarballs and sacks-'o-bits from vendors; these might be:

for d in /d/open-src     \
         /d/vendor-stuff \
         ; do
    mkdir -p $d
done

As ever, it is very easy to specify some other pathnames that you prefer.

Bootstrapping the first few packages...

In this step, ARK installs itself, as well a few essential external packages. Essentially, run:

cd $ARK_SRC/ARK/arkbase
./arkcmd package reveal --verbose arkbase

Cloning a box from a similar ARK host

If you've got another box that is the same flavor a newclient, you can do something like (from oldclient):

cd /._disk1
sudo /bin/tar cf - .-our \
  | ssh newclient 'cd /._disk1 && sudo /bin/tar xf -'

cd /usr/local
sudo /bin/tar cf - * .-ark-deploy \
  | ssh newclient 'cd /usr/local && sudo /bin/tar xf -'

cd /
sudo /bin/tar cf - .-ark-deploy \
  | ssh newclient 'cd / && sudo /bin/tar xf -'

The above is standard operating procedure for me. A subsequent ark package reveal --hosts=newclient ALL is then typically enough to make `newclient' a first-class citizen.

ToDo: say something about `ark-invasive' ...?