bglenden@mandrill.cv.nrao.edu (Brian Glendenning) (09/12/90)
Can anyone tell me what the usual tricks are when dealing with
readonly source hierarchies (i.e. we have 5 different binary formats
and we're tired of maintaining several versions of identical source
trees (the sources are changing somewhat rapidly and the extra copies
take not insignificant amounts of disk)). I'm not clear what the best,
or even usual, solution to this (presumably) common problem is.
--
Brian Glendenning - National Radio Astronomy Observatory
bglenden@nrao.edu bglenden@nrao.bitnet (804) 296-0286aglew@crhc.uiuc.edu (Andy Glew) (09/12/90)
If you don't want to be able to prepare all the binaries
simultaneously, use SCCS or RCS and version names, with a different
name (or set of names) for each different binary version.
In general, version names can be divided into two classes:
historical names, which once put in place never change, and current
names, like 'Latest-Greatest-Version-For-Foo', which change every time
an update or bugfix is made.
If you want to be able to prepare all the binaries simultaneously, try
using link farms - whether symlinks or hardlinks depends on your
situation.
Create a single, central, RCS tree.
Create several "views" of the source tree, containing the current
source for a particular version. Share files common to more than one
view using links. Let every developer have her own view.
In the paper "Boxes, Links, and Parallel Trees: Elements of a
Configuration Management System", USENIX Workshop Proceedings,
Software Management. USENIX Association, April 3-4, 1989, New
Orleans, Louisiana, I describe such a CM system I devised for UNIX
system development at Gould. IMHO it was pretty good - developers who
got used to it recreated it on their own when we moved from BSD to
System V.
As I recall at the workshop, half of the audience thought it was a
really neat idea, and the other half though it was all old hat.
"Doesn't everybody do things that way".
It featured, in particular, RCS wrappers to move between several
overlaid RCS trees, hardlink cloning (saved space and permitted
chroot), and chroot "boxes" to totally isolate one developer from
another developers changes which might happen to get installed on the
development system by accident... (reducing problems like "I didn't
think my compiler bugfix would cause you to spend a week chasing down
a bug in the OS"...)
There are a number of commercial systems out there, but rolling your
own (1) isn't hard, and (2) gets you more features than most
commercial systems I've seen.
--
Andy Glew, a-glew@uiuc.edu [get ph nameserver from uxc.cso.uiuc.edu:net/qi]chris@mimsy.umd.edu (Chris Torek) (09/12/90)
In article <BGLENDEN.90Sep11155146@mandrill.cv.nrao.edu> bglenden@mandrill.cv.nrao.edu (Brian Glendenning) writes: >Can anyone tell me what the usual tricks are when dealing with >readonly source hierarchies (i.e. we have 5 different binary formats >and we're tired of maintaining several versions of identical source >trees (the sources are changing somewhat rapidly and the extra copies >take not insignificant amounts of disk)). I'm not clear what the best, >or even usual, solution to this (presumably) common problem is. One solution, operating now at UC Berkeley and almost working here at UMCP-CSD (there are still some programs that must be cleaned up): 1. Obtain pmake (Adam de Boor's `parallel make') as modified for 4.3BSD-reno (these mods may be optional; I have not looked at the unmodified version). 2. Put symlinks in your kernel, or a hack into pmake for `pretend symlinks' of some sort (see step 4 below). 3. Change all your makefiles. This is the hard part. In the process, you generally want to move each program's source into a separate subdirectory, and move the source version of the manual for that program to the same directory. In some cases this requires thought and/or use of special pmake features. In most cases, however, you wind up with a situation like one of these: a) program /usr/bin/foo is built from one C source file. The Makefile in /usr/src/bin/foo reads, in its entirety: PROG= foo .include <bsd.prog.mk> (you can of course add comments and blank lines if you like). b) program /usr/bin/foo is built from several C source files. The Makefile in /usr/src/bin/foo reads: PROG= foo SRCS= foo.c bar.c baz.c .include <bsd.prog.mk> c) program /usr/sbin/foo is built from one C source file, but its manual goes in section 8: PROG= foo MAN8= foo.0 .include <bsd.prog.mk> d) program foo has no manpage (e.g., the C preprocessor): add the line NOMAN= noman (or `NOMAN=an_island' if you prefer :-) ). <bsd.prog.mk> simply keys off the existence of this make-variable and avoids building and installing a man page. e) directory makefiles: /usr/src/bin/Makefile reads something like SUBDIR= cat cp csh ed ls sh stty test .include <bsd.subdir.mk> f) library makefiles: these look something like LIB= foo SRCS= foofile.c fooio.c foosubr.c MAN3= fooopen.0 fooread.0 foocompute.0 MLINKS= fooopen.0 fooclose.0 \ fooread.0 foowrite.0 .include <bsd.lib.mk> g) complicated makefiles: you will need to use ${.CURDIR} to give full path names for source files, because of step 4 below. Read the pmake documentation for details. Machine-specific source files (such as the VAX `arff' console-media manipulation program) go in subdirectories as usual, and the parent directory's makefile gets a bit uglier: .if ${MACHINE} == "sun4" SUBDIR+=installboot .elif ${MACHINE} == "tahoe" SUBDIR+=dlmpcc enpload .elif ${MACHINE} == "vax" SUBDIR+=arff rxformat .endif When you have completed this step, you can simply cd /usr/src; make depend; make; make install; make clean and everything will be recompiled and reinstalled. (This takes a few hours or so....) 4. This is the magic part. All of the above is simply preliminary work. Mount /usr/src read-only. Make /usr/obj a symlink if necessary, or simply mount something on /usr/obj. [on the machine on which /usr/src is to be kept, where it is mounted read/write:] $ cd /usr/src $ make cleandir # remove any leftover stuff [on every machine that it to use /usr/src] $ mkdir /usr/obj; find /usr/src/* -type d -print | > sed 's,/usr/src/\(.*\),mkdir /usr/obj/\1\ > ln -s /usr/obj/\1 /usr/src/\1/obj,' | sh This takes a while. When it is done, run $ cd /usr/src; make depend; make; make install on all the machines sharing /usr/src. Note that 4.3BSD-reno comes with a new manual page scheme, but it is not necessary to buy into this. A small modification to <bsd.man.mk> allows using the old method without changing *any* makefiles. (By not sharing /usr/share/mk, or by putting ${MACHINE} tests in these files, you can even split between new and old on different architectures.) Although step 3 above is painful, it is worth it: it separates the *specifications* for the sources for each program (which differ for each program) from the *semantics* of how to make each program (which are almost always identical; the few exceptions get bigger makefiles). If the semantics change---e.g., if dependencies are to be maintained by some other method than `mkdep'---virtually no changes to /usr/src are necessary. Only the `master' /usr/share/mk/*.mk files (and their sources in /usr/src/share/mk) are affected, and, occasionally, those exceptional makefiles. Want to install with default owner `root' instead of `bin'? Just change <bsd.own.mk> and <bsd.lib.mk>. Want to build libraries with extra debugging information? Just change <bsd.lib.mk>. And so on.... This scheme may not be the best, but it works. It solves two problems: shrinking the makefiles (moving common semantics to a commmon place), and sharing the sources (one source tree suffices for any number of architectures). If you do cross-compilation you can even handle that, if a bit grotesquely, by making /usr/obj itself a symlink and pointing it to different object trees. (What is really needed here is Plan-9 style per-process mounts.) -- In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 405 2750) Domain: chris@cs.umd.edu Path: uunet!mimsy!chris
chris@mimsy.umd.edu (Chris Torek) (09/13/90)
In article <26504@mimsy.umd.edu> I wrote: > [on every machine that it to use /usr/src] > $ mkdir /usr/obj; find /usr/src/* -type d -print | > > sed 's,/usr/src/\(.*\),mkdir /usr/obj/\1\ > > ln -s /usr/obj/\1 /usr/src/\1/obj,' | sh Only the `mkdir' part of the sed need be done on every machine. The symlinks should only be made on the master source machine: Change the last two lines to > sed 's,/usr/src/\(.*\),mkdir /usr/obj/\1' | sh (Incidentally, this will make more `obj' symlinks, and sometimes more directories, than actually necessary. <bsd.prog.mk> now has a `make obj' that makes the symlinks. The extra directories in /usr/obj are probably not worth worrying about.) -- In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 405 2750) Domain: chris@cs.umd.edu Path: uunet!mimsy!chris