[gnu.emacs] shared source question

steve@CFHT.CFHT.HAWAII.EDU (Steven Smith) (05/12/89)

I have been supporting GNU emacs at my site for about nine months.
Currently I need to have emacs on HP9000/3xx, HP9000/8xx, Sun3/xxx,
Sun4/xxx networked together(NFS).  Support for the various flavors of X10, 
X11, Sunview and (soon I hope) NEWS/X11 are usefull.  Also various OS releases
are coming in all the time.

My question is this, is it possible (or even a good thing?) to use *one* 
NFS mountable copy of the source tree (for any given version of emacs) 
and a master Makefile to generate all the needed executables, thereby
saving 40+ meg of disc for each combination?

If anyone has any experience, good or bad, with such a beast please 
reply be email (to: steve@cfht.hawaii.edu).  I'll summarize if needed.

mahalo,

steve

lnz@LUCID.COM (Leonard N. Zubkoff) (06/05/89)

In reply to:

    My question is this, is it possible (or even a good thing?) to use *one*
    NFS mountable copy of the source tree (for any given version of emacs)
    and a master Makefile to generate all the needed executables, thereby
    saving 40+ meg of disc for each combination?

    If anyone has any experience, good or bad, with such a beast please
    reply be email (to: steve@cfht.hawaii.edu).  I'll summarize if needed.


I maintain GNU Emacs for eight different machine types here: Apollo
68020/68030, Apollo PRISM, sun3, sun4, sun386, sun2, vax-ultrix, and
vax-4.3bsd.  On the Apollo's, I can build a single compound executable (cmpexe)
that can be run on both the 68020/68030 and PRISM based nodes, so users don't
need to know what type of node they're using, and a single Emacs tree can
be shared very easily.

Unfortunately, the Suns and Vaxes do not support a notion of compound
executables, so more work is required to share a common Emacs tree, but it is
possible and I've been running such a system here for a couple of months
without any problems.

In order to share the same GNU Emacs tree across all our NFS'd Suns and Vaxes,
and to have only a single executable which users need to know about, I've made
"emacs" and all the other executables be shell scripts that indirect to the
name of the executable concatenated to the machine type.  The Sun "/bin/arch"
command returns one of sun3, sun4, sun386, and sun2, so this part was easy.  On
the Vaxes I created "/bin/arch" as a shell script that just echoes "vax-ultrix"
or "vax-4.3bsd" as appropriate.

To allow easy building of the 6 NFS-shared Emacs's, I've developed a couple of
simple shell scripts.  First, copy "lisp/version.el" to "lisp/version.el-dist",
and install the "etc/Makefile.x" reproduced below.

Next, create your "config.h" files by naming them "config.h.<machine-type>", as
in "config.h.sun3", "config.h.sun4", etc.  Then, I just run the
"build-common-emacs" script reproduced below.  It in turn invokes a remote
shell on the appropriate machines and runs the script "build-local-emacs".
Be sure to change 18.54 below to whatever version you're using.

		Leonard N. Zubkoff


Note: I keep the GNU Emacs tree as "/gnuemacs" on every machine, so the scripts
below reflect that; you should modify them to refer to the location of your
Emacs tree.

Note: You probably want to munge "src/ymakefile" to skip the "inc-version"
step, and then just manually edit "version.el-dist" before each full rebuild.
With NFS, you must beware of removing the last link to any executable that's in
use, as it may then kill the user's process when it tries to page in.


			***** build-common-emacs *****

#! /bin/csh -f

date

echo "***** Installing Trampolines for Executables *****"
set executables = (emacs etc/test-distrib etc/etags etc/ctags etc/loadst \
		   etc/make-docfile etc/digest-doc etc/sorted-doc \
		   etc/movemail etc/cvtmail etc/fakemail etc/yow etc/env \
		   etc/server etc/emacsclient etc/emacstool)
foreach file ($executables)
    cat > $file <<!
#! /bin/csh -f
exec /gnuemacs/$file.\`/bin/arch\` \$*
!
    chmod u=rwx,g=rx,o=rx $file
end

echo "***** Building GNU Emacs for Sun3 *****"
rsh trilogy	    -n "cd /gnuemacs; build-local-emacs"

echo "***** Building GNU Emacs for Sun4 *****"
rsh anita-bryant    -n "cd /gnuemacs; build-local-emacs"

echo "***** Building GNU Emacs for Sun386 *****"
rsh friday-13	    -n "cd /gnuemacs; build-local-emacs"

echo "***** Building GNU Emacs for Sun2 *****"
rsh dresden	    -n "cd /gnuemacs; build-local-emacs"

echo "***** Building GNU Emacs for VAX/Ultrix3.0 *****"
rsh donnerparty	    -n "cd /gnuemacs; build-local-emacs"

echo "***** Building GNU Emacs for VAX/4.3BSD *****"
rsh wedtech	    -n "cd /gnuemacs; build-local-emacs"

date


***** build-local-emacs *****

#! /bin/csh -f
set machine = `/bin/arch`
if ($machine == sun3 || $machine == sun4 \
	|| $machine == sun386 || $machine == sun2) then
    set extra_programs = emacstool.x
    set extra_executables = etc/emacstool
else
    set extra_programs =
    set extra_executables =
endif
umask 022
(cd lisp; sed "s/18.54.0/18.54.0.$machine/" version.el-dist > version.el)
(cd src; rm -f config.h xmakefile; rm -f *.o; ln -s config.h.$machine config.h)
(cd etc; make -f Makefile.x all $extra_programs)
(cd src; make; mv xemacs ../emacs.x)
rm -f etc/DOC lisp/version.el src/xmakefile src/*.o
rm -f src/config.h src/temacs src/emacs-18.54.0.$machine
set executables = (emacs etc/test-distrib etc/etags etc/ctags etc/loadst \
		   etc/make-docfile etc/digest-doc etc/sorted-doc \
		   etc/movemail etc/cvtmail etc/fakemail etc/yow etc/env \
		   etc/server etc/emacsclient $extra_executables)
foreach file ($executables)
    mv $file.x $file.$machine
end


			  ***** etc/Makefile.x *****

# Modified on 1-May-89 by Leonard N. Zubkoff:
#   Changed CFLAGS from "-g" to "-O".
#   Build executables with name $file.x.

CFLAGS= -O
# For Xenix.  Needed for movemail
#  LOADLIBES= -lx
# For Iris 4d.  Needed for loadst.
#  LOADLIBES = -lmld

EXECUTABLES = test-distrib.x etags.x ctags.x loadst.x make-docfile.x \
   digest-doc.x sorted-doc.x movemail.x cvtmail.x fakemail.x yow.x env.x \
   server.x emacsclient.x

all: ${EXECUTABLES}

distclean:
	-rm -f ${EXECUTABLES} *~ \#* DOC* core

# This justs verifies that the non-ASCII characters
# in the file `testfile' have no been clobbered by
# whatever means were used to copy and distribute Emacs.
# If they were clobbered, all the .elc files were clobbered too.
test-distrib.x: test-distrib.c
	$(CC) -o test-distrib.x test-distrib.c
	./test-distrib.x

etags.x: etags.c
	$(CC) -o etags.x ${CFLAGS} -DETAGS etags.c $(LOADLIBES)

ctags.x: etags.c
	$(CC) -o ctags.x ${CFLAGS} -DCTAGS etags.c $(LOADLIBES)

loadst.x: loadst.c ../src/config.h
	$(CC) -o loadst.x ${CFLAGS} loadst.c $(LOADLIBES)

make-docfile.x: make-docfile.c
	$(CC) -o make-docfile.x ${CFLAGS} make-docfile.c $(LOADLIBES)

digest-doc.x: digest-doc.c
	$(CC) -o digest-doc.x ${CFLAGS} digest-doc.c $(LOADLIBES)

sorted-doc.x: sorted-doc.c
	$(CC) -o sorted-doc.x ${CFLAGS} sorted-doc.c $(LOADLIBES)

#
movemail.x: movemail.c ../src/config.h
	$(CC) -o movemail.x ${CFLAGS} movemail.c $(LOADLIBES)

cvtmail.x: cvtmail.c
	$(CC) -o cvtmail.x ${CFLAGS} cvtmail.c $(LOADLIBES)

fakemail.x: fakemail.c ../src/config.h
	$(CC) -o fakemail.x ${CFLAGS} fakemail.c $(LOADLIBES)

yow.x: yow.c
	$(CC) -o yow.x ${CFLAGS} yow.c $(LOADLIBES)

# this is silly -- just use emacs to edit this file!
# (in any case this program doesn't preserve alphabetical ordering,
#  which is why I'm removing it)
#addyow: addyow.c
#	$(CC) -o addyow.x ${CFLAGS} addyow.c

env.x: env.c ../src/config.h
	$(CC) -o env.x -DEMACS ${CFLAGS} env.c $(LOADLIBES)

server.x: server.c ../src/config.h
	$(CC) -o server.x ${CFLAGS} server.c $(LOADLIBES)

emacsclient.x: emacsclient.c ../src/config.h
	$(CC) -o emacsclient.x ${CFLAGS} emacsclient.c $(LOADLIBES)

# This one is NOT included in EXECUTABLES.
# See ../src/ymakefile.
emacstool.x: emacstool.c
	$(CC) emacstool.c -o emacstool.x -g -lsuntool -lsunwindow -lpixrect $(LOADLIBES)

allbery@ncoast.org (Brandon S. Allbery) (06/07/89)

In your message of Sun, 04 Jun 89 18:13:48 PDT, you write:
+---------------
|     My question is this, is it possible (or even a good thing?) to use *one*
|     NFS mountable copy of the source tree (for any given version of emacs)
|     and a master Makefile to generate all the needed executables, thereby
|     saving 40+ meg of disc for each combination?
| 
| I maintain GNU Emacs for eight different machine types here: Apollo
| 68020/68030, Apollo PRISM, sun3, sun4, sun386, sun2, vax-ultrix, and
| vax-4.3bsd.  On the Apollo's, I can build a single compound executable (cmpexe)
| that can be run on both the 68020/68030 and PRISM based nodes, so users don't
| need to know what type of node they're using, and a single Emacs tree can
| be shared very easily.
| 
| Unfortunately, the Suns and Vaxes do not support a notion of compound
| executables, so more work is required to share a common Emacs tree, but it is
| possible and I've been running such a system here for a couple of months
| without any problems.
+---------------

Dumb question:  could Gnu /bin/ld be made capable of creating compound
executables of some kind?  (Not that I have seen the source to know the
reality of the situation, but could encapsulated COFF files be a model of
some kind for this?)

If I'm barking up the wrong tree, spare the lists and send me email; I'll go
away quietly.

++Brandon

lnz@LUCID.COM (Leonard N. Zubkoff) (06/07/89)

No doubt GNU LD could be modified to handle compound executables, or a separate
utility could be used to produce them from individual architecture-specific
executables.  The problem, however, is that the kernel must be modified to
understand how to select and run the correct executable from a compound
executable.  So when there is a GNU kernel, adding this support could be
done rather easily I expect; without source to the kernel, there's no way
to do this.

		Leonard

herme@MCC.COM (Manuel Hermenegildo) (06/07/89)

We have been using a relatively simple method in order to deal with
the problem of having one set of source files and executables for
different machines in an NFS environment. This is a "low tech"
solution that doesn't need any modification of the loader or kernel.
We use it, for example, in a Prolog implementation that uses one set
of source files but runs on many different machines (suns, vaxen, and
multiprocessors -Sequent).  I will describe the method specialized for
emacs, but it is of general application. If you know of a better
method let me know. If you want more info on this one, I will be happy
to provide it.

The first part of the trick is to avoid duplication of source files by
using symbolic links: all versions of emacs can be kept under a common
directory tree.  The top directory in the tree (let's call it
"emacs-dir") contains all the source files (".c" files), a global
Makefile, the directories for the non-machine-specific files --e.g.
lisp source directory--- and a directory called "objcode". In the
objcode directory there are subdirectories for each machine that is
supported (e.g. "sun3", "sun2", "balance", "bobcat", etc.). Each one
of these directories contains >symbolic links< to all the source files
in the "emacs-dir" directory and a specialized copy of the makefile.
Running "make" in each subdirectory in "objcode" creates a
machine-specific version of emacs without duplicating the source
files. Thus, after running make on each "objcode" subdirectory there
will be different machine-specific versions of emacs

....emacs-dir/objcode/sun3/emacs
....emacs-dir/objcode/sun2/emacs
....emacs-dir/objcode/balance/emacs
....emacs-dir/objcode/bobcat/emacs

without having duplicated the source files and libraries. 

The second part of the trick is to select the right executable when
the user types "emacs", depending on which machine he or she is
running on. A simple way to do this is to define at login time (i.e.
in the ".login" file) an environment variable called "hardware" which
contains the type of hardware that the login shell is running on (the
type of hardware can be determined automatically --I can post that
also if someone is interested), and then add to the standard set of
paths a path to the appropriate version of emacs

set path= (....emacs-dir/objcode/$hardware $path)

that way the correct version is selected every time when the user
types "emacs". In general, rather than doing this specifically for
emacs, we organize our "bin" files into subdirectories 

~/bin/sun3
~/bin/sun4
~/bin/balance

which contain links to the right executables for different packages
and set the user path to be  

set path= (~/bin/$hardware $path)

Based on having a correct value in the "$hardware" variable the
creation of the subdirectories (such as "....emacs/objcode/sun4") and
the links to the source files is also automated by using a special
makefile at the top level, so that when you type "make" at the top
level the corresponding "objcode" subdirectory for the machine you are
running on is created (if it didn't exist before), the symbolic links
made, and compilation is automatically started in that subdirectory.
Creating the different versions then simply involves starting shells
on the different machines and typing "make" at the "emacs-dir" level.

Again, I will be happy to provide more info or hear of a better solution.

Manuel Hermenegildo