[comp.os.research] System Call Interception

gingell@Sun.COM (Rob Gingell) (04/08/90)

In article <2278@darkstar.ucsc.edu> mbj@spice.cs.cmu.edu (Michael Jones) writes:
>I'm interested in descriptions of past or current systems which provide a
>mechanism for programs to intercept system calls in user code.
>...
>For instance, I know that Tenex was one of the early systems to provide such
>a mechanism (via the TFORK jsys).  I know that it was used in part for
>Arpanet filesystem support, possibly in conjunction with the National
>Software Works project.  But the only literature I have which describes it
>is the Tops-20 Monitor Calls manual, which obviously says nothing about what
>it was used for and how such applications were structured.  
>...
>Any information on such systems would be greatly appreciated.

There an article about the TENEX usage of system call trapping (or JSYS
trapping in TENEX/TOPS-20 terms) in the January 1978 issue of IEEE Computer.
It describes the "Resource Sharing Executive" (RSEXEC) program that was
an attempt at building some distributed resource sharing through the
ARPAnet TENEX community.  It did this by trapping various JSYSs and then
performing translation or augmentation of the basic system operation in
order to present the picture of a global (to the ARPAnet) TENEX name space.

Under an RSEXEC environment, you could use an "augmented" file name syntax
that permitted you to reference the resources of another system, e.g.:

	@type [CASE-10]<RANDOM>FILE-OF-SOME-INTEREST.TXT

or:

	@copy [UTAH-10]<SYSTEM>INFORMATION.FILE [BBN-TENEX]LPT:

to print files across the country (and reference some venerable TENEX's :-).
I believe you could also configure pseudo-environments out of various
network resources, and RSEXEC administered the charade by, as always,
translating operations into operations for both the local and remote
system(s).

RSEXEC was in operation from (I guess) around 1974 until probably the
mid-to-late '70's.  JSYS trapping seems now to be used only for program
debugging and tracing.  More information on JSYS trapping is the
proceedings of the 1975 NCC, AFIPS Conference Volume 44.

SunOS contains an augmented ptrace() for doing system call trapping.
It's use is for program debugging and monitoring via the program "trace".

System V Release 4 has a comparable facility through the /proc file system
and is used to implement the program "truss".

aw0g+@andrew.cmu.edu (Aaron Wohl) (05/02/90)

In tops20 here at cmu (back when we 7 machines) tfork was used mostly
for debugging.  Mike Fryd wrote an enviornment using tfork for students
to take the mastery exams (they had to write a program).  The exam enviornment
prevented access to files that might be used to cheet.

Some tops20 programs had paths compiled into them.  Sometimes they
didn't give very good error messages.  I made up tfork program called
nodir that trapped all file accesses and logged them and redirected
them to the default directory.  It was also usefull to log all file
accesses to help figure out build dependancies.

Debugging was the most usefull however.  It settled arguments between
Phil Almquist (the exec (ie shell)) maintainer and me (the monitor
maintainer) over who had botched a particular call.

System call intercepts are used extensivly in macintosh debugging
today.  To some extent with TMON, and fequently with 'THE debugger'.
Most of the memory blocks allocated by the mac os are relocatable.
A call to most system calls can sometimes cause relocation and
garbage collection.  It is real simple have programs that die when
memory is low and a compact happens.  To test this typically all
the system calls that can ever garbage collect are intercepted and
the location of all the relocatable blocks is shifted around.  Space
is left between blocks and filled with psuedo randum numbers that
are later checked.

If you get some big pile of junk program and it suddenly stops and
says 'system call error' it is real nice to get a backtrace of the
recent system calls.

On a tops20 all of the tops10 system calls are intercepted.  When the
first one happens PA1050, a tops10 system call emulator is loaded
and started.  The compilation/assemble/link process for many many
years thought it was on a tops10.

Aaron