[comp.compilers] TOPLAS: Linking Programs Incrementally

johnl@iecc.cambridge.ma.us (John R. Levine) (03/21/91)

The January TOPLAS has an interesting article by Russell Quong and Mark Linton
on incremental linking.  They noted that in the edit-compile-link-debug
cycle, most of the linked modules don't change from one link to another, so
they made a modified linker that tries to replace modules in place.  It leaves
some space (size determined heuristically) after modules that have changed so
that moderate growth doesn't require relinking.  They found that by allocating
24% slop space, they could relink in place 97% of the time, and that relinking
in place was much faster, for some medium sized programs it took under two
seconds as opposed to about 30 seconds for a regular ld link.

I see that this work mostly took place in 1986-87.  Does anyone know of
anything more recent along these lines?

Regards,
John Levine, johnl@iecc.cambridge.ma.us, {spdcc|ima|world}!iecc!johnl

-- 
Send compilers articles to compilers@iecc.cambridge.ma.us or
{ima | spdcc | world}!iecc!compilers.  Meta-mail to compilers-request.

mjs@hpfcso.fc.hp.com (Marc Sabatella) (03/23/91)

>The January TOPLAS has an interesting article by Russell Quong and Mark Linton
>on incremental linking.

This was, as noted, rather old work; we have it as a Stanfaord technical
report dated December 1987 (CSL-TR-87-341).

There is also an article on "SunPro: Engineering A Practical Program
Development Environment" in "Advanced Programming Environments.
Proceedings Of An International Workshop" by Adamsm Gramlich, Muchnick,
and Tirfing; this is dated even earlier (1986).  The implementation of
incremental linking was discussed only peripherally.  The basic idea
seemed to be to use slop, as in Quong and Linton's work, but rather than
keep relocation and other information necessary for quick updates in
memory, it appears to be stored in the executable file.  This seems
reasonable to me: the data in memory was probably going to to be swapped
out anyhow, and this made it easier to work in a clustered or NFS
environment, when you can't guarantee all links would be on the same
machine.  It would also work across reboots.

I tried prototyping something along the same lines as my interpretation of
SunPro.  I saw improvements more or less like Quong and Linton did in the
non-overflow case (albeit with slightly more overhead, as I had to re-read
and parse the symbol table and associated relocation tables), but in the
overflow case, I got little improvement.  This was probably because Quong
and Linton actually kept the entire program's text and data in memory, so
in the case of overflow, there was no need to re-read the .o's that had
not changed, whereas I still had to go back and re-read them, or at least
re-read the original executable.

--------------
Marc Sabatella (marc@hpmonk.fc.hp.com)
-- 
Send compilers articles to compilers@iecc.cambridge.ma.us or
{ima | spdcc | world}!iecc!compilers.  Meta-mail to compilers-request.