[comp.lang.forth] Metacompiling

wmb@MITCH.ENG.SUN.COM (Mitch Bradley) (12/03/90)

> But show me a metacompiler that someone besides the author can use.

Admittedly, metacompilers are tricky, but I don't think that it is
as bad as all that.  I was able to understand the MVP metacompiler
well enough to "clone" it just by looking at the source code that it
was supposed to accept (before the real MVP metacompiler went on sale).
Ditto for Mike Perry's Forth 79 metacompiler.  I did this when I had
been programming in Forth for less than a year.

Considering the general question of assembler source vs. metacompiler
source for Forth kernels:

There are 4 reasons for regenerating a Forth kernel:

1) Making simple mods and additions to a Forth kernel that is already
   working on a particular machine.  In this case, having the source
   in assembler form (e.g. MASM) is a reasonable solution.

2) Generating a "stripped-down" system that omits words that are not
   needed by a particular Forth application. Assembler source is okay
   for this task, but a "stripper" or "turnkey compiler" is a better
   tool.  A "stripper" is a program that finds all the words used by
   the application, and automatically generates a new system that
   contains only those words.

3) Changing virtual machine implementation details (e.g. threading
   technique, header format) without changing the target instruction
   set.  Assembler source is weak for this application, as most macro
   assemblers are not particularly good for creating the kinds of
   data structures that you often want to use in Forth kernels.

4) Retargeting a Forth kernel to run on a different processor.
   Assembler source is really pitiful for this application.

Metacompilers are significantly better than macro assemblers for tasks
3 and 4, and at least as good for tasks 1 and 2.

One problem with macro assemblers is that there are so many different
ones, even for the same processor.  (Forth itself is pretty bad in this
respect too).

For tasks 3 and 4, it is absolutely necessary to understand in great
detail the data structures used to implement the Forth kernel, and the
representation details of those data structures on the target processor.
You also have to understand the loader/downloader format for that
processor and its development system or operating system.  This is the
hard part, not the use of the metacompiler.

Sometimes it makes sense to use a hybrid approach.  For example, on the
SPARC processor, I use the Unix SPARC assembler to assemble the divide
subroutine, and the metacompiler imports the resulting object file into
the target kernel.  On some other systems, I "finesse" the problem of
"understanding the loader format" by using a C program to "wrap" the
system-independent Forth dictionary image.  The C compiler suite
understands the loader format, and the C library routines know how
to do system I/O.  Using techniques like these, my metacompiled 680x0
Forth has been ported to something like 40 different systems, from
embedded custom boards to workstations (and it's by no means a
"minimal" Forth).  I can often port it to a new system in 5 or 10
minutes; it rarely takes more than an evening.  Retargeting the
metacompiler from 680x0 to SPARC took 2 evenings; 1 evening to write
the assembler/disassembler, and a couple of hours to establish the
metacompiler byte ordering and alignment primitives for the different
processor.

Cheers,
Mitch Bradley, wmb@Eng.Sun.COM

Mitch.Bradley@ENG.SUN.COM (06/07/91)

> Does someone can
> give me a more definitive examples/demonstarions of metacompiler?

Henry Laxen wrote a series of 3 articles about metacompilation.  They
appeared in Forth Dimensions, Volume 4 No. 6, Volume 5 No. 2, and
Volume 5 No. 3.

Back issues of Forth Dimensions are available from the Forth Interest
Group, P.O. Box 8231, San Jose, CA 95155,  Phone: (408)277-0668
FAX: (408)286-8922

The price (overseas air mail) is $18 for any volume from volume 1 through
volume 6, and $25 for any volume from volume 7 through volume 12.

Forth Interest Group also sells back issues of FORML conference proceedings:

$40  each of 1980, 1982, 1983, 1984, 1985, 1986
$34  each of 1988, 1988 Australian FORML
$50  each of 1987, 1989, 1990
$55  1981

Numerous other Forth-related titles are available too.

Add $3.00 to your order for handling.

Mitch.Bradley@Eng.Sun.COM