[comp.compilers] Compiler with adjustable parsers

hackeron@ATHENA.MIT.EDU (03/01/90)

Does anyone know of a compiler/language that allows you to specify changes
to how the language is parsed (in part at least) from withing the program.
I'm thinking of something like having statements in the language that define
how to recognize data types.  for example one could have a data type set
that is usually defined as follows:

set1 : set[real] := { 23.0 12.0 34.4 2.3 34.6 2.58 387.2 }

but have a statement in the language that changes the syntax to :

set1$set.real = ( 23.0 12.0 34.4 2.3 34.6 2.58 387.2)

where the things that change are the terminals that separate the sections and
delimit the sections of the definition.

Are there such compilers that can change their parsing grammars (or augment 
them) at compile time ?

							---Harris
[Fifteen years ago Ned Irons' IMP72 let you stick BNF in the middle of the
program.  It worked, but led to some pretty obscure programs since everyone
redefined the syntax all over the place.  The compiler was also very slow
since it had to use Earley's algorithm to handle ambiguous syntax.  I don't
know if anyone does this sort of thing now.  -John]
-- 
Send compilers articles to compilers@esegue.segue.boston.ma.us
{spdcc | ima | lotus}!esegue.  Meta-mail to compilers-request@esegue.
Please send responses to the author of the message, not the poster.

grunwald@foobar.Colorado.EDU (Dirk Grunwald) (03/03/90)

Well, Scheme and Lisp let you do this using extend-syntax and
defmacro, respectively.

Also, wasn't there a language done at Harvard called Extensible
Language or EL or something like that?
[EL/1, see a subsequent message -John]
-- 
Send compilers articles to compilers@esegue.segue.boston.ma.us
{spdcc | ima | lotus}!esegue.  Meta-mail to compilers-request@esegue.
Please send responses to the author of the message, not the poster.

arnold@audiofax.com (Arnold Robbins) (03/03/90)

In article <1990Mar1.143905.14553@esegue.segue.boston.ma.us> hackeron@ATHENA.MIT.EDU writes:
>Does anyone know of a compiler/language that allows you to specify changes
>to how the language is parsed (in part at least) from withing the program.

Didn't Algol-68 allow the programmer to change the precedence of
operators and such?
-- 
Arnold Robbins -- Senior Research Scientist - AudioFAX
2000 Powers Ferry Road, #220 / Marietta, GA. 30067    
INTERNET: arnold@audiofax.com	Phone: +1 404 933 7600
UUCP:	  emory!audfax!arnold	Fax:   +1 404 933 7606
-- 
Send compilers articles to compilers@esegue.segue.boston.ma.us
{spdcc | ima | lotus}!esegue.  Meta-mail to compilers-request@esegue.
Please send responses to the author of the message, not the poster.

steve@hubcap.clemson.edu ("Steve" Stevenson) (03/03/90)

hackeron@ATHENA.MIT.EDU writes:

>Does anyone know of a compiler/language that allows you to specify changes
>to how the language is parsed (in part at least) from withing the program.

Way back in the Jurassic age, there was a lot of work by Cheatham and
Wegbreit in ``extensible languages.'' Sorry, no direct references come to
mind, although I seem to recall a CACM article by Wegbreit on ``EL1''

-- 
Steve (really "D. E.") Stevenson           steve@hubcap.clemson.edu
Department of Computer Science,            (803)656-5880.mabell
Clemson University, Clemson, SC 29634-1906
-- 
Send compilers articles to compilers@esegue.segue.boston.ma.us
{spdcc | ima | lotus}!esegue.  Meta-mail to compilers-request@esegue.
Please send responses to the author of the message, not the poster.

dhw@itivax.iti.org (David H. West) (03/03/90)

In article <1990Mar1.143905.14553@esegue.segue.boston.ma.us> hackeron@ATHENA.MIT.EDU writes:
>Does anyone know of a compiler/language that allows you to specify changes
>to how the language is parsed (in part at least) from withing the program.

Edinburgh-derived Prologs have a precedence grammar that can be
changed at runtime (though not all implementations let you change
everything); the interesting part of it is that you can change
operators between prefix, infix and postfix, and the system will, if
you wish, respect the new conventions on output also.  This makes
reading and writing (certains kinds of) unusually-formatted stuff
almost painless.   I once threw together a tiny natural-language-like 
interface in a great hurry by this means. (Emphasis on TINY).

Some functional languages of UK origin have similar facilities.
-- 
Send compilers articles to compilers@esegue.segue.boston.ma.us
{spdcc | ima | lotus}!esegue.  Meta-mail to compilers-request@esegue.
Please send responses to the author of the message, not the poster.

pgl@cup.portal.com (03/03/90)

> Does anyone know of a compiler/language that allows you to specify changes
> to how the language is parsed (in part at least) from withing the program.

Sure: Prolog allows (re)defining operators.  In addition, IBM-Prolog
allows specifying various other syntax items, by changing its scan
tables and its table of "assignable atoms".  For example, it allows
the Edinburgh syntax:
	a([X|_y],_) :- b(X), !, c(X,_y) .
and, by simply changing these tables and defining different operators,
the default IBM syntax, modified for keyboards without square brackets:
	a({X!*y},*) <- b(X) & / & c(X,*y) .

What's more, you can input a clause in one syntax, change the syntax and
then get the clause back in the new syntax (using clause/2 or ax(*,*)).

The parser is quite fast because during processing, all atoms must
be hashed anyway, so comparing against an "assignable atom" is just
a pointer comparison.

- peter ludemann	pgl@cup.portal.com	[standard disclaimer]
-- 
Send compilers articles to compilers@esegue.segue.boston.ma.us
{spdcc | ima | lotus}!esegue.  Meta-mail to compilers-request@esegue.
Please send responses to the author of the message, not the poster.

ipser@vaxa.isi.edu (Ed Ipser) (03/03/90)

In article <1990Mar1.143905.14553@esegue.segue.boston.ma.us> 
hackeron@ATHENA.MIT.EDU writes:
>Does anyone know of a compiler/language that allows you to specify changes
>to how the language is parsed (in part at least) from withing the program.
>I'm thinking of something like having statements in the language that define
>how to recognize data types.  for example one could have a data type set
>that is usually defined as follows:

The proceedings of the '69 and '71 Extensible Language Symposiums were
published in SIGPLAN issues volume 4, number 8, August '69 and
volume 6, number 12, December '71, respectively.  I think that you will 
find what you are looking for in these issues.
-- 
Send compilers articles to compilers@esegue.segue.boston.ma.us
{spdcc | ima | lotus}!esegue.  Meta-mail to compilers-request@esegue.
Please send responses to the author of the message, not the poster.

PIRINEN@CC.HELSINKI.FI (Pekka P. Pirinen) (03/04/90)

In article <1990Mar1.143905.14553@esegue.segue.boston.ma.us>, hackeron@ATHENA.MIT.EDU writes:
> Does anyone know of a compiler/language that allows you to specify changes
> to how the language is parsed (in part at least) from withing the program.

Most Lisp dialects have this ability.  Lisp has evolved as the language of the
AI research people, and devising new languages has always been a
favorite technique of AI -- which is why it's easy in Lisp.  (Yes, I
believe in evolution: Lisp programs are sexier!)

But then, the Lisp approach to parsing is unusual:
 - the structure of a program is defined in terms of data structures
 - the compiler is required to take these data structures as input
Therefore the compiler usually works quite independently of the parser
(which, I guess, makes this reply inappropriate for comp.compilers :-)
 - data structures have an external representation in terms of
   sequences of characters
The agent in a Lisp system that creates data structures by reading
sequences of characters is called 'the reader'.  Most Lisps have a
programmable reader, meaning you can define the reader to call a
user-defined function when, say, encountering a certain character.  This
function can then read more characters (possibly by recursive calls to
the reader), construct and return a data object.  Since the full power
of the language is available, this is a pretty powerful feature!

For a modern implementation of this, see Guy L. Steele, Jr.: _Common
Lisp, the Language_ chapter 22.1.


Pekka P. Pirinen
University of Helsinki

Internet: pirinen@cc.helsinki.fi
BITNET:   pirinen@finuh
UUCP:     mcvax!cc.helsinki.fi!pirinen
-- 
Send compilers articles to compilers@esegue.segue.boston.ma.us
{spdcc | ima | lotus}!esegue.  Meta-mail to compilers-request@esegue.
Please send responses to the author of the message, not the poster.