[comp.os.minix] Lex

ast@cs.vu.nl (Andy Tanenbaum) (06/23/89)

In comp.sources.unix there is a version of lex posted.
It has only 7 parts, quite unusual for that group.
Anybody volunteer to take a look at it and see if it can be ported to
MINIX? 

Andy Tanenbaum (ast@cs.vu.nl)

LANCER%GACVAX1.BITNET@vm1.nodak.edu (Lance A. Brown) (06/24/89)

Hi all,
        I would be pleased to take a look at the lex code and try to port it
over to Minix as I was already planning on doing this.  The only problem is
I would need to have some kind soul get the files for me from comp.sources.unix
   and mail them to me.

Thanks,
Lance Brown
LANCER@GACVAX1.BITNET

steve@basser.oz (Stephen Russell) (06/25/89)

In article <2796@ast.cs.vu.nl> ast@cs.vu.nl (Andy Tanenbaum) writes:
>
>In comp.sources.unix there is a version of lex posted.
>It has only 7 parts, quite unusual for that group.
>Anybody volunteer to take a look at it and see if it can be ported to
>MINIX? 

I've just moved it MSDOS compiled with Turbo C. As it comes out the
box, it needs a small text/large data model. Sum of the sizes of the
TC object files is:

	text 34686 data 20301 bss 28599 Total 83586

Problem is the large structures allocated dynamically - more details
below. The major bss hogs are some large arrays in main.c and scan.c,
which can be safely shrunk, particularly the 16K of input buffering in
the scanner.

I also compiled it with small text/small data, like Minix split I&D.
It failed allocating those big structures. Object sizes were:

	text 29349 data 20221 bss 27913 Total 77483

Same comments as above.

Here is a condensed summary of running flex on its own scanner
definition.  Note that this is a pretty meaty scanner (114 rules) so
probably represents an upper limit on memory usage.  I've starred the
interesting lines - those showing "n/m" indicate dynamic tables with
"m" initial entries that could be shrunk by changing #defines in the
header file.

	flex version 2.1 (beta) usage statistics:
	*  777/2000 NFA states
	*  315/1000 DFA states (1339 words)
	*  21/40 start conditions
	*  23/100 character classes needed 190/500 words of storage, 38 reused
	   6489 state/nextstate pairs created
	*  374/1000 base-def entries created
	   1291/2000 (peak 3034) nxt-chk entries created
	   649/2500 (peak 2419) template nxt-chk entries created
	   3498 total table entries needed

In fact, flex is written to grow (most of?) these dynamic tables as
needed, so much smaller initial sizes should work on Minix, at least
for medium-sized lex definitions.

A few other things:

- it assumes C compiler supports 32 char external identifiers.
- it uses a YACC generated parser, so would need to ported on some UNIX
  box to get the YACC output.
- generated scanners use large (16K) input buffer. This can be changed
  with an appropriate #define in the skeleton file.
- the documentation assumes you know how to use lex.

Summary: should be possible to port. Code size is reasonable, but
would require some tuning to shrink the tables to fit in 64K.

agrusow@exunido.uucp (Michael Vishchers) (06/26/89)

In article <2796@ast.cs.vu.nl> ast@cs.vu.nl (Andy Tanenbaum) writes:
>
>In comp.sources.unix there is a version of lex posted.
>It has only 7 parts, quite unusual for that group.
>Anybody volunteer to take a look at it and see if it can be ported to
>MINIX? 

A few weeks ago, I posted my port of the previous flex version to
Comp.binaries.atari.st. I was planning to send the sources (after porting to
MINIX and making some cosmetic cleanups) to Comp.sources.atari.st .

I have not looked at the new version yet, so I do not know how much trouble
a volunteer would have. (BTW: the port of the previous version went quite
straightforward, using Sozobon C).

I think at least I will not have the time for the new port in the next weeks,
but if there is enough interest (and my machine doesn't break), I might try
the old version today under MINIX and post the sources (or diffs ?) to this
group as well

Michael

 ______________________________________________________________________
/ Michael Vishchers                           | History repeats itself.\
| michaelV@owl.ls5.informatik.uni-dortmund.de | Has to.                |
| michaelV%owl.ls5@UNIDO.uucp                 | No one listens.        |
| (I think)                                   |        (steve turner)  |
\______________________________________________________________________/
/ Home address:: Frohnhauser Str. 233    D-4300 Essen 1    West-Germany\
________________________________________________________________________
 

V61%DHDURZ1.BITNET@cunyvm.cuny.edu (Ronald Lamprecht) (06/27/89)

I have a little bit experience with flex : I have ported the old version
of flex to OS9,TOS,...

Therefore I will port the new version to Minix ST (ACK compiler, 'no' data
size restrictions). Perhaps we can combine the different cdiffs for PC and ST
with #ifdef's and post together to this group.

(I will try to contact Stephen Russell)

Bitnet:  V61@DHDURZ1                               Ronald Lamprecht
UUCP:    ...!unido!DHDURZ1.bitnet!V61              Theoretische Physik
ARPAnet: V61%DHDURZ1.BITNET@CUNYVM.CUNY.EDU       (Heidelberg, West Germany)