rsalz@bbn.com (Rich Salz) (10/11/90)
Submitted-by: Vern Paxson <vern@cs.cornell.edu> Posting-number: Volume 23, Issue 40 Archive-name: flex2.3/part04 #! /bin/sh # This is a shell archive. Remove anything before this line, then feed it # into a shell via "sh file" or similar. To overwrite existing files, # type "sh file -c". # The tool that generated this appeared in the comp.sources.unix newsgroup; # send mail to comp-sources-unix@uunet.uu.net if you want that tool. # Contents: MISC/Atari.patches flex.1 # Wrapped by rsalz@litchi.bbn.com on Wed Oct 10 13:24:00 1990 PATH=/bin:/usr/bin:/usr/ucb ; export PATH echo If this archive is complete, you will see the following message: echo ' "shar: End of archive 4 (of 10)."' if test -f 'MISC/Atari.patches' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'MISC/Atari.patches'\" else echo shar: Extracting \"'MISC/Atari.patches'\" \(29474 characters\) sed "s/^X//" >'MISC/Atari.patches' <<'END_OF_FILE' X(Message inbox:32) XDate: Mon, 03 Jul 89 21:15:32 CET XFrom: V61%DHDURZ1.BITNET@lbl.gov XSubject: Flex, bug fix, improvments, patches for Minix & TOS XTo: vern@lbl-csam.arpa X XAt first I have to thank you for your wonderful program. I had ported the Xold version to OS9,TOS (Atari ST) and Minix and the new version 2.1 Beta Xto Minix and TOS. X XWhile porting and using flex I detected a bug and made some improvements. XI have included a shared, compressed and uuencoded file contaning all cdiffs Xand additional files (Sorry, but I'm on EBCDIC-Bitnet) and a short discussion Xof the changes. Even some of the TOS specific changes might be of general Xinterest ! X XI posted these cdiffs to the minix discussion group, but I think it's up Xto you to post them to the unix-sources group. If you plan to post even Xthe TOS compiler specific patches please contact me because there might be Xfurther compiler (P.D.) additions. If you have an interest I could also Xport the new version to OS9 -- this is a little bit more difficult, because XOS9 uses CR as end of line character (the EOL char. is coded into the Xinitscan.c tables,...). It is necessary to change all occurences of '\n' to Xmacros and variables and it's useful to add a new -n options (see commented Xline in main.c) X X X XThe changes: (1.7.89 RAL) X X - Bug fix: The original flex didn't like trailing spaces in exclusive start X condition lists ! If you add an trailing space to line 68 in scan.l X X "%x SECT2 SECT2PROLOG SECT3 CODEBLOCK PICKUPDEF SC CARETISBOL NUM QUOTE " X X you get a misleading error message: X X "Syntax error at line 69: bad start condition list" X X This bug can either be fixed in parse.y or in scan.l . I have chosen the X last because there the fix is minimal: Just change the rule (line 128) X X "\n" to "{OPTWS}\n" X X - Enhancements: X - new option "-?" that provides some help information about the other X flags (main.c) X - new option "-aTMPPATH" that allows a redefinition of the standard X path for the temporary file (this might be rather large >200k if X F,f options are selected). (main.c, flexdef.h (l.376)) X - hexdump of illegal characters -- this proved to be a useful debugging X tool especialy if invisible control characters occur which weren't X covered by the rules. (scan.l fprintf statement line 129,...) X X - Patches due to TOS X - General: TOS uses CR,LF as end of line char., Flex wants only a single X LF as EOL char. Therefore all I/O must be translated using f* calls. X This is done everywhere besides the YY_INPUT macro (flex.skel (scan.c), X line 31) that uses a low level 'read'. This should be definitly changed X to fread, so that all I/O calls occur on the same level. X - the short action_file_name has been "flexXXXXXX.tmp", but that's too X much for TOS,MSDOS ! I changed it to "flexXXXX.tmp" in main.c (patch X covered by the -a option additions) X - some compilers don't like external names that are ambiguous within X the first 8 characters. I defined macros that change all these long X external names to names that are unique within the first 8 characters. X Just define SHORT_EXTERNAL_NAMES to use this feature (flexdef.h) X - some statement changes that some compiler don't like: X typedef *xxx[] -> typedef **xxx (flexdef.h.,l.308) X "/*" -> '/','*' within a comment in (scan.l, l.209) X - changed short "lexyy.c" to "lex_yy.c" what's more similar to the unix X "lex.yy.c" (main.c). X - a few further really compiler dependent changes provided with X #ifdef ATARI && LATTICE res. TURBO braces. X X - Additional TOS only files X - Makefile.tos: Common makefile for all TOS compilers. If you add further X ones please email me the new makefile. X - flex.lnk: Lattice - GST linker argument extension file X - flex.tlk: Turbo linker argument extension file X X XAdditional remarks: X XI didn't add a new initscan.c (= flex -ist scan.l). The current one is good Xenough for a first compilation. With this first version of flex you can Xrebuild your own scan.c and the final flex version ! X XMinix ST : X - I had to "chmem =70000 cv" (>50000) to get flex linked X - 'memset' (PC 1.3, EFTH40,...) is necessary X - chmem =90000 flex may be sufficient X XMinix PC : X It should be possible to port Flex to Minix PC. The current sizes of flex X are: X Minix ST (ACK) Lattice (TOS) Turbo (TOS) X X size 75300 83305 57957 X compilation time 22' 15' 3'40" X flex -is scan.l 1'49" 43" 30" X X The Minix ST size includes the bad generated code using only a subset of X the 68000 commands, long addresses only and a huge relocation table. X Therefore the PC size will be <64 k ! More serious is the fact that I had X to chmem =90000 flex to get scan.l converted to scan.c . But I never saw X a more complex lex source than scan.l -- so it should be possible to X reduce some array sizes without limitation for all day usage. X X No one volunteered yet for a Minix PC port -- but if someone will try it X I would provide him with a new scan.c and some hints. X XTOS: X Don't forget to adapt the flexskel path within flexdef.h ! X X XBitnet: V61@DHDURZ1 Ronald Lamprecht XUUCP: ...!unido!DHDURZ1.bitnet!V61 Theoretische Physik XARPAnet: V61%DHDURZ1.BITNET@CUNYVM.CUNY.EDU (Heidelberg, West Germany) X(Message inbox:36) XDate: Wed, 05 Jul 89 21:16:07 CET XFrom: V61%DHDURZ1.BITNET@csa2.lbl.gov XSubject: Re: What is TOS XTo: vern@lbl-csam.arpa X XTOS is the name of the Atari ST operating system that is very similar Xto MSDOS (Both use CR,LF as end of line character). Therefore I have Xbeen astonished that no EOL convertion porblems occur within MSDOS. X XI have been aware of the double buffering when changing read to fread and I Xaccept your argument of a possible slow down. But if you integrate the other XAtari - TOS changes, please insert a X #ifdef ATARI X fread .... X #else X read .... X #endif Xin flex.skel . X XBitnet: V61@DHDURZ1 Ronald Lamprecht XUUCP: ...!unido!DHDURZ1.bitnet!V61 Theoretische Physik XARPAnet: V61%DHDURZ1.BITNET@CUNYVM.CUNY.EDU (Heidelberg, West Germany) X X X X Xecho x - Makefile_cdiff Xsed '/^X/s///' > Makefile_cdiff << '/' XX*** Src_2.1/Makefile Thu Jun 28 00:06:42 1989 XX--- Makefile Thu Jul 3 02:12:48 1989 XX*************** XX*** 5,10 **** XX--- 5,11 ---- XX # Porting considerations: XX # XX # For System V Unix machines, add -DSYS_V to CFLAGS. XX+ # For Minix (ST), add -DSYS_V to CFLAGS XX # For Vax/VMS, add -DSYS_V to CFLAGS. XX # For MS-DOS, add "-DMS_DOS -DSYS_V" to CFLAGS. Create \tmp if not present. XX # You will also want to rename flex.skel to something with a three XX*************** XX*** 21,28 **** XX SKELETON_DIR = /usr/local/lib XX SKELETON_FILE = flex.skel XX SKELFLAGS = -DDEFAULT_SKELETON_FILE=\"$(SKELETON_DIR)/$(SKELETON_FILE)\" XX! CFLAGS = -O XX! LDFLAGS = -s XX XX FLEX_FLAGS = XX FLEX = ./flex XX--- 22,29 ---- XX SKELETON_DIR = /usr/local/lib XX SKELETON_FILE = flex.skel XX SKELFLAGS = -DDEFAULT_SKELETON_FILE=\"$(SKELETON_DIR)/$(SKELETON_FILE)\" XX! CFLAGS = -O -DSYS_V XX! LDFLAGS = XX XX FLEX_FLAGS = XX FLEX = ./flex XX*************** XX*** 57,63 **** XX yylex.c XX XX flex : $(FLEXOBJS) XX! $(CC) $(CFLAGS) -o flex $(LDFLAGS) $(FLEXOBJS) XX XX first_flex: XX cp initscan.c scan.c XX--- 58,65 ---- XX yylex.c XX XX flex : $(FLEXOBJS) XX! $(CC) $(CFLAGS) -o flex $(FLEXOBJS) $(LDFLAGS) XX! chmem =150000 flex XX XX first_flex: XX cp initscan.c scan.c X/ Xecho x - flex.skel_cdif Xsed '/^X/s///' > flex.skel_cdif << '/' XX*** Src_2.1/flex.skel Thu Jun 28 00:19:20 1989 XX--- flex.skel Thu Jul 2 22:18:31 1989 XX*************** XX*** 28,34 **** XX * is returned in "result". XX */ XX #define YY_INPUT(buf,result,max_size) \ XX! if ( (result = read( fileno(yyin), buf, max_size )) < 0 ) \ XX YY_FATAL_ERROR( "read() in flex scanner failed" ); XX #define YY_NULL 0 XX #define yyterminate() return ( YY_NULL ) XX--- 28,34 ---- XX * is returned in "result". XX */ XX #define YY_INPUT(buf,result,max_size) \ XX! if ( (result = fread( buf,1, max_size, yyin )) < 0 ) \ XX YY_FATAL_ERROR( "read() in flex scanner failed" ); XX #define YY_NULL 0 XX #define yyterminate() return ( YY_NULL ) X/ Xecho x - flexdef.h_cdif Xsed '/^X/s///' > flexdef.h_cdif << '/' XX*** Src_2.1/flexdef.h Thu Jun 28 00:43:27 1989 XX--- flexdef.h Thu Jul 3 02:45:50 1989 XX*************** XX*** 26,31 **** XX--- 26,40 ---- XX XX /* @(#) $Header: flexdef.h,v 2.0 89/06/20 15:49:50 vern Locked $ (LBL) */ XX XX+ #ifdef ATARI XX+ #define SYS_V XX+ #define abs(x) ((x) < 0 ? -(x) : (x)) XX+ #define SHORT_FILE_NAMES XX+ #ifdef TURBO XX+ #define SHORT_EXTERNAL_NAMES XX+ #endif XX+ #endif XX+ XX #ifndef FILE XX #include <stdio.h> XX #endif XX*************** XX*** 41,47 **** XX #endif XX XX #ifndef VMS XX! char *memset(); XX #else XX /* memset is needed for old versions of the VMS C runtime library */ XX #define memset(s, c, n) \ XX--- 50,58 ---- XX #endif XX XX #ifndef VMS XX! #ifndef ATARI && TURBO XX! char *memset(); XX! #endif XX #else XX /* memset is needed for old versions of the VMS C runtime library */ XX #define memset(s, c, n) \ XX*************** XX*** 81,91 **** XX--- 92,129 ---- XX #define true 1 XX #define false 0 XX XX+ #ifdef ATARI XX+ #define DEFAULT_SKELETON_FILE "D:\\include\\flexskel" XX+ #endif XX+ XX XX #ifndef DEFAULT_SKELETON_FILE XX #define DEFAULT_SKELETON_FILE "flex.skel" XX #endif XX XX+ #ifdef SHORT_EXTERNAL_NAMES XX+ /* avoid long external names that are ambiguous within the first 8 characters */ XX+ #define current_mns c__mns XX+ #define current_max_rules c__max_rules XX+ #define current_max_state_type c__max_state_type XX+ #define current_max_scs c__max_scs XX+ #define current_max_dfa_size c__max__size XX+ #define current_max_xpairs c__max_xpairs XX+ #define current_max_template_xpairs c__max_template_xpairs XX+ #define current_max_dfas c__max_dfas XX+ #define current_maxccls c__maxccles XX+ #define current_max_ccl_tbl_size c__max_ccl_tbl_size XX+ #define indent_puts ind_puts XX+ #define indent_put2s ind_put2s XX+ #define gen_next_compressed_state gen_n_comressed_state XX+ #define gen_next_match gen_n_match XX+ #define gen_next_state gen_n_state XX+ #define variable_trailing_context_rules var_tr_context_rules XX+ #define variable_trailing_rule var_tr_rule XX+ #define backtrack_report backtr_report XX+ #define backtrack_file backtr_file XX+ #endif XX+ XX /* special chk[] values marking the slots taking by end-of-buffer and action XX * numbers XX */ XX*************** XX*** 305,311 **** XX int int_val; XX } ; XX XX! typedef struct hash_entry *hash_table[]; XX XX #define NAME_TABLE_HASH_SIZE 101 XX #define START_COND_HASH_SIZE 101 XX--- 343,349 ---- XX int int_val; XX } ; XX XX! typedef struct hash_entry **hash_table; XX XX #define NAME_TABLE_HASH_SIZE 101 XX #define START_COND_HASH_SIZE 101 XX*************** XX*** 372,378 **** XX extern int datapos, dataline, linenum; XX extern FILE *skelfile, *yyin, *temp_action_file, *backtrack_file; XX extern char *infilename; XX! extern char action_file_name[]; XX XX XX /* variables for stack of states having only one out-transition: XX--- 410,416 ---- XX extern int datapos, dataline, linenum; XX extern FILE *skelfile, *yyin, *temp_action_file, *backtrack_file; XX extern char *infilename; XX! extern char *action_file_name; XX XX XX /* variables for stack of states having only one out-transition: X/ Xecho x - main.c_cdiff Xsed '/^X/s///' > main.c_cdiff << '/' XX*** Src_2.1/main.c Thu Jun 28 00:30:39 1989 XX--- main.c Thu Jul 3 02:27:47 1989 XX*************** XX*** 81,96 **** XX FILE *temp_action_file; XX FILE *backtrack_file; XX int end_of_buffer_state; XX! #ifndef SHORT_FILE_NAMES XX! char action_file_name[] = "/tmp/flexXXXXXX"; XX! #else XX! char action_file_name[] = "flexXXXXXX.tmp"; XX! #endif XX! XX #ifndef SHORT_FILE_NAMES XX static char outfile[] = "lex.yy.c"; XX #else XX! static char outfile[] = "lexyy.c"; XX #endif XX static int outfile_created = 0; XX XX--- 81,91 ---- XX FILE *temp_action_file; XX FILE *backtrack_file; XX int end_of_buffer_state; XX! char *action_file_name; XX #ifndef SHORT_FILE_NAMES XX static char outfile[] = "lex.yy.c"; XX #else XX! static char outfile[] = "lex_yy.c"; XX #endif XX static int outfile_created = 0; XX XX*************** XX*** 328,333 **** XX--- 323,329 ---- XX { XX int i, sawcmpflag, use_stdout; XX char *arg, *skelname = NULL, *flex_gettime(), clower(), *mktemp(); XX+ char *tmp_action =(char *)0, *malloc(); XX XX printstats = syntaxerror = trace = spprdflt = interactive = caseins = false; XX backtrack_report = performance_report = ddebug = fulltbl = fullspd = false; XX*************** XX*** 349,354 **** XX--- 345,355 ---- XX for ( i = 1; arg[i] != '\0'; ++i ) XX switch ( arg[i] ) XX { XX+ case 'a': XX+ if ( i != 1 ) XX+ flexerror( "-a flag must be given separately" ); XX+ tmp_action = &arg[i+1]; XX+ goto get_next_arg; XX case 'b': XX backtrack_report = true; XX break; XX*************** XX*** 445,452 **** XX printstats = true; XX break; XX XX! default: XX! lerrif( "unknown flag %c", (int) arg[i] ); XX break; XX } XX XX--- 446,458 ---- XX printstats = true; XX break; XX XX! case '?': XX! flexinfo(0); XX! break; XX! XX! default: XX! fprintf(stderr,"flex : unknown flag %c\n", (int) arg[i] ); XX! flexinfo(1); XX break; XX } XX XX*************** XX*** 454,459 **** XX--- 460,493 ---- XX ; XX } XX XX+ XX+ /* if you change the default tmp file names don't forget to change the XX+ initialization for i, too ! XX+ XX+ */ XX+ #ifndef SHORT_FILE_NAMES XX+ i = 10; XX+ if (!tmp_action) i += 5; XX+ #else XX+ i = 12; XX+ #endif XX+ if (tmp_action) XX+ i += strlen(tmp_action) + 1; XX+ if((action_file_name = malloc(i+1)) == (char *)0) XX+ flexerror("No memory for action_file_name"); XX+ *action_file_name = (char) NULL; XX+ if (tmp_action) XX+ strcat(action_file_name,tmp_action); XX+ #ifndef SHORT_FILE_NAMES XX+ else XX+ strcat(action_file_name,"/tmp"); XX+ strcat(action_file_name,"/flexXXXXXX"); XX+ #else XX+ if (tmp_action) XX+ strcat(action_file_name,"/"); XX+ strcat(action_file_name,"flexXXXX.tmp"); XX+ #endif XX+ XX if ( (fulltbl || fullspd) && usemecs ) XX flexerror( "full table and -cm don't make sense together" ); XX XX*************** XX*** 520,526 **** XX if ( (skelfile = fopen( skelname, "r" )) == NULL ) XX lerrsf( "can't open skeleton file %s", skelname ); XX XX! (void) mktemp( action_file_name ); XX XX if ( (temp_action_file = fopen( action_file_name, "w" )) == NULL ) XX lerrsf( "can't open temporary action file %s", action_file_name ); XX--- 554,562 ---- XX if ( (skelfile = fopen( skelname, "r" )) == NULL ) XX lerrsf( "can't open skeleton file %s", skelname ); XX XX! #ifndef ATARI && LATTICE XX! (void) mktemp( action_file_name ); XX! #endif XX XX if ( (temp_action_file = fopen( action_file_name, "w" )) == NULL ) XX lerrsf( "can't open temporary action file %s", action_file_name ); XX*************** XX*** 566,571 **** XX--- 602,640 ---- XX } XX XX XX+ flexinfo(status) XX+ int status; XX+ { XX+ fprintf(stderr,"Syntax : FLEX inp_file\n"); XX+ fprintf(stderr,"Function: fast lexical analyzer generator V%s\n",flex_version); XX+ fprintf(stderr,"Options : a dir_path : directory path for temporary files\n"); XX+ fprintf(stderr," - b : generate backtracking information to lex.backtrack\n"); XX+ fprintf(stderr," - c : compressed table, no equiv., no meta equiv.classes\n"); XX+ fprintf(stderr," e : equivalence classes\n"); XX+ fprintf(stderr," F : fast table\n"); XX+ fprintf(stderr," |f : full table\n"); XX+ fprintf(stderr," |m : meta equivalence classes\n"); XX+ fprintf(stderr," - d : generate debugging scanner\n"); XX+ fprintf(stderr," - F : fast table\n"); XX+ fprintf(stderr," - f : full (not compressed) table\n"); XX+ fprintf(stderr," - I : generate interactive scanner\n"); XX+ fprintf(stderr," - i : generate case-insensitive scanner\n"); XX+ fprintf(stderr," - L : supress #line directives\n"); XX+ /* fprintf(stderr," - n hexnum : generate scanner using <hexnum> as newline char.\n");*/ XX+ fprintf(stderr," - p : generate performance report to stderr\n"); XX+ fprintf(stderr," - S skeleton_path : file path for skeleton file\n"); XX+ fprintf(stderr," - s : suppress echo of unmatched scanner input to stdout\n"); XX+ fprintf(stderr," - T : run flex in trace mode\n"); XX+ #ifdef ATARI XX+ fprintf(stderr," - t : place result on stdout instead of lex_yy.c\n"); XX+ #else XX+ fprintf(stderr," - t : place result on stdout instead of lex.yy.c\n"); XX+ #endif XX+ fprintf(stderr," - v : print statistics of generated scanner\n"); XX+ fprintf(stderr," default = -cem\n"); XX+ exit(status); XX+ } XX+ XX /* readin - read in the rules section of the input file(s) XX * XX * synopsis X/ Xecho x - scan.l_cdiff Xsed '/^X/s///' > scan.l_cdiff << '/' XX*** Src_2.1/scan.l Thu Jun 30 19:42:00 1989 XX--- scan.l Thu Jul 2 22:19:26 1989 XX*************** XX*** 125,132 **** XX XX {SCNAME} RETURNNAME; XX ^{OPTWS}\n ++linenum; /* allows blank lines in section 1 */ XX! \n ++linenum; return ( '\n' ); XX! . synerr( "illegal character" ); BEGIN(RECOVER); XX XX XX <C_COMMENT>"*/" ECHO; BEGIN(0); XX--- 125,136 ---- XX XX {SCNAME} RETURNNAME; XX ^{OPTWS}\n ++linenum; /* allows blank lines in section 1 */ XX! {OPTWS}\n ++linenum; return ( '\n' ); XX! . { XX! synerr( "illegal character" ); XX! fprintf(stderr,"Char : $%x\n",yytext[yyleng-1]); XX! BEGIN(RECOVER); XX! } XX XX XX <C_COMMENT>"*/" ECHO; BEGIN(0); XX*************** XX*** 206,212 **** XX <SECT2>^{OPTWS}\n ++linenum; /* allow blank lines in section 2 */ XX XX /* this horrible mess of a rule matches indented lines which XX! * do not contain "/*". We need to make the distinction because XX * otherwise this rule will be taken instead of the rule which XX * matches the beginning of comments like this one XX */ XX--- 210,216 ---- XX <SECT2>^{OPTWS}\n ++linenum; /* allow blank lines in section 2 */ XX XX /* this horrible mess of a rule matches indented lines which XX! * do not contain '/','*'. We need to make the distinction because XX * otherwise this rule will be taken instead of the rule which XX * matches the beginning of comments like this one XX */ X/ Xecho x - Makefile.tos Xsed '/^X/s///' > Makefile.tos << '/' XX# make file for "flex" tool XX XX# @(#) $Header: Makefile,v 2.3 89/06/20 17:27:12 vern Exp $ (LBL) XX XX# Porting considerations: XX# XX# For System V Unix machines, add -DSYS_V to CFLAGS. XX# For Vax/VMS, add -DSYS_V to CFLAGS. XX# For MS-DOS, add "-DMS_DOS -DSYS_V" to CFLAGS. Create \tmp if not present. XX# You will also want to rename flex.skel to something with a three XX# character extension, change SKELETON_FILE below appropriately, XX# For Amiga, add "-DAMIGA -DSYS_V" to CFLAGS. XX# XX# A long time ago, flex was successfully built using Microsoft C and XX# the following options: /AL, /stack:10000, -LARGE, -Ml, -Mt128, -DSYS_V XX XX XX# the first time around use "make first_flex" XX XX# The following definitions must be set according to your compiler - XX# examples for a Lattice Compiler with GST assembler and TURBO C with XX# assembler are provided below and must just be updated (don't forget to XX# update the linker argument extension files (*.lnk,*.tlk), too) : XX# XX#CCPATH = path to compiler directory without trailing \ XX#CHPATH = path to header files without trailing \ XX#CC = filename of the compiler XX#CFLAGS = compiler option flags XX#CIEXT = extension of C sources that should be used for input filenames XX#ASMPATH = path to assembler directory without trailing \ XX#ASM = filename of the assembler XX#AFLAGS = assembler option flags XX#AIEXT = extension of assembler sources that should be used for assembler XX# input filenames XX#AEXT = general assembler filename extension XX#LNKPATH = path to linker directory without trailing \ XX#LINK = filename of the linker XX#LFLAG0 = first option (full pathname of C startupcode) XX#LFLAG1 = further options + option flag for argument extension filename XX#LFLAG2 = further options + option flag for output-filename XX#LNKEXT = extension of linker argument extension file XX#OIEXT = extension of objects that should be used for linker input files XX#OEXT = general object file extension XX XX# Lattice definitions XXCCPATH = d:\latt XXCHPATH = d:\latt\include XXCC = lc.ttp XXCFLAGS = -h -n -cw -cc -i$(CHPATH)\ -g$(CCPATH)\ -dLATTICE -dATARI XXCIEXT = XXASMPATH = d:\gst XXASM = assem.ttp XXAFLAGS = -nolist -errors errors.out XXAIEXT = XXAEXT = .asm XXLNKPATH = d:\gst XXLINK = ld.ttp XXLFLAG0 = XXLFLAG1 = -with XXLFLAG2 = -nolist -sec -mem 200 -prog XXLNKEXT = .lnk XXOIEXT = XXOEXT = .bin XX XX# Turbo definitions XX#CCPATH = d:\turbo XX#CHPATH = d:\turbo\include XX#CC = tcc.prg XX#CFLAGS = -GJMPRZ -H=$(CHPATH)\ -w- -DTURBO -DATARI XX#CIEXT = .c XX#ASMPATH = d:\turbo XX#ASM = mas.prg XX#AFLAGS = XX#AIEXT = .s XX#AEXT = .s XX#LNKPATH = d:\turbo XX#LINK = tlink.ttp XX#LFLAG0 = $(LNKPATH)\lib\tcstart.o XX#LFLAG1 = -I= XX#LFLAG2 = -O= XX#LNKEXT = .tlk XX#OIEXT = .o XX#OEXT = .o XX XX# Other definitions XX# (not used for Atari because of short argument string - defined in flexdef.h XX XXSKELETON_DIR = /usr/lib XXSKELETON_FILE = flex.skel XXSKELFLAGS = -DDEFAULT_SKELETON_FILE=\"$(SKELETON_DIR)/$(SKELETON_FILE)\" XX XX XXFLEX = d:\etc\flex.ttp XXFLEX_FLAGS = -is XXYACC = d:\etc\yacc.ttp XXRENAME = d:\bin\rename XXDEL = d:\bin\del XX XX# Internal definitions XXLNK = $(LNKPATH)\$(LINK) XX XXFLEXOBJS = \ XX ccl$(OEXT) \ XX dfa$(OEXT) \ XX ecs$(OEXT) \ XX gen$(OEXT) \ XX main$(OEXT) \ XX misc$(OEXT) \ XX nfa$(OEXT) \ XX parse$(OEXT) \ XX scan$(OEXT) \ XX sym$(OEXT) \ XX tblcmp$(OEXT) \ XX yylex$(OEXT) XX XXFLEX_C_SOURCES = \ XX ccl.c \ XX dfa.c \ XX ecs.c \ XX gen.c \ XX main.c \ XX misc.c \ XX nfa.c \ XX parse.c \ XX scan.c \ XX sym.c \ XX tblcmp.c \ XX yylex.c XX XXflex : $(FLEXOBJS) XX $(LNK) $(LFLAG0) $(LFLAG1)flex$(LNKEXT) $(LFLAG2)flex.ttp XX XXfirst_flex: XX cp initscan.c scan.c XX make $(MFLAGS) flex XX XXparse.h parse.c : parse.y XX $(YACC) -d parse.y XX $(DEL) parse.c XX $(RENAME) y_tab.c parse.c XX $(DEL) parse.h XX $(RENAME) y_tab.h parse.h XX XXscan.c : scan.l XX $(FLEX) $(FLEX_FLAGS) scan.l XX $(RENAME) lex_yy.c scan.c XX XXscan$(OEXT) : scan.c parse.h flexdef.h XX $(CCPATH)\$(CC) $(CFLAGS) scan$(CIEXT) XX XXmain$(OEXT) : main.c flexdef.h XX $(CCPATH)\$(CC) $(CFLAGS) main$(CIEXT) XX XXccl$(OEXT) : ccl.c flexdef.h XX $(CCPATH)\$(CC) $(CFLAGS) ccl$(CIEXT) XX XXdfa$(OEXT) : dfa.c flexdef.h XX $(CCPATH)\$(CC) $(CFLAGS) dfa$(CIEXT) XX XXecs$(OEXT) : ecs.c flexdef.h XX $(CCPATH)\$(CC) $(CFLAGS) ecs$(CIEXT) XX XXgen$(OEXT) : gen.c flexdef.h XX $(CCPATH)\$(CC) $(CFLAGS) gen$(CIEXT) XX XXmisc$(OEXT) : misc.c flexdef.h XX $(CCPATH)\$(CC) $(CFLAGS) misc$(CIEXT) XX XXnfa$(OEXT) : nfa.c flexdef.h XX $(CCPATH)\$(CC) $(CFLAGS) nfa$(CIEXT) XX XXparse$(OEXT) : parse.c flexdef.h XX $(CCPATH)\$(CC) $(CFLAGS) parse$(CIEXT) XX XXsym$(OEXT) : sym.c flexdef.h XX $(CCPATH)\$(CC) $(CFLAGS) sym$(CIEXT) XX XXtblcmp$(OEXT) : tblcmp.c flexdef.h XX $(CCPATH)\$(CC) $(CFLAGS) tblcmp$(CIEXT) XX XXyylex$(OEXT) : yylex.c flexdef.h XX $(CCPATH)\$(CC) $(CFLAGS) yylex$(CIEXT) XX XXflex.man : flex.1 XX nroff -man flex.1 >flex.man XX XXlint : $(FLEX_C_SOURCES) XX lint $(FLEX_C_SOURCES) > flex.lint XX XXdistrib : XX mv scan.c initscan.c XX chmod 444 initscan.c XX $(MAKE) $(MFLAGS) clean XX XXclean : XX rm -f core errs flex *$(OEXT) parse.c *.lint parse.h flex.man tags XX XXtags : XX ctags $(FLEX_C_SOURCES) XX XXvms : flex.man XX $(MAKE) $(MFLAGS) distrib XX XXtest : XX $(FLEX) $(FLEX_FLAGS) scan.l XX $(RENAME) lex_yy.c scan.ctest XX cmp scan.c scan.ctest XX X/ Xecho x - Readme2 Xsed '/^X/s///' > Readme2 << '/' XXThe changes: (1.7.89 RAL) XX XX - Bug fix: The original flex didn't like trailing spaces in exclusive start XX condition lists ! If you add an trailing space to line 68 in scan.l XX XX "%x SECT2 SECT2PROLOG SECT3 CODEBLOCK PICKUPDEF SC CARETISBOL NUM QUOTE " XX XX you get a misleading error message: XX XX "Syntax error at line 69: bad start condition list" XX XX This bug can either be fixed in parse.y or in scan.l . I have chosen the XX last because there the fix is minimal: Just change the rule (line 128) XX XX "\n" to "{OPTWS}\n" XX XX - Enhancements: XX - new option "-?" that provides some help information about the other XX flags (main.c) XX - new option "-aTMPPATH" that allows a redefinition of the standard XX path for the temporary file (this might be rather large >200k if XX F,f options are selected). (main.c, flexdef.h (l.376)) XX - hexdump of illegal characters -- this proved to be a useful debugging XX tool especialy if invisible control characters occur which weren't XX covered by the rules. (scan.l fprintf statement line 129,...) XX XX - Patches due to TOS XX - General: TOS uses CR,LF as end of line char., Flex wants only a single XX LF as EOL char. Therefore all I/O must be translated using f* calls. XX This is done everywhere besides the YY_INPUT macro (flex.skel (scan.c), XX line 31) that uses a low level 'read'. This should be definitly changed XX to fread, so that all I/O calls occur on the same level. XX - the short action_file_name has been "flexXXXXXX.tmp", but that's too XX much for TOS,MSDOS ! I changed it to "flexXXXX.tmp" in main.c (patch XX covered by the -a option additions) XX - some compilers don't like external names that are ambiguous within XX the first 8 characters. I defined macros that change all these long XX external names to names that are unique within the first 8 characters. XX Just define SHORT_EXTERNAL_NAMES to use this feature (flexdef.h) XX - some statement changes that some compiler don't like: XX typedef *xxx[] -> typedef **xxx (flexdef.h.,l.308) XX "/*" -> '/','*' within a comment in (scan.l, l.209) XX - changed short "lexyy.c" to "lex_yy.c" what's more similar to the unix XX "lex.yy.c" (main.c). XX - a few further really compiler dependent changes provided with XX #ifdef ATARI && LATTICE res. TURBO braces. XX XX - Additional TOS only files XX - Makefile.tos: Common makefile for all TOS compilers. If you add further XX ones please email me the new makefile. XX - flex.lnk: Lattice - GST linker argument extension file XX - flex.tlk: Turbo linker argument extension file XX XX XXAdditional remarks: XX XXI didn't add a new initscan.c (= flex -ist scan.l). The current one is good XXenough for a first compilation. With this first version of flex you can XXrebuild your own scan.c and the final flex version ! XX XXMinix ST : XX - I had to "chmem =70000 cv" (>50000) to get flex linked XX - 'memset' (PC 1.3, EFTH40,...) is necessary XX - chmem =90000 flex may be sufficient XX XXMinix PC : XX It should be possible to port Flex to Minix PC. The current sizes of flex XX are: XX Minix ST (ACK) Lattice (TOS) Turbo (TOS) XX XX size 75300 83305 57957 XX compilation time 22' 15' 3'40" XX flex -is scan.l 1'49" 43" 30" XX XX The Minix ST size includes the bad generated code using only a subset of XX the 68000 commands, long addresses only and a huge relocation table. XX Therefore the PC size will be <64 k ! More serious is the fact that I had XX to chmem =90000 flex to get scan.l converted to scan.c . But I never saw XX a more complex lex source than scan.l -- so it should be possible to XX reduce some array sizes without limitation for all day usage. XX XX No one volunteered yet for a Minix PC port -- but if someone will try it XX I would provide him with a new scan.c and some hints. XX XXTOS: XX Don't forget to adapt the flexskel path within flexdef.h ! XX XX X/ Xecho x - flex.lnk Xsed '/^X/s///' > flex.lnk << '/' XX* XX* XX* linker control file for flex.ttp XX* XX* XX* XXINPUT d:\latt\lib\startup.bin XX* XXINPUT ccl.bin XXINPUT dfa.bin XXINPUT ecs.bin XXINPUT gen.bin XXINPUT misc.bin XXINPUT nfa.bin XXINPUT parse.bin XXINPUT sym.bin XXINPUT tblcmp.bin XXINPUT main.bin XXINPUT yylex.bin XXINPUT scan.bin XX* XXLIBRARY d:\latt\lib\clib.bin XX X/ Xecho x - flex.tlk Xsed '/^X/s///' > flex.tlk << '/' XXccl.o XXdfa.o XXecs.o XXgen.o XXmisc.o XXnfa.o XXparse.o XXsym.o XXtblcmp.o XXyylex.o XXmain.o XXscan.o XXd:\turbo\lib\tcstdlib.lib ; standard lib XXd:\turbo\lib\tcextlib.lib ; extended lib XXd:\turbo\lib\tctoslib.lib ; TOS lib XXd:\turbo\lib\tcgemlib.lib ; AES and VDI lib XX-S=200000 X/ END_OF_FILE if test 29474 -ne `wc -c <'MISC/Atari.patches'`; then echo shar: \"'MISC/Atari.patches'\" unpacked with wrong size! fi # end of 'MISC/Atari.patches' fi if test -f 'flex.1' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'flex.1'\" else echo shar: Extracting \"'flex.1'\" \(20799 characters\) sed "s/^X//" >'flex.1' <<'END_OF_FILE' X.TH FLEX 1 "26 May 1990" "Version 2.3" X.SH NAME Xflex - fast lexical analyzer generator X.SH SYNOPSIS X.B flex X.B [-bcdfinpstvFILT8 -C[efmF] -Sskeleton] X.I [filename ...] X.SH DESCRIPTION X.I flex Xis a tool for generating X.I scanners: Xprograms which recognized lexical patterns in text. X.I flex Xreads Xthe given input files, or its standard input if no file names are given, Xfor a description of a scanner to generate. The description is in Xthe form of pairs Xof regular expressions and C code, called X.I rules. flex Xgenerates as output a C source file, X.B lex.yy.c, Xwhich defines a routine X.B yylex(). XThis file is compiled and linked with the X.B -lfl Xlibrary to produce an executable. When the executable is run, Xit analyzes its input for occurrences Xof the regular expressions. Whenever it finds one, it executes Xthe corresponding C code. X.LP XFor full documentation, see X.B flexdoc(1). XThis manual entry is intended for use as a quick reference. X.SH OPTIONS X.I flex Xhas the following options: X.TP X.B -b XGenerate backtracking information to X.I lex.backtrack. XThis is a list of scanner states which require backtracking Xand the input characters on which they do so. By adding rules one Xcan remove backtracking states. If all backtracking states Xare eliminated and X.B -f Xor X.B -F Xis used, the generated scanner will run faster. X.TP X.B -c Xis a do-nothing, deprecated option included for POSIX compliance. X.IP X.B NOTE: Xin previous releases of X.I flex X.B -c Xspecified table-compression options. This functionality is Xnow given by the X.B -C Xflag. To ease the the impact of this change, when X.I flex Xencounters X.B -c, Xit currently issues a warning message and assumes that X.B -C Xwas desired instead. In the future this "promotion" of X.B -c Xto X.B -C Xwill go away in the name of full POSIX compliance (unless Xthe POSIX meaning is removed first). X.TP X.B -d Xmakes the generated scanner run in X.I debug Xmode. Whenever a pattern is recognized and the global X.B yy_flex_debug Xis non-zero (which is the default), the scanner will Xwrite to X.I stderr Xa line of the form: X.nf X X --accepting rule at line 53 ("the matched text") X X.fi XThe line number refers to the location of the rule in the file Xdefining the scanner (i.e., the file that was fed to flex). Messages Xare also generated when the scanner backtracks, accepts the Xdefault rule, reaches the end of its input buffer (or encounters Xa NUL; the two look the same as far as the scanner's concerned), Xor reaches an end-of-file. X.TP X.B -f Xspecifies (take your pick) X.I full table Xor X.I fast scanner. XNo table compression is done. The result is large but fast. XThis option is equivalent to X.B -Cf X(see below). X.TP X.B -i Xinstructs X.I flex Xto generate a X.I case-insensitive Xscanner. The case of letters given in the X.I flex Xinput patterns will Xbe ignored, and tokens in the input will be matched regardless of case. The Xmatched text given in X.I yytext Xwill have the preserved case (i.e., it will not be folded). X.TP X.B -n Xis another do-nothing, deprecated option included only for XPOSIX compliance. X.TP X.B -p Xgenerates a performance report to stderr. The report Xconsists of comments regarding features of the X.I flex Xinput file which will cause a loss of performance in the resulting scanner. X.TP X.B -s Xcauses the X.I default rule X(that unmatched scanner input is echoed to X.I stdout) Xto be suppressed. If the scanner encounters input that does not Xmatch any of its rules, it aborts with an error. X.TP X.B -t Xinstructs X.I flex Xto write the scanner it generates to standard output instead Xof X.B lex.yy.c. X.TP X.B -v Xspecifies that X.I flex Xshould write to X.I stderr Xa summary of statistics regarding the scanner it generates. X.TP X.B -F Xspecifies that the X.ul Xfast Xscanner table representation should be used. This representation is Xabout as fast as the full table representation X.ul X(-f), Xand for some sets of patterns will be considerably smaller (and for Xothers, larger). See X.B flexdoc(1) Xfor details. X.IP XThis option is equivalent to X.B -CF X(see below). X.TP X.B -I Xinstructs X.I flex Xto generate an X.I interactive Xscanner, that is, a scanner which stops immediately rather than Xlooking ahead if it knows Xthat the currently scanned text cannot be part of a longer rule's match. XAgain, see X.B flexdoc(1) Xfor details. X.IP XNote, X.B -I Xcannot be used in conjunction with X.I full Xor X.I fast tables, Xi.e., the X.B -f, -F, -Cf, Xor X.B -CF Xflags. X.TP X.B -L Xinstructs X.I flex Xnot to generate X.B #line Xdirectives in X.B lex.yy.c. XThe default is to generate such directives so error Xmessages in the actions will be correctly Xlocated with respect to the original X.I flex Xinput file, and not to Xthe fairly meaningless line numbers of X.B lex.yy.c. X.TP X.B -T Xmakes X.I flex Xrun in X.I trace Xmode. It will generate a lot of messages to X.I stdout Xconcerning Xthe form of the input and the resultant non-deterministic and deterministic Xfinite automata. This option is mostly for use in maintaining X.I flex. X.TP X.B -8 Xinstructs X.I flex Xto generate an 8-bit scanner. XOn some sites, this is the default. On others, the default Xis 7-bit characters. To see which is the case, check the verbose X.B (-v) Xoutput for "equivalence classes created". If the denominator of Xthe number shown is 128, then by default X.I flex Xis generating 7-bit characters. If it is 256, then the default is X8-bit characters. X.TP X.B -C[efmF] Xcontrols the degree of table compression. X.IP X.B -Ce Xdirects X.I flex Xto construct X.I equivalence classes, Xi.e., sets of characters Xwhich have identical lexical properties. XEquivalence classes usually give Xdramatic reductions in the final table/object file sizes (typically Xa factor of 2-5) and are pretty cheap performance-wise (one array Xlook-up per character scanned). X.IP X.B -Cf Xspecifies that the X.I full Xscanner tables should be generated - X.I flex Xshould not compress the Xtables by taking advantages of similar transition functions for Xdifferent states. X.IP X.B -CF Xspecifies that the alternate fast scanner representation (described in X.B flexdoc(1)) Xshould be used. X.IP X.B -Cm Xdirects X.I flex Xto construct X.I meta-equivalence classes, Xwhich are sets of equivalence classes (or characters, if equivalence Xclasses are not being used) that are commonly used together. Meta-equivalence Xclasses are often a big win when using compressed tables, but they Xhave a moderate performance impact (one or two "if" tests and one Xarray look-up per character scanned). X.IP XA lone X.B -C Xspecifies that the scanner tables should be compressed but neither Xequivalence classes nor meta-equivalence classes should be used. X.IP XThe options X.B -Cf Xor X.B -CF Xand X.B -Cm Xdo not make sense together - there is no opportunity for meta-equivalence Xclasses if the table is not being compressed. Otherwise the options Xmay be freely mixed. X.IP XThe default setting is X.B -Cem, Xwhich specifies that X.I flex Xshould generate equivalence classes Xand meta-equivalence classes. This setting provides the highest Xdegree of table compression. You can trade off Xfaster-executing scanners at the cost of larger tables with Xthe following generally being true: X.nf X X slowest & smallest X -Cem X -Cm X -Ce X -C X -C{f,F}e X -C{f,F} X fastest & largest X X.fi X.IP X.B -C Xoptions are not cumulative; whenever the flag is encountered, the Xprevious -C settings are forgotten. X.TP X.B -Sskeleton_file Xoverrides the default skeleton file from which X.I flex Xconstructs its scanners. You'll never need this option unless you are doing X.I flex Xmaintenance or development. X.SH SUMMARY OF FLEX REGULAR EXPRESSIONS XThe patterns in the input are written using an extended set of regular Xexpressions. These are: X.nf X X x match the character 'x' X . any character except newline X [xyz] a "character class"; in this case, the pattern X matches either an 'x', a 'y', or a 'z' X [abj-oZ] a "character class" with a range in it; matches X an 'a', a 'b', any letter from 'j' through 'o', X or a 'Z' X [^A-Z] a "negated character class", i.e., any character X but those in the class. In this case, any X character EXCEPT an uppercase letter. X [^A-Z\\n] any character EXCEPT an uppercase letter or X a newline X r* zero or more r's, where r is any regular expression X r+ one or more r's X r? zero or one r's (that is, "an optional r") X r{2,5} anywhere from two to five r's X r{2,} two or more r's X r{4} exactly 4 r's X {name} the expansion of the "name" definition X (see above) X "[xyz]\\"foo" X the literal string: [xyz]"foo X \\X if X is an 'a', 'b', 'f', 'n', 'r', 't', or 'v', X then the ANSI-C interpretation of \\x. X Otherwise, a literal 'X' (used to escape X operators such as '*') X \\123 the character with octal value 123 X \\x2a the character with hexadecimal value 2a X (r) match an r; parentheses are used to override X precedence (see below) X X X rs the regular expression r followed by the X regular expression s; called "concatenation" X X X r|s either an r or an s X X X r/s an r but only if it is followed by an s. The X s is not part of the matched text. This type X of pattern is called as "trailing context". X ^r an r, but only at the beginning of a line X r$ an r, but only at the end of a line. Equivalent X to "r/\\n". X X X <s>r an r, but only in start condition s (see X below for discussion of start conditions) X <s1,s2,s3>r X same, but in any of start conditions s1, X s2, or s3 X X X <<EOF>> an end-of-file X <s1,s2><<EOF>> X an end-of-file when in start condition s1 or s2 X X.fi XThe regular expressions listed above are grouped according to Xprecedence, from highest precedence at the top to lowest at the bottom. XThose grouped together have equal precedence. X.LP XSome notes on patterns: X.IP - XNegated character classes X.I match newlines Xunless "\\n" (or an equivalent escape sequence) is one of the Xcharacters explicitly present in the negated character class X(e.g., "[^A-Z\\n]"). X.IP - XA rule can have at most one instance of trailing context (the '/' operator Xor the '$' operator). The start condition, '^', and "<<EOF>>" patterns Xcan only occur at the beginning of a pattern, and, as well as with '/' and '$', Xcannot be grouped inside parentheses. The following are all illegal: X.nf X X foo/bar$ X foo|(bar$) X foo|^bar X <sc1>foo<sc2>bar X X.fi X.SH SUMMARY OF SPECIAL ACTIONS XIn addition to arbitrary C code, the following can appear in actions: X.IP - X.B ECHO Xcopies yytext to the scanner's output. X.IP - X.B BEGIN Xfollowed by the name of a start condition places the scanner in the Xcorresponding start condition. X.IP - X.B REJECT Xdirects the scanner to proceed on to the "second best" rule which matched the Xinput (or a prefix of the input). X.B yytext Xand X.B yyleng Xare set up appropriately. Note that X.B REJECT Xis a particularly expensive feature in terms scanner performance; Xif it is used in X.I any Xof the scanner's actions it will slow down X.I all Xof the scanner's matching. Furthermore, X.B REJECT Xcannot be used with the X.I -f Xor X.I -F Xoptions. X.IP XNote also that unlike the other special actions, X.B REJECT Xis a X.I branch; Xcode immediately following it in the action will X.I not Xbe executed. X.IP - X.B yymore() Xtells the scanner that the next time it matches a rule, the corresponding Xtoken should be X.I appended Xonto the current value of X.B yytext Xrather than replacing it. X.IP - X.B yyless(n) Xreturns all but the first X.I n Xcharacters of the current token back to the input stream, where they Xwill be rescanned when the scanner looks for the next match. X.B yytext Xand X.B yyleng Xare adjusted appropriately (e.g., X.B yyleng Xwill now be equal to X.I n X). X.IP - X.B unput(c) Xputs the character X.I c Xback onto the input stream. It will be the next character scanned. X.IP - X.B input() Xreads the next character from the input stream (this routine is called X.B yyinput() Xif the scanner is compiled using X.B C++). X.IP - X.B yyterminate() Xcan be used in lieu of a return statement in an action. It terminates Xthe scanner and returns a 0 to the scanner's caller, indicating "all done". X.IP XBy default, X.B yyterminate() Xis also called when an end-of-file is encountered. It is a macro and Xmay be redefined. X.IP - X.B YY_NEW_FILE Xis an action available only in <<EOF>> rules. It means "Okay, I've Xset up a new input file, continue scanning". X.IP - X.B yy_create_buffer( file, size ) Xtakes a X.I FILE Xpointer and an integer X.I size. XIt returns a YY_BUFFER_STATE Xhandle to a new input buffer large enough to accomodate X.I size Xcharacters and associated with the given file. When in doubt, use X.B YY_BUF_SIZE Xfor the size. X.IP - X.B yy_switch_to_buffer( new_buffer ) Xswitches the scanner's processing to scan for tokens from Xthe given buffer, which must be a YY_BUFFER_STATE. X.IP - X.B yy_delete_buffer( buffer ) Xdeletes the given buffer. X.SH VALUES AVAILABLE TO THE USER X.IP - X.B char *yytext Xholds the text of the current token. It may not be modified. X.IP - X.B int yyleng Xholds the length of the current token. It may not be modified. X.IP - X.B FILE *yyin Xis the file which by default X.I flex Xreads from. It may be redefined but doing so only makes sense before Xscanning begins. Changing it in the middle of scanning will have Xunexpected results since X.I flex Xbuffers its input. Once scanning terminates because an end-of-file Xhas been seen, X.B Xvoid yyrestart( FILE *new_file ) Xmay be called to point X.I yyin Xat the new input file. X.IP - X.B FILE *yyout Xis the file to which X.B ECHO Xactions are done. It can be reassigned by the user. X.IP - X.B YY_CURRENT_BUFFER Xreturns a X.B YY_BUFFER_STATE Xhandle to the current buffer. X.SH MACROS THE USER CAN REDEFINE X.IP - X.B YY_DECL Xcontrols how the scanning routine is declared. XBy default, it is "int yylex()", or, if prototypes are being Xused, "int yylex(void)". This definition may be changed by redefining Xthe "YY_DECL" macro. Note that Xif you give arguments to the scanning routine using a XK&R-style/non-prototyped function declaration, you must terminate Xthe definition with a semi-colon (;). X.IP - XThe nature of how the scanner Xgets its input can be controlled by redefining the X.B YY_INPUT Xmacro. XYY_INPUT's calling sequence is "YY_INPUT(buf,result,max_size)". Its Xaction is to place up to X.I max_size Xcharacters in the character array X.I buf Xand return in the integer variable X.I result Xeither the Xnumber of characters read or the constant YY_NULL (0 on Unix systems) Xto indicate EOF. The default YY_INPUT reads from the Xglobal file-pointer "yyin". XA sample redefinition of YY_INPUT (in the definitions Xsection of the input file): X.nf X X %{ X #undef YY_INPUT X #define YY_INPUT(buf,result,max_size) \\ X { \\ X int c = getchar(); \\ X result = (c == EOF) ? YY_NULL : (buf[0] = c, 1); \\ X } X %} X X.fi X.IP - XWhen the scanner receives an end-of-file indication from YY_INPUT, Xit then checks the X.B yywrap() Xfunction. If X.B yywrap() Xreturns false (zero), then it is assumed that the Xfunction has gone ahead and set up X.I yyin Xto point to another input file, and scanning continues. If it returns Xtrue (non-zero), then the scanner terminates, returning 0 to its Xcaller. X.IP XThe default X.B yywrap() Xalways returns 1. Presently, to redefine it you must first X"#undef yywrap", as it is currently implemented as a macro. It is Xlikely that X.B yywrap() Xwill soon be defined to be a function rather than a macro. X.IP - XYY_USER_ACTION Xcan be redefined to provide an action Xwhich is always executed prior to the matched rule's action. X.IP - XThe macro X.B YY_USER_INIT Xmay be redefined to provide an action which is always executed before Xthe first scan. X.IP - XIn the generated scanner, the actions are all gathered in one large Xswitch statement and separated using X.B YY_BREAK, Xwhich may be redefined. By default, it is simply a "break", to separate Xeach rule's action from the following rule's. X.SH FILES X.TP X.I flex.skel Xskeleton scanner. X.TP X.I lex.yy.c Xgenerated scanner (called X.I lexyy.c Xon some systems). X.TP X.I lex.backtrack Xbacktracking information for X.B -b Xflag (called X.I lex.bck Xon some systems). X.TP X.B -lfl Xlibrary with which to link the scanners. X.SH "SEE ALSO" X.LP Xflexdoc(1), lex(1), yacc(1), sed(1), awk(1). X.LP XM. E. Lesk and E. Schmidt, X.I LEX - Lexical Analyzer Generator X.SH DIAGNOSTICS X.I reject_used_but_not_detected undefined Xor X.LP X.I yymore_used_but_not_detected undefined - XThese errors can occur at compile time. They indicate that the Xscanner uses X.B REJECT Xor X.B yymore() Xbut that X.I flex Xfailed to notice the fact, meaning that X.I flex Xscanned the first two sections looking for occurrences of these actions Xand failed to find any, but somehow you snuck some in (via a #include Xfile, for example). Make an explicit reference to the action in your X.I flex Xinput file. (Note that previously X.I flex Xsupported a X.B %used/%unused Xmechanism for dealing with this problem; this feature is still supported Xbut now deprecated, and will go away soon unless the author hears from Xpeople who can argue compellingly that they need it.) X.LP X.I flex scanner jammed - Xa scanner compiled with X.B -s Xhas encountered an input string which wasn't matched by Xany of its rules. X.LP X.I flex input buffer overflowed - Xa scanner rule matched a string long enough to overflow the Xscanner's internal input buffer (16K bytes - controlled by X.B YY_BUF_MAX Xin "flex.skel"). X.LP X.I scanner requires -8 flag - XYour scanner specification includes recognizing 8-bit characters and Xyou did not specify the -8 flag (and your site has not installed flex Xwith -8 as the default). X.LP X.I Xfatal flex scanner internal error--end of buffer missed - XThis can occur in an scanner which is reentered after a long-jump Xhas jumped out (or over) the scanner's activation frame. Before Xreentering the scanner, use: X.nf X X yyrestart( yyin ); X X.fi X.LP X.I too many %t classes! - XYou managed to put every single character into its own %t class. X.I flex Xrequires that at least one of the classes share characters. X.SH AUTHOR XVern Paxson, with the help of many ideas and much inspiration from XVan Jacobson. Original version by Jef Poskanzer. X.LP XSee flexdoc(1) for additional credits and the address to send comments to. X.SH DEFICIENCIES / BUGS X.LP XSome trailing context Xpatterns cannot be properly matched and generate Xwarning messages ("Dangerous trailing context"). These are Xpatterns where the ending of the Xfirst part of the rule matches the beginning of the second Xpart, such as "zx*/xy*", where the 'x*' matches the 'x' at Xthe beginning of the trailing context. (Note that the POSIX draft Xstates that the text matched by such patterns is undefined.) X.LP XFor some trailing context rules, parts which are actually fixed-length are Xnot recognized as such, leading to the abovementioned performance loss. XIn particular, parts using '|' or {n} (such as "foo{3}") are always Xconsidered variable-length. X.LP XCombining trailing context with the special '|' action can result in X.I fixed Xtrailing context being turned into the more expensive X.I variable Xtrailing context. For example, this happens in the following example: X.nf X X %% X abc | X xyz/def X X.fi X.LP XUse of unput() invalidates yytext and yyleng. X.LP XUse of unput() to push back more text than was matched can Xresult in the pushed-back text matching a beginning-of-line ('^') Xrule even though it didn't come at the beginning of the line X(though this is rare!). X.LP XPattern-matching of NUL's is substantially slower than matching other Xcharacters. X.LP X.I flex Xdoes not generate correct #line directives for code internal Xto the scanner; thus, bugs in X.I flex.skel Xyield bogus line numbers. X.LP XDue to both buffering of input and read-ahead, you cannot intermix Xcalls to <stdio.h> routines, such as, for example, X.B getchar(), Xwith X.I flex Xrules and expect it to work. Call X.B input() Xinstead. X.LP XThe total table entries listed by the X.B -v Xflag excludes the number of table entries needed to determine Xwhat rule has been matched. The number of entries is equal Xto the number of DFA states if the scanner does not use X.B REJECT, Xand somewhat greater than the number of states if it does. X.LP X.B REJECT Xcannot be used with the X.I -f Xor X.I -F Xoptions. X.LP XSome of the macros, such as X.B yywrap(), Xmay in the future become functions which live in the X.B -lfl Xlibrary. This will doubtless break a lot of code, but may be Xrequired for POSIX-compliance. X.LP XThe X.I flex Xinternal algorithms need documentation. END_OF_FILE if test 20799 -ne `wc -c <'flex.1'`; then echo shar: \"'flex.1'\" unpacked with wrong size! fi # end of 'flex.1' fi echo shar: End of archive 4 \(of 10\). cp /dev/null ark4isdone MISSING="" for I in 1 2 3 4 5 6 7 8 9 10 ; do if test ! -f ark${I}isdone ; then MISSING="${MISSING} ${I}" fi done if test "${MISSING}" = "" ; then echo You have unpacked all 10 archives. rm -f ark[1-9]isdone ark[1-9][0-9]isdone else echo You still must unpack the following archives: echo " " ${MISSING} fi exit 0 exit 0 # Just in case... -- Please send comp.sources.unix-related mail to rsalz@uunet.uu.net. Use a domain-based address or give alternate paths, or you may lose out.