olson@sax.cs.uiuc.edu (Robert Olson) (08/21/90)
Well, due to the large response I'm posting the scripts here. Realize that this is just a snapshot of my current work. I make no promises, though I am interested in what you think and if it works for you. If there is continued interest I may put the effort into making it more usable. Enjoy! #! /bin/sh # This is a shell archive, meaning: # 1. Remove everything above the #! /bin/sh line. # 2. Save the resulting text in a file. # 3. Execute the file with /bin/sh (not csh) to create the files: # Gen # This archive created: Tue Aug 21 11:36:30 1990 export PATH; PATH=/bin:$PATH if test ! -d 'Gen' then echo shar: creating directory "'Gen'" mkdir 'Gen' fi echo shar: entering directory "'Gen'" cd 'Gen' if test ! -d 'src' then echo shar: creating directory "'src'" mkdir 'src' fi echo shar: entering directory "'src'" cd 'src' echo shar: extracting "'Makefile'" '(923 characters)' if test -f 'Makefile' then echo shar: will not over-write existing file "'Makefile'" else cat << \SHAR_EOF > 'Makefile' # # Robert Olson, University of Illinois at Urbana # rolson@uiuc.edu # Copyright (c) 1990 by Robert Olson # All rights reserved # This program may not be sold, but may be distributed # provided this header is included. # # # $Author: olson $ # $Revision: 1.3 $ # $State: Exp $ # $Locker: olson $ # $Source: /home/reed/Pablo/Parser/cplus/Gen/src/RCS/Makefile,v $ # $Log: Makefile,v $ # Revision 1.3 90/08/21 11:21:14 olson # added the legalese # # Revision 1.2 90/08/17 14:10:02 olson # fixed make clean # # Revision 1.1 90/08/17 14:04:31 olson # First rev # CC = gcc CFLAGS = all: tokenize tokens.pl TOKENIZE_OBJ = lextoken.o tokenize.o tokenize: $(TOKENIZE_OBJ) $(CC) $(CFLAGS) -o $@ $(TOKENIZE_OBJ) $(LIBS) tokenize.o: tokens.h lextoken.o: tokens.h .PRECIOUS: lextoken.c tokens.h tokens.pl: lextoken.l maketokens.pl lextoken.l clean: rm -f *.o *~ \#* tokenize tokens.pl tokens.h SHAR_EOF if test 923 -ne "`wc -c < 'Makefile'`" then echo shar: error transmitting "'Makefile'" '(should have been 923 characters)' fi fi # end of overwriting check echo shar: extracting "'lextoken.l'" '(2609 characters)' if test -f 'lextoken.l' then echo shar: will not over-write existing file "'lextoken.l'" else cat << \SHAR_EOF > 'lextoken.l' %Start COMMENT %{ /* * * Robert Olson, University of Illinois at Urbana * rolson@uiuc.edu * Copyright (c) 1990 by Robert Olson * All rights reserved * This program may not be sold, but may be distributed * provided this header is included. * * $Author: olson $ * $Revision: 1.2 $ * $State: Exp $ * $Locker: olson $ * $Source: /home/reed/Pablo/Parser/cplus/Gen/src/RCS/lextoken.l,v $ * $Log: lextoken.l,v $ * Revision 1.2 90/08/21 11:21:16 olson * added the legalese * * Revision 1.1 90/08/17 14:04:32 olson * First rev * * */ #include "tokens.h" %} delim [ \t\n] ws {delim}+ digit [0-9] expon ([Ee][+\-]?{digit}+) %% <INITIAL>[ \t\n] {} <INITIAL>\"[^"]*\" { return STRINGCONST; } <INITIAL>('[^']')|('\\[a-z']') { return CHARCONST; } <INITIAL>"/*" { BEGIN(COMMENT); } <COMMENT>"*/" { BEGIN(INITIAL); } <COMMENT>.|\n { } <INITIAL>static { return STATIC; } <INITIAL>typedef { return TYPEDEF; } <INITIAL>extern { return EXTERN; } <INITIAL>struct { return STRUCT; } <INITIAL>union { return UNION; } <INITIAL>#.*$ { return DIRECTIVE; } <INITIAL>"!=" { return BANGEQUAL; } <INITIAL>"!" { return BANG; } <INITIAL>"%" { return MOD; } <INITIAL>"%=" { return MODEQUAL; } <INITIAL>"&&" { return ANDAND; } <INITIAL>"&=" { return ANDEQUAL; } <INITIAL>"&" { return AND; } <INITIAL>"(" { return LPAREN; } <INITIAL>")" { return RPAREN; } <INITIAL>"*=" { return STAREQUAL; } <INITIAL>"+=" { return PLUSEQUAL; } <INITIAL>"+" { return PLUS; } <INITIAL>"-=" { return MINUSEQUAL; } <INITIAL>"->" { return POINTSTO; } <INITIAL>"-" { return MINUS; } <INITIAL>"." { return DOT; } <INITIAL>"/" { return SLASH; } <INITIAL>":" { return COLON; } <INITIAL>";" { return SEMICOLON; } <INITIAL>"<<=" { return LEFTSHIFTEQUAL; } <INITIAL>"<<" { return LEFTSHIFT; } <INITIAL>"<=" { return LESSEQUAL; } <INITIAL>"<" { return LESSTHAN; } <INITIAL>"=" { return EQUAL; } <INITIAL>">>=" { return RIGHTSHIFTEQUAL; } <INITIAL>">=" { return GREATEREQUAL; } <INITIAL>">>" { return RIGHTSHIFT; } <INITIAL>">" { return GREATERTHAN; } <INITIAL>"?" { return QUESTIONMARK; } <INITIAL>"[" { return LBRACKET; } <INITIAL>"]" { return RBRACKET; } <INITIAL>"^=" { return CARETEQUAL; } <INITIAL>"," { return COMMA; } <INITIAL>"^" { return CARET; } <INITIAL>"{" { return LBRACE; } <INITIAL>"|=" { return BAREQUAL; } <INITIAL>"||" { return BARBAR; } <INITIAL>"|" { return BAR; } <INITIAL>"}" { return RBRACE; } <INITIAL>[a-zA-Z_*$][a-zA-Z_*$0-9]+ { return WORD; } <INITIAL>[+-]?(0|([1-9][0-9]*)) { return INTEGER; } <INITIAL>[+-]?{digit}+(\.{digit}+)?{expon}? { return REAL; } <INITIAL>. { return UNKNOWN; } %% SHAR_EOF if test 2609 -ne "`wc -c < 'lextoken.l'`" then echo shar: error transmitting "'lextoken.l'" '(should have been 2609 characters)' fi fi # end of overwriting check echo shar: extracting "'tokenize.pl'" '(1180 characters)' if test -f 'tokenize.pl' then echo shar: will not over-write existing file "'tokenize.pl'" else cat << \SHAR_EOF > 'tokenize.pl' # # Robert Olson, University of Illinois at Urbana # rolson@uiuc.edu # Copyright (c) 1990 by Robert Olson # All rights reserved # This program may not be sold, but may be distributed # provided this header is included. # # # $Author: olson $ # $Revision: 1.3 $ # $State: Exp $ # $Locker: olson $ # $Source: /home/reed/Pablo/Parser/cplus/Gen/src/RCS/tokenize.pl,v $ # $Log: tokenize.pl,v $ # Revision 1.3 90/08/21 11:21:20 olson # added the legalese # # Revision 1.2 90/08/17 14:09:11 olson # fixed headers # # Revision 1.1 90/08/17 14:04:35 olson # First rev # # sub init_tokenizer { local($input_file) = @_; print "input file is $input_file dir is $perlprog_dir\n"; if (! -f $input_file) { die "Input file $input_file not found\n"; } open(INPUT, "$perlprog_directory/tokenize $input_file|") || die "Cannot open pipe: $!"; print "initialized tokenizer\n"; } sub next_token { local(*token, *val) = @_; local($end); $token = <INPUT>; $end = !defined($token); if ($end) { print "got EOF\n"; return 0; } $val = <INPUT>; chop $token; chop $val; return 1; } 1; SHAR_EOF if test 1180 -ne "`wc -c < 'tokenize.pl'`" then echo shar: error transmitting "'tokenize.pl'" '(should have been 1180 characters)' fi chmod +x 'tokenize.pl' fi # end of overwriting check echo shar: extracting "'tokenize.c'" '(1695 characters)' if test -f 'tokenize.c' then echo shar: will not over-write existing file "'tokenize.c'" else cat << \SHAR_EOF > 'tokenize.c' /* * Robert Olson, University of Illinois at Urbana * rolson@uiuc.edu * Copyright (c) 1990 by Robert Olson * All rights reserved * This program may not be sold, but may be distributed * provided this header is included. * * $Author: olson $ * $Revision: 1.3 $ * $State: Exp $ * $Locker: olson $ * $Source: /home/reed/Pablo/Parser/cplus/Gen/src/RCS/tokenize.c,v $ * $Log: tokenize.c,v $ * Revision 1.3 90/08/21 11:21:19 olson * added the legalese * * Revision 1.2 90/08/17 14:09:11 olson * fixed headers * * */ #include "tokens.h" #include <stdio.h> extern int errno; extern char *sys_errlist[]; extern char yytext[]; extern FILE *yyin, *yyout; main(int argc, char **argv) { int rc; char *inputFile; FILE *fp; char *pgm = argv[0]; int withNames; argc--; argv++; withNames = 0; while (argc > 0) { if (**argv == '-') { if (strcmp(*argv, "-names") == 0) { withNames = 1; } else { fprintf(stderr, "Invalid flag %s. Usage: %s [-names] file.l\n", *argv, pgm); exit(1); } } else break; argc--; argv++; } if (argc > 0) { inputFile = argv[0]; fp = fopen(inputFile, "r"); if (fp == NULL) { fprintf(stderr, "Cannot open %s: %s\n", inputFile, sys_errlist[errno]); exit(1); } yyin = fp; } setbuf(yyout, 0); if (withNames) { while ((rc = yylex()) > 0) { printf("%s\n%s\n", tokenNames[rc], yytext); } } else { while ((rc = yylex()) > 0) { printf("%d\n%s\n", rc, yytext); } } } yywrap() { } SHAR_EOF if test 1695 -ne "`wc -c < 'tokenize.c'`" then echo shar: error transmitting "'tokenize.c'" '(should have been 1695 characters)' fi fi # end of overwriting check echo shar: extracting "'process.pl'" '(10764 characters)' if test -f 'process.pl' then echo shar: will not over-write existing file "'process.pl'" else cat << \SHAR_EOF > 'process.pl' # # Robert Olson, University of Illinois at Urbana # rolson@uiuc.edu # Copyright (c) 1990 by Robert Olson # All rights reserved # This program may not be sold, but may be distributed # provided this header is included. # # # $Author: olson $ # $Revision: 1.3 $ # $State: Exp $ # $Locker: olson $ # $Source: /home/reed/Pablo/Parser/cplus/Gen/src/RCS/process.pl,v $ # $Log: process.pl,v $ # Revision 1.3 90/08/21 11:21:18 olson # added the legalese # # Revision 1.2 90/08/17 14:09:08 olson # fixed headers # # Revision 1.1 90/08/17 14:04:34 olson # First rev # # # $phfile_name=""; $hfile_name=""; $ccfile_name=""; $base_name = ""; %valid_types = ("int", 1, "short", 1, "float", 1, "double", 1, "char", 1, "FILE", 1, "YYSTYPE", 1); $n_variables = 0; %all_variable_index = (); @all_variable_names = (); @all_variable_types = (); @all_variable_inits = (); @all_variable_extern_flags = (); @all_variable_static_flags = (); @all_variable_typedef_flags = (); $n_functions = 0; %all_function_index = (); @all_function_names = (); @all_function_types = (); @all_function_args = (); @all_function_bodies = (); sub is_valid_type { local($type) = @_; local($flag); $flag = $valid_types{$type}; if (defined($flag) && $flag) { # print "type $type is valid\n"; return 1; } else { # print "type $type is invalid\n"; return 0; } } sub add_function { local($nargs) = 0 + @_; die "Wrong number of args ($nargs) to add_function" if $nargs != 4; local($name, $type, $args, $body) = @_; local($function_no) = $n_functions++; $all_function_index{$name} = $function_no; $all_function_names[$function_no] = $name; $all_function_types[$function_no] = $type; $all_function_args[$function_no] = $args; $all_function_bodies[$function_no] = $body; return $function_no; } sub add_variable { local($nargs) = 0 + @_; die "Wrong number of args ($nargs) to add_variable" if $nargs != 5; local($name, $type, $init, $eflag, $sflag, $tflag) = @_; local($variable_no); local($varname); ($varname) = ($name =~ /(\w+)/); # print "determined varname=\"$varname\"\n"; $variable_no = $n_variables++; if (defined($old = $all_variable_index{$varname})) { print "var $name already defined, adding new\n"; } $all_variable_index{$varname} = $variable_no; $all_variable_names[$variable_no] = $name; $all_variable_types[$variable_no] = $type; $all_variable_extern_flags[$variable_no] = $eflag; $all_variable_static_flags[$variable_no] = $sflag; $all_variable_typedef_flags[$variable_no] = $tflag; $all_variable_inits[$variable_no] = $init; return $variable_no; } sub init_process { ($base_name) = @_; print "init_process, base is $base_name\n"; $phfile_name = "$base_name-private.h"; $hfile_name = "$base_name.h"; $ccfile_name = "$base_name.cc"; open(PHFILE, ">$phfile_name") || die "Cannot open $phfile_name: $!\n"; open(HFILE, ">$hfile_name") || die "Cannot open $hfile_name: $!\n"; open(CCFILE, ">$ccfile_name") || die "Cannot open $ccfile_name: $!\n"; select(PHFILE); $| = 1; select(HFILE); $| = 1; select(CCFILE); $| = 1; select(STDOUT); $| = 1; print "calling header writing rtns\n"; do write_ccfile_header(); do write_hfile_header(); } sub finish_process { local($i); print "function info:\n"; for ($i = 0; $i < $n_functions; $i++) { printf("Function %d: name=%s type=%s args=%s\n", $i, $all_function_names[$i], $all_function_types[$i], $all_function_args[$i]); # print " Body: $all_function_bodies[$i]\n"; } print "variable info:\n"; for ($i = 0; $i < $n_variables; $i++) { printf("Variable %d: name=%s type=%s eflag=%d sflag=%d tflag=%d\n", $i, $all_variable_names[$i], $all_variable_types[$i], $all_variable_extern_flags[$i], $all_variable_static_flags[$i], $all_variable_typedef_flags[$i]); # print " Init: $all_variable_inits[$i]\n"; } print "index contents:\n"; for $key (keys(%all_variable_index)) { print "key=$key value=$all_variable_index{$key}\n"; } print "variable info from index:\n"; for $i (values(%all_variable_index)) { printf("Variable %d: name=%s type=%s eflag=%d sflag=%d tflag=%d\n", $i, $all_variable_names[$i], $all_variable_types[$i], $all_variable_extern_flags[$i], $all_variable_static_flags[$i], $all_variable_typedef_flags[$i]); # print " Init: $all_variable_inits[$i]\n"; } do write_variables(); do write_functions(); do write_hfile_trailer(); close(PHFILE); close(HFILE); close(CCFILE); } sub write_functions { local($key, $idx); local($name, $type, $args, $body); local($write_body); for $idx (0 .. @all_function_names - 1) { $name = $all_function_names[$idx]; $type = $all_function_types[$idx]; $args = $all_function_args[$idx]; $body = $all_function_bodies[$idx]; print "got func $idx name $name type $type\n"; print "args=\"$args\"\n"; # print "body=\"$body\"\n"; if ($body ne "") { $body =~ s/;/;\n/g; $body =~ s/^\s*#\s*(ifdef|elif|if)/# \1 /; if (($ptrstuff) = ($name =~ /([*&]+)/)) { local($newname) = $name; $newname =~ s/[*&]+//; print HFILE "\t$type $ptrstuff $newname $args;\n"; print CCFILE "$type $ptrstuff $base_name :: $newname $args\n"; } else { print HFILE "\t$type $name $args;\n"; print CCFILE "$type $base_name :: $name $args\n"; } print CCFILE "$body\n"; } else { # Assume its an extern decl. print HFILE "\textern $type $name $args;\n"; } } } sub write_variables { local($key); local($idx); local($name, $type, $init, $eflag, $sflag, $tflag); $constructor = "\tlexer = lex;\n" if ($output_mode eq "yacc"); $destructor = ""; for $key (keys %all_variable_index) { print "writing info for $key\n"; $idx = $all_variable_index{$key}; $name = $all_variable_names[$idx]; $type = $all_variable_types[$idx]; $init = $all_variable_inits[$idx]; $eflag = $all_variable_extern_flags[$idx]; $sflag = $all_variable_static_flags[$idx]; $tflag = $all_variable_typedef_flags[$idx]; if ($tflag) { next; } if ($init eq "") { print HFILE "\t$type $name;\n"; } elsif ($init =~ /std(in|out)/) { local($newname) = $name; local($newinit) = $init; $newinit =~ s/[{}]//g; $newname =~ s/\*//; print HFILE "\t$type $name;\n"; $constructor .= "$newname = $newinit;\n"; } elsif (($init =~ /{/) || ($init =~ /"/)) # } keep indenter happy { local($newname) = $name; $newname =~ s/\*//; print HFILE "\tstatic $type $name;\n"; print CCFILE "$type $base_name::$newname = $init;\n"; } else { local($newname) = $name; $newname =~ s/\*//; print HFILE "\t$type $name;\n"; $constructor .= "\t$newname = $init;\n"; } } if ($output_mode eq "yacc") { print CCFILE "$base_name::$base_name($other_class_name *lex)\n{\n"; } else { print CCFILE "$base_name::$base_name()\n{\n"; } print CCFILE "$constructor"; print CCFILE "}\n"; print CCFILE "$base_name::~$base_name()\n{\n"; print CCFILE "$destructor"; print CCFILE "}\n"; } sub write_ccfile_header { print "writing ccfile header\n"; print CCFILE "#include \"$hfile_name\"\n"; if ($other_class_name ne "") { print CCFILE "#include \"$other_class_name.h\"\n"; } print CCFILE "#define yylex lexer->yylex\n" if ($output_mode eq "yacc"); print CCFILE "#define yylval (parser->yylval)\n" if ($output_mode eq "lex"); print CCFILE "\n"; } sub write_hfile_header { local($optional_include) = ""; if ($other_class_name ne "") { $optional_include = "#include \"$other_class_name.h\"\nclass $other_class_name;\n"; } print "writing hfile header\n"; print HFILE <<End; #pragma once #ifndef ${base_name}_h #define ${base_name}_h #include "$phfile_name" extern int yywrap(); $optional_include class ${base_name} { public: End if ($output_mode eq "yacc") { print HFILE <<End; $other_class_name *lexer; ${base_name}($other_class_name *lex); ~$base_name(); End } elsif ($output_mode eq "lex") { print HFILE <<End; $other_class_name *parser; void setParser($other_class_name *par) { parser = par; } ; ${base_name}(); ~$base_name(); End } else { print HFILE <<End; ${base_name}(); ~$base_name(); End } } sub write_hfile_trailer { print HFILE <<End; }; #endif End } sub add_anonstruct_typedef { local($which, $def, $name) = @_; print HFILE "\ttypedef $which $def $name; /* ANON */\n"; $valid_types{$name} = 1; } sub process_simple_decl { local($type, *vars, *inits, *funcargs, *funcbodies, $extern_flag, $static_flag, $typedef_flag, $typedef_done) = @_; if ($typedef_flag) { return if ($typedef_done); print "doing typedef...\n"; if (@vars != 1) { die "Malformed typedef\n"; } print HFILE "\ttypedef $type $vars[0];\n"; $valid_types{$vars[0]} = 1; return; } for ($i = 0; $i < @vars; $i++) { if ($funcargs[$i] ne "") { if ($funcbodies[$i] ne "") { print "doing defined fun $vars[$i]\n"; do process_defined_function($type, $vars[$i], $funcargs[$i], $funcbodies[$i]); } else { print "doing declared fun $vars[$i]\n"; do process_declared_function($type, $vars[$i], $funcargs[$i]); } } else { if ($inits[$i] ne "") { do process_initialized_variable($type, $vars[$i], $inits[$i], $extern_flag, $static_flag); } else { do process_uninitialized_variable($type, $vars[$i], $extern_flag, $static_flag); } } } } sub process_defined_function { local($type, $name, $decl, $body) = @_; do add_function($name, $type, $decl, $body); } sub process_declared_function { local($type, $name, $decl) = @_; do add_function($name, $type, $decl, ""); } sub process_initialized_variable { local($type, $name, $initialization, $extern_flag, $static_flag) = @_; do add_variable($name, $type, $initialization, $extern_flag, $static_flag); } sub process_uninitialized_variable { local($type, $name, $extern_flag, $static_flag) = @_; do add_variable($name, $type, "", $extern_flag, $static_flag); } sub process_struct_decl { local($which, $name, $def, $tflag) = @_; local($i); print "got struct decl for $which $name\n"; if ($def ne "") { print HFILE "\t$which $name\n\t\t$def;\n"; } } sub process_directive { local($line) = @_; print PHFILE "$line\n"; } sub process_other { local($other) = @_; warn "got other token $other"; } 1; SHAR_EOF if test 10764 -ne "`wc -c < 'process.pl'`" then echo shar: error transmitting "'process.pl'" '(should have been 10764 characters)' fi fi # end of overwriting check echo shar: extracting "'generate-class'" '(5230 characters)' if test -f 'generate-class' then echo shar: will not over-write existing file "'generate-class'" else cat << \SHAR_EOF > 'generate-class' #!/usr/local/bin/perl # # Robert Olson, University of Illinois at Urbana # rolson@uiuc.edu # Copyright (c) 1990 by Robert Olson # All rights reserved # This program may not be sold, but may be distributed # provided this header is included. # # $Author: olson $ # $Revision: 1.3 $ # $State: Exp $ # $Locker: olson $ # $Source: /home/reed/Pablo/Parser/cplus/Gen/src/RCS/generate-class,v $ # $Log: generate-class,v $ # Revision 1.3 90/08/21 11:21:15 olson # added the legalese # # Revision 1.2 90/08/17 14:09:05 olson # fixed headers # # Revision 1.1 90/08/17 14:04:32 olson # First rev # $class_name = "Foo"; $input_file = ""; do 'getopts.pl' || die "cant do getopts.pl"; do process_args(); do "$perlprog_directory/process.pl" || die "cant do process.pl: $!"; do "$perlprog_directory/parser-rtns.pl" || die "cant do parser-rtns.pl"; do "$perlprog_directory/tokens.pl" || die "cant do tokens.pl"; do "$perlprog_directory/tokenize.pl" || die "cant do tokenize.pl"; do initialize(); do init_process($class_name); do main(); do finish_process(); exit; sub main { local(@funcs) = (); local(@funcbodies) = (); local(@rest); local($line); local($extern_flag, $static_flag); local($token, $token_value) = ("", ""); # print "before loop\n"; while (&next_token(*token, *token_value)) { $extern_flag = 0; $static_flag = 0; $typedef_flag = 0; # print "in loop, token=$token val=$token_value\n"; while ($token == &TYPEDEF || $token == &EXTERN || $token == &STATIC) { if ($token == &TYPEDEF) { $typedef_flag = 1; } if ($token == &EXTERN) { $extern_flag = 1; } if ($token == &STATIC) { $static_flag = 1; } &next_token(*token, *token_value); } if ($token == &DIRECTIVE) { # Process a preprocessor directive do process_directive($token_value); } elsif ($token == &STRUCT || $token == &UNION) { # Here: struct | union . [ aggrname ] [ { aggr body } ] varlist ; # where varlist is as above local(@result); local(@vars, @inits, @funcargs, @funcbodies); local($which) = $token_value; local($aggrname, $aggrdef); ($token, $token_value) = &read_aggr_def(*aggrname, *aggrdef); # print "done read_aggr_def, token_value=$token_value\n"; if ($token != &SEMICOLON) { &read_decl($token, $token_value, *vars, *inits, *funcargs, *funcbodies); } local($typedef_done) = 0; if ($typedef_flag && $aggr_name eq "") { if (@vars != 1) { die "Malformed typedef\n"; } do add_anonstruct_typedef($which, $aggrdef, $vars[0]); $typedef_done = 1; } if ($aggrdef ne "" && !$typedef_flag) { do process_struct_decl($which, $aggrname, $aggrdef, $typedef_flag); } do process_simple_decl("$which $aggrname", *vars, *inits, *funcargs, *funcbodies, $extern_flag, $static_flag, $typedef_flag, $typedef_done); } elsif ($token == &WORD) { if (&is_valid_type($token_value)) { # Here: typename . varlist ; # varlist: varname [= initializer] ... # # Or, its a function declaration/definition # # typename . funcname ( args ) , funcname ( args ) ... ; # or # typename . funcname ( args ) { body } local(@vars, @inits); local(@funcargs, @funcbodies); $type = $token_value; # print "got type=$type\n"; &next_token(*token, *token_value); &read_decl($token, $token_value, *vars, *inits, *funcargs, *funcbodies); do process_simple_decl($type, *vars, *inits, *funcargs, *funcbodies, $extern_flag, $static_flag, $typedef_flag); } else { # Here I just assume that this is an implicitly int-defined # typename. # print "got word $token_value at toplevel\n"; local(@vars, @inits); local(@funcargs, @funcbodies); &read_decl($token, $token_value, *vars, *inits, *funcargs, *funcbodies); do process_simple_decl("int", *vars, *inits, *funcargs, *funcbodies, $extern_flag, $static_flag, $typedef_flag); } } else { &syntax_error("toplevel", $token, $token_value); } } } sub process_args { local($name); do Getopts('c:m:i:d:'); if ($opt_c ne "") { $class_name = $opt_c; } if ($opt_m eq "lex") { $output_mode = "lex"; } elsif ($opt_m eq "yacc") { $output_mode = "yacc"; } elsif ($opt_m ne "") { die "Unknown mode $opt_m: valid modes are \"lex\" and \"yacc\"\n"; } if ($opt_d eq "") { $perlprog_directory = "."; } else { $perlprog_directory = $opt_d; } die "Cannot access perl prog directory $perprog_directory: $!" if (! -d $perlprog_directory); if ($output_mode ne "") { if ($opt_i eq "") { die "for lex/yacc modes, must specify -i otherclassname\n"; } $other_class_name = $opt_i; } $nargs = @ARGV; if ($nargs == 0) { $input_file = ""; } elsif ($nargs == 1) { $input_file = $ARGV[0]; } else { die "Usage: $0 [-c name] [-m lex|yacc] [-d perlprog-directory] [file]\n"; } } sub initialize { do init_tokenizer($input_file); } SHAR_EOF if test 5230 -ne "`wc -c < 'generate-class'`" then echo shar: error transmitting "'generate-class'" '(should have been 5230 characters)' fi chmod +x 'generate-class' fi # end of overwriting check echo shar: extracting "'parser-rtns.pl'" '(5394 characters)' if test -f 'parser-rtns.pl' then echo shar: will not over-write existing file "'parser-rtns.pl'" else cat << \SHAR_EOF > 'parser-rtns.pl' # # Robert Olson, University of Illinois at Urbana # rolson@uiuc.edu # Copyright (c) 1990 by Robert Olson # All rights reserved # This program may not be sold, but may be distributed # provided this header is included. # # # $Author: olson $ # $Revision: 1.3 $ # $State: Exp $ # $Locker: olson $ # $Source: /home/reed/Pablo/Parser/cplus/Gen/src/RCS/parser-rtns.pl,v $ # $Log: parser-rtns.pl,v $ # Revision 1.3 90/08/21 11:21:17 olson # added the legalese # # Revision 1.2 90/08/17 14:09:07 olson # fixed headers # # Revision 1.1 90/08/17 14:04:33 olson # First rev # # # # This parses # aggr_def: [aggrname] [ { aggr_body} ] [ varlist ]; # varlist: var | var, varlist # var: name [ = initializer ] # do 'tokens.pl' ; sub read_aggr_def { local(*aggr_name, *aggr_def) = @_; local($token, $token_value); local($brace_depth, $done); local($name, $init); &next_token(*token, *token_value); if ($token == &WORD) { $aggr_name = $token_value; &next_token(*token, *token_value); } if ($token == &LBRACE) { # Parse an aggr definition $aggr_def = $token_value; for ($brace_depth = 1; $brace_depth > 0; ) { &next_token(*token, *token_value); $brace_depth++ if ($token == &LBRACE); $brace_depth-- if ($token == &RBRACE); $aggr_def = &concat_tokens($aggr_def, $token_value); } &next_token(*token, *token_value); } print "token=$token val=$token_value\n"; print "got aggr name \"$aggr_name\"\n"; print "got aggr def \"$aggr_def\"\n"; return ($token, $token_value); } # Here: typename . varlist ; # varlist: varname [= initializer] ... # # Or, its a function declaration/definition # # typename . funcname ( args ) , funcname ( args ) ... ; # or # typename . funcname ( args ) { body } sub read_decl { local($token, $token_value, *vars, *inits, *funcargs, *funcbodies) = @_; local($done); local($args, $body); local($brace_depth); @vars = (); @inits = (); @funcnames = (); @funcargs = (); @funcbodies = (); print "read_decl: token=$token val=$token_value\n"; return if ($token == &SEMICOLON); if ($token != &WORD) { &syntax_error("read_decl", $token, $token_value); } for ($done = 0; !$done; ) { $name = $token_value; $init = ""; $body = ""; $args = ""; &next_token(*token, *token_value); if ($token == &LBRACKET) { # An array variable $name = &concat_tokens($name, $token_value); local($mydone) = 0; while (!$mydone) { for ($brace_depth = 1; $brace_depth > 0; ) { &next_token(*token, *token_value); $brace_depth++ if ($token == &LBRACKET); $brace_depth-- if ($token == &RBRACKET); $name = &concat_tokens($name, $token_value); } &next_token(*token, *token_value); $mydone = ($token != &LBRACKET); } } if ($token == &EQUAL) { # got an initializer, read until next comma or semicolon, # or until matching braces if there are any &next_token(*token, *token_value); if ($token == &LBRACE) { $init = $token_value; for ($brace_depth = 1; $brace_depth > 0; ) { &next_token(*token, *token_value); $brace_depth++ if ($token == &LBRACE); $brace_depth-- if ($token == &RBRACE); $init = &concat_tokens($init, $token_value); } &next_token(*token, *token_value); } else { while ($token != &COMMA && $token != &SEMICOLON) { $init = &concat_tokens($init, $token_value); &next_token(*token, *token_value); } } } elsif ($token == &LPAREN) { # A function declaration... $args = $token_value; for ($brace_depth = 1; $brace_depth > 0; ) { &next_token(*token, *token_value); $brace_depth++ if ($token == &LPAREN); $brace_depth-- if ($token == &RPAREN); $args = &concat_tokens($args, $token_value); } &next_token(*token, *token_value); if ($token == &LBRACE) { # A function definition $body = $token_value; for ($brace_depth = 1; $brace_depth > 0; ) { &next_token(*token, *token_value); $brace_depth++ if ($token == &LBRACE); $brace_depth-- if ($token == &RBRACE); $body = &concat_tokens($body, $token_value); } push(vars, $name); push(inits, $init); push(funcbodies, $body); push(funcargs, $args); # We return here, since you cant have more than one defn. return; } } print "got var name=\"$name\" init=\"$init\" \n"; print "got args=\"$args\" body=\"$body\"\n"; push(vars, $name); push(inits, $init); push(funcbodies, $body); push(funcargs, $args); if ($token == &COMMA) { &next_token(*token, *token_value); } else { $done = 1; } } print "Exiting from read_decl\n"; } sub syntax_error { local($where, $token, $token_value) = @_; local($name) = &get_token_name($token); die "Syntax error in $where token = $name val=$token_value at input line $.\n"; } sub concat_tokens { local($first, $next) = @_; local($lastchar, $firstchar); $lastchar = substr($first, -1, 1); $firstchar = substr($next, 0, 1); if ($firstchar eq "#") { return $first . "\n" . $next . "\n"; } elsif (($lastchar =~ /\w/) && ($firstchar =~ /\w/)) { return $first . " " . $next; } else { return $first . $next; } } 1; SHAR_EOF if test 5394 -ne "`wc -c < 'parser-rtns.pl'`" then echo shar: error transmitting "'parser-rtns.pl'" '(should have been 5394 characters)' fi fi # end of overwriting check echo shar: extracting "'maketokens.pl'" '(1768 characters)' if test -f 'maketokens.pl' then echo shar: will not over-write existing file "'maketokens.pl'" else cat << \SHAR_EOF > 'maketokens.pl' #!/usr/local/bin/perl # # # Robert Olson, University of Illinois at Urbana # rolson@uiuc.edu # Copyright (c) 1990 by Robert Olson # All rights reserved # This program may not be sold, but may be distributed # provided this header is included. # # $Author: olson $ # $Revision: 1.3 $ # $State: Exp $ # $Locker: olson $ # $Source: /home/reed/Pablo/Parser/cplus/Gen/src/RCS/maketokens.pl,v $ # $Log: maketokens.pl,v $ # Revision 1.3 90/08/21 11:21:16 olson # added the legalese # # Revision 1.2 90/08/17 14:09:06 olson # fixed headers # # Revision 1.1 90/08/17 14:04:32 olson # First rev # die "Usage: $0 file\n" if (@ARGV != 1); $input_file = $ARGV[0]; open(INPUT, "<$input_file") || die "Cannot open input file $input_file: $!\n"; @names = (); open(HFILE, ">tokens.h") || die "cannot open tokens.h: $!\n"; open(PFILE, ">tokens.pl") || die "cannot open tokens.pl: $!\n"; print PFILE <<End; # # This file automatically generated by maketokens.pl from $input_file # End print HFILE <<End; /* * This file automatically generated by maketokens.pl from $input_file */ End push(names, "NONE"); while (<INPUT>) { if (($name) = /.*return\s*([A-Z]+).*$/) { push(names, $name); } } for ($i = 0; $i < @names; $i++) { $name = $names[$i]; print HFILE "#define $name $i\n"; print PFILE "sub $name { return $i; }\n"; } print HFILE "\n"; print HFILE "static char *tokenNames[] = {\n"; for ($i = 0; $i < @names; $i++) { $name = $names[$i]; print HFILE "\t\"$name\",\n"; print PFILE "\$token_names[$i] = \"$name\";\n"; } print HFILE "};\n"; print PFILE <<End; sub get_token_name { local(\$token) = @_; return \$token_names[\$token]; } End print PFILE "1;\n"; close(HFILE); close(PFILE); SHAR_EOF if test 1768 -ne "`wc -c < 'maketokens.pl'`" then echo shar: error transmitting "'maketokens.pl'" '(should have been 1768 characters)' fi chmod +x 'maketokens.pl' fi # end of overwriting check echo shar: done with directory "'src'" cd .. if test ! -d 'Example' then echo shar: creating directory "'Example'" mkdir 'Example' fi echo shar: entering directory "'Example'" cd 'Example' echo shar: extracting "'parser.y'" '(1199 characters)' if test -f 'parser.y' then echo shar: will not over-write existing file "'parser.y'" else cat << \SHAR_EOF > 'parser.y' %{ /* * Robert Olson, University of Illinois at Urbana * rolson@uiuc.edu * Copyright (c) 1990 by Robert Olson * All rights reserved * This program may not be sold, but may be distributed * provided this header is included. * * $Author: olson $ * $Revision: 1.3 $ * $State: Exp $ * $Locker: $ * $Source: /home/reed/Pablo/Parser/cplus/Gen/Example/RCS/parser.y,v $ * $Log: parser.y,v $ * Revision 1.3 90/08/21 11:20:49 olson * Added the legalese * * Revision 1.2 90/08/17 13:52:36 olson * fixed headers * * Revision 1.1 90/08/17 13:50:59 olson * First revision * */ #include <stdio.h> %} %token PLUS MINUS INTEGER SEMICOLON %% input: input_line | input_line input ; input_line: expr SEMICOLON { printf("got value %d\n", $1); } ; expr: plus_expr { process_plus($1); } | minus_expr { process_minus($1); } | num ; plus_expr: num PLUS expr { $$ = $1 + $3; } ; minus_expr: num MINUS expr { $$ = $1 - $3; } ; num: INTEGER { $$ = $1; } ; %% process_plus(int v) { printf("plus got %d\n", v); } process_minus(int v) { printf("minus got %d\n", v); } yywrap() { } yyerror(char *s) { fprintf(stderr, "Error %s\n", s); } SHAR_EOF if test 1199 -ne "`wc -c < 'parser.y'`" then echo shar: error transmitting "'parser.y'" '(should have been 1199 characters)' fi fi # end of overwriting check echo shar: extracting "'lex.l'" '(764 characters)' if test -f 'lex.l' then echo shar: will not over-write existing file "'lex.l'" else cat << \SHAR_EOF > 'lex.l' %{ /* * Robert Olson, University of Illinois at Urbana * rolson@uiuc.edu * Copyright (c) 1990 by Robert Olson * All rights reserved * This program may not be sold, but may be distributed * provided this header is included. * * $Author: olson $ * $Revision: 1.3 $ * $State: Exp $ * $Locker: $ * $Source: /home/reed/Pablo/Parser/cplus/Gen/Example/RCS/lex.l,v $ * $Log: lex.l,v $ * Revision 1.3 90/08/21 11:20:46 olson * Added the legalese * * Revision 1.2 90/08/17 13:52:29 olson * fixed headers * * Revision 1.1 90/08/17 13:50:58 olson * First revision * */ #include "parser.h" extern int yylval; %} %% ";" { return SEMICOLON; } "+" { return PLUS; } "-" { return MINUS; } [0-9]+ { yylval = atoi(yytext); return INTEGER; } SHAR_EOF if test 764 -ne "`wc -c < 'lex.l'`" then echo shar: error transmitting "'lex.l'" '(should have been 764 characters)' fi fi # end of overwriting check echo shar: extracting "'Makefile'" '(1756 characters)' if test -f 'Makefile' then echo shar: will not over-write existing file "'Makefile'" else cat << \SHAR_EOF > 'Makefile' # # Robert Olson, University of Illinois at Urbana # rolson@uiuc.edu # Copyright (c) 1990 by Robert Olson # All rights reserved # This program may not be sold, but may be distributed # provided this header is included. # # # $Author: olson $ # $Revision: 1.4 $ # $State: Exp $ # $Locker: $ # $Source: /home/reed/Pablo/Parser/cplus/Gen/Example/RCS/Makefile,v $ # $Log: Makefile,v $ # Revision 1.4 90/08/21 11:34:58 olson # fixed GENERATE defn # # Revision 1.3 90/08/21 11:20:48 olson # Added the legalese # # Revision 1.2 90/08/17 13:54:53 olson # fixed make clean # # Revision 1.1 90/08/17 13:50:59 olson # First revision # # GENERATE = ../src/generate-class -d ../src PROTOIZE = protoize C++ = g++ C++FLAGS = YACC = yacc YFLAGS = -d LEX = lex LFLAGS = PARSER = Parser LEXER = Lexer all: example $(LEXER).cc $(LEXER).h: lex.prot $(GENERATE) -m lex -i $(PARSER) -c $(LEXER) lex.prot $(PARSER).cc $(PARSER).h: parser.prot $(GENERATE) -m yacc -i $(LEXER) -c $(PARSER) parser.prot EXAMPLE_OBJ = main.o Parser.o Lexer.o .PRECIOUS: lex.c parser.c example: $(EXAMPLE_OBJ) $(C++) $(C++FLAGS) -o $@ $(EXAMPLE_OBJ) $(LIBS) #PARSER_H = parser.h main.o: Parser.h Lexer.h $(PARSER_H) Lexer.o: Lexer.h Parser.h $(PARSER_H) Parser.o: Parser.h Lexer.h $(PARSER_H) lex.prot: $(PARSER_H) parser.h: parser.c clean: rm -f a.out core *.o lex.c parser.c *.prot rm -f example y.tab.c y.tab.h *~ \#* *.save rm -f Lexer.cc Lexer.h Lexer-private.h rm -f Parser.cc Parser.h Parser-private.h .SUFFIXES: .prot .cc .cc.o: $(C++) $(C++FLAGS) -c $< .y.c: $(YACC) $(YFLAGS) $< -mv y.tab.h $*.h -sed '/#[ ]*line.*\.y/d' y.tab.c > $*.c .l.c: $(LEX) $(LFLAGS) -t $< > $@ .c.prot: $(PROTOIZE) -c "$(CFLAGS)" $< cp $< $@ SHAR_EOF if test 1756 -ne "`wc -c < 'Makefile'`" then echo shar: error transmitting "'Makefile'" '(should have been 1756 characters)' fi fi # end of overwriting check echo shar: extracting "'main.cc'" '(852 characters)' if test -f 'main.cc' then echo shar: will not over-write existing file "'main.cc'" else cat << \SHAR_EOF > 'main.cc' /* * Robert Olson, University of Illinois at Urbana * rolson@uiuc.edu * Copyright (c) 1990 by Robert Olson * All rights reserved * This program may not be sold, but may be distributed * provided this header is included. * * $Author: olson $ * $Revision: 1.3 $ * $State: Exp $ * $Locker: $ * $Source: /home/reed/Pablo/Parser/cplus/Gen/Example/RCS/main.cc,v $ * $Log: main.cc,v $ * Revision 1.3 90/08/21 11:20:47 olson * Added the legalese * * Revision 1.2 90/08/17 13:52:39 olson * fixed headers * * Revision 1.1 90/08/17 13:50:59 olson * First revision * */ #include "Parser.h" #include "Lexer.h" Parser *parser; Lexer *lexer; main() { int rc; lexer = new Lexer; parser = new Parser(lexer); lexer->setParser(parser); rc = parser->yyparse(); return rc; } int yywrap() { return parser->yywrap(); } SHAR_EOF if test 852 -ne "`wc -c < 'main.cc'`" then echo shar: error transmitting "'main.cc'" '(should have been 852 characters)' fi fi # end of overwriting check echo shar: done with directory "'Example'" cd .. echo shar: extracting "'README'" '(1673 characters)' if test -f 'README' then echo shar: will not over-write existing file "'README'" else cat << \SHAR_EOF > 'README' This is generate-class, a perl script for generating C++ classes from lex and yacc output . It requires the perl program (written by Larry Wall, and available from tut.cis.ohio-state.edu and your local comp.sources.unix archives) and protoize. Protoize is a utility distributed with the Gnu C compiler which converts K&R type declarations to ANSI declarations. It also uses lex. To install generate-class, go to the src subdirectory and do a make. This builds the lex tokenizer that the script uses (I originally used a perl tokenizer, but went to lex because it was more powerful and easier to modify). There is an example lex/yacc parser in Example that you can look at. The arguments to generate-class are: -c name Sets the class name of the generated code -m mode Sets the mode for this run. Mode is either "lex" or "yacc" -- the generated classes have some dependencies on how they are going to be used, namely the lexer wants to have a pointer to the parser and vice versa. -i name The class name of the related object (eg the Lexer class will want to be given the name of the parser object). -d pathname The directory in which the auxilliary perl scripts reside. The script currently generates a lot of output -- this distribution is just a snapshot of my code. This is currently the only documentation as well, mainly since I just wrote this for my own use and thought others might find it useful. At some point I plan to write up a little more about it, but I don't have the time now and want to get it out since there is quite a bit of interest. Let me know what you think of the scripts. --bob 8/21/90 rolson@uiuc.edu SHAR_EOF if test 1673 -ne "`wc -c < 'README'`" then echo shar: error transmitting "'README'" '(should have been 1673 characters)' fi fi # end of overwriting check echo shar: extracting "'AboutPerl'" '(1976 characters)' if test -f 'AboutPerl' then echo shar: will not over-write existing file "'AboutPerl'" else cat << \SHAR_EOF > 'AboutPerl' I have been asked what perl is. Larry Wall says it better than I in the manpage for perl: Perl is an interpreted language optimized for scanning arbi- trary text files, extracting information from those text files, and printing reports based on that information. It's also a good language for many system management tasks. The language is intended to be practical (easy to use, effi- cient, complete) rather than beautiful (tiny, elegant, minimal). It combines (in the author's opinion, anyway) some of the best features of C, sed, awk, and sh, so people familiar with those languages should have little difficulty with it. (Language historians will also note some vestiges of csh, Pascal, and even BASIC-PLUS.) Expression syntax corresponds quite closely to C expression syntax. Unlike most Unix utilities, perl does not arbitrarily limit the size of your data--if you've got the memory, perl can slurp in your whole file as a single string. Recursion is of unlimited depth. And the hash tables used by associative arrays grow as necessary to prevent degraded performance. Perl uses sophisticated pattern matching techniques to scan large amounts of data very quickly. Although optimized for scanning text, perl can also deal with binary data, and can make dbm files look like associative arrays (where dbm is available). Setuid perl scripts are safer than C programs through a dataflow tracing mechanism which prevents many stupid security holes. If you have a problem that would ordinarily use sed or awk or sh, but it exceeds their capa- bilities or must run a little faster, and you don't want to write the silly thing in C, then perl may be for you. There are also translators to turn your sed and awk scripts into perl scripts. OK, enough hype. SHAR_EOF if test 1976 -ne "`wc -c < 'AboutPerl'`" then echo shar: error transmitting "'AboutPerl'" '(should have been 1976 characters)' fi fi # end of overwriting check echo shar: done with directory "'Gen'" cd .. # End of shell archive exit 0 -- Bob Olson University of Illinois at Urbana/Champaign Internet: rolson@uiuc.edu UUCP: {uunet|convex|pur-ee}!uiucdcs!olson "You can't win a game of chess with an action figure!" AMA #522687 DoD #28 <<Dee's address: dee%fazer.uucp@harp.cs.uiuc.edu>>