[comp.lang.perl] yacc stack overflow

emv@ox.com (Ed Vielmetti) (03/08/91)

what does this error message mean?

<emv@crane> smoke
yacc stack overflow in file /u1/emv/bin/smoke at line 238, next token "$verbose"
Execution of /u1/emv/bin/smoke aborted due to compilation errors.

	I get so tired of utilities with arbitrary, undocumented,
	compiled-in limits.  Don't you?

it's a really weird one; i have a giant if {} elsif {} elsif {} thing
going, and adding one statement somewhere will cause the above error
to appear.

perl 3, patchlevel 37.

-- 
 Msen	Edward Vielmetti
/|---	moderator, comp.archives
	emv@msen.com

louie@sayshell.umd.edu (Louis A. Mamakos) (03/08/91)

I just cranked up the value of YYMAXDEPTH in perl.c (from perl.y), and
'fixed' this problem.  It at least let me use the find2perl script
posted earlier.  To really do this fix right, the Makefile needs to
pass '-DYYMAXDEPTH=300' to cc when it compiles perl.c.

louie

mostek@motcid.UUCP (Frank B. Mostek) (03/08/91)

emv@ox.com (Ed Vielmetti) writes:

>what does this error message mean?

><emv@crane> smoke
>yacc stack overflow in file /u1/emv/bin/smoke at line 238, next token "$verbose"
>Execution of /u1/emv/bin/smoke aborted due to compilation errors.

>stuff...

Change the value of YYMAXDEPTH.  I think the default is 150.  I can't
remember what I changed mine to.  Or, you may want to use the
__RUNTIME_YYMAXDEPTH option, which allocates stacks at runtime.  Or,
you can use YYINCREMENT which uses malloc(). Or, forget about the tools
and write your own parser from scratch.

Have fun :)
-- 
************************************************************
Frank Mostek			uunet!motcid!amethyst!mostek
Software Consultant 		(708)632-7191
************************************************************

allbery@NCoast.ORG (Brandon S. Allbery KB8JRR) (03/13/91)

As quoted from <EMV.91Mar8010709@poe.aa.ox.com> by emv@ox.com (Ed Vielmetti):
+---------------
| <emv@crane> smoke
| yacc stack overflow in file /u1/emv/bin/smoke at line 238, next token "$verbose"
| Execution of /u1/emv/bin/smoke aborted due to compilation errors.
| 
| 	I get so tired of utilities with arbitrary, undocumented,
| 	compiled-in limits.  Don't you?
| 
| it's a really weird one; i have a giant if {} elsif {} elsif {} thing
| going, and adding one statement somewhere will cause the above error
| to appear.
+---------------

Yacc is one of those utilities with an arbitrary, compiled-in limit:  the
generated parser can handle only so much before it does this.  I hit it with
PL41 and terminfo.pl (yes, it will eventually be posted.  If things will let
up at the office for long enough, that is.  grrr...).

Question, which should be replied via MAIL:  if this limit is in
/usr/lib/yaccpar and I produce a modified version that dynamically allocates
yacc stack space, can I post the diffs without getting AT&T on my case?

++Brandon
-- 
Me: Brandon S. Allbery			    Ham: KB8JRR on 40m, 10m when time
Internet: allbery@NCoast.ORG		      permits; also 2m, 220, 440, 1200
America OnLine: KB8JRR // Delphi: ALLBERY   AMPR: kb8jrr.AmPR.ORG [44.70.4.88]
uunet!usenet.ins.cwru.edu!ncoast!allbery          KB8JRR @ WA8BXN.OH

oz@yunexus.yorku.ca (Ozan Yigit) (03/14/91)

In article <1991Mar13.012932.1710@NCoast.ORG>
allbery@ncoast.ORG (Brandon S. Allbery KB8JRR) writes:

>Yacc is one of those utilities with an arbitrary, compiled-in limit ...

Does that limit exist on PD byacc? [berkeley yacc] If not, maybe you
would be better off using that one.

oz

rbj@uunet.UU.NET (Root Boy Jim) (03/14/91)

In article <1991Mar13.012932.1710@NCoast.ORG> allbery@ncoast.ORG (Brandon S. Allbery KB8JRR) writes:
?As quoted from <EMV.91Mar8010709@poe.aa.ox.com> by emv@ox.com (Ed Vielmetti):
?
?Question, which should be replied via MAIL:  if this limit is in
?/usr/lib/yaccpar and I produce a modified version that dynamically allocates
?yacc stack space, can I post the diffs without getting AT&T on my case?

Of course. I'm surprised that someone who's been around as long as you
have would even ask. It's done all the time.

Of course, you could also use bison.

-- 
		[rbj@uunet 1] stty sane
		unknown mode: sane

allbery@NCoast.ORG (Brandon S. Allbery KB8JRR) (03/14/91)

As quoted from <1991Mar13.012932.1710@NCoast.ORG> by allbery@NCoast.ORG (Brandon S. Allbery KB8JRR):
+---------------
| /usr/lib/yaccpar and I produce a modified version that dynamically allocates
| yacc stack space, can I post the diffs without getting AT&T on my case?
+---------------

Actually, the answer *may* be "yes" --- AT&T has already permitted generated
y.tab.c's to be distributed.

In any case, I got two main suggestions:  byacc (which is freely hackable) and
bison (which automatically grows the stack).  In my case, bison is out:  I
had the same problem with some home-grown parsers... which must be linked with
proprietary libraries.  And I've explained to a brick wall (read: rms) why
that can't change.

I *have* byacc, so that's an idea.  But many people don't have it, and do have
Perl, and therefore have a problem.  Suggestions?

++Brandon
-- 
Me: Brandon S. Allbery			    Ham: KB8JRR on 40m, 10m when time
Internet: allbery@NCoast.ORG		      permits; also 2m, 220, 440, 1200
America OnLine: KB8JRR // Delphi: ALLBERY   AMPR: kb8jrr.AmPR.ORG [44.70.4.88]
uunet!usenet.ins.cwru.edu!ncoast!allbery          KB8JRR @ WA8BXN.OH

lwall@jpl-devvax.jpl.nasa.gov (Larry Wall) (03/14/91)

In article <1991Mar13.234823.12036@NCoast.ORG> allbery@ncoast.ORG (Brandon S. Allbery KB8JRR) writes:
: As quoted from <1991Mar13.012932.1710@NCoast.ORG> by allbery@NCoast.ORG (Brandon S. Allbery KB8JRR):
: +---------------
: | /usr/lib/yaccpar and I produce a modified version that dynamically allocates
: | yacc stack space, can I post the diffs without getting AT&T on my case?
: +---------------
: 
: Actually, the answer *may* be "yes" --- AT&T has already permitted generated
: y.tab.c's to be distributed.
: 
: In any case, I got two main suggestions:  byacc (which is freely hackable) and
: bison (which automatically grows the stack).  In my case, bison is out:  I
: had the same problem with some home-grown parsers... which must be linked with
: proprietary libraries.  And I've explained to a brick wall (read: rms) why
: that can't change.
: 
: I *have* byacc, so that's an idea.  But many people don't have it, and do have
: Perl, and therefore have a problem.  Suggestions?

Find the line in your Makefile that says "mv y.tab.c ..." and change
the mv to perly.fixer.  Install the following script as perly.fixer.
See if it helps.  :-)

I note that Sun's yaccpar already does dynamic stack allocation.  This
script is built to fix 4.3BSD's yaccpar--lemme know if yours differs.

I don't like arbitrary limits either...

Larry

#!/bin/sh
: make a subdirectory, cd to it, and run this through sh.
echo 'If this kit is complete, "End of kit" will echo at the end'
echo Extracting perly.fixer
sed >perly.fixer <<'!STUFFY!FUNK!' -e 's/X//'
X#!/bin/sh
X
Xinput=$1
Xoutput=$2
Xtmp=/tmp/f$$
X
Xegrep 'YYSTYPE[ 	]*yyv\[ *YYMAXDEPTH *\];
Xshort[ 	]*yys\[ *YYMAXDEPTH *\] *;
Xyyps *= *&yys\[ *-1 *\];
Xyypv *= *&yyv\[ *-1 *\];
Xif *\( *\+\+yyps *> *&yys\[ *YYMAXDEPTH *\] *\)' $input >$tmp
Xset `wc -l $tmp`
X
Xcase "$1" in
X5) echo "Patching perly.c to allow dynamic yacc stack allocation";;
X*) mv $input $output; rm -f $tmp; exit;;
Xesac
X
Xcat >$tmp <<'END'
X/YYSTYPE[ 	]*yyv\[ *YYMAXDEPTH *\];/c\
Xint yymaxdepth = YYMAXDEPTH;\
XYYSTYPE *yyv; /* where the values are stored */\
Xshort *yys;\
Xshort *maxyyps;
X
X/short[ 	]*yys\[ *YYMAXDEPTH *\] *;/d
X
X/yyps *= *&yys\[ *-1 *\];/d
X
X/yypv *= *&yyv\[ *-1 *\];/c\
X\	if (!yyv) {\
X\	    yyv = (YYSTYPE*) malloc(yymaxdepth * sizeof(YYSTYPE));\
X\	    yys = (short*) malloc(yymaxdepth * sizeof(short));\
X\	    maxyyps = &yys[yymaxdepth];\
X\	}\
X\	yyps = &yys[-1];\
X\	yypv = &yyv[-1];
X
X
X/if *( *\+\+yyps *> *&yys\[ *YYMAXDEPTH *\] *)/c\
X\		if( ++yyps >= maxyyps ) {\
X\		    int tv = yypv - yyv;\
X\		    int ts = yyps - yys;\
X\
X\		    yymaxdepth *= 2;\
X\		    yyv = (YYSTYPE*)realloc((char*)yyv,\
X\		      yymaxdepth*sizeof(YYSTYPE));\
X\		    yys = (short*)realloc((char*)yys,\
X\		      yymaxdepth*sizeof(short));\
X\		    yyps = yys + ts;\
X\		    yypv = yyv + tv;\
X\		    maxyyps = &yys[yymaxdepth];\
X\		}
X
X/yacc stack overflow.*}/d
X/yacc stack overflow/,/}/d
XEND
X
Xsed -f $tmp <$input >$output
Xrm -rf $tmp $input
!STUFFY!FUNK!
echo ""
echo "End of kit"
: I do not append .signature, but someone might mail this.
exit

ronald@robobar.co.uk (Ronald S H Khoo) (03/14/91)

oz@yunexus.yorku.ca (Ozan Yigit) writes:

> allbery@ncoast.ORG (Brandon S. Allbery KB8JRR) writes:
> >Yacc is one of those utilities with an arbitrary, compiled-in limit ...
> 
> Does that limit exist on PD byacc? [berkeley yacc] If not, maybe you
> would be better off using that one.

I quote from /usr/local/src/perl/perl.c:

    static char yysccsid[] = "@(#)yaccpar	1.8 (Berkeley) 01/20/90";

    #ifdef YYSTACKSIZE
    #ifndef YYMAXDEPTH
    #define YYMAXDEPTH YYSTACKSIZE
    #endif
    #else
    #ifdef YYMAXDEPTH
    #define YYSTACKSIZE YYMAXDEPTH
    #else
    #define YYSTACKSIZE 500
    #define YYMAXDEPTH 500
    #endif
    #endif
    [...]
    if (yyssp >= yyss + yystacksize - 1)
    {
        goto yyoverflow;
    }
    [...]
yyoverflow:
    yyerror("yacc stack overflow");

So it looks like that's the case too.  Ho hum.

-- 
Ronald Khoo <ronald@robobar.co.uk> +44 81 991 1142 (O) +44 71 229 7741 (H)

lwall@jpl-devvax.jpl.nasa.gov (Larry Wall) (03/17/91)

In article <1991Mar14.080812.19719@robobar.co.uk> ronald@robobar.co.uk (Ronald S H Khoo) writes:
: oz@yunexus.yorku.ca (Ozan Yigit) writes:
: 
: > allbery@ncoast.ORG (Brandon S. Allbery KB8JRR) writes:
: > >Yacc is one of those utilities with an arbitrary, compiled-in limit ...
: > 
: > Does that limit exist on PD byacc? [berkeley yacc] If not, maybe you
: > would be better off using that one.
: 
: I quote from /usr/local/src/perl/perl.c:
: 
...
:     if (yyssp >= yyss + yystacksize - 1)
:     {
:         goto yyoverflow;
:     }
:     [...]
: yyoverflow:
:     yyerror("yacc stack overflow");
: 
: So it looks like that's the case too.  Ho hum.

If someone wants to modify the perly.fixer script I just sent out to fix
this yaccpar too, I'll be glad to include it.

Larry

allbery@NCoast.ORG (Brandon S. Allbery KB8JRR) (03/20/91)

As quoted from <125497@uunet.UU.NET> by rbj@uunet.UU.NET (Root Boy Jim):
+---------------
| Of course. I'm surprised that someone who's been around as long as you
| have would even ask. It's done all the time.
+---------------

"Done all the time" != "okay to do".  Look at how long the yacc and lex
sources were on the DECUS tapes....

And AT&T is being extraordinarily obtuse of late.  (Cf. the nonsense about the
"backing store" patent.)

++Brandon
-- 
Me: Brandon S. Allbery			    Ham: KB8JRR on 2m, 220, 440, 1200
Internet: allbery@NCoast.ORG		(QRT on HF until local problems fixed)
America OnLine: KB8JRR // Delphi: ALLBERY   AMPR: kb8jrr.AmPR.ORG [44.70.4.88]
uunet!usenet.ins.cwru.edu!ncoast!allbery          KB8JRR @ WA8BXN.OH

meissner@osf.org (Michael Meissner) (03/21/91)

In article <1991Mar20.005444.27907@NCoast.ORG> allbery@NCoast.ORG (Brandon S. Allbery KB8JRR) writes:

| As quoted from <125497@uunet.UU.NET> by rbj@uunet.UU.NET (Root Boy Jim):
| +---------------
| | Of course. I'm surprised that someone who's been around as long as you
| | have would even ask. It's done all the time.
| +---------------
| 
| "Done all the time" != "okay to do".  Look at how long the yacc and lex
| sources were on the DECUS tapes....
| 
| And AT&T is being extraordinarily obtuse of late.  (Cf. the nonsense about the
| "backing store" patent.)

However, they did explictly state in writting that as of System V.3,
the output of yacc and lex did not require a UNIX license to run the
resulting program....
--
Michael Meissner	email: meissner@osf.org		phone: 617-621-8861
Open Software Foundation, 11 Cambridge Center, Cambridge, MA, 02142

Considering the flames and intolerance, shouldn't USENET be spelled ABUSENET?

worley@compass.com (Dale Worley) (03/21/91)

   X-Name: Brandon S. Allbery KB8JRR

   And AT&T is being extraordinarily obtuse of late.  (Cf. the
   nonsense about the "backing store" patent.)

That's hardly obtuse -- it's just that AT&T has discovered that
there's money to be made in software.  Perhaps "backing store" is not
patentable -- and perhaps it is.  The cost of sending out a few
threatening letters is far less than the profits to be made if the
patent is upheld.

Dale

Dale Worley		Compass, Inc.			worley@compass.com
--
A friend told me about how the software project he was working on got
canceled: "The business was estimated at $100 million within 4 years,
but this being an American company, that wasn't good enough -- a year
is as far as time goes."

rbj@uunet.UU.NET (Root Boy Jim) (03/21/91)

In article <1991Mar20.005444.27907@NCoast.ORG> allbery@ncoast.ORG (Brandon S. Allbery KB8JRR) writes:
>As quoted from <125497@uunet.UU.NET> by rbj@uunet.UU.NET (Root Boy Jim):
>+---------------
>| Of course. I'm surprised that someone who's been around as long as you
>| have would even ask. It's done all the time.
>+---------------
>
>"Done all the time" != "okay to do".  Look at how long the yacc and lex
>sources were on the DECUS tapes....

Actually "done repeatedly in broad daylight" =~ "okay to do", at least
in the civil world. It can be argued, and has been successfully, that
failure to exercise one's rights is tantamount to releasing them.

I know nothing of the DECUS case.

>And AT&T is being extraordinarily obtuse of late.  (Cf. the nonsense about the
>"backing store" patent.)

There is no money in hassling people who already have paid for source
from posting diffs which can only be used by other people who have
also paid for source. Whether the "trade secret" has been compromised
in this situation is legally nebulous. Too delicate to risk an
adverse legal decision on. 

They could actually make money from the backing store thing.
Let us pray that they won't.

However, there is no reason to confuse the two issues.

>++Brandon
>-- 
>Me: Brandon S. Allbery			    Ham: KB8JRR on 2m, 220, 440, 1200
>Internet: allbery@NCoast.ORG		(QRT on HF until local problems fixed)
>America OnLine: KB8JRR // Delphi: ALLBERY   AMPR: kb8jrr.AmPR.ORG [44.70.4.88]
>uunet!usenet.ins.cwru.edu!ncoast!allbery          KB8JRR @ WA8BXN.OH
-- 
		[rbj@uunet 1] stty sane
		unknown mode: sane