beshers@cs.columbia.edu (Clifford Beshers) (03/22/91)
Subject: perl.4.0.beta on HPUX 7.05 -- turn of -O on eval.c and ttoke.c
--text follows this line--
I thought this readership might appreciate the warning about the
following problem. The files compile without the optimizer, and
the result appears to work without troubles.
The following files gave the optimizer trouble when
compiling perl.4.0.beta on HPUX 7.05 running on the new 68040
chip.
The machine in question has 32 MB of memory, so running out of
memory is not a reasonable complaint.
I have appended my config.sh.
****************************************************************
open> what `whence cc`
/bin/cc:
$Revision: 66.6.1.1 $
open>
open> uname -a
HP-UX open 7.05 B 9000/380 open
open> make
...
cc -c -O eval.c
Global optimizer found error in "eval": Out of Memory
...
cc -c -DTAINT -O teval.c
Global optimizer found error in "eval": Out of Memory
...
cc -c -DTAINT -O ttoke.c
Global optimizer found error in "yylex": out of space - unable to allocate memor
y for internal use
*** Error code 1
****************************************************************
config.sh
****************************************************************
#!/bin/sh
# config.sh
# This file was produced by running the Configure script.
kit_has_binaries='false'
d_eunice='undef'
define='define'
eunicefix=':'
loclist='
cat
cp
echo
expr
grep
mkdir
mv
rm
sed
sort
tr
uniq
'
expr='/bin/expr'
sed='/bin/sed'
echo='/bin/echo'
cat='/bin/cat'
rm='/bin/rm'
mv='/bin/mv'
cp='/bin/cp'
tail=''
tr='/usr/bin/tr'
mkdir='/bin/mkdir'
sort='/bin/sort'
uniq='/usr/bin/uniq'
grep='/usr/local/bin/grep'
trylist='
Mcc
bison
cpp
csh
egrep
nroff
test
yacc
'
test='test'
inews=''
egrep='/usr/local/bin/egrep'
more=''
pg=''
Mcc='Mcc'
vi=''
mailx=''
mail=''
cpp='/lib/cpp'
perl=''
emacs=''
ls=''
rmail=''
sendmail=''
shar=''
smail=''
tbl=''
troff=''
nroff='/usr/bin/nroff'
uname=''
uuname=''
line=''
chgrp=''
chmod=''
lint=''
sleep=''
pr=''
tar=''
ln=''
lpr=''
lp=''
touch=''
make=''
date=''
csh='/bin/csh'
bash=''
ksh=''
lex=''
flex=''
bison='/usr/local/gnu/bin/bison'
Log='$Log'
Header='$Header'
Id='$Id'
alignbytes='2'
bin='.'
byteorder='4321'
contains='grep'
cppstdin='cc -E'
cppminus='-'
d_bcmp='define'
d_bcopy='define'
d_bzero='define'
d_castneg='define'
castflags='0'
d_charsprf='undef'
d_chsize='undef'
d_crypt='define'
cryptlib=''
d_csh='define'
d_dosuid='undef'
d_dup2='define'
d_fchmod='define'
d_fchown='define'
d_fcntl='define'
d_flexfnam='define'
d_flock='undef'
d_getgrps='define'
d_gethent='define'
d_getpgrp='define'
d_getpgrp2='define'
d_getprior='undef'
d_htonl='undef'
d_index='undef'
d_killpg='define'
d_lstat='define'
d_memcmp='define'
d_memcpy='define'
d_mkdir='define'
d_msg='define'
d_msgctl='define'
d_msgget='define'
d_msgrcv='define'
d_msgsnd='define'
d_ndbm='define'
d_odbm='define'
d_open3='define'
d_readdir='define'
d_rename='define'
d_rmdir='define'
d_select='define'
d_sem='define'
d_semctl='define'
d_semget='define'
d_semop='define'
d_setegid='undef'
d_seteuid='undef'
d_setpgrp='define'
d_setpgrp2='define'
d_setprior='undef'
d_setregid='undef'
d_setresgid='define'
d_setreuid='undef'
d_setresuid='define'
d_setrgid='undef'
d_setruid='undef'
d_shm='define'
d_shmat='define'
d_shmctl='define'
d_shmdt='define'
d_shmget='define'
d_socket='define'
d_sockpair='undef'
d_oldsock='undef'
socketlib=''
d_statblks='define'
d_stdstdio='define'
d_strctcpy='define'
d_strerror='define'
d_symlink='define'
d_syscall='define'
d_truncate='define'
d_vfork='define'
d_voidsig='define'
d_tosignal=''
d_volatile='define'
d_vprintf='define'
d_charvspr='undef'
d_wait4='undef'
d_waitpid='define'
gidtype='gid_t'
i_fcntl='undef'
i_grp='define'
i_niin='define'
i_sysin='undef'
i_pwd='define'
d_pwquota='undef'
d_pwage='undef'
d_pwchange='undef'
d_pwclass='undef'
d_pwexpire='undef'
d_pwcomment='undef'
i_sys_file='define'
i_sysioctl='define'
i_time='undef'
i_sys_time='define'
i_sys_select='undef'
d_systimekernel='undef'
i_utime='define'
i_varargs='define'
i_vfork='undef'
inclPath=''
intsize='4'
libc='/lib/libc.a'
nm_opts=''
libndir=''
i_my_dir='undef'
i_ndir='undef'
i_sys_ndir='undef'
i_dirent='define'
i_sys_dir='undef'
d_dirnamlen='undef'
ndirc=''
ndiro=''
mallocsrc='malloc.c'
mallocobj='malloc.o'
usemymalloc='y'
mansrc=''
manext=''
models='none'
split=''
small=''
medium=''
large=''
huge=''
optimize='-O'
ccflags=''
cppflags=''
ldflags=''
cc='cc'
libs='-lnet -lndir -lndbm -lm -lBSD'
n=''
c='\c'
package='perl'
randbits='15'
scriptdir='.'
sig_name='ZERO HUP INT QUIT ILL TRAP IOT EMT FPE KILL BUS SEGV SYS PIPE ALRM TERM USR1 USR2 CLD PWR VTALRM PROF IO WINDOW STOP TSTP CONT TTIN TTOU'
spitshell='cat'
shsharp='true'
sharpbang='#!'
startsh='#!/bin/sh'
stdchar='unsigned char'
uidtype='uid_t'
void=''
voidhave='1'
voidwant='1'
w_localtim='1'
w_s_timevl='1'
w_s_tm='1'
yacc='/usr/bin/yacc'
lib=''
privlib='.'
CONFIG=true
--
-----------------------------------------------
Clifford Beshers
450 Computer Science Department
Columbia University
New York, NY 10027
beshers@cs.columbia.edubeshers@cs.columbia.edu (Clifford Beshers) (03/22/91)
In article <11307@dog.ee.lbl.gov> milburn@me10.lbl.gov (John Milburn) writes: From: milburn@me10.lbl.gov (John Milburn) Newsgroups: comp.sys.hp Date: 22 Mar 91 03:37:15 GMT In article <BESHERS.91Mar21184234@division.cs.columbia.edu> beshers@cs.columbia.edu (Clifford Beshers) writes: >Subject: perl.4.0.beta on HPUX 7.05 -- turn of -O on eval.c and ttoke.c Gee, the same thing happened with perl rev 3. Give yourself enough swap and enough user memory, an a LOT of time, and you really can optimize eval.c and toke.c. At least you could with 3.0. -- John Milburn milburn@me10.lbl.gov (415) 486-6969 "Koko, will there be gnomes and dwarves for Lebee to wrestle with?" "Yes Mishu, and also trolls and mutants we may spar with!" -SNL This isn't enough? Physical memory: 32768 Maximum user memory: 28972 Free memory: 23692 User memory utilization: 18% Swap space configured: 43008 Enabled via swapon(1m): 43008 Currently free swap space: 34120 Swap space utilization: 20% -- ----------------------------------------------- Clifford Beshers 450 Computer Science Department Columbia University New York, NY 10027 beshers@cs.columbia.edu
milburn@me10.lbl.gov (John Milburn) (03/22/91)
In article <BESHERS.91Mar21184234@division.cs.columbia.edu> beshers@cs.columbia.edu (Clifford Beshers) writes: >Subject: perl.4.0.beta on HPUX 7.05 -- turn of -O on eval.c and ttoke.c Gee, the same thing happened with perl rev 3. Give yourself enough swap and enough user memory, an a LOT of time, and you really can optimize eval.c and toke.c. At least you could with 3.0. -jem -- John Milburn milburn@me10.lbl.gov (415) 486-6969 "Koko, will there be gnomes and dwarves for Lebee to wrestle with?" "Yes Mishu, and also trolls and mutants we may spar with!" -SNL
milburn@me10.lbl.gov (John Milburn) (03/23/91)
beshers@cs.columbia.edu (Clifford Beshers) writes: >In article <11307@dog.ee.lbl.gov> milburn@me10.lbl.gov (John Milburn) writes: > Gee, the same thing happened with perl rev 3. Give yourself enough > swap and enough user memory, an a LOT of time, and you really can > optimize eval.c and toke.c. At least you could with 3.0. >This isn't enough? > Physical memory: 32768 Maximum user memory: 28972 > Free memory: 23692 User memory utilization: 18% > Swap space configured: 43008 Enabled via swapon(1m): 43008 > Currently free swap space: 34120 Swap space utilization: 20% No, its not. I needed about 80 Megs of swap to optimize eval.c. I only did it once, then switched to +O1 for eval.c and toke.c. I guess the -O level of optimization in hp-ux is over ambitious. -jem -- John Milburn milburn@me10.lbl.gov (415) 486-6969 "I am successful because I am the only person in my city who is not heavily addicted to powerful narcotics." -Cerebus
piet@cs.ruu.nl (Piet van Oostrum) (03/25/91)
>>>>> milburn@me10.lbl.gov (John Milburn) (JM) writes:
JM> No, its not. I needed about 80 Megs of swap to optimize eval.c. I only
JM> did it once, then switched to +O1 for eval.c and toke.c. I guess the
JM> -O level of optimization in hp-ux is over ambitious.
Besides that, I believe it's broken. A few weeks ago I spent a whole day
trying to find a bug in a program. At the end of the day I found that the
makefile had -O in it. After changing that to +O1 the program run flowlessly.
I do never use -O. I just don't trust it.
--
Piet* van Oostrum, Dept of Computer Science, Utrecht University,
Padualaan 14, P.O. Box 80.089, 3508 TB Utrecht, The Netherlands.
Telephone: +31 30 531806 Uucp: uunet!mcsun!ruuinf!piet
Telefax: +31 30 513791 Internet: piet@cs.ruu.nl (*`Pete')mike@hpfcso.FC.HP.COM (Mike McNelly) (03/27/91)
> JM> No, its not. I needed about 80 Megs of swap to optimize eval.c. I only > JM> did it once, then switched to +O1 for eval.c and toke.c. I guess the > JM> -O level of optimization in hp-ux is over ambitious. > > Besides that, I believe it's broken. A few weeks ago I spent a whole day > trying to find a bug in a program. At the end of the day I found that the > makefile had -O in it. After changing that to +O1 the program run flowlessly. > I do never use -O. I just don't trust it. > -- > Piet* van Oostrum, Dept of Computer Science, Utrecht University, > Padualaan 14, P.O. Box 80.089, 3508 TB Utrecht, The Netherlands. > Telephone: +31 30 531806 Uucp: uunet!mcsun!ruuinf!piet > Telefax: +31 30 513791 Internet: piet@cs.ruu.nl (*`Pete') It's quite possible that the optimizer had a bug in it. It's also quite possible that the source program had a bug that was unmasked by optimization (this happens a lot). Until the problem is tracked down no one will ever know for sure. Mike McNelly mike@fc.hp.com
khb@chiba.Eng.Sun.COM (Keith Bierman fpgroup) (03/27/91)
....optimizer is unsafe.... I've commented on this before. In general, in my experience application codes are at fault far more often than optimizer errors. This same topic has come up in comp.benchmarks. Since Robert posted it publically, I see no reason not to repost it here. Sometimes it is no ones "bug" it is just an interesting interaction ... typically machines with wide accumulators (ala the 8087 and 68881) user code and register allocators can blend to provide really hard to track down bugs .... which in some sense are not bugs. The allocator did a good job. The user had no way to know (or control) how big the temps are, etc. I usually argue against such wide registers for just such reasons.... From: metzger@convex.com (Robert Metzger) Newsgroups: comp.benchmarks Subject: Re: benchmarks and interprocedural optimization Date: 23 Mar 91 18:21:54 GMT Organization: Convex Computer Corporation, Richardson, Tx. Nntp-Posting-Host: bach.convex.com In article <1991Mar21.213144.14636@nas.nasa.gov> eugene@wilbur.nas.nasa.gov (Eugene N. Miya) writes: >In article <1991Mar21.184428.8226@convex.com> metzger@convex.com >(Robert Metzger) writes: >>One other note: interprocedural optimization will render many industry >>standard benchmarks meaningless. > >Just getting back...... >You should be careful about this. > >The problem is how companies like yours go about testing fixes and >optimizations. Standard test suites exist for language syntax conformance, >deviance, but no such standard set of tests exists for language optimizations. >There needs to be some testing consistency for all this fancy software, >otherwise, how do we have any assurance said features are working. > >That's one reason why I just regard this as another form of testing, >and nothing special. Of course, no company wants embarassing test results. I appreciate your kind concern, but I'm not sure what your're warning me about :-) I'll elaborate on my point: 1) Compilers that perform interprocedural optimization render meaningless the results of simple-minded benchmarks. For example, one of the first things a well-meaning technical-sales-support person at CONVEX did when he got a crack at the Application Compiler was to run whetstone through it. Sure enough, since whetstone attempts to measure call overhead and the APC attempts to minimize call overhead, the results made it look like CONVEX had released a new generation of hardware. We didn't release those numbers to the public, but there's nothing that prevents an unsophisticated buyer from doing the same thing and being misled. (Of course if he's buying million dollar computers based on the results of such stupid benchmarks, maybe he deserves what he gets.) 2) Compilers that perform interprocedural optimization cannot be fooled by subroutine calls into not performing optimizations. This is a common trick in some widely used benchmarks. Well, an IPO compiler will certainly perform thorough IP side-effect analysis, and it will know exactly what is going on inside the procedure. It may automatically perform procedure inlining as well. Procedure calls no longer prevent compilers from moving assignments and uses, or eliminating them altogether, where procedure compilers could not do so. Procedure calls are no longer firewalls. 3) Compilers that perform interprocedural optimization perform a lot more compile-time evaluation of your code. Basically, if you don't read your all data from a data file at runtime, the compiler could potentially evaluate your expressions at compile time and replace your application with a bunch of PRINT statements. I will be the first to say that testing interprocedural optimizers is very challenging. In some senses, a 100,000 line application is ONE test case. Which means you have to run hundreds of applications and millions of lines of code through the compiler before you trust it. On the other hand, our experience with an interprocedural compiler is that the compiler find a dozens or hundreds of bugs in every application for every one bug the application finds in the compiler. Logic errors, abuse of language features, and numerically unstable algorithms are all exposed by this new technology. From my perspective, 99.99% of all scientific/ engineering FORTRAN applications are shot through with numerous errors, and I'm far more concerned about the reliability of the applications than the compilers at this point. -- Robert Metzger CONVEX Computer Corp. Richardson, Texas Generic Disclaimer: I write software, not CONVEX Corporate Policies. "The only legitimate function of government is to protect its citizens from harm or property loss by violence, stealth, or fraud. All else is tyranny." -- ---------------------------------------------------------------- Keith H. Bierman kbierman@Eng.Sun.COM | khb@chiba.Eng.Sun.COM SMI 2550 Garcia 12-33 | (415 336 2648) Mountain View, CA 94043
jim@tiamat.fsc.com ( IT Manager) (03/30/91)
In article <7370347@hpfcso.FC.HP.COM>, mike@hpfcso.FC.HP.COM (Mike McNelly) writes: > > It's quite possible that the optimizer had a bug in it. It's also quite > possible that the source program had a bug that was unmasked by > optimization (this happens a lot). Until the problem is tracked down no > one will ever know for sure. We ran into a snag with -O that pointed out some poor programming habits (or probably just a mistake we over looked). As it happened, I left out the code to initialize a variable before use. I never noticed, cause apparently, without -O, the compiler decided to be nice and initialize it for me. Once the program was "finished", and I added -O to the Makefile, it no longer worked. After some playing, I finally figured out the error, and added the code to initialize the variable, and compiling with -O worked just fine. Apparently, the optimizer said "you want things optimized, and initializing something for you takes extra cycles, so I won't do it - Nyah!" (insert picture of compiler with its toungue sticking out :-). I never got a warning or error message in either case, so either I don't have the warning message level set right, or the compiler just doesn't tell you about these things I should remember to do in the first place. ------------- James B. O'Connor jim@tiamat.fsc.com Ahlstrom Filtration, Inc. 615/821-4022 x. 651
jthomas@nmsu.edu (James Thomas) (03/30/91)
In article <821@tiamat.fsc.com> jim@tiamat.fsc.com ( IT Manager) writes: jim> In article <7370347@hpfcso.FC.HP.COM>, mike@hpfcso.FC.HP.COM (Mike McNelly) writes: > > It's quite possible that the optimizer had a bug in it. It's also quite > possible that the source program had a bug that was unmasked by > optimization (this happens a lot). Until the problem is tracked down no > one will ever know for sure. jim> We ran into a snag with -O that pointed out some poor programming jim> habits (or probably just a mistake we over looked). As it happened, I jim> left out the code to initialize a variable before use. ... jim> I never got a warning or error message in either case, so either I jim> don't have the warning message level set right, or the compiler just jim> doesn't tell you about these things I should remember to do in the jim> first place. lint may be painful, but that's exactly what it's for :-) Try it, you might like it :-) Jim
mjs@hpfcso.FC.HP.COM (Marc Sabatella) (04/04/91)
>As it happened, I left out the code >to initialize a variable before use. I never noticed, cause apparently, >without -O, the compiler decided to be nice and initialize it for me. Once >the program was "finished", and I added -O to the Makefile, it no longer >worked. After some playing, I finally figured out the error, and added the >code to initialize the variable, and compiling with -O worked just fine. >Apparently, the optimizer said "you want things optimized, and initializing >something for you takes extra cycles, so I won't do it - Nyah!" (insert >picture of compiler with its toungue sticking out :-). > >I never got a warning or error message in either case, so either I don't >have the warning message level set right, or the compiler just doesn't tell >you about these things I should remember to do in the first place. We'd love to take credit for adding the initialization code, and warn when we add it, but that is not what happened. The compiler added no such code. Normally, local variables not declared "static" or "register" are allocated on the stack. Often, those locations on the stack just happen to contain zeroes, particularly early in the execution of the program, before the stack has become cluttered. When optimized, however, such locals are put in registers, which "happen" to be zero a whole lot less often. So the compiler is not actually as smart as you imply, nor is it deliberately mocking your programming style or sticking its tongue out at you :-) BTW, lint can't do it all. A classic example of something it can't do: an unitialized character array, "initialized" via a "strncpy" that does not copy a terminating null character. Of course, this array is unlikely to be allocated to a register when optimized, but optimization may affect the stack contents, as can seemingly unrelated changes to other routines in your program.