[comp.sys.mac] Broken compilers

stew@endor.UUCP (05/12/87)

In article <746@apple.UUCP> dgold@apple.UUCP (David Goldsmith) writes:
>Well, er, ah, hmmm, it seems that there was this low memory global called
>BasicGlob, which was reserved for use by the system, and System 4.1 is now
>using it (it's been renamed ExpandMem), but, er, uh, ahem, it seems that
>MacTerminal (against our own rules), was USING BasicGlob.  So it is
>incompatible with System 4.1.
>-- 
>David Goldsmith
>Apple Computer, Inc.

Although I don't know if MegaMax was the culprit in this case, this is
representative of a serious problem with some development systems,
particularly MegaMax C and LightSpeed C.

MacTerminal's not the only application broken by the reuse of
BasicGlob.  It seems that MegaMax used BasicGlob to store a pointer to
the Application's globals.  Thus, any application compiled with
MegaMax C is broken under System 4.1.  I don't use MegaMax C anymore,
but the current release version of my program, ChemDraw 1.0, was
compiled with MegaMax and consequently fails under System 4.1.

WARNING: There is another similar disaster waiting to happen.

All programs compiled with LightSpeed C WILL BREAK, GUARANTEED 100%,
when Apple comes out with a 32 bit operating system for the Mac II or
future hardware.  This is true of LightSpeed C versions 1.02 and 2.01.
The problem is that they use a BSET instruction to manipulate Handle
state flags.  Apple has repeatedly warned Developers (e.g., in Tech
Note #2, dated January 21, 1986) that this is a no-no.

DEVELOPERS: Do not distribute programs compiled with LightSpeed C
unless you are prepared to deal with your users complaining that your
program doesn't work when Apple improves its operating system.

TO ALL CURRENT AND POTENTIAL DEVELOPMENT SYSTEM DEVELOPERS: If it is
important for developers to follow the guidelines, then it is
important squared for development system runtimes to follow the rules.
Otherwise, you will make many, many programs break.  Someone somewhere
is going to test your standard disclaimer of liability, and you will
be in for a legal nightmare.

In a way, although I was seriously damaged by it, I am glad that Apple
broke programs which used BasicGlob.  I am somewhat afraid that the
large number of programs which will break will deter Apple from
providing a 32 bit operating system.

Needless to say, I no longer plan to sell programs built with MegaMax
or Lightspeed C.  I hope that the MPW runtimes follow the rules...
It would be nice if the developers of MPW and of new Macintosh
hardware and operating system software could cooperate to head off problems
like these in the future.

I apologize to everyone for all this verbiage, but I AM PISSED at both
MegaMax and LightSpeed for screwing me over like this.  I can only
thank my lucky stars that I discovered the problems with LightSpeed in
time to avoid a costly mistake.

Stew Rubenstein - Harvard University Chemistry Department
                - Cambridge Scientific Computing, Inc.
                - rubenstein@harvard.harvard.edu
                - seismo!harvard!rubenstein

olson@harvard.UUCP (05/12/87)

In article <1953@husc6.UUCP> stew@endor.UUCP (Stew Rubenstein) writes:
>...
>All programs compiled with LightSpeed C WILL BREAK, GUARANTEED 100%,
>when Apple comes out with a 32 bit operating system for the Mac II or
>future hardware.  This is true of LightSpeed C versions 1.02 and 2.01.
>The problem is that they use a BSET instruction to manipulate Handle
>state flags.  Apple has repeatedly warned Developers (e.g., in Tech
>Note #2, dated January 21, 1986) that this is a no-no.
>...
>

Is this true of the Mac-style support libraries (MacLib) or the unix-style
support libraries, or the code that runs just before main(), or all three,
or WHAT?  This seems very important!  I'm hoping it's only in the unix
libraries, since I don't use them anyway.  This should be a correctable
problem.  Is the code that comes before main() in LSC replaceable by the
end-user (of the compiler)?

How about it, THINK?  What's the scoop?

-Eric
olson@endor.harvard.edu
olson@endor.UUCP

jww@sdcsvax.UCSD.EDU (Joel West) (05/12/87)

In article <1953@husc6.UUCP>, stew@endor.harvard.edu (Stew Rubenstein) writes:
> TO ALL CURRENT AND POTENTIAL DEVELOPMENT SYSTEM DEVELOPERS: If it is
> important for developers to follow the guidelines, then it is
> important squared for development system runtimes to follow the rules.
> Otherwise, you will make many, many programs break.  

> Needless to say, I no longer plan to sell programs built with MegaMax
> or Lightspeed C.  I hope that the MPW runtimes follow the rules...

LSC is great fun and very quick for prototyping, but this points
up the advantange of MPW for final compilation of commercial products.

The event horizon for major system changes at Apple seems to be 6-18
months.  That is, if a major incompatible system software change
is planned, MPW programs will probably not break for at least that long.
Plus, now that Apple has compatibility tech notes (they didn't
when MacTerminal came out), I suspect most of their new software
conforms to those rules per management orders.  

Following the guidelines at the third-party companies isn't hard, and
I suspect most decent firms will do so.  As for code quality, a recent
benchmark (due in Byte next fall) showed that Lightspeed C and Aztec C
were comparable for a register version of the sieve.  Unofficially,
MPW C produced a comparable result, with less than a 2% variation for
all three.
-- 
	Joel West
	{ucbvax,ihnp4}!sdcsvax!jww	(ihnp4!gould9!joel if I ever fix news)
	jww@sdcsvax.ucsd.edu	if you must

earleh@dartvax.UUCP (Earle R. Horton) (05/13/87)

In article <1953@husc6.UUCP>, stew@endor.harvard.edu (Stew Rubenstein) writes:
> 
> All programs compiled with LightSpeed C WILL BREAK, GUARANTEED 100%,
> when Apple comes out with a 32 bit operating system for the Mac II or
> future hardware.  This is true of LightSpeed C versions 1.02 and 2.01.
> The problem is that they use a BSET instruction to manipulate Handle
> state flags.  Apple has repeatedly warned Developers (e.g., in Tech
> Note #2, dated January 21, 1986) that this is a no-no.

Is this a general compiler problem, or is it, rather, specific to some
of the runtime libraries?  The above explanation does not make this
clear.  I would appreciate some detail as to where the offending BSET
instruction gets inserted into the typical C program.  In particular,
does a program using only "MacTraps" have this problem?

-- 
****************************************************************************
*  Dot seegnachur?  I don' got to show you no steenkeen dot seegnachur!!   *
****************************************************************************

lampson@crvax1.dec.com (Mike @DDO - Central Area Software Support) (05/15/87)

Speaking of broken compilers, watch out for MS-BASIC Compiler V1.0. 
Applications compiled with this compiler *will NOT run* under AppleShare.
Microsoft has promised a fix soon.

As far as compatibility with System 4.1, I don't know; I'm afraid to
try.

_Mike

---
Mike Lampson
Digital Equipment Corp.
Arlington Hts, IL
				...decwrl!crvax1.dec.com!lampson
				lampson%crvax1.dec@decwrl.dec.com

alcmist@well.UUCP (Frederick Wamsley) (05/15/87)

There are actually two (at least) problems with Lightspeed C's code
generation which can cause compatibility problems.  Besides using BSET
on the high-order bits of master pointers, they're also generating
self-modifying code for some low-level I/O operations.

When I asked, Think's tech support confirmed both problems and said
that both would be cleared up in the next release.

Of course, if you have a product going out the door tomorrow, this won't
help you.  But I was pleased to find Think aware of compatibility issues.
-- 
Fred Wamsley  {dual,hplabs}!well!alcmist;well!alcmist@lll-crg.arpa;
CIS 72247,3130; GEnie FKWAMSLEY;ATT Mail fwamsley; other uucp 
uw-beaver!uw-june!bcsaic!asymet!fred; USPS - why bother?
"uucp mail is not in general unreliable" - Lauren Weinstein

stew@endor.harvard.edu (Stew Rubenstein) (05/16/87)

In article <6186@dartvax.UUCP> earleh@dartvax.UUCP (Earle R. Horton) writes:
>In article <1953@husc6.UUCP>, stew@endor.harvard.edu (Stew Rubenstein) writes:
>> 
>> All programs compiled with LightSpeed C WILL BREAK, GUARANTEED 100%,
>> when Apple comes out with a 32 bit operating system for the Mac II or
>Is this a general compiler problem, or is it, rather, specific to some
>of the runtime libraries?

When I said "all" I meant "all."  Even if you don't even include MacTraps.
The problem is in that little bit of code in segment 1 that gets prepended
to all (that means every one without exception) applications.

rpd@apple.UUCP (05/16/87)

In article <1995@husc6.UUCP>, stew@endor.harvard.edu (Stew Rubenstein) writes:
> In article <6186@dartvax.UUCP> earleh@dartvax.UUCP (Earle R. Horton) writes:
> >In article <1953@husc6.UUCP>, stew@endor.harvard.edu (Stew Rubenstein) writes:
> >> 
> >> All programs compiled with LightSpeed C WILL BREAK, GUARANTEED 100%,
> >> when Apple comes out with a 32 bit operating system for the Mac II or
> >Is this a general compiler problem, or is it, rather, specific to some
> >of the runtime libraries?
> 
> When I said "all" I meant "all."  Even if you don't even include MacTraps.
> The problem is in that little bit of code in segment 1 that gets prepended
> to all (that means every one without exception) applications.
The claim that all LSC applications execute code that assumes a 24 bit
addressing model is simply not true.  I am the primary developer at Apple of
the "A/UX Toolbox", the software that allows A/UX (Apple's UNIX) programs to
use the mac toolbox.  The A/UX Toolbox supports running programs that are
compiled as UNIX executables or Macintosh binaries.  In either case, the
application must be "well behaved".  This means, among other things, that
the application will be running in a 32 bit environment and cannot play nasty
games with the high bits of pointers.  Several months ago, I asked a friend
who uses LSC (or was that LSD?  No, that was another friend) to write a simple
program for me to test.  This program ran fine.  It is a very simple program
that only makes a few QuickDraw calls.  Sorry, I don't know what version of
LSC was used.  It is entirely possible that there are 32 bit violations in
code generated from LSC.  I only know that you don't necessarily get them in
every application.
						-- Rick Daley
						rpd@apple.UUCP
						ucbvax!sun!apple!rpd

stew@endor.UUCP (05/16/87)

In article <776@apple.UUCP> rpd@apple.UUCP (Rick Daley) writes:
>In article <1995@husc6.UUCP>, stew@endor.harvard.edu (Stew Rubenstein) writes:
>> In article <6186@dartvax.UUCP> earleh@dartvax.UUCP (Earle R. Horton) writes:
>> >In article <1953@husc6.UUCP>, stew@endor.harvard.edu (Stew Rubenstein) writes:
>> >> 
>> >> All programs compiled with LightSpeed C WILL BREAK, GUARANTEED 100%,
>> >> when Apple comes out with a 32 bit operating system for the Mac II or
>> >Is this a general compiler problem, or is it, rather, specific to some
>> >of the runtime libraries?
>> 
>> When I said "all" I meant "all."  Even if you don't even include MacTraps.
>> The problem is in that little bit of code in segment 1 that gets prepended
>> to all (that means every one without exception) applications.
>The claim that all LSC applications execute code that assumes a 24 bit
>addressing model is simply not true.

Forgive me, you are right.  I was able to construct a small program
which skipped over the code which BSET's handles.  Here is a program
which will fail:

main()
{
	char *p = "test";
}

Any application which has a CREL resource in it will fail.  I don't
know if this is a necessary condition, but it is sufficient.  LSC
Developers - check your applications.  Let's take an informal poll:
Let me know if your application has any CREL resources.  If so, it
will fail.  Let me know if it doesn't, also, so I can get a reasonably
accurate poll.  So far, I have checked eight programs which I have
around that were built with LSC.  Every one has at least one CREL.

It's right there, at CODE0001+$78, BSET #7, (A3), where A3 is a handle
to the code resource being loaded.

>  I am the primary developer at Apple of
>the "A/UX Toolbox", the software that allows A/UX (Apple's UNIX) programs to
>use the mac toolbox.
>						-- Rick Daley
>						rpd@apple.UUCP
>						ucbvax!sun!apple!rpd

Now I'm REALLY worried!  Surely you are doing more testing???
Please don't tell me that this is typical of development on A/UX Toolbox...

rpd@apple.UUCP (Rick Daley) (05/18/87)

In article <2003@husc6.UUCP>, stew@endor.harvard.edu (Stew Rubenstein) writes:
> In article <776@apple.UUCP> rpd@apple.UUCP (Rick Daley) writes:
> >In article <1995@husc6.UUCP>, stew@endor.harvard.edu (Stew Rubenstein) writes:
> >> In article <6186@dartvax.UUCP> earleh@dartvax.UUCP (Earle R. Horton) writes:
> >> >In article <1953@husc6.UUCP>, stew@endor.harvard.edu (Stew Rubenstein) writes:
> >> >> 
> >> >> All programs compiled with LightSpeed C WILL BREAK, GUARANTEED 100%,
> >> >> when Apple comes out with a 32 bit operating system for the Mac II or
> >> >Is this a general compiler problem, or is it, rather, specific to some
> >> >of the runtime libraries?
> >> 
> >> When I said "all" I meant "all."  Even if you don't even include MacTraps.
> >> The problem is in that little bit of code in segment 1 that gets prepended
> >> to all (that means every one without exception) applications.
> >
> >The claim that all LSC applications execute code that assumes a 24 bit
> >addressing model is simply not true.
> 
> Forgive me, you are right.  I was able to construct a small program
> which skipped over the code which BSET's handles.  Here is a program
> which will fail:
> 
> main()
> {
>       char *p = "test";
> }
> 
> Any application which has a CREL resource in it will fail.
> ...
> It's right there, at CODE0001+$78, BSET #7, (A3), where A3 is a handle
> to the code resource being loaded.

Ok, here's what's happening.  Your sample program does, in fact, run under
the A/UX Toolbox.  Here's why...  The LSC startup code does do the horrible
nasty bset instruction you described.  Then, later on, they probably do a bclr
instruction to unlock the block.  However, they never use the invalid address
they created by doing the first bset.  It turns out that addresses in the A/UX
heap will end up having their high bits clear (at least for now).  So, this
code will work under A/UX, even though it is not really 32-bit clean.
The only ways this could could fail would be:
        1) If something uses the screwy master pointer before the LSC code
    does the bclr.  I don't think that this can happen in this case.
	2) If a heap compaction occurs, and the offending block gets moved.
    Since LSC sets the wrong bit, the Memory Manager still considers the block
    to be relocatable.
        3) If a future implementation moves the heap to a high enough address
    such that the high bits are actually set.
Don't get the idea that I am defending the LSC code.  They should (and will,
according to an article posted recently) clean this code up.  In fact, it
was pointless of me to correct your original statement.  It just irritates
me when someone uses absolute-sounding statements that aren't correct.
I think it was the "100%" that got to me.  According to rumors floating on
the net, LSC actually does much worse things than this.  People have posted
claims that LSC produces self-modifying code.  This is not guaranteed to work
on any Mac II, even if you aren't running A/UX.  I hope that everyone out
there who owns LSC gives Think a call about this.

> Now I'm REALLY worried!  Surely you are doing more testing???
> Please don't tell me that this is typical of development on A/UX Toolbox...

I'd like to make a few points here...
	1) I suspect that you could test every LSC app available under A/UX
    without finding one that crashes as a result of this glitch.  As an aside,
    the "Scarab of RA!" shareware program which was recently posted to the net
    runs fine under the A/UX Toolbox.  The about box in this program says that
    it was developed with LSC.  This program takes up nearly 200K!
	2) Very early into this project, it was decided that A/UX would not
    support 24 bit addressing.  This means that the A/UX Toolbox cannot be
    highly compatible with "ill behaved" applications.  The idea is not that
    we are supporting a virtual Macintosh environment which will run all of
    your old mac binaries.  The main thrust of this product is to provide
    the ability to put a Macintosh user interface on UNIX programs.  Since
    it turned out to be easy to support mac binaries too, we did it.
    Unfortunately, most commercial applications are not 32 bit clean.  These
    apps will have to be cleaned up before they will run under A/UX.  The
    changes tend to be quite trivial, although a few programmers really
    bought into the whole 24 bit idea.  Anyway, the point of my mumblings
    is that we are testing mainly to see if the A/UX Toolbox works the way
    it should, not to try to support every slimy trick a programmer can get
    away with on the mac.  Therefore, we do most of our testing using special
    test software, not commercial products.
	3) We are hoping that developers will respond to the A/UX Toolbox
    by releasing new versions of their software that work cleanly.  This
    finally gives them incentive and a testing vehicle to follow more of
    the advice given in various Tech notes.  If this happens, everyone
    wins.  When Apple finally produces a version of the native mac OS that
    uses 32 bit addressing, hopefully compatibility with cruel apps will be
    less of an issue.  Users will win by getting software that is designed,
    rather that taped together.  An awful lot of the work involved in
    producing new versions of system files and ROMs is dealing with
    backwards-compatibility with screwy apps.

All this talk about A/UX compatibility has probably got people wondering.
Unfortunately, the documentation for the A/UX Toolbox is still in alpha
draft.  When that stabilizes, I will (if the powers that be say ok) post at
least the section on compatibility.  Until then, I can give a few hints...
	1) The main problem will definitely be 24-bit assumptions.  Like the
    Tech notes say, don't use bset instead of HLock.  Also, don't use data
    structures that pack pointers and bytes into a common 32 bit field.
    Finally, if you ever want to use Lo3Bytes to mask flag bits off a
    dereferenced handle, use the new StripAddress trap instead.
	2) Also like the Tech notes say, don't use privileged instructions.
    If you want to look at the sr, use "move ccr" instructions, not "move sr".
	3) Don't mess with the hardware yourself.  A/UX could not be robust
    if applications were allowed arbitrary access to hardware.
	4) Try to avoid playing around in the data structures described in
    inside Macintosh.  Most of them are supported, but some won't be.
    Example:  The event queue is manipulated by the A/UX kernel and will
    not be directly accessible to an application.
	5) You will be able to check a flag to see if you are executing
    under A/UX.  If you really want to save some time by breaking the above
    rules, you can still run (slower) under A/UX by doing things cleanly
    only when you need to.
	6) I have yet to see a program produced by MacApp that failed to run
    under the A/UX Toolbox. (As long as the built-in debugger is not used)
	7) If anyone has software that they would like to have tested under
    A/UX, I'm willing to give it a try.  If I get buried with requests, I'll
    have to give it up.  Just send me a copy and I'll try it.  Please note:
	    a) Please try to test it on a generic Mac II first.
	    b) If you want me to destroy my copy after testing, say so.
	    c) If it does something weird that you think might cause trouble,
	tell me.  That will save me the trouble of trying to figure out what
	is happening.
	    d) Please give me a full e-mail address.  This Vax is pretty messed
	up and can't deal with foo@bar stuff.
	    e) If you don't trust e-mail, send me a disk.  If you want it back,
	include a pre-paid self-addressed whatchamacallit.
	    f) My e-mail address is:
		rpd@apple.UUCP
		    or
		...!ucbvax!sun!apple!rpd
	    g) My US mail address is:
		Rick Daley
		Apple Computer
		MS 27 AJ
		10500 North DeAnza
		Cupertino, CA 95014

Since some of the above is quite opinionated, I'll suspend my usual distaste
for disclaimers...
The above opinions are mine, not necessarily Apple's.
Apple will just have to get their own.

jww@sdcsvax.UCSD.EDU (Joel West) (05/19/87)

Yes, LSC does give self-modifying code that screws up the 68020
cache, according to my friends at Levco (the Prodigy folks.)

Apparently the PB calls get built on the stack (to get the sync/async
bit, probably).  You have two different PB calls in a row and the 68020
says "Oh, it's the same address, it's in cache, so I don't have to
fetch the value from the memory address".

Real subtle to debug but Think is aware of it, not terribly worried, but
apparently plans to correct it in the next release.
-- 
	Joel West
	{ucbvax,ihnp4}!sdcsvax!jww	(ihnp4!gould9!joel if I ever fix news)
	jww@sdcsvax.ucsd.edu	if you must