svc@well.UUCP (Leonard Rosenthol) (11/01/89)
In article <8852@hoptoad.uucp> tim@hoptoad.UUCP (Tim Maroney) writes: >In article <8835@hoptoad.uucp>, tim@hoptoad.uucp (Tim Maroney) writes: >>The moral: watch out for traps that require glue or put the glue in a >>segment that is never unloaded. > >I get a different moral out of this -- don't unload segments. I use so >many function pointers in my code that it's impossible anyway, so I >don't have this problem. As far as I'm concerned, UnloadSeg was >relevant only to the Mac 128K.... > Are you serious, Tim?!?!? UnloadSeg is as useful today (maybe even more so) than it was back in the days of the 128K Mac...By using UnloadSeg and keeping careful watch of which segs get used the most to keep them around and possibly preloaded one is able to not only get lots of code into very small MF partitions but can also keep their heap from getting fragmented. I think more programmers should start using UnloadSeg so that we can have more small partition applications and therefore have more and more of them running - sure I know about virtual memory, but I prefer the real thing! So get with it folks - there is such as thing as UnloadSeg - USE IT!!! -- +--------------------------------------------------+ Leonard Rosenthol | GEnie : MACgician Lazerware, inc. | MacNet: MACgician UUCP: svc@well.UUCP | ALink : D0025
6600pete@hub.UUCP (11/01/89)
From article <14386@well.UUCP>, by svc@well.UUCP (Leonard Rosenthol): > By using UnloadSeg and > keeping careful watch of which segs get used the most to keep them around and > possibly preloaded one is able to not only get lots of code into very small > MF partitions but can also keep their heap from getting fragmented. > I think more programmers should start using UnloadSeg so that we can > have more small partition applications and therefore have more and more of them > running - sure I know about virtual memory, but I prefer the real thing! > So get with it folks - there is such as thing as UnloadSeg - USE IT!!! This would be a good article for Macker. How to use UnloadSeg safely. It would cover o when to do it o what to do in the rest of the code to make sure you don't access any globals accessed while their segment is swapped out o how to make sure there will always be space to load a swapped-out segment. Takers? Mail me for submission guidelines. Pete Gontier : pete@cavevax.ucsb.edu; outgoing .UUCP addresses bounce Editor, Macker : Online Macintosh Programming Journal; mail for subscription Hire this kid : Mac, DOS, C, Pascal, asm, excellent communication skills Underground : Internet BBS via rlogin 128.11.41.100 -l bbs
tim@hoptoad.uucp (Tim Maroney) (11/02/89)
In article <2825@hub.UUCP> 6600pete@hub.UUCP writes: >This would be a good article for Macker. How to use UnloadSeg safely. It would >cover > o when to do it > o what to do in the rest of the code to make sure you don't access > any globals accessed while their segment is swapped out Oh boy. The problem with MacTutor is now and always has been that the staff lack technical expertise, yet try to run a technical magazine. It would be great if your zine could cover that gap. But this question is (I don't know how to phrase this tactfully) silly. Globals and segmentation have absolutely nothing to do with each other. Globals don't live in a segment, they live in a single block of storage allocated by the OS when the application is launched. Nothing personal or inflammatory is intended, but if you are going to make this kind of extremely basic error, what makes you any more qualified than Dave Smith to run such a magazine? > o how to make sure there will always be space to load a swapped-out > segment. This is a bit more perceptive. Answer is: you have to make sure there is always a contiguous chunk of storage which is free and which is as large as the combination of the largest unloadable segments which may need to be simultaneously loaded. If this sounds difficult to compute, it's because it is; it would be easy to overlook a possible combination of segments which could occur at some point in execution. Nor would testing do much good, since all the tester could tell you would be "It just quit all of a sudden while I was doing X!" Doing X is likely to take you here, there, and everywhere in the code. -- Tim Maroney, Mac Software Consultant, sun!hoptoad!tim, tim@toad.com "Something was badly amiss with the spiritual life of the planet, thought Gibreel Farishta. Too many demons inside people claiming to believe in God." -- Salman Rushdie, THE SATANIC VERSES
tim@hoptoad.uucp (Tim Maroney) (11/02/89)
In article <8852@hoptoad.uucp> tim@hoptoad.UUCP (Tim Maroney) writes: >I get a different moral out of this -- don't unload segments. I use so >many function pointers in my code that it's impossible anyway, so I >don't have this problem. As far as I'm concerned, UnloadSeg was >relevant only to the Mac 128K.... In article <14386@well.UUCP> svc@well.UUCP (Leonard Rosenthol) writes: > Are you serious, Tim?!?!? UnloadSeg is as useful today (maybe even >more so) than it was back in the days of the 128K Mac...By using UnloadSeg and >keeping careful watch of which segs get used the most to keep them around and >possibly preloaded one is able to not only get lots of code into very small >MF partitions but can also keep their heap from getting fragmented. The heap fragmentation issue is obsolete, now that the OS moves code segments into high memory. It is impossible for the heap to become fragmented with this operation, assuming that nothing moved into high memory is ever moved back down (which is how it should be done; that's the purpose of the high memory operation). And sure, you can move things into smaller partitions this way, as long as you don't care about certain minor facts like keeping to the interface guidelines. User operations are supposed to begin taking effect instantaneously. If they require fetching in a large resource from the application file, then they are anything but instantaneous. You also ignored the function pointer issue. I received a letter from a person at Apple that I respect and who is usually right, but not this time. He says that you should put all function-pointer fetching routines into the main code segment: that way, they will be jump table pointers. This may work fine for code that uses one or two function pointers and calls them all itself. However, just imagine passing a function pointer into the jump table to the vertical retrace manager or as a completion routine to the file manager. Can you say "intermittent crash when A5 is not CurrentA5"? I knew you could. And imagine code (if you must; I don't have to imagine) that has lots of function pointers -- you're going to move *all* assignments to them into the main code segment? I don't think so! You're also asking for errors. Suppose you forget just once to put one of your function pointer fetches into the main code segment. Normally, it won't have any problems. But then, just very occasionally and not so you can figure out what's happening, that code segment will wind up relocating and you will have a function pointer to inner space. The tester or user who finds this little bug is not going to be able to tell you what happened; users, in particular, tend to forget everything that happened right before a crash due to emotional shock, and even if they don't, they can only give you the broadest area of functionality in which it occurred. And with this kind of bug, even if they remember exactly, the problem can't be replicated reliably. A similar kind of bug will happen (as I just noted to Pete Gontier) if you inadequately provide guarantees that a segment can be reloaded when you need it. And as recently pointed out by another poster, you have yet another potential intermittent error if traps glue happens to be in an unloaded segment. This has the same replicability problems as the others. > I think more programmers should start using UnloadSeg so that we can >have more small partition applications and therefore have more and more of them >running - sure I know about virtual memory, but I prefer the real thing! I prefer instant response and program reliability, as well as the ability to use function pointers freely in my code. There are really very few people who need to use more than two or three programs in conjunction with the Finder at once, and those people can get SIMMs pretty cheap these days. Real RAM hogs like MPW Shell and FullWrite may need to use it, but my programs rarely get over 250K. >So get with it folks - there is such as thing as UnloadSeg - USE IT!!! No thanks! -- Tim Maroney, Mac Software Consultant, sun!hoptoad!tim, tim@toad.com "Prisons are built with stones of Law, Brothels with bricks of Religion." - Blake, "The Marriage of Heaven and Hell"
6600pete@hub.UUCP (11/02/89)
From article <8877@hoptoad.uucp>, by tim@hoptoad.uucp (Tim Maroney): > In article <2825@hub.UUCP> 6600pete@hub.UUCP writes: >>This would be a good article for Macker. How to use UnloadSeg safely. It would >>cover >> o when to do it >> o what to do in the rest of the code to make sure you don't access >> any globals accessed while their segment is swapped out > > But this question > is (I don't know how to phrase this tactfully) silly. Globals and > segmentation have absolutely nothing to do with each other. You're right, of course. It was something I worried about when I first started dragging units around in the Lightspeed Pascal project window. I should have phrased it better. > [ answer to the question about segmentation vs. globals ] You sound like someone who is quite capable of writing and article on the subject. Why don't you? >> o how to make sure there will always be space to load a swapped-out >> segment. > > This is a bit more perceptive. Answer is: [ the answer ] Hey! Save this for the article. Now your readers have lost X amount of interest in the answer because they've seen a capsulized version already. Pete Gontier : pete@cavevax.ucsb.edu; outgoing .UUCP addresses bounce Editor, Macker : Online Macintosh Programming Journal; mail for subscription Hire this kid : Mac, DOS, C, Pascal, asm, excellent communication skills Underground : Internet BBS via rlogin 128.11.41.100 -l bbs
lippin@spam.berkeley.edu (The Apathist) (11/02/89)
Recently tim@hoptoad.UUCP (Tim Maroney) wrote: >And sure, you can move things into smaller partitions this way, as long >as you don't care about certain minor facts like keeping to the >interface guidelines. User operations are supposed to begin taking >effect instantaneously. If they require fetching in a large resource >from the application file, then they are anything but instantaneous. Remember, those segments aren't getting swapped out unless you needed the memory for something else. When your user is pushing the program to where this would matter, the difference between unloading and not unloading is the difference between running slowly and not running at all. I think it's clear which one the user wants. >You also ignored the function pointer issue. I received a letter from >a person at Apple that I respect and who is usually right, but not this >time. He says that you should put all function-pointer fetching >routines into the main code segment: that way, they will be jump table >pointers. This may work fine for code that uses one or two function >pointers and calls them all itself. However, just imagine passing a >function pointer into the jump table to the vertical retrace manager or >as a completion routine to the file manager. Can you say "intermittent >crash when A5 is not CurrentA5"? I knew you could. Both of you got this one wrong. Any development system worth its salt, and certainly any I've used, will create function pointers as pointers into the jump table, precisely so that they're always valid. They're truly absolute pointers, so they don't care one whit about CurrentA5 (although your code might; that's the well-known gotcha of interrupt tasks, and has established solutions.) [user-confusion caused by bugs that don't happen deleted] >I prefer instant response and program reliability, as well as the >ability to use function pointers freely in my code. There are really >very few people who need to use more than two or three programs in >conjunction with the Finder at once, and those people can get SIMMs >pretty cheap these days. Real RAM hogs like MPW Shell and FullWrite >may need to use it, but my programs rarely get over 250K. Wow! You make it sound like having teeth pulled! It's not so difficult as all that. Tom's three easy rules to a long and happy life using UnloadSeg: 1. Put the trap glue in your main segment. 2. Put your low memory emergency code in your main segment. 3. Put your main event loop in your main segment, and have it call UnloadSeg on all other segments, often. Clever programmers can improve somewhat on this, depending on the structure of their particular programs. But this will work well, and your users would appreciate it, if somebody explained the difference to them. After all, this is the computer for the rest of them. --Tom Lippincott lippin@math.berkeley.edu "You have no power here! Now begone, before someone drops a house on you, too!" --Glinda
oster@dewey.soe.berkeley.edu (David Phillip Oster) (11/02/89)
I recently HAD to use UnloadSeg(). The marketing manager called me on one
of the products I was writing and said, "Help, the program is now so big
it will only allow 2400 records on a 1Meg MacPlus. The box says 2600. It
will take six weeks to make new boxes and we need to ship NOW!"
So, I had to use unloadseg to get space when the user's files got large.
Here is the big problem: There MUST be space to load a required segment.
If the o.s. can't find space, it will crash. You are making an innocent,
arbitrary procedure call, and _kaboom_. Hard to debug and hard to fix.
Some work arounds: 1.) you can do a setjump in your main loop, and a
longjump in your growzone routine. This should sort of work. Note though,
your growzone function is called whenever anything needs memory including
the o.s. doing a longjump from inside the o.s. is a good way to leave it
in an inconstant state. For your own code, you can use Signal (see the
tech notes,) and declare a cleanup routine that will get executed during
the longjump, but it isn't easy to retrofit this kind of thing into an
almost done 40,000 line program.
2.) your development system will let you find out what segment (by number)
each procedure is in. You can add a few:
if(NIL == GetResource('CODE', MAINDOCUPDATESEGNUM)){
Error(IAMSORRYDAVE);
}
that is, the program makes a probe to see if it will be able to get the
segment it needs to do a command. If it can't the user sees an error
message. (This assumes the error message code is in the main segment.)
This is technique I used. My programs are object oriented, so my message
dispatcher can return the error code MESSAGEUNDELIVERABLENOTATHOME, if I
sen d a message to an object whose code can not be read.
This has some problems: I have to manage my own
symbols for my segments, which might get out of date as I rearrange my
procedures. I wish there were a system call: CanLoadSeg(procedureName) which
returns NIL or a handle to the segment. It always bothers me when I see a
function with an obvious inverse that is not implemented. (Yes, I know I
could write it, but would it work in System 7?)
My next program will use technique 1, unless you good folk can tell me a
better way to do things.
I usually only unload segments in my main loop, but occasionally, when I
need extra room (for example during a Save or a spool Print.) I call a
procedure that unlocks all purgable segments except 1: the save segment
for save or the print segment for Print.
This issue is only a small part of the larger issue of memory management
in Mac programs: for example: should your grow zone function throw away
undo information as a last ditch attempt to make extra room? How about if
you need the room while in the middle of executing a ReDo command?
The simple answer is: don't print "max sizes handled" on your boxes. Have
your program enforce conservative limits on the user. Maybe I'm just
taking "the power to be your best" too much to heart.
You may have noticed that I answer more questions than I ask. Am I trying
too hard on this issue?
> The mac is a detour in the inevitable march of mediocre computers.
> drs@bnlux0.bnl.gov (David R. Stampf)
--- David Phillip Oster -master of the ad hoc odd hack.
Arpa: oster@dewey.soe.berkeley.edu
Uucp: {uwvax,decvax}!ucbvax!oster%dewey.soe.berkeley.edu
Summary/Conclusion: if your mac program calls a procedure in a segment
that isn't loaded, the segment loader will call LoadSeg() to fetch it. If
LoadSeg() fails, the mac crashes. This can bite you even if you never call
UnloadSeg() since a segment only gets loaded when you call it for the
first time (which might be AFTER you read a giant file.)time@oxtrap.oxtrap.UUCP (Tim Endres) (11/03/89)
Why does every answer here need to be black and white? I have developed many programs for the Mac. Some use UnloadSeg, others never do. It depends on the application. A serious programmer looks at the application, not some religious artifact, when making these choices.
zben@umd5.umd.edu (Ben Cranston) (11/03/89)
In article <88528@hoptoad.uucp> tim@hoptoad.UUCP (Tim Maroney) writes: > [Apple says] you should put all function-pointer fetching > routines into the main code segment: that way, they will be jump table > pointers. This may work fine for code that uses one or two function > pointers and calls them all itself. However, just imagine passing a > function pointer into the jump table to the vertical retrace manager or > as a completion routine to the file manager. Can you say "intermittent > crash when A5 is not CurrentA5"? I knew you could. Segments should be used for large, transitory functions like initialization, disk prepping, making an environment up from scratch the first time a user calls the application, etc. They should not be used for vertical retrace tasks or IO completion tasks unless the birth, life, and death of these tasks take place entirely while control is contained in that segment > And imagine code (if you must; I don't have to imagine) that has lots > of function pointers -- you're going to move *all* assignments to them > into the main code segment? I don't think so! The alternative, termed 'Jedi' by Tom Lippincott in message <1989Nov1.204513.1183@agate.berkeley.edu>, would be for the development system to ALWAYS use jump table pointers for references to functions, even when the caller is within the same segment as the callee. It's only a few cycles, but it is a cost. > And as recently pointed out by another poster, you have yet another > potential intermittent error if traps glue happens to be in an unloaded > segment. This has the same replicability problems as the others. The same diatribe for traps glue: it should be in the main segment unless it is ONLY used by routines within a single segment, in which case it can safely be linked with that segment. Come on. The Unisys 1100 system, written in the 1960s for crying out loud, AUTOMATICALLY moved shared routines up the segment tree as far as possible. Yet the same people who term that machine a dinosaur have to manually place their library routines. It would be funny if it were not so pathetic. On that machine I was once bitten very badly COMPARING function pointers when one was redirected through the jump table and the other was not. I ended up having to temporarily place those functions back in the main segment until I reprogrammed all the code to use an integer index, and a case at the very end when the functions were actually being called. We seem to have two extremes, neither one of which is terribly useful: Tim is saying "segments are too bothersome, I can't be bothered to actually *design* a segmentation scheme, instead I'll make the users of my programs buy enough physical memory not to need it". Others are saying "put everything into segments and beat on the development systems manufacturers to make everything actually *work* in segments: function pointers, IO completion routines, VBL tasks, traps glue, etc". My claim is that segmentation can be a useful technique to reduce the memory requirements of applications. In order to use it effectively you have to THINK about what you are doing. I don't see a lot of that in the hacker environment. -- Sig DS.L ('ZBen') ; Ben Cranston <zben@Trantor.UMD.EDU> * Computer Science Center Network Infrastructures Group * University of Maryland at College Park
vallon@sbcs.sunysb.edu (Justin Vallon) (11/03/89)
In article <1989Nov2.053139.23967@agate.berkeley.edu> lippin@math.berkeley.edu writes: >Recently tim@hoptoad.UUCP (Tim Maroney) wrote: >>You also ignored the function pointer issue. I received a letter from >>a person at Apple that I respect and who is usually right, but not this >>time. He says that you should put all function-pointer fetching >>routines into the main code segment: that way, they will be jump table >>pointers. This may work fine for code that uses one or two function >>pointers and calls them all itself. However, just imagine passing a >>function pointer into the jump table to the vertical retrace manager or >>as a completion routine to the file manager. Can you say "intermittent >>crash when A5 is not CurrentA5"? I knew you could. > >Both of you got this one wrong. Any development system worth its >salt, and certainly any I've used, will create function pointers as >pointers into the jump table, precisely so that they're always valid. >They're truly absolute pointers, so they don't care one whit about >CurrentA5 (although your code might; that's the well-known gotcha of >interrupt tasks, and has established solutions.) > I seem to sense some confusion here. I think you two are arguing about two different things. Tim said "put all function-pointer FETCHING routines into the main code segment". I think this is a little wrong. I think person at Apple said, or meant to say, that you should put any function that will be referenced by a function-pointer into the main segment. When the main segment is loaded (when you are launched), all main-segment jump-table entries will be set at the time of _LoadSeg. Remember that when _LoadSeg is called, ALL jt entries that reference that segment are updated to a single JMP to the address of the routine in the locked code segment. Since you never UnloadSeg(main-seg), these jt entries will never require a LoadSeg, or A5 to get to your routine in the always- loaded main segment. --------------------------------------------------------------------------- -Justin vallon@sbcs.sunysb.edu ---------------------------------------------------------------------------
lippin@spam.berkeley.edu (The Apathist) (11/03/89)
Recently vallon@sblw.UUCP (Justin Vallon) wrote: >Remember that when _LoadSeg is called, ALL jt entries that reference that >segment are updated to a single JMP to the address of the routine in the >locked code segment. Since you never UnloadSeg(main-seg), these jt entries >will never require a LoadSeg, or A5 to get to your routine in the always- >loaded main segment. This is true, but misses the point. It would be quite an inconvenience to put all routines that are referenced by pointers into the main segment. However, this isn't necessary -- even when a segment is unloaded, a pointer into the jump table is still valid. Jumping at it will cause the segment to be loaded. As has already been pointed out, this would be a problem at interrupt time, so one must avoid unloading code (and, for that matter, other resources) that will be needed during pending interrupts. I seem also to have caused some confusion in my statement that "a true Jedi's development system will use pointers into the jump table." I did not mean to imply that this was a Rare and Special Thing, but rather that the most common ones do, but I'd check first when using a more obscure system. --Tom Lippincott lippin@math.berkeley.edu "Those who understand will require no further explanation." --Saul Bellow
amanda@intercon.com (Amanda Walker) (11/03/89)
In article <32323@ucbvax.BERKELEY.EDU>, oster@dewey.soe.berkeley.edu (David Phillip Oster) writes: > CanLoadSeg(procedureName) which > returns NIL or a handle to the segment. It always bothers me when I see a > function with an obvious inverse that is not implemented. (Yes, I know I > could write it, but would it work in System 7?) Hmm. Actually, if your development system implements function pointers as jump table entries (which I believe MPW does, unless you tell it otherwise), this shouldn't be too hard. You can look at the contents of the function pointer (its entry in the jump table), and get either the actual address if it's loaded or the segment # if not... This assumes the jump table format won't change, but as long as System 7 runs current binaries, life should be pretty mellow (this should even work under A/UX, come to think of it...). -- Amanda Walker <amanda@intercon.com>
jeffl@NCoast.ORG (Jeff Leyser) (11/07/89)
This post has absolutely nothing to do with Macintosh Computers. But I
just couldn't resist.....
In post <8877@hoptoad.uucp>, tim@hoptoad.UUCP (Tim Maroney) says:
!is (I don't know how to phrase this tactfully) silly. Globals and
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
!Nothing personal or inflammatory is intended, but if you are going to
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
!--
!Tim Maroney, Mac Software Consultant, sun!hoptoad!tim, tim@toad.com
^^^^^^^^^^^
This is obviously a forgery, and should be ignored!
--
Jeff Leyser jeffl@NCoast.ORG