jtn@potomac.ads.com (John T. Nelson) (03/25/91)
Here's an interesting problem. Let's say I want to write a text editor. This editor will want to handle files that might be very small (a couple of bytes) to perhaps several megabytes of text (big .Hqx files). To do this, I obviously want to use malloc. Problem is that the Mac OS mallocs (or NewPtr if you like) its space out of the application heap whose size is apparently determined by the application's suggested size (available from command-I in the Finder). Okay so far? Now... in order to handle those BIG files this means that I need an app size that's extremely large (several megabytes) but that means I cripple the application because then it won't run on smaller Macs with perhaps only two meg of memory. If I set the app size too small then it doesn't run on those big files. It would be nice to newPtr my space from a "free" area whose upper limit was not fixed and tied to the application's expectations. What does one do in these sort of situations or have I missed something conceptually? I would like to be able to run an app without telling the OS how much memory I need because I don't know how much. Many thanks for all of your help now and in the past. I remain yours... Dark Hacker! =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= ORGANIZATION: BLETCH (Bletcherous League of Evil Twisted Computer Hackers) UUCP: kzin!speaker@mimsy.umd.edu INTERNET: jtn@potomac.ads.com SPOKEN: Dark Hacker PHONE: (703) 243-1611 The Mythos of Dark Hacker: "Controlled by the sinister and shadowy "suits" Dark Hacker now employs the tools of computer science to free himself from the suit's will. By day he is a lackey... but at night when the city sleeps he becomes.... DARK HACKER!" =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
peirce@outpost.UUCP (Michael Peirce) (03/25/91)
In article <1991Mar24.172706.20807@potomac.ads.com>, jtn@potomac.ads.com (John T. Nelson) writes: > > > Here's an interesting problem. Let's say I want to write a text > editor. This editor will want to handle files that might be very > small (a couple of bytes) to perhaps several megabytes of text (big > .Hqx files). To do this, I obviously want to use malloc. Problem is > that the Mac OS mallocs (or NewPtr if you like) its space out of the > application heap whose size is apparently determined by the > application's suggested size (available from command-I in the Finder). > > Okay so far? > > Now... in order to handle those BIG files this means that I need an > app size that's extremely large (several megabytes) but that means I > cripple the application because then it won't run on smaller Macs with > perhaps only two meg of memory. If I set the app size too small then > it doesn't run on those big files. It would be nice to newPtr my > space from a "free" area whose upper limit was not fixed and tied to > the application's expectations. > > What does one do in these sort of situations or have I missed > something conceptually? I would like to be able to run an app without > telling the OS how much memory I need because I don't know how much. A couple of strategies come to mind: (1) write a smart text file handler. This would read a portion of the text into a "small" buffer and keep the rest on disk. This is how MPW does it and it can work very well if designed properly. (2) If you only need this space briefly you could ask MultiFinder for it. Of course, MultiFinder may not have it to give out if there are a number of Apps running. (3) keep everything in memory and force your users to change the App Size to handle big files - and ship AppSizer with it to make life easier for them :-) ;-) -- michael -- Michael Peirce -- outpost!peirce@claris.com -- Peirce Software -- Suite 301, 719 Hibiscus Place -- Macintosh Programming -- San Jose, California 95117 -- & Consulting -- (408) 244-6554, AppleLink: PEIRCE
Jim.Spencer@p510.f22.n282.z1.fidonet.org.org (Jim Spencer) (03/25/91)
John T. Nelson writes in a message to All JTN> Now... in order to handle those BIG files this means that I need JTN> an app size that's extremely large (several megabytes) but that JTN> means I cripple the application because then it won't run on JTN> smaller Macs with perhaps only two meg of memory. If I set the JTN> app size too small then it doesn't run on those big files. It JTN> would be nice to newPtr my space from a "free" area whose upper JTN> limit was not fixed and tied to the application's expectations. JTN> JTN> What does one do in these sort of situations or have I missed JTN> something conceptually? I would like to be able to run an app JTN> without telling the OS how much memory I need because I don't JTN> know how much. What you are missing is that if you really plan to handle really big files, on the order of several megabytes, you need to come up with a way to only read part of the file into memory at a time. Even those with lots of RAM won't go for a text processer which needs really huge amounts of memory just so the entire file can be in RAM at the same time.
pete@adele.inria.fr (Pete Keleher) (03/27/91)
In article <669949449.1@mmug.fidonet.org.org>, Jim.Spencer@p510.f22.n282.z1.fidonet.org.org (Jim Spencer) writes: |> |> John T. Nelson writes in a message to All |> |> JTN> Now... in order to handle those BIG files this means that I need |> JTN> an app size that's extremely large (several megabytes) but that |> JTN> means I cripple the application because then it won't run on |> JTN> smaller Macs with perhaps only two meg of memory. If I set the |> JTN> app size too small then it doesn't run on those big files. It |> JTN> would be nice to newPtr my space from a "free" area whose upper |> JTN> limit was not fixed and tied to the application's expectations. |> JTN> |> JTN> What does one do in these sort of situations or have I missed |> JTN> something conceptually? I would like to be able to run an app |> JTN> without telling the OS how much memory I need because I don't |> JTN> know how much. |> |> What you are missing is that if you really plan to handle really big |> files, on the order of several megabytes, you need to come up with a |> way to only read part of the file into memory at a time. Even those |> with lots of RAM won't go for a text processer which needs really |> huge amounts of memory just so the entire file can be in RAM at the |> same time. I disagree. You are saying that each application should manage its own virtual memory. It is precisely to avoid this that we have application-independant virtual memory supported by the system. If you want to read a 10meg file, read into memory and let the system page it on and off of disk. I agree totally with earlier comments to the effect that the the ability to dynamically change your address space size is a major lack of 7.0. -- Pete Keleher INRIA, B.P. 105, 78153 Rocquencourt Cedex, France. Tel.: +33 (1) 39-63-52-93; fax: +33 (1) 39-63-53-30; e-mail: pete@sor.inria.fr
dorner@pequod.cso.uiuc.edu (Steve Dorner) (03/28/91)
In article <2029@seti.inria.fr> pete@adele.inria.fr (Pete Keleher) writes: >|> with lots of RAM won't go for a text processer which needs really >|> huge amounts of memory just so the entire file can be in RAM at the >|> same time. > >I disagree. You are saying that each application should manage its own >virtual memory. It is precisely to avoid this that we have >application-independant virtual memory supported by the system. If you want >to read a 10meg file, read into memory and let the system page it on and >off of disk. Wrong several counts. First, it takes a while to read 10 meg into memory; users don't want to wait for it. Second, it takes a ton of disk for all those (probably useless anyway) pages. Third, an application can be smart about anticipating a user's needs; a paging system cannot be nearly so smart. >I agree totally with earlier comments to the effect that the the ability to >dynamically change your address space size is a major lack of 7.0. Not if you really go gung-ho for your 'swapping is wonderful' thesis; just give everybody 20 or 30 meg of virtual space, and 'let the system page it on and off of disk'. -- Steve Dorner, U of Illinois Computing Services Office Internet: s-dorner@uiuc.edu UUCP: uunet!uiucuxc!uiuc.edu!s-dorner
time@ice.com (Tim Endres) (03/29/91)
In article <2029@seti.inria.fr>, pete@adele.inria.fr (Pete Keleher) writes: > > I disagree. You are saying that each application should manage its own > virtual memory. It is precisely to avoid this that we have > application-independant virtual memory supported by the system. If you want > to read a 10meg file, read into memory and let the system page it on and > off of disk. OH Please do! And my commercial product will beat yours hands down EVERY time! When you force the user to sit through a 10Megabyte move of data from one place on their hard disk to another place JUST so they can read the first page of the file, your product is history! Just another classic example of a "computing thoery" mindset missing the "practical world" mark by several miles. As I have told every student of programming I have ever taught: "Use your head for more than syntax checking!" We do not hire CS majors. They make terrible programmers. :) tim. ------------------------------------------------------------- Tim Endres | time@ice.com ICE Engineering | uupsi!ice.com!time 8840 Main Street | Voice FAX Whitmore Lake MI. 48189 | (313) 449 8288 (313) 449 9208
urlichs@smurf.sub.org (Matthias Urlichs) (03/31/91)
Yet another virtual-memory vs. the-application-manages-its-data war... I suggest: When in doubt, get both. A text editor is a very good example how an application benefits greatly by doing its personal memory management. The editor isn't going to be any more complex just because not all of the text is in memory at any one time, and the user can't look at the whole text at the same time anyway. ;-). A color picture editor, however, will be a _lot_ easier to program&debug if it can just allocate the picture as a big chunk o' memory, use the standard 32-bit QD calls, and let the system handle swapping the actual data if necessary. Sweeping generalizations as to which method is the all-out better one aren't going to help. (IMHO.) -- Matthias Urlichs -- urlichs@smurf.sub.org -- urlichs@smurf.ira.uka.de /(o\ Humboldtstrasse 7 - 7500 Karlsruhe 1 - FRG -- +49-721-621127(0700-2330) \o)/
pete@adele.inria.fr (Pete Keleher) (04/02/91)
In article <1CE00001.binbdde@tbomb.ice.com>, time@ice.com (Tim Endres) writes: |> |> In article <2029@seti.inria.fr>, pete@adele.inria.fr (Pete Keleher) writes: |> > |> > I disagree. You are saying that each application should manage its own |> > virtual memory. It is precisely to avoid this that we have |> > application-independant virtual memory supported by the system. If you want |> > to read a 10meg file, read into memory and let the system page it on and |> > off of disk. |> |> OH Please do! And my commercial product will beat yours hands down |> EVERY time! When you force the user to sit through a 10Megabyte move |> of data from one place on their hard disk to another place JUST so |> they can read the first page of the file, your product is history! |> |> Just another classic example of a "computing thoery" mindset missing the |> "practical world" mark by several miles. As I have told every student |> of programming I have ever taught: |> |> "Use your head for more than syntax checking!" |> |> We do not hire CS majors. They make terrible programmers. :) |> |> tim. How often do you read 10meg files, Tim? Do you want every application to throw out a clean simple design in favor of re-implementing system services just so that 10meg file can be read in a little faster? Meanwhile, code complexity is greatly increased, code size as well. Computer science majors are taught to think in terms of abstractions. Optimize for the common case. If the 10meg file is your common case, then you aren't looking for a general purpose editor. -- Pete Keleher INRIA, B.P. 105, 78153 Rocquencourt Cedex, France. Tel.: +33 (1) 39-63-52-93; fax: +33 (1) 39-63-53-30; e-mail: pete@sor.inria.fr
amanda@visix.com (Amanda Walker) (04/03/91)
In article <2043@seti.inria.fr> pete@adele.inria.fr (Pete Keleher) writes:
How often do you read 10meg files, Tim? Do you want every application to
throw out a clean simple design in favor of re-implementing system services
just so that 10meg file can be read in a little faster? Meanwhile, code
complexity is greatly increased, code size as well.
Generally speaking, a small increase in code complexity can vastly improve
the range of conditions under which a program will perform well. In a
commercial application, this kind of investment is well worth it.
Computer science majors are taught to think in terms of abstractions.
Optimize for the common case. If the 10meg file is your common case, then
you aren't looking for a general purpose editor.
True, but if your product can't handle "unusual" cases, it needs
improvement. Customers don't care how spartan the code is--if it doesn't
let them get their work done, it's broken.
It's amazing what the commercial environment can do for your sense of
pragmatism :)...
--
Amanda Walker amanda@visix.com
Visix Software Inc. ...!uunet!visix!amanda
--
"I have never seen anything fill up a vacuum so fast and still suck."
-- Rob Pike commenting on the X Window System