melling@cs.psu.edu (Michael D Mellinger) (06/09/91)
Does anyone have a parallel make program running on the NeXT? I have a lab full of NeXT's that I want to put to use :-). By my calculations, the 040 NeXT runs a little more than 1/2 as fast as our $10,000,000 IBM 3090 when compiling programs(compared standard C programs). So, if I get two or three machines running together, I should be in good shape. -Mike
lee@pipe.cs.wisc.edu (Soo Lee) (06/09/91)
In article <bq5Hy#*q@cs.psu.edu> melling@cs.psu.edu (Michael D Mellinger) writes: > >Does anyone have a parallel make program running on the NeXT? I have >a lab full of NeXT's that I want to put to use :-). By my >calculations, the 040 NeXT runs a little more than 1/2 as fast as our >$10,000,000 IBM 3090 when compiling programs(compared standard C >programs). So, if I get two or three machines running together, I >should be in good shape. > >-Mike Distributed processing questions have been issused based on the number of processors rather than communication between task partitioned. This is not quite correct without considering communication bottleneck. In case of MIMD configuaration, raw performance is normally limited by the communication bottleneck. In this case, ethernet has only 1MB/sec sustained transfer rate. Soo lee@cs.wisc.edu
ddj@zardoz.club.cc.cmu.edu (Doug DeJulio) (06/09/91)
In article <bq5Hy#*q@cs.psu.edu> melling@cs.psu.edu (Michael D Mellinger) writes: > >Does anyone have a parallel make program running on the NeXT? I have >a lab full of NeXT's that I want to put to use :-). By my >calculations, the 040 NeXT runs a little more than 1/2 as fast as our >$10,000,000 IBM 3090 when compiling programs(compared standard C >programs). So, if I get two or three machines running together, I >should be in good shape. Yes and no. I compiled gnu-make, which has support for parallel compiles. When I use "gmake -j3" (do three jobs at the same time) combined with "cc -pipe" on my 20 meg NeXTstation, I get an average of a 20% speedup in compiles. Note that you have to be careful about writing your makefiles. You have to have your dependancies all mapped out. The order in which things are compiled can't be depended upon, beyond what's neccessitated by dependencies. I've seen too many makefiles from packages off the net that don't work when used with gnumake -j#, where # is greater than 1. Gnu-make has support for splitting the compile up among a cluster of machines, but since I only have one machine I didn't try to port this functionality to the NeXT (if somone wants to give me a second machine, I'll do it...). Note that the syntax used by gnu-make and the make provided by NeXT (basically CMU CS Make, I think) are different. They have some similar extensions, but the makefiles that take advantage of their extensions are incompatible between them. So, you can't use gnu-make on IB-generated makefiles. Yet. This one I might try to tackle this summer. "This looks like a job for Zilla!" -- Doug DeJulio dd26+@andrew.cmu.edu
scott@mcs-server.gac.edu (Scott Hess) (06/10/91)
In article <1991Jun9.152606.12802@cs.cmu.edu> ddj@zardoz.club.cc.cmu.edu (Doug DeJulio) writes: In article <bq5Hy#*q@cs.psu.edu> melling@cs.psu.edu (Michael D Mellinger) writes: > >Does anyone have a parallel make program running on the NeXT? I have >a lab full of NeXT's that I want to put to use :-). By my >calculations, the 040 NeXT runs a little more than 1/2 as fast as our >$10,000,000 IBM 3090 when compiling programs(compared standard C >programs). So, if I get two or three machines running together, I >should be in good shape. Yes and no. I compiled gnu-make, which has support for parallel compiles. When I use "gmake -j3" (do three jobs at the same time) combined with "cc -pipe" on my 20 meg NeXTstation, I get an average of a 20% speedup in compiles. More than likely, not much more will be reasonable, without an investment in a faster NFS server and the like. I've found that the biggest limiting factor in compiles isn't generally the CPU speed, but the memory and disk capabilities. The compiling itself doesn't actually seem to take that long . . . this might be a limitation of the GNU compiler, as GNU tends to be sort of liberal with memory usage (and in a low-memory system, memory==disk, so that makes it worse). Anyhow, getting down to the main point, I've done makes on different projects on different machines, but since all the makes are yelling at the same server for their fodder, it didn't seem to be substantially faster than single makes on the same machine . . . That was a NeXT serving, though, maybe a DEC or Sun machine with optimized NFS stuff could do better . . . Later, -- scott hess scott@gac.edu Independent NeXT Developer Graduated GAC Undergrad! <I still speak for nobody> Note: I have moved home for a time. My email address will still be valid. Any SnailMail should be redirected, along with phone calls. At the least, my parents can tell you how to get hold of me, or forward any mail . . . Old: PO 829, GAC, St. Peter, MN 56082 (507) 933-8466 New: RR#4 Box 227 Pipestone, MN 56164 (507) 825-2788
soward@inslab.uky.edu (John Soward) (06/10/91)
In article <SCOTT.91Jun9143313@mcs-server.gac.edu> scott@mcs-server.gac.edu (Scott Hess) writes: > In article <1991Jun9.152606.12802@cs.cmu.edu> ddj@zardoz.club.cc.cmu.edu (Doug DeJulio) writes: > In article <bq5Hy#*q@cs.psu.edu> melling@cs.psu.edu (Michael D Mellinger) writes: > > > >Does anyone have a parallel make program running on the NeXT? I have > >a lab full of NeXT's that I want to put to use :-). By my > >calculations, the 040 NeXT runs a little more than 1/2 as fast as our > >$10,000,000 IBM 3090 when compiling programs(compared standard C > >programs). So, if I get two or three machines running together, I > >should be in good shape. > > Yes and no. I compiled gnu-make, which has support for parallel > compiles. When I use "gmake -j3" (do three jobs at the same time) > combined with "cc -pipe" on my 20 meg NeXTstation, I get an average of > a 20% speedup in compiles. > > More than likely, not much more will be reasonable, without an > investment in a faster NFS server and the like. I've found that > the biggest limiting factor in compiles isn't generally the CPU > speed, but the memory and disk capabilities. The compiling itself > doesn't actually seem to take that long . . . this might be a > limitation of the GNU compiler, as GNU tends to be sort of > liberal with memory usage (and in a low-memory system, memory==disk, > so that makes it worse). > > Anyhow, getting down to the main point, I've done makes on different > projects on different machines, but since all the makes are > yelling at the same server for their fodder, it didn't seem to be > substantially faster than single makes on the same machine . . . > That was a NeXT serving, though, maybe a DEC or Sun machine with > optimized NFS stuff could do better . . . > > Later, > -- > scott hess scott@gac.edu > Independent NeXT Developer Graduated GAC Undergrad! > <I still speak for nobody> Oh, now, see, I think you missed the point as well as the OS classes, he was running 3 compiles simultaneously on the *same* machine, Which may initially seem like a fauxpas, but in reality will get a larger percentage of the overall cpu, since Mach will interrupt you ocmpile just to see what else there is to do, this way that other thing to do will be to work on another part of your compile... -- ________________________________________________________________________ / "no, they're totally different. Their song goes 'bomp-bomp-bomp' / / but mine goes 'bomp-bomp-Bomp'...totally differnt" -V. Ice. / / john@ukpr.uky.edu | soward@s.ms.uky.edu | sysjohn@ukcc.uky.edu / /_______________________________________________________________________/
scott@mcs-server.gac.edu (Scott Hess) (06/10/91)
In article <1991Jun10.035748.3335@ms.uky.edu> soward@inslab.uky.edu (John Soward) writes: In article <SCOTT.91Jun9143313@mcs-server.gac.edu> scott@mcs-server.gac.edu (Scott Hess) writes: > In article <1991Jun9.152606.12802@cs.cmu.edu> ddj@zardoz.club.cc.cmu.edu (Doug DeJulio) writes: > In article <bq5Hy#*q@cs.psu.edu> melling@cs.psu.edu (Michael D Mellinger) writes: > > > >Does anyone have a parallel make program running on the NeXT? I have > >a lab full of NeXT's that I want to put to use :-). By my ^^^^^^^^^^^^^^^^^^ > >calculations, the 040 NeXT runs a little more than 1/2 as fast as our > >$10,000,000 IBM 3090 when compiling programs(compared standard C > >programs). So, if I get two or three machines running together, I ^^^^^^^^^^^^ > >should be in good shape. > > Yes and no. I compiled gnu-make, which has support for parallel > compiles. When I use "gmake -j3" (do three jobs at the same time) > combined with "cc -pipe" on my 20 meg NeXTstation, I get an average of > a 20% speedup in compiles. > [I describe a reasonable reason for why you'd only get about 20% speedup - basically that compiles are more disk/memory limited than CPU limited. -scott] Oh, now, see, I think you missed the point as well as the OS classes, he was running 3 compiles simultaneously on the *same* machine, Which may initially seem like a fauxpas, but in reality will get a larger percentage of the overall cpu, since Mach will interrupt you ocmpile just to see what else there is to do, this way that other thing to do will be to work on another part of your compile... I think we both missed the point (note the highlighted places above). I _don't_ think that my response has anything to do with whatever OS classes I've taken - either way. I described why a networked parallel make might not get much more performance (which would, I guess, address the original question), while apparently the first respondant was talking about running multiple compiles on the same machine. Please enlighten me as to where the contradiction is? On the other hand, all of the points I made do apply, in spades. After all, multiple compiles on the same machine will have _more_ competition for the limited resources of memory and disk, since now the swapfiles are in the same place. So there is sort of a built-in limit to the performance increase. I would suspect that on the same machine, you will see more performance increase (in a parallel-make, of course) from the processes being able to continually use the bottle-neck resources than from the higher use of the CPU. For instance, if you accept that compiles are disk-limited, then if you run one compile, the second would slow things down due to contention. But, if one took longer than the other, the one that kept running would be able to have the disk all to itself when the first one is done (and before another compile is scheduled). Rather than having an extreme usage pattern (either using heavily, or not at all), you should get a more even pattern, where the disk is being used almost all the time. I used to use the idea of runnning more processes at a time when I was on a VMS system, as I could then get a higher portion of the CPU. Consider a system running a couple long processes in the background (ok, 2). Then, if I run one process of the same caliber, I'll get 1/3 of the CPU (or therabouts). If I run 2, meanwhile, I'll get 1/4 for each, for 1/2 total. If I run more than that, I'll get the majority of the CPU. But, on a NeXT that won't really make that much difference. In general, you're only running one process, as it's not a time-sharing system. So, when you run multiple compiles, they are stealing CPU from each other, as opposed to some unrelated process (unless you happen to be trying to work in the foreground, in which case you probably don't _want_ them to steal CPU from you!). If you have two compiles running at the same time, Mach is no more efficient switching from one to the other than it is switching from the only compile running back to the same compile (if there's nothing else to do). So the only gains you might get would be to steal CPU from a non-compile process, or by the improved disk access stats. Anyhow, -- scott hess scott@gac.edu Independent NeXT Developer Graduated GAC Undergrad! <I still speak for nobody> Note: I have moved home for a time. My email address will still be valid. Any SnailMail should be redirected, along with phone calls. At the least, my parents can tell you how to get hold of me, or forward any mail . . . Old: PO 829, GAC, St. Peter, MN 56082 (507) 933-8466 New: RR#4 Box 227 Pipestone, MN 56164 (507) 825-2788
crum@alicudi.usc.edu (Gary L. Crum) (06/10/91)
Competition between compilation processes for disk I/O is addressed significantly by caching, and NeXT Mach is pretty good at using available memory as a disk cache. I am using pmake on a four-processor Silicon Graphics system (4D/340VGX), and it works quite well. For me, a pmake(1) compile generally finishes in less than one fourth the time that make(1) requires for the same job. Silicon Graphics has included pmake(1) with IRIX for some time now. Maybe NeXT will enable Mach multiple processor support and include GNU-make in Release 3. Hope so. I would also very much like to see multiple display support extended beyond systems with NeXTdimension boards, e.g. so one can put multiple '040 boards in a cube and control multiple monochrome megapixel displays from a single keyboard and mouse. (If you also want these things, then please express your interest in them.) Gary