rayan@ai.toronto.edu (Rayan Zachariassen) (04/09/89)
Whether in the short or long term, we are looking at the high-end SGI boxes as compute servers and timesharing boxes. In this context we are totally uninterested in their graphics aspects. I'd like to hear from sites that are already doing this, if any, with any comments about this kind of use of the SGI boxes. I'm especially interested in comparisons with other systems prior to purchase, and differences between expectations and reality after you got the box in. I gather these things have just started shipping so the field is probably still meagre... In order to get technical details out of the local salescritter we have to ask specific questions, so I'd also like some general answers to the following to get us going: Our major worry (in the fine SGI tradition...) is with the software, in particular we consider any System V based box to start out with a negative (this is our reality not our religion). We understand SGI is committed to SV. Does this mean they will track AT&T SV releases directly, or that whatever SV-based OS that MIPS comes up with will shortly appear on the 4Ds? The filesystem is a worry. We're happy it isn't SV but unhappy at the apparent gratuitous incompatibility with the BSD F^nS. Our tools are unlikely to work, right? It also seems like a less robust design. Will it go away in favour of something else that is largely compatible with F^nS ((Fat)Fast File System)? I note that MIPS ships FFS with their rice-computer OS, how come SGI seems to be waiting for SVR4 to do the same? During testing on the personal iris, some anomalies showed up that could be explained by the scheduler or VM being tuned for a single-user workstation environment. For example running a certain (non-graphics) program would cause lost ether packets and horrible response time on the iris, but the same program is apparently wellbehaved on other machines. Similarly, logging out of the PI causes lost packets. Anyone experienced similar anomalies on the 4D/2xx? Anyone using them for timesharing? How does the fine-grained multiprocessing support (threads libraries, compiler support etc.) differ qualitatively from other implementations (MachOS, Sequent, Encore, Sun)? Can one use a 4D to serve root and swap for a SunOS 4.0 workstation? How is the hardware reliability on the 4Ds? Any other pertinent comments from customers are welcome. The kind of configuration that is of interest is a 4D/240S with minimal extra stuff (small SCSI, cartridge), to which we'll add the storage subsystem w/ a few gigs of disk. Users would have access via the ether. In the timesharing application we would want to potentially support at least twice the work our Sun4/280Ss are being asked to do (which is 30 users + 30 workstations, mostly light activity but occasional developers and long-running and/or large jobs) which it does well when it works. Please REPLY BY MAIL! I will summarize if interesting info appears. Thanks, rayan AI/NA/Theory, DCS, U of Toronto
rayan@AI.TORONTO.EDU (Rayan Zachariassen) (04/10/89)
Because if one ignores the graphics then the boxes become a competitive hardware platform for general purpose computing. SGI keeps thinking of themselves as a graphics company (at least that's the reaction we get from high-ups when they see our lack of interest in the graphics). This is a pity because with the right OS they could make a killing in our kind of market. To answer your second question, UNIX is indeed the right OS, but System V in particular is NOT. If (say) a multiprocessor 4.3+BSD ran on the 4Ds and competitive hardware price was maintained, they'd be the only game in town for us. We even frown on berkeleyized SV because the admin stuff is different, but I guess we can live with it if we have to. If there is a future 4BSD-mips release, I'd love to see a port to the 4D platform. rayan
blbates@AERO4.LARC.NASA.GOV (Bates TAD/HRNAB ms294 x2601) (04/10/89)
If you are not interested in graphics, why buy a SGI machine?
Buy something from anybody else, it is bound to be better, cheaper,
and better supported than a SGI machine.
Do you not like UNIX or just System V in particular?
--
Brent L. Bates
NASA-Langley Research Center
M.S. 294
Hampton, Virginia 23665-5225
(804) 864-2854
E-mail: blbates@aero4.larc.nasa.gov or blbates@aero2.larc.nasa.govmadd@adt.UUCP (jim frost) (04/10/89)
> If you are not interested in graphics, why buy a SGI machine? >Buy something from anybody else, it is bound to be better, cheaper, >and better supported than a SGI machine. The high-end 4D machines have very good price/performance even without the graphics (so do the Personals but without graphics new offerings from DEC etc are better apparently). If they beat other systems in price/performance (they beat quite a few), then the graphics is just a bonus. Operators can play flight while waiting for backups :-). Seriously, I believe that a properly tuned/configured 4D/2xx would make a fantastic multiuser machine for the money. The biggest problem is the lack of serial ports, which can be fixed by either a cheap machine as a front-end or a standalone terminal server such as encore's annex box. Exactly how well this will perform is up in the air; all of our SGI machines are obviously tuned to work single-user/single application and perform rather poorly if you break these constraints. I haven't looked into correcting this because we generally have one user, one or more machines. Since the machines are mostly SysV and are intended to be workstations, there are some real problems with using them as multiuser: * 'tar' is not a backup program, no matter who thinks so. We copy entire filesystems between machines for redundancy and back up from one of the Suns, but our data space is less then a half-gigabyte in general, not the case on large multiuser machines. * SysV "ps" is too painful to use when managing a system with lots of things running. Someone ought to build an "sps". Funky shell scripts could fix this but when you need to know what's going on, you usually are in too much trouble to waste that kind of time and resources to find out (eg some idiot accidentally spawned two hundred jobs and filled the proc table; never done it myself :-). * There are several programs (ftp has given me trouble in the past) which don't clean up utmp correctly, bothersome but not fatal. * The filesystem does not appear to be BSD FFS, something which becomes an issue real fast with a lot of users. It doesn't have the 14 character bugaboo that bothers me so much though. * If you use Sun's yp, you're in for a lot of fun. It's not the default for getpwent etc. Aside from the problems that caused, we've had few problems with it. * The graphics is not in the least bit secure. Neither is anyone else's that I've worked with. That's all I can think of at the moment. Some of the above may have been fixed; the OS version we use on most of our 4D's is out-of-date and we've yet to see an update. I'd be interested in hearing about performance if someone has tuned a 4D for multiuser. jim frost madd@bu-it.bu.edu
phil@BRL.MIL (Phil Dykstra) (04/10/89)
> Because if one ignores the graphics then the boxes become a competitive > hardware platform for general purpose computing. Yes, even if you totally ignore the graphics, the CPU power per dollar in the SGI's is still impressive. W.r.t. BSD Unix on SGI machines, I agree that this would sure be nice! I despise SysV. I think our hope of liberation will come with R4, which will finally make System V a usable OS [By then of course 4.4 BSD will be out and SysV will have to play catch up again, but it will still be a major improvement]. - Phil
jmb@patton.SGI.COM (Jim Barton) (04/11/89)
In article <89Apr9.160219edt.38129@neat.ai.toronto.edu>, rayan@ai.toronto.edu (Rayan Zachariassen) writes: > Our major worry (in the fine SGI tradition...) is with the software, in > particular we consider any System V based box to start out with a negative > (this is our reality not our religion). We understand SGI is committed to SV. Sounds like religion - for SGI it's commercial reality, as it is for every other successful computer vendor. Oh well, why tilt at windmills? > Does this mean they will track AT&T SV releases directly, or that whatever > SV-based OS that MIPS comes up with will shortly appear on the 4Ds? IRIX != UMIPS. We are committed to tracking AT&T releases. IRIX currently is at V.3.1, with V.3.2 late this year. We will implement V.4 as expediently as possible. We will also consider any major OSF/1 feature which adds value to the system. > The filesystem is a worry. We're happy it isn't SV but unhappy at the > apparent gratuitous incompatibility with the BSD F^nS. Our tools are > unlikely to work, right? It also seems like a less robust design. Will it > go away in favour of something else that is largely compatible with > F^nS ((Fat)Fast File System)? I note that MIPS ships FFS with their > rice-computer OS, how come SGI seems to be waiting for SVR4 to do the same? Why worry? The SGI ExtentFileSystem is >faster< than the BSD FFS. For instance, on the Jim Barton extra-special-whizzy single-and-multi-process blow-out-the-buffer-cache benchmark (substantiated by the AIM II disk benchmark and other tests), UMIPS 3.0 FFS on the M/120 runs about 15% slower than IRIX 3.0 EFS on the >exact same hardware<. And we've put alot of work into EFS since the 3.0 release ... As to robust, it also duplicates superblocks, has cylinder groups, bitmaps and the like, but it can use >all< your disk fairly effectively (try that with FFS!) I'll be happy to send you a copy of the benchmark, as well. > During testing on the personal iris, some anomalies showed up that could > be explained by the scheduler or VM being tuned for a single-user workstation > environment. For example running a certain (non-graphics) program would > cause lost ether packets and horrible response time on the iris, but the > same program is apparently wellbehaved on other machines. Similarly, logging > out of the PI causes lost packets. Anyone experienced similar anomalies > on the 4D/2xx? Anyone using them for timesharing? Main problem is with the window manager, which is pretty heavyweight. For all that nice display and all, it takes lots of memory, which has to be fought over with the application you are running in a limited memory system. If you really aren't interested in graphics, don't start the window manager, and the performance will be very good (you >said< server, right?). Try running your same application on a Sun 4 with NeWS and 8Mb of memory (assuming you can get Sun to sell you one) and amuse yourself with the results. As to the lost packets, I believe that this bug is fixed in the latest release available to the field. > How does the fine-grained multiprocessing support (threads libraries, compiler > support etc.) differ qualitatively from other implementations (MachOS, > Sequent, Encore, Sun)? Its better, of course! :-). The thread implementation has been published in USENIX proceedings, etc.. It provides a much more natural model for multi threaded applications than any other model I know of. We also support a layer of synchronization using spinlocks and semaphores that religiously avoids kernel interaction. Remember that syncrhonization latency is the chief problem in getting high performance from fine-grained parallelism. On top of this are some of our primitives, plus the Sequent m_* routines for simple parallel programming. In the environment, we support a multi-process asynchronous debugger which works on normal and "threaded" processes (Sun, Mach don't!). The profiler handles a threaded process correctly. All this is integrated with the normal high-performance MIPS compilers. > Can one use a 4D to serve root and swap for a SunOS 4.0 workstation? I assume so. We are currently at NFS 3.2, so if that's all it needs, it should work. > How is the hardware reliability on the 4Ds? Reliability is good. We publish a demonstrated MTBF number for all machines. PowerSeries products are rated for at least 6000 hours. > Any other pertinent comments from customers are welcome. The kind of > configuration that is of interest is a 4D/240S with minimal extra stuff > (small SCSI, cartridge), to which we'll add the storage subsystem w/ a > few gigs of disk. Users would have access via the ether. In the > timesharing application we would want to potentially support at least > twice the work our Sun4/280Ss are being asked to do (which is 30 users > + 30 workstations, mostly light activity but occasional developers and > long-running and/or large jobs) which it does well when it works. You may want to buy the storage from us. We currently support >10Gb of storage on a single machine through large-capacity SMD drives. Since 4 processors with twice the power of a Sun 4 are in the same box, I would think the load you describe would be easily handled. In my lab, we use a 4D/120 with 2 extra processors (an "unofficial" 4D/140). We have a 150Mb SCSI cartridge, 9-track, E-net, etc. The disk configuration, from /etc/motd, is: Maddog 4D/140S IRIX 4D-3.2A (Alpha 7) ASD Compute/File Server =============================================================================== CDC Sabre 9720-1230 1.2Gb SMD xyl1d1s0 / xyl1d1s6 /usr build tree Fuji Eagle 2351 400Mb SMD xyl1d2s1 /usr/tmp xyl1d2s6 /f user data Toshiba MK156FB 156Mb SCSI dks0d1s7 /e MIPS source Fuji Eagle 2351 400Mb SMD xyl1d0s7 /d user data Hitachi 514-38 380Mb ESDI ips0d3s7 /g user data Hitachi 512-17 150Mb ESDI ips0d0s7 /vme0 BRL, MIPS bench Hitachi DK514C 380Mb SCSI dks0d2s7 /vme0/jmb/other user data =======================> 3.1 Gb and counting <================================= This machine is used by > 30 workstations and lot's of users, performs as a build machine, as well as supporting our development environment, with lots of NFS filesystems, constant E-net traffic and more. Since a 4D/240 is twice as fast as this machine, you should have no problems. > Please REPLY BY MAIL! I will summarize if interesting info appears. I thought the net might be interested in the quasi-official SGI answer. > Thanks, > > rayan > > AI/NA/Theory, DCS, U of Toronto My pleasure. -- Jim Barton Silicon Graphics Computer Systems "UNIX: Live Free Or Die!" jmb@sgi.sgi.com, sgi!jmb@decwrl.dec.com, ...{decwrl,sun}!sgi!jmb "I used to be disgusted, now I'm just amused." - Elvis Costello, 'Red Shoes' --
blbates@aero4.larc.nasa.gov (Bates TAD/HRNAB ms294 x2601) (04/11/89)
Aren't there any other companies with multiprocessors? I would find the pains and hassle of having a SGI machine not worth the trouble. I like UNIX and 4BSD, but don't care for System V. AT&T needs to learn a lot from Berkeley. What about Connection, Convex, and others? Those are multiprocessor machines. -- Brent L. Bates NASA-Langley Research Center M.S. 294 Hampton, Virginia 23665-5225 (804) 864-2854 E-mail: blbates@aero4.larc.nasa.gov or blbates@aero2.larc.nasa.gov
madd@adt.UUCP (jim frost) (04/11/89)
>In article <89Apr9.160219edt.38129@neat.ai.toronto.edu>, rayan@ai.toronto.edu (Rayan Zachariassen) writes: >>We understand SGI is committed to SV. > >Sounds like religion - for SGI it's commercial reality, as it is for every >other successful computer vendor. DEC? Seriously, not every vendor who supported SysV was successful, nor (more to the point) does every successful vendor support SysV, even throwing out those companies who don't care about UNIX at all. Most of the very successful vendors support SysV but add a lot of the BSD functionality -- job control and sockets are the biggest ones; almost everything else just affects performance. >We will also consider any major OSF/1 feature which adds value >to the system. I don't think you'll see anything in OSF/1, which is going to be pretty vanilla IBM AIX; OSF/2 is supposed to have a lot of new functionality but that's vaporware for awhile. I'll be surprised to see OSF/1 out before this time next year no matter what the official word is. >The SGI ExtentFileSystem is >faster< than the BSD FFS. Could we get some info on your benchmark? I'm particularly interested in how each FS was tuned. I tend to believe the results considering the FS throughput our SGI's have, but tuning can be everything. I'd also like to know what you do to keep fragmentation down when the FS fills up; I'm curious. Our biggest complaint about SGI performance is that it degrades substantially over time. I'm fairly certain that this is a VM problem since it happens with every large application I've run, including some which have pretty clean usage and do *not* have this problem under 4.3 BSD. The system returns to its former spunkiness after reboot. I might expect that it's related to 4Sight except that logout/login doesn't correct the problem. Speaking of 4Sight, it would be very useful to some people if SGI would provide a method of accessing graphics without the window manager. 4Sight eats up a lot of memory ("heavyweight" as you say) as well as some graphics resources (particularly bitplanes) which applications could make use of. jim frost madd@bu-it.bu.edu
lamy@ai.utoronto.ca (Jean-Francois Lamy) (04/11/89)
How does one dump/restore an SGI server with about 4Gigs? (we currently have essentially unattended dumps at night over the network, using the Berkeley rdump/rrestore combo, to a pair of Exabytes). Is anything remotely similar possible with SGI machines?
madd@adt.UUCP (jim frost) (04/12/89)
> Aren't there any other companies with multiprocessors? Sure. Sequent, Encore, and IBM come to mind immediately. SGI beats all of them in price/performance although the Encore is set up for heavy use, not as a workstation, and performs very well as a large multiuser machine, making it very competitive in that kind of market. I hear bad things about Sequent reliability and IBM <-> UNIX still strikes me as an oxymoron even with AIX and OSF so you can basically throw them out. What it comes down to is "what are you looking for". SGI beats them all in the workstation and superworkstation market, no questions asked, competitive even without the geometry engine. Even though it's SysV, it's not very SysV and many BSD programs can be built with few changes. You can configure a 4D as multiuser but it probably won't support many. Encore is not in the workstation market, but instead has machines which support hundreds of online users in a very cost-effective manner. Unfortunately their best market is education where there isn't always a lot of money to be had. There are probably other companies which have similar products but I haven't dealt with them at all. Apologies if I missed any big ones. >What about Connection, Convex, and others? >Those are multiprocessor machines. I think you can throw out Connection. It's not a multiprocessor machine like you're accustomed to and the entire design of the system is such that it probably won't ever support UNIX (makes for good reading though -- twelve dimensional architecture). It's really a great thing for heavy-duty parallel computation tasks in batch mode, not interactive or multiuser. jim frost madd@bu-it.bu.edu
jmb@patton.SGI.COM (Jim Barton) (04/12/89)
In article <89Apr11.122420edt.38132@neat.ai.toronto.edu>, lamy@ai.utoronto.ca (Jean-Francois Lamy) writes: > How does one dump/restore an SGI server with about 4Gigs? > (we currently have essentially unattended dumps at night over the network, > using the Berkeley rdump/rrestore combo, to a pair of Exabytes). Is > anything remotely similar possible with SGI machines? You can buy officially supported Exabyte-style drives from SGI, and put your own 2Gb tapes in them. The 'bru' command should support dump/restore to this device, as well as tar and cpio. One of my personal favorites to save tape and speed the backup is to compress the data, for example: find . -newer lastbackup -print | cpio -oc | compress -v -f |\ rsh remote "dd bs=16k of=/dev/tape" Compression is slow, but Exabyte tape drives are slower, so there you are ... -- jmb
jmb@patton.SGI.COM (Jim Barton) (04/12/89)
In article <8904111520.AA05578@adt.uucp>, madd@adt.UUCP (jim frost) writes: > Could we get some info on your benchmark? I'm particularly interested > in how each FS was tuned. I tend to believe the results considering > the FS throughput our SGI's have, but tuning can be everything. I'd > also like to know what you do to keep fragmentation down when the FS > fills up; I'm curious. Send me some personal mail and I'll send you a copy of the benchmark I used. The test was done on a clean filesystem on both, with no other activity going on. Both systems were "stock" as delivered from the manufacturer. > Our biggest complaint about SGI performance is that it degrades > substantially over time. I'm fairly certain that this is a VM problem > since it happens with every large application I've run, including some > which have pretty clean usage and do *not* have this problem under 4.3 > BSD. The system returns to its former spunkiness after reboot. I > might expect that it's related to 4Sight except that logout/login > doesn't correct the problem. I've never seen this; do you have any quantitative data? What release are you running? Our big server (maddog) is a multi-user machine running builds, etc., all the time. We've never seen it slow down over time. If this really happens, I really want to fix it! > jim frost > madd@bu-it.bu.edu -- Jim Barton Silicon Graphics Computer Systems "UNIX: Live Free Or Die!" jmb@sgi.sgi.com, sgi!jmb@decwrl.dec.com, ...{decwrl,sun}!sgi!jmb "I used to be disgusted, now I'm just amused." - Elvis Costello, 'Red Shoes' --
zombie@voodoo.UUCP (Mike York) (04/13/89)
In article <8904111520.AA05578@adt.uucp> madd@adt.UUCP (jim frost) writes: >Our biggest complaint about SGI performance is that it degrades >substantially over time. I'm fairly certain that this is a VM problem >since it happens with every large application I've run, including some >which have pretty clean usage and do *not* have this problem under 4.3 >BSD. The system returns to its former spunkiness after reboot. I >might expect that it's related to 4Sight except that logout/login >doesn't correct the problem. We saw this problem with 3.0 on a 4D/70GT, but haven't noticed it since we've upgraded to 3.1. As I recall, SGI told us the problem was due to 4Sight and that rebooting *was* the only work around. Now, if SGI could only speed up picking on the 70GT to be as fast as picking on the 4D/20... -- Mike York Boeing Computer Services, Renton, Washington (206) 234-7724 uw-beaver!ssc-vax!voodoo!zombie
juancho@dgp.toronto.edu (John Buchanan) (04/13/89)
In article <30349@sgi.SGI.COM> jmb@patton.SGI.COM (Jim Barton) writes: >In article <89Apr9.160219edt.38129@neat.ai.toronto.edu>, rayan@ai.toronto.edu (Rayan Zachariassen) writes: >> During testing on the personal iris, some anomalies showed up that could >> be explained by the scheduler or VM being tuned for a single-user workstation >> environment. For example running a certain (non-graphics) program would >> cause lost ether packets and horrible response time on the iris, but the >> same program is apparently wellbehaved on other machines. Similarly, logging >> out of the PI causes lost packets. Anyone experienced similar anomalies >> on the 4D/2xx? Anyone using them for timesharing? > >Main problem is with the window manager, which is pretty heavyweight. For all >that nice display and all, it takes lots of memory, which has to be fought >over with the application you are running in a limited memory system. If >you really aren't interested in graphics, don't start the window manager, >and the performance will be very good (you >said< server, right?). Try >running your same application on a Sun 4 with NeWS and 8Mb of memory (assuming >you can get Sun to sell you one) and amuse yourself with the results. In the above text rayan forgot to metion that we were not using the console during much of this testing. (At least the testing that I was involved in.) We were trying to run a ray tracing package which is very floating point intensive. When I tried to log in it reminded me of the old days around here when 30-60 of us lived on a vax780. In short there was no real time response at all. This is a cause for concern since it does pertain to the server aspect of these machines. >I thought the net might be interested in the quasi-official SGI answer. > >> Thanks, >> >> rayan >> >> AI/NA/Theory, DCS, U of Toronto > >My pleasure. > >-- Jim Barton >Silicon Graphics Computer Systems "UNIX: Live Free Or Die!" >jmb@sgi.sgi.com, sgi!jmb@decwrl.dec.com, ...{decwrl,sun}!sgi!jmb > > "I used to be disgusted, now I'm just amused." > - Elvis Costello, 'Red Shoes' >--
rpaul@dasys1.UUCP (Rod Paul) (04/15/89)
At work we've been running 3 of the 4D series machines for just over a year now, our current configuration is: 2 4D/70G's 8M ram 2x380Mb drives 1 CS12 16M ram 1.2 gig drive serial-port-adapter 9-track Also on our net is an Abekas A60 (don't do graphics without one). We're gunna receive a personal next week, and upgrade all machines to 16M ram (NeWS is quite a hog)... The only bitch we've ever realy had with SGI has been the gen-lock boards not putting out REAL broadcast quality, thus the Abekas. Our CS12 has been rebuilt 1.5 times, kind of a bitch, but SGI have allways been real good about support, and the most down time we've ever had is about 4 days. We do a lot of 3D computer animation on these machines using Wavedick, sorry Wavefront, software. There are never more than about 4 users on the server at once, but it exports via NFS to the other machines and is accessed constantly. Our future plans will be to upgrade to 240's as servers and slap personals on them (if we find the personals reliable). In general I'm very happy with SGI, their libraries and the direction they're going with networks, i.e. distributed graphics library (DGL). I've found both the hotline and local service guy's very responsive and have even received the ocasional call from out west asking how things are going. Compared to the nightmares I've heard about SUN, APPOLO and DEC regarding servicing, SGI blows them away. Peripherals? Sure they're expensive, but I don't have to spend time dicking with drivers and finding out the hard way how well some product integrates. Let SGI do it first, and pay them for it. Also if it screws up, they'll replace it, probably quite fast. Guess I'm sounding like a sales guy, but of all the machines I've worked on in the past seven years, SGI are going in a decent direction, at least where computers and graphics are concerned (allthough I'm not sold on NeWS yet). Cheers, Rod. -- Rodian Paul | Big Electric Cat Public UNIX | Just say YES to UII ! ..!cmcl2!dasys1!rpaul |
msc@ramoth.SGI.COM (Mark Callow) (04/17/89)
In article <538@voodoo.UUCP>, zombie@voodoo.UUCP (Mike York) writes: > In article <8904111520.AA05578@adt.uucp> madd@adt.UUCP (jim frost) writes: > >Our biggest complaint about SGI performance is that it degrades > >substantially over time. I'm fairly certain that this is a VM problem > >since it happens with every large application I've run, including some > >which have pretty clean usage and do *not* have this problem under 4.3 > >BSD. The system returns to its former spunkiness after reboot. I > >might expect that it's related to 4Sight except that logout/login > >doesn't correct the problem. > > We saw this problem with 3.0 on a 4D/70GT, but haven't noticed it since > we've upgraded to 3.1. As I recall, SGI told us the problem was due to > 4Sight and that rebooting *was* the only work around. > > Now, if SGI could only speed up picking on the 70GT to be as fast as > picking on the 4D/20... > If 4Sight was the problem (and it was a problem on early versions of 3.0 due to memory leaks) it would be cleared up by logging out and logging in again. If a reboot was necessary something else was (is?) wrong. -- -Mark