celeste@coherent.com (Celeste C. Stokely) (05/09/89)
Paul Elliot is asking if he needs a more powerful server,server disks,
local memory, or local scsi disks in an environment of lisp hacking. The
answer is *YES*.
Paul is running 4.0 and I'm at 3.5, but the answers will be much the same.
I have 13 3/60s(diskfull), most actively developing in lisp, and 4 3/50s
(diskless) doing much simpler work (frame and email). I also have a 3/280
fileserver.
All the 3/60s have 20-24MB of memory, the 3/280 has 16MB. No lisp jobs or
compilations are allowed on the server. All compilations are on the 3/60s.
All 3/60 disks are SCSI, mainly WrenIV, but a few 141MB. All 3/60s have
from 60-100MB of swap locally. The 3/60s are now awesomely fast. They
used to have 12MB of memory and 30MB of swap and they choked on lisp.
The 3/280 has a super-eagle, a Hitachi 892MB, and a CDC Saber 1.2GB. The
eagle and Hitachi are each on a XY451. The saber is on a XY753. The server
is almost completely an NFS server. (It also handles email and news.) The
server is rarely overloaded, as a result.
The problem for us is the network. We have all these machines strung on 1
poor, tired cheapernet. My users constantly read and write 13-15MB files
over the net. Our net utilization "idles" at 25% and regularly peaks to
70%. The users say "the server is too slow" but I can absolutely see that
it's the net.
Real Soon Now we will be going to thick ethernet, and will be splitting
into 2 nets, load-balancing the heavy net users on each net.
So, the bottom line for highest productivity in lisp development for us
has been: Add all the memory you can, go diskfull, configure lots of
local swap space, and load-balance the net. A major, major win was also
saying "no compiles on the server". That speeded up NFS service
enormously.
We're about to spring for some sparc machines (desktop and server) to just
add some raw MIPS to the equation, too. The 60s and the 280 have been
great workhorses, but 12-16MIPS is pretty intoxicating when your compile
at 3MIPS takes 2 hours! ;-)
Hope this helps.
..Celeste Stokely
Coherent Thought Inc.
UUCP: ...!{ames,sun,uunet}!coherent!celeste Domain: celeste@coherent.com
Internet: coherent!celeste@ames.arpa or ...@uunet.uu.net
VOICE: 415-493-8805 FAX: 415-493-1555
SNAIL:3350 W. Bayshore Rd. #205, Palo Alto CA 94303cdr@decwrl.dec.com (Carl Rigney) (05/09/89)
In article <550@trlamct.oz> you write: >X-Sun-Spots-Digest: Volume 7, Issue 261, message 5 of 16 > >We currently have network running SunOS 4.0 consisting of: > >fileserver Sun3/260 16M memory xylogics 451 > 1 x fujitsu eagle > 1 x fujitsu swallow If you can put faster disks on this, get Sun's new SMD-4 controller, which is supposed to double disk throughput. I don't think it will work with Eagles, though. Seriously think about getting Sun's Pegasus CPU upgrade (the 68030 that was announced 4/13) - it has a reworked I/O cache. It's supposed to significantly improve I/O, as well as giving you 7 MIPS instead of 3. >With the following diskless clients. > >1 x client Sun3/60 12M memory (lisp programming) >3 x clients Sun3/50 4M memory (lisp programming) >2 x clients Sun3/50 4M memory (C & fortran) 4 MB for SunOS 4.0 and Lisp is intensely masochistic. Are your kernals configured as small as possible? Someone just posted an article to Sun-Spots that says a diskful 4MB machine runs SLOWER than a diskless 4 MB, and a diskless 8MB machine runs twice as fast as either. That might mean you have to go 3rd party for the 3/50 memory, though. If you have the money you might see about replacing them with 8 MB 3/80's (a 68030 with cheaper memory), or even 16 MB. Another possibility you might investigate is getting one of CDC's new Wren Runner SCSI drives - these have 10.5ms access time compared to the 20-30 MB for Sun's dogs. They're also synchronous, which only helps if you have a 3/80 or 4/60; the older suns only support asynch SCSI (boo). But more memory is better than more disk, generally. >The problem is getting access to swap space on the disk. Have you split the /export/swap across both disks on your server? Also, have you looked at your ethernet to see what's going on? How much load does traffic show? Would segmenting your net cut down, or are your machines all that's on your wire? >The alternatives appear to be : > >1. New high speed disk for existing server. A waste unless you get the better controller, I suspect. >2. New more powerful fileserver with high speed disk. Upgrading the 3/260 (or just buying a 3/470 or 480) should be OK. Upgrading is cheaper, probably, but Sun's being very aggresive about Pegasus pricing. >3. Providing 3/60s or equivalent with 20 Megs for the lisp programmers > and upgrading the 3/50s. 16MB 3/80s might be good. Beware that that's all the memory they can have, though. If you have the money and your lisp works on the SPARC architecture you might consider a 16MB 4/60 - 12 MIPS for the price of a 3/60, with synchronous SCSI! Putting a Wren runner on one of those ought to be a good time! Of course, SPARC binaries are larger than 680x0, so you'll need that extra memory... >4. Providing local SCSI disks giving each client local swap space > (dataless client configuration) This is the least effective answer, probably. Especially if you buy your disks from Sun. >I personally feel that a more powerful fileserver is still going to suffer >a bottleneck at the disk with clients fighting for access to swap space, >even if a new server provided twice the disk I/O speed compared to the >existing server. You want two fast disks so you can spread the load among them, or better yet two controllers. First put as much memory as possible in your workstations, then buy a 3/480-32 with 2 SMD-4 controllers and 4 892-MB SMD Disks, or upgrade your existing 3/260 to as close to that as you can afford. For the fullblown system you're probably looking at ~ $100k before discount, but your friendly Sun sales rep would know better than I. And get Perfect Byte (or *someone*'s!) 2.3GB 8mm Tape drive for backups. Caveat! Your mileage could differ with the 3/480, 4/60, and 3/80 - after intense study we're getting the first two, but I haven't had hands-on experience yet. I'll be interested to hear what everyone else has to say. --Carl Rigney cdr@amdcad.AMD.COM {ames decwrl gatech pyramid sun uunet}!amdcad!cdr 408-749-2453 The Network is the network. The Computer is the computer. Sorry about the confusion.
david@sun.com (David L. Kensiski 929-8844) (05/10/89)
In SunSpots v7 n261 (25 April 89), munnari!trlamct.oz.au!paul@uunet.uu.net (Paul Elliott) said: > We will be adding some extra clients over the next few months and wish to > upgrade the existing hardware and/or buy a more powerful server. > > What we would like to know is where to spend the money. Options 3 (upgrading the [RAM on the] 3/50s) and 4 (providing local SCSI disks), which is actually option 5 (some combination of the above), would probably net you the biggest change in system performance. My experience with SunOS 4.0 is that you want more than 4 MB of RAM on your workstation, especially if you use suntools. Upgrading the RAM on the 3/50s would reduce the number of page faults you will receive, keeping you from swapping as much. Sun doesn't provide RAM upgrades to 3/50s, but there are a number of third party vendors that do. I recently read that Helios offers a RAM upgrade that does not require soldering to the CPU, hence it does not invalidate the Sun warranty or service contract. Local SCSI disks will keep any swapping that does happen off the network. A small disk is all you need if you install a dataless configuration - 50 MB should do fine. In a former life, all of our 3/50s had local SCSI disks, ranging from 70 to 300 MB. The 20 to 30 MB partitions left after putting root and swap on the 70 MB disks was basically worthless - you could put man pages or emacs source on it, but you wouldn't want much more since, in my opinion, 3/50s don't make good file servers. David L. Kensiski 1772 Tribute Road Martin Marietta Data Systems Sacramento, CA 95815 UUCP: uunet.uu.com!mmsac!david VOICE: (916) 929-8844 UUCP: sun.com!sacto!mmsac!david
lmb@vicom.com (Larry Blair) (05/10/89)
In article <550@trlamct.oz> Paul writes:
=X-Sun-Spots-Digest: Volume 7, Issue 261, message 5 of 16
=fileserver Sun3/260 16M memory xylogics 451
= 1 x fujitsu eagle
= 1 x fujitsu swallow
The 451 is slow junk. Your problems with your server capacity are related
to the I/O bottleneck. You can easily triple I/O throughput with a
caching controller. Sun is selling the new SMD-4, which I haven't seen
yet. I've used both the Interphase 4200 and the Ciprico 3220 and they
work great. Be sure to do a tunefs -a 16 to get the full caching
advantage. I have been absolutely amazed at how much capacity our 3/160
w/16mb have exhibited since we started to use the 4200's.
There are several other things that you can do to improve throughput. Run
more nfsd's. If you are running 4.0.1, get more memory. If you are
running 3.X, increase the number of buffers.
=4. Providing local SCSI disks giving each client local swap space
= (dataless client configuration)
This may be a good idea for the Lisp machines. They may not run faster
after you fix your throughput problem, but they won't slow down the other
clients.
=Looking at the alternatives above we could spend money on a more powerful
=fileserver or a similar amount on additional memory and local disks for
=each client. For money spent which alternative gives the best performance
=improvement.
I would wait a few months and see what comes out on the market. There are
several new companies working hard on high-performance, low cost
fileservers.
=I personally feel that a more powerful fileserver is still going to suffer
=a bottleneck at the disk with clients fighting for access to swap space,
=even if a new server provided twice the disk I/O speed compared to the
=existing server.
Right-o. The new controllers will at least double your throughput. I doubt
greatly if you are cpu bound.
--
Larry Blair ames!vsi1!lmb lmb@vicom.comhedrick@geneva.rutgers.edu (Charles Hedrick) (05/10/89)
I don't think anybody really has a complete answer to this question. However I have at least some informed comments. First, fileservers in general may or may not be a waste of money, but *your* fileserver seems to be. You have a 3/260 with a XY451 controller. Our results (which agree with Sun's configuration guide, by the way) suggest that a 3/260 isn't all that much faster than a 3/160 when used as a file server. (It's a dandy machine for real computing though.) Furthermore, it is being crippled by the XY451. You'd almost certainly get better throughput from one of the 3rd party fast SCSI disks on a 3/60 than from this file server, and at a fraction of the cost. Both Sun and other tests agree that the Sun shoebox is similar to network access to an unloaded file server. Sun originally (back when the 3/50 first came out) said that both were about 1/2 the speed of a "real" SMD disk. However SCSI technology has improved since those tests. (Those were the days of the XY451 and Eagles.) It appears that with good embedded SCSI disk controllers you can now get almost as good results out of local SCSI disks as SMD (indeed some results suggest better, but they tend to be comparing with less than optimal SMD controllers), at least in single-user configurations. In the worst case, it seems that swapping from a file server is a disaster. The worst case assumes - that all of your clients are swapping their hearts out at the same time - that you're comparing to local swapping disks using the best SCSI technology, and not the old Sun shoeboxes - that you have no more I/O bandwidth in your file server than on a client. In that case, 6 clients are going to see less than 1/6 the swapping performance they would see if they had local disks. However, you may not have that worst case. Here are things which may combine to make a file server reasonable: - your clients may not all need to swap at the same time. If they are using Lisp, this means you've put lots of memory in them. Note that this strategy has to be used on all your machines. If you have a couple of machines without enough memory, then they may swap continuously. In that case they'll degrade even machines that have enough memory. Enough memory means enough to keep the swap rate down, not to totally eliminate swapping. Even the 12M machine will need to swap now and then, and when it does, it will get killed by the 4M machines. - you may be able to afford more I/O throughput on the server than on an individual client. E.g. you may be able to use more disks in parallel, or more memory as a cache (both on the controller and in main memory). You'd certainly do better with one of the new generation of SMD controllers, particularly if you're running a release of SunOS that doesn't do overlapped seek on the Xylogics controllers. At the moment we are still using 3/50's and 3/60's in diskless configurations. However all of the proposals I've made so far for Sparcstation 1's have involved local 100MB disks.
david@sun.com (David L. Kensiski 929-8844) (05/10/89)
Bob Sutterfield <bob@cis.ohio-state.edu> writes: >sacto!mmsac!cygnus!david@sun.com (David L. Kensiski) writes: > I recently read that Helios offers a RAM upgrade that does not > require soldering to the CPU, hence it does not invalidate the Sun > warranty or service contract. > It's the other way around: Helios is the only memory upgrade that I > know of that requires soldered modifications to the 3/50 board, > rendering Sun unwilling to service it if it breaks. Page 38 of the April 17 issue of InfoWorld contians the article: Helios Announces Reversible Add-On RAM Cards For Sun-3/50 The article explains that Helios has previously offered boards that were soldered to the CPU board, but also that Helios just announced a new reversible board that is not in violation of Sun's warranties.
gerry%seq1.keele.ac.uk@nsfnet-relay.ac.uk (G.D.Pratt) (05/11/89)
Hi, Does anyone have experience of using anything other than a Sun as a fileserver for discless clients? This would be everthing - nd, root and NFS mounts. I saw some discussion about about using a Gould but I assume this was just for NFS mounts. tia, gerry pratt - workstation support - university of keele - england email: gerry@uk.ac.keele.seq1
aarons%cogs.sussex.ac.uk@nsfnet-relay.ac.uk (Aaron Sloman) (05/11/89)
munnari!trlamct.oz.au!paul@uunet.uu.net (Paul Elliott) writes: [I've abbreviated quite a bit] > fileserver Sun3/260 16M memory xylogics 451 > 1 x fujitsu eagle > 1 x fujitsu swallow > 1 x client Sun3/60 12M memory (lisp programming) > 3 x clients Sun3/50 4M memory (lisp programming) > 2 x clients Sun3/50 4M memory (C & fortran) > The performance of the above network can deteriorate markedly as swapping > activity increases. > > We will be adding some extra clients over the next few months and wish to > upgrade the existing hardware and/or buy a more powerful server. > > What we would like to know is where to spend the money. Here is an unconventional answer: Do as much of your AI-type programming as possible on a powerful central server with a lot of memory (> 32 Mbytes if possible) and use your 3/50s and 3/60s only to manage windowing and graphics. This works fine with a networked window manager. This recommendation is based on the following assumptions: 1. each user is mostly typing into an editor and using a relatively small working set much of the time, but occasionally requires a large working set, causing paging on a small machine. 2. programs are rarely CPU-bound (not true if you are doing image processing or neural net simulations, for example). 3. each user is going to need a garbage collection (GC) every now and again, and when that happens (especially on a 4 Mbyte machine running a big program) there will be an awful lot of paging across ethernet to the server. This will slow EVERYTHING down. 4. Disk traffic is enormously reduced if enough memory is available during a GC to avoid paging. 5. If a lot of the memory is allocated to a process when it requires a GC this will generally interfere with other processes FAR less than all the paging would. 6. Assuming (2) above, the gains obtained from 5, plus the fact that the server has a much faster processor, considerably outweigh the losses resulting from having to share the server CPU. (This argument becomes even stronger if the server is a Sun-4, or something even more powerful.) 7. The only way to get equivalent performance without putting all your lisp processes on the server is to buy so much memory for each diskless machine that it does not need to page during a GC. This will generally be a far more expensive solution. The above argument will not apply to all patterns of work. Eg. if your program is doing a LOT of graphics and screen manipulation, doing it over ethernet from the server could be worse than the paging during garbage collections. However, my belief is that for many kinds of teaching and research, the best approach is to time share a large powerful central machine using relatively cheap workstations to handle windowing and graphics. We do this, and even have some VDU users logging through to the server. Aaron Sloman, School of Cognitive and Computing Sciences, Univ of Sussex, Brighton, BN1 9QN, England INTERNET: aarons%uk.ac.sussex.cogs@nsfnet-relay.ac.uk aarons%uk.ac.sussex.cogs%nsfnet-relay.ac.uk@relay.cs.net JANET aarons@cogs.sussex.ac.uk BITNET: aarons%uk.ac.sussex.cogs@uk.ac or aarons%uk.ac.sussex.cogs%ukacrl.bitnet@cunyvm.cuny.edu UUCP: ...mcvax!ukc!cogs!aarons or aarons@cogs.uucp
bob@cis.ohio-state.edu (Bob Sutterfield) (05/11/89)
sacto!mmsac!cygnus!david@Sun.COM (David L. Kensiski 929-8844): Page 38 of the April 17 issue of InfoWorld contians the article: Helios Announces Reversible Add-On RAM Cards For Sun-3/50 The article explains that Helios has previously offered boards that were soldered to the CPU board, but also that Helios just announced a new reversible board that is not in violation of Sun's warranties. About 20 minutes after I sent my message I went to pick up the day's paper mail, which contained the May 1989 SunObserver with a description of just such a product. I guess I'm just behind the times :-)
sgf@cfm.brown.edu (05/16/89)
The basic question here is how to most efficiently distribute disk, memory, and server cycles. You want to avoid buying another server or upgrading CPU whenever significant performance gains can be had through cheaper means. For both OS 3.5 and 4.0: Most people want at least an 8MB machine. Except for the sys-admin, there isn't really anyone who can use 4MB painlessly in today's application and development environments. 4MB machines put an unfair paging load on the server. For OS 3.5: The server bottleneck is the >>ie<<, not CPU or disk (well, disk can be if you're runniing slow controllers). Let's assume that you've got a 451 with 2 SMD disks and a Ciprico (my preference in SMD controllers) with 2 SMD-E disks on a 3/280 that serves 4MB 3/50's. If the 50's average VM usage is 12MB then neither the CPU nor the disk I/O capacity of the 280 will be anywhere near max'ed. It'll be the poor 16b ie chugging along on its slow driver which slows the server (the bulk of the server load is nd page faults with some swapping). (AND there's no point in a server with more than 4MB if you're not allowing user logins). For OS 4.0: (...and the above descibed hardware...) (I haven't been willing to try this until I upgrade the 50's to at least 8MB.) There's been a lot of talk from people at Sun claiming that 4MB 50's can run well under 4.0 with proper tuning. I think that they can limp along (I have used them, but I don't want them on my network, which is CFD research...). The most painful effect of a 4MB 4.0 50 is on the server load. Even if the buf cache on the server goes into memory pig mode (makes the disk I/O faster, saves some CPU), 4.0 has the nasty effect of increasing the server CPU required for a client page fault (no RG, NFS can't possibly run in as few cycles as nd) to the point that a 3/280 CPU max's out before the ie (I postulate this based on some hard performance monitoring on two networks: 3.5 3/280 serving 4MB 3/50's (our network); multiple 4.0 4/280's serving 8MB 3/60's (browncs). I hope to confirm it soon by installing 4.0.1 on our network just before popping in the memory upgrades.). I assume that a 4/280 with enough load will once again max out the ie before anything else (anybody have a driver for the 32b Interphase Eagle? :-) The bottom line is that (especially given the product line pricing structure - gee, isn't it a shame when companies get too big...): 1. Check to see where your bottlenecks are and what is causing the load (paging or normal NFS operations). 2. If the load is paging and the limit is the ie definitely put more memory in the clients. 3. If the limit is disk I/O get a faster controller and/or more disk and distribute the client root/swap evenly among all disks. (and if the load is paging put more memory in the clients.) There is basically no reason for a server to be limited by disk I/O. 4. If the load is paging and the limit is CPU (especially under 4.0) put small (minimize cost) SCSI disks on the clients for swap space (after being sure that they have a reasonable amount of memory). There are plenty of suppliers of small (5.25" form factor) enclosures with integral power supplies, so grow your own shoeboxes. One thing that irks me is that (as an example) Sun COULD sell a modified 3/280 with >>3X<< the server performance capacity for an addtional $12,600 list, and at the SAME MARGIN (no additional disk; based on the Dec. 1, 1988 price list). The same sort of thing could be done for any of the servers (of course it would mean selling fewer servers :-(. Maybe now that X-terminals are getting popular Sun will take the NeWT (NeWS Terminal) out of mothballs... Sam Fulcomer, Head Zookeeper Brown University Center for Fluid Mechanics, Turbulence and Computation sgf@cfm.brown.edu uunet!brunix!sgf
bob@cis.ohio-state.edu (Bob Sutterfield) (05/17/89)
In SunSpots v7 n280, sacto!mmsac!cygnus!david@sun.com (David L. Kensiski 929-8844) writes:
I recently read that Helios offers a RAM upgrade that does not
require soldering to the CPU, hence it does not invalidate the Sun
warranty or service contract.
It's the other way around: Helios is the only memory upgrade that I know
of that requires soldered modifications to the 3/50 board, rendering Sun
unwilling to service it if it breaks.bdr@earth.cray.com (Brian Reitz) (05/18/89)
In SunSpots v7 n280, sacto!mmsac!cygnus!david@sun.com (David L. Kensiski 929-8844) writes: >> I recently read that Helios offers a RAM upgrade that does not >> require soldering to the CPU, hence it does not invalidate the Sun >> warranty or service contract. > >It's the other way around: Helios is the only memory upgrade that I know >of that requires soldered modifications to the 3/50 board, rendering Sun >unwilling to service it if it breaks. Well yes and no. The early Helios boards were of the soldered type but their new design is of the NO Soldering type. All you do is pull four PAL's (U800 - U803) and pull the two PLCC's (SUN PCHK (U1020) and SUN MMU (U3201)). Plug their board into the sockets that the PLCC's came from, and insert the PLCC's back into the sockets on the Helios board. Toss the PAL's (well you best save them for a rainy day) change the EEPROM and your back in business!! The Helios board will fit with either the old dimple tops or the new style 3/50's. Brian Reitz Cray Research Inc. bdr@cray.com