ray@dsiramd.dsir.govt.nz (Ray Brownrigg) (06/20/89)
Having been reading this group for just a few weeks now, I suspect I might have just missed discussions on some or all of the following. If so I would appreciate mail summarising the findings. Otherwise, please e-mail me and I will summarise. 1) Recent benchmarking discussions, particularly the SPARC/MIPS and DS3100/M-120 comparisons have mostly failed to mention multi-user capabilities, although I realise the context switching, register architecture, and cache buffering threads may have resulted from an initial discussion of this topic. Does anyone have experience of the relative multi-user performance of the SS1 vs SS330 vs RC2030 vs M-120, with say VAX/780 (or MVII) as a reference? I realise this may well be very environment dependent, but in a general scientific/academic environment it is basically impossible to "take your application to a sample system and use that as YOUR benchmark", when your 'application' is say 4 scientists out of a pool of 20, wanting to do interactive editing, compiling, executions, as well as other package-based applications. This of course leads to: 2) What standard benchmarks purport to give a feel for the multiuser capabilities of a system? I understand the MUSBUS and AIM benchmarks may well do this, but I do not have much detailed information on the benchmarks, and even less information (i.e. none) on results for the above-mentioned systems. My specific interest is in making comparisons between VAX/780 (or MVII) running VMS and a small RISC workstation (say SS1 or RC2030). I note that the MIPS Performance Brief Issue 3.8 (June 1989) does not mention either of these tests. Now given this is comp.arch, my real question is the following: 3) Are there any *architectural* reasons for being wary of potential multi-user performance of say a SPARCstation 1, or am I justified in saying: 'Well, if you have 12 VUPs, then the system should be capable of delivering (say) 2 VUPs to each of 6 users, that should be enough to convince them to move from sharing a MicroVAX II under VMS.'? For example, I realise that to give each user 2 VUPs, then each user should have something like 4 (or 6, or 8, depending on the applications used) megabytes of memory, and 48 Mb is not possible on SS1 yet, but would a 16Mb system be significantly insufficient (given the statistical benefits of multiplicity)? Please e-mail any data and I will summarise. -- Ray Brownrigg domain: ray@dsiramd.dsir.govt.nz Applied Maths Div, DSIR ACSnet: ray@dsiramd.nz[@munnari] PO Box 1335 System: OLIVETTI/AT&T 3B2/400B+, System V R3.0 Wellington, New Zealand "unx -rules -OK"
ray@dsiramd.dsir.govt.nz (Ray Brownrigg) (07/06/89)
Summary of responses to my posting regarding multi-user benchmarks, and the feasibility of replacing some of the services offered by a VMS-based MicroVAX II with (say) a SPARCstation 1 running (say) 6 simultaneous users at peak load. I have retained the authors words, just edited out repeated or less important information. ------ From: chris@yarra.oz MUSBUS is a UNIX benchmark - no way to run it under VMS. Same is with AIM to my knowledge but I have no practical experience with it. ... AIM 2 has several fundamental flaws ... AIM 3 is much better. ... MUSBUS is in public domain and you can easily obtain the source code to run it yourself. You can also easily customise it in the sense of changing your job mix to reflect your real expected load. ... The benchmark is also a good tool for shaking and checking quality of UNIX ports. You just increase and increase load and wait when it breaks. Some architectures and UNIX ports are known to break supprisingly easy above 64 simulated users. By the way to really test a machine you should run the benchmark way past the saturation ankle on CPU utilization and elapsed time degradation curves. I suggest that you run it up to the number of simulated users yielding elapsed time degradation value of at least 2.5. ... Many a hardare vendor will only publish results on the linear section of the curves which is pretty much useless for the purpose of assessing where the machine saturates and how severely. Chris Jankowski chris@yarra.oz chris@yarra.oz.au Pyramid Technology Australia - Melbourne ------ From: khb@Sun.COM Personally I think MUSBUS and AIM are nice but misguided. ... No one in their right mind buys a DS3100 or a SS-1 as a multiuser machine. Neither has the IO bandwidth, or any of the other goodies that dominate in real multiuser designs. But then I think of the Convex C120 as an adequate single user machine. ... The old mainframes had 1MIPS of CPU power, but could handle 100 users. The fact that there were dozens of minicomputer class machines (PP's, channel controllers, etc) made this "magic possible". A workstation is designed (primarily) to handle a single person quite well. DS3100 and SS-1 rely on SCSI IO. SS-1 has a buss, so it is theorectically possible to have more suitable IO for a server, but it seems unlikely that an IPI device will be made available:> The better solution is to buy a machine designed to be a server. VMS is more efficient about sharing resources than unix. ... ... Single user machines are optimized to handle a few context switches quite fast. Serious multiuser machines face 100's or 1000's of context switches a second ... hence different MMU designs are called for (among other considerations, like IO). ... Keith H. Bierman Marketing Technical Specialist ! kbierman@sun.com ------ From: rnovak@mips.com ... AIM and Neal Nelson are proprietary benchmarks. You typically have to pay large fees to see results from these benchmarks (or whisper in your salesman's ear). MUSBUS is probably the best, but I haven't had a chance to run that on the M/120 yet. ... Keep your ear to the ground about SPEC. Robert E. Novak MIPS Computer Systems, Inc. ------ From: mash@mips.com ... If you look at MIPS Magazine over the last few months, here is some (albeit not great) data on mulit-user performance. ALso, UNIX Review magazine's Tested Mettle reports include multi-user, and so do Digital Reviews. Note that the SS1 is 12 Dhrystone-mips, not VUPS. It's more like 9 VUPS. In the past, execution of kernel code is not something that Sun-4s have excelled at, due to a) use of virtually-tagged caches that needto be flushed more often than you'd like. b) register windows not matching the way the kernel works, and (minor) c) cost of saving large register file. john mashey mash@mips.com ------ As a rejoinder, and perhaps to elicit further comment, we are proposing to provide the RISC workstation as a *supplementary* service to the MVII, primarily for heavy CPU users (the MVII is charged for on a CPU-second basis). Given that in the first instance access to the workstation will be through ASCII terminals (perhaps some TEK4014 emulators) over ethernet (a TCP/IP capable terminal server), I suggest that the total cpu load may be not too much more than a single intensive user can apply using multiple active windows. If this is the case then the problem boils down to one of context switching - a single user cannot be typing into several windows at any one time. Analyzing the situation though, 6 users all typing together would result in a peak in the order of 50 (?) context switches per second, surely within the capability of a 12-16 (integer) MIPS processor? -- Ray Brownrigg domain: ray@dsiramd.dsir.govt.nz Applied Maths Div, DSIR ACSnet: ray@dsiramd.nz[@munnari] PO Box 1335 System: OLIVETTI/AT&T 3B2/400B+, System V R3.0 Wellington, New Zealand "unx -rules -OK"
ray@dsiramd.dsir.govt.nz (Ray Brownrigg) (07/26/89)
(Apologies for the repost, but this apparently didn't get as far as it should have.) Summary of responses to my posting regarding multi-user benchmarks, and the feasibility of replacing some of the services offered by a VMS-based MicroVAX II with (say) a SPARCstation 1 running (say) 6 simultaneous users at peak load. I have retained the authors words, just edited out repeated or less important information. ------ >From: chris@yarra.oz MUSBUS is a UNIX benchmark - no way to run it under VMS. Same is with AIM to my knowledge but I have no practical experience with it. ... AIM 2 has several fundamental flaws ... AIM 3 is much better. ... MUSBUS is in public domain and you can easily obtain the source code to run it yourself. You can also easily customise it in the sense of changing your job mix to reflect your real expected load. ... The benchmark is also a good tool for shaking and checking quality of UNIX ports. You just increase and increase load and wait when it breaks. Some architectures and UNIX ports are known to break supprisingly easy above 64 simulated users. By the way to really test a machine you should run the benchmark way past the saturation ankle on CPU utilization and elapsed time degradation curves. I suggest that you run it up to the number of simulated users yielding elapsed time degradation value of at least 2.5. ... Many a hardare vendor will only publish results on the linear section of the curves which is pretty much useless for the purpose of assessing where the machine saturates and how severely. Chris Jankowski chris@yarra.oz chris@yarra.oz.au Pyramid Technology Australia - Melbourne ------ >From: khb@Sun.COM Personally I think MUSBUS and AIM are nice but misguided. ... No one in their right mind buys a DS3100 or a SS-1 as a multiuser machine. Neither has the IO bandwidth, or any of the other goodies that dominate in real multiuser designs. But then I think of the Convex C120 as an adequate single user machine. ... The old mainframes had 1MIPS of CPU power, but could handle 100 users. The fact that there were dozens of minicomputer class machines (PP's, channel controllers, etc) made this "magic possible". A workstation is designed (primarily) to handle a single person quite well. DS3100 and SS-1 rely on SCSI IO. SS-1 has a buss, so it is theorectically possible to have more suitable IO for a server, but it seems unlikely that an IPI device will be made available:> The better solution is to buy a machine designed to be a server. VMS is more efficient about sharing resources than unix. ... ... Single user machines are optimized to handle a few context switches quite fast. Serious multiuser machines face 100's or 1000's of context switches a second ... hence different MMU designs are called for (among other considerations, like IO). ... Keith H. Bierman Marketing Technical Specialist ! kbierman@sun.com ------ >From: rnovak@mips.com ... AIM and Neal Nelson are proprietary benchmarks. You typically have to pay large fees to see results from these benchmarks (or whisper in your salesman's ear). MUSBUS is probably the best, but I haven't had a chance to run that on the M/120 yet. ... Keep your ear to the ground about SPEC. Robert E. Novak MIPS Computer Systems, Inc. ------ >From: mash@mips.com ... If you look at MIPS Magazine over the last few months, here is some (albeit not great) data on mulit-user performance. ALso, UNIX Review magazine's Tested Mettle reports include multi-user, and so do Digital Reviews. Note that the SS1 is 12 Dhrystone-mips, not VUPS. It's more like 9 VUPS. In the past, execution of kernel code is not something that Sun-4s have excelled at, due to a) use of virtually-tagged caches that needto be flushed more often than you'd like. b) register windows not matching the way the kernel works, and (minor) c) cost of saving large register file. john mashey mash@mips.com ------ As a rejoinder, and perhaps to elicit further comment, we are proposing to provide the RISC workstation as a *supplementary* service to the MVII, primarily for heavy CPU users (the MVII is charged for on a CPU-second basis). Given that in the first instance access to the workstation will be through ASCII terminals (perhaps some TEK4014 emulators) over ethernet (a TCP/IP capable terminal server), I suggest that the total cpu load may be not too much more than a single intensive user can apply using multiple active windows. If this is the case then the problem boils down to one of context switching - a single user cannot be typing into several windows at any one time. Analyzing the situation though, 6 users all typing together would result in a peak in the order of 50 (?) context switches per second, surely within the capability of a 12-16 (integer) MIPS processor? -- Ray Brownrigg domain: ray@dsiramd.dsir.govt.nz Applied Maths Div, DSIR ACSnet: ray@dsiramd.nz[@munnari] PO Box 1335 System: OLIVETTI/AT&T 3B2/400B+, System V R3.0 Wellington, New Zealand "unx -rules -OK"