bass@dmsd.UUCP (John Bass) (04/24/86)
Subject: Re: Delphi Mac Digest V2 #11 Summary: The art of benchmarks Sometimes it would be nice to erase ones posting, after a talk with Micah we understand each other a bit better. I had assumed Brecher had a closer relationship to Micah -- this was in error. Thus some of my anger was miss directed and not deserving the beating I gave Micah. They have a quality product and like the rest of us are just interested in shipping value for the $$$ - I am sorry for any problems caused. My last article was written as something of an offensive defense to an ongoing series of comments by Mr Brecher which unfairly cast the MacSCSI product in an unfavorable light (as well as derivative products shipped by other dealers such as Warp 9). Generally it is not good practice to for one vendor to publicly knock another vendor -- it makes a lot of bad feelings and bad press. Having not followed this group for several weeks (we have been on and off the net because of our own down time and trashed news from both our feeds) I was more than upset to find Mr. Brecher promoting HIS/MICHA's product with a campain specifically against the MacSCSI product. I thus gave both Micah and Brecher a hard rap for the BS since they are coming from behind as a late starter in the disk drive upgrade game --I felt Mr Brechers comments were specifically directed to upset buyers and disrupt the established reputation of Warp 9 and my product. Micah is specifically targeting users and dealers upset by Warp 9's direct sale to HighTech end users who don't need to pay for dealer support. There is a lot of moaning about high prices and support, the two go hand in hand since support staff are often more expensive than the product. Both Warp 9 and I feel that technical users (who do not need the support and overhead of the local sales team) should be able to buy product at a fair price (IE a single markup to cover the OEM's overhead). We advise both users and dealers that if they REALLY need detailed SUPPORT then either buy through a dealer and get the hand holding they need or hire a local guru to do the installation and support at some fee that is mutually agreeble. This is called unbundling support from the basic product price. This practice also drives prices down and makes margins lower somewhere -- either at the dealer or OEM. Three side effects result -- 1) HighTech buyers get good deals and are generally happly, 2) LowTech buyers with HighTech friends make out ok as well, 3) LowTech buyers looking for detailed help from the OEM's are not likely to get the kind of support they need. LowTech buyers can sometimes get good service and support from their local store or dealer at normal shop rates or as goodwill. Many stores are rightfully upset when the LowTech buyer expects them to spend hours of support time as goodwill for a product they didn't sell -- they would be smarter to be upfront and set a reasonable time&materials rate for ALL support and shift more into the services role than the sales role -- thus using support and service as a revenue center in their store instead of an overhead dept. This can improve the customer base since they will draw sales from stores that are less customer service oriented. Some users will be confused for having to PAY for what was a FREE service -- this can be handled by discounting within the store and/or offering "support and training coupons" when sales are made at list. Some stores are VERY upfront about their VALUE-ADDED role in the market place and stress the support they offer at LIST price vs. support from Cash&Carry discount stores. A lot of customer awareness needs to be generated in this area. My posting generated about the same discomfort in the management at Micah as I had from Brecher's ongoing remarks -- they called here HOT screaming lawsuits, bad faith, false statements, etc to one of my staff while I was gone. When I returned Tony Roumell's call and we both understood that each of us don't like this kind of BS -- several things were settled. First Mr.Brecher is not an on staff employee of Micah but a contractor selling his own services. Second that both of us would suggest to Mr Brecher that as an agent of Micah he should not to place either Micah or us in such a postition again where one or both of us need to defend ourself from a highly negative public posting by the other parties agent(s). Third that all parties (Micah, Warp 9, DMS Design, and Fastime) endevor to ship a quality product although we disagree on required levels of support in various markets and other issues about discount direct sale and value-added services from OEM's and dealers. Tony corrected several statements I had made in error. They are now in full production having shipped "a couple hundred" units instead of the "several dozen" I was aware of. He also claimed to be 100% mac plus compatable -- I don't know if this means that Micah's hostadapter just works with a mac plus or that their hostadapter provides a 100% hardware and software compatable external mac plus scsi port on a 512k system like our MacSCSI Plus hostadapter (anybody who has a Micah drive know?). I also called Mr Brecher and tried to explain a much different view point for setting performance and interleave values than his. His goal is to build THE BENCHMARK that clearly states WHO IS BEST based on HIS definition of a STANDARD WORKLOAD. I generally distrust ANY benchmark that has someone elses FIXED view of a STANDARD WORKLOAD. System usages vary too greatly between users, applications, and system environments. More on this later. The following is a direct reply to various comments from Mr Brecher found in several articles in the Delphi Digest that promted the heated posting. The posting resulted in a number of questions by people that called/wrote me -- I will try again to explain my alternate viewpoint on this issue of performance and interleave values. > From: BRECHER (6660) > Subject: Re: Macintosh Hard Drives > Date: 20-MAR 05:29 MUGS Online > > .... If speed is a criterion, forget > Warp 9; it uses basically the highly inefficient MacSCSI driver > software (it also requires a boot floppy). For speed among > currently-available internal drives, consider HyperDrive or > MicahDrive. Informal tests reported to but not verified by me > indicate MicahDrive to be somewhat faster. The MicahDrive uses 1:1 > sector interleave (i.e., no interleave); I believe that current > HyperDrives use 2:1. .... > > Claimer: I wrote the MicahDrive ROM/driver/manager. There was NO need to single us out other than Warp 9 is the 2nd largest shipper of Mac internal disk subsystems behind General Computer. We are also the lowest cost since we do direct mail order sales to technically self-supporting end users -- thus bypassing the 40% markup that the store/dealer needs to pay their overheads (this by the way has many dealers red hot across the US). If speed were the ONLY requirement in racing any land vehicle everyone would be using some sort of rocket with dozens of ways to try and keep it on the ground. In autocross racing I prefer my hot corvair ... it handles quite well both on the track and in the hills -- it has ample horsepower (about 200) and a light weight (about 2200lbs). This makes it quite responsive against mussle cars with large block V8's that weigh much more and handle poorly -- for drag events I would use another car more suitable for drags. Atlas rockets perform poorly in autocrosses, and would be great in some class of drag events if you could refit and refuel them between heats (not an easy chore). For all around driving my Corvairs are great on the street, track and hills -- and cost much less than the puesdo "FAST" cars that have "STATUS". Nothing about disk subsystem performance is as cut and dried either! As for the various versions of our MacSCSI drivers, each performs quite well in the intended application, each version better than the previous -- all are VERY COST COMPETITIVE. Mr Brecher's coments about "the highly inefficient MacSCSI driver" are both WRONG and OUT OF LINE. We set the hardware interleave slower for a reason -- given a reasonable controller and properly tuned transfer loop the MacSCSI Pin board will transfer faster than a 4:1 interleave instead of the 11:1 (or slower if single sector) default we ship. We published the driver as single sector because several of the early SASI controllers don't handle multisector read/write command properly. These controllers are common on the used market here for under $100. We did the FIRST SCSI disk subsystem on a MAC and published the MEAGER EARLY version from last summer in DDJ. That propted dozens of competitors within weeks of the article to start building the BETTER product learning from our mistakes. We showed Apple one of our boards early last summer and they quickly added the NCR5380 onto the main logic board and offered it as the MacPlus (clearly defining SCSI as the standard interface). We have learned from our and others mistakes as well -- we have fixed the bugs in the early driver that we evolved from the Aztec C ram disk driver and have evolved the hardware to emulate the MacPlus addressing to get both hardware and software compatability with other MacPlus SCSI compatable products. The original MacSCSI pin board is relatively SLOW because of the software strobes used instead of the Psuedo-DMA transfers used in ALL later hostadapters including the MacPlus. When I did the original design the Ncr5380 had been on the street a few months and making the Psuedo DMA work required some understanding that wasn't well documented at that time. We figured it out a few months later, but well after the DDJ article was off to press. The SAFE PIO transfer loop is pretty slow but works on on all controllers and is REQUIRED on most controllers to pass command and status data. A faster unfolded PIO or PsuedoDMA transfer loop without edge checking is safe for most data transfers -- but breaks on several controllers where the micro can cause delays in the data transfer rate. Thus you need to have a detailed understanding of the controller to optimize the SCSI software to it. Warp 9 never planned to ship the original MacSCSI pin boards -- the sudden removal of the AP 64 pin clip from the market caused them to take a sudden last minute change of plans. They have planned to ship a MacPlus compatable host adapter all along -- now that the AP vs. General Computer thing is getting settled, AP clips are being delivered again in small quantity. I expect Warp 9 will shift back to the MacSCSI Plus board shortly. In Jan Warp 9 wanted product ASAP and we had little run time on the OMTI 3100 controller so we opted to play it safe and do full edge following, lest some glitch cause problems on hundreds of systems in the field. They have since optimized the transfer rate on the MacSCSI pin board based on more confidence in the OMTI controller. All of this is really history since the pin board is a DEAD product given the MacSCSI Plus -- We will be offering a low cost upgrade for owners of our Original boards shortly, Mirror/Warp 9 have talked about doing the same. We offer unbundled parts to build either an internal and/or external upgrade at less than $900, and Warp 9 is offering about the same for a packaged unit with value added services and software (but no sources) -- Micah's offering is primarily directed at their dealer channel and their prices were quoted at something higher than ours ???? (something around/under 1,495???) which is a FAIR price for that market given current costs for production, sales, and support. > > I haven't seen any Mac SCSI floppy disks (technically, the Bernoulli Boxes use > floppy media; but they perform like hard disks). The name Mac SCSI is our product/trademark name (TM application pending). We don't offer or ship a floppy product. Mr Brecher should not try and confuse the world by using MacSCSI as both our product name and a generic term. > The test is designed to measure hardware and disk driver performance, with no > influence from the file system (HFS/MFS) or volume/file/Finder configuration. > The program issues I/O requests directly to the disk driver. > > The benchmark consists of three parts: > > (1) 100 reads of 32KB of data from the start of the volume; > (2) 100 writes of 32KB of data to the start of the volume; > --the above two tests measure data transfer speed; 32KB was chosen to > be a reasonably large chunk, but not so large that it would cross a > cylinder boundary (thus not requiring any head movement). > > (3) 40 iterations of: read one 512-byte block from an offset of 1MB, followed > by read of one 512-byte block from start of volume; > --the above test measures access time, i.e., seek or head movement speed. > The test is non-destructive -- test (2) writes the data that was read > in test ( 1). Test (3) is bypassed if the volume size is less than > 1MB+512bytes. The volume tested is that from which the program is > run. The problem here is that we have a fundamentally diffent view of how the Mac is being used. I assume that 512/1mb systems will use the switcher to remove the long loading times for the VERY large Mac applications, or that a RAM disk will be used for the high activity programs and files in development environments like working under something like the Aztec C shell. The REMAINING disk traffic is generally short requests thru the filesystem where the burst transfer rate should have little effect. Consider that many filesystem requests to the driver are are short, a few blocks -- the PIO transfer for the 512 bytes from the NCR5380 (from the data buffer in the disk controller) is at best about 1ms using blind reads with the best possible hardware pseudo-dma, about 2ms using SAFER hardware pseudo-dma and about 4-5ms for an edge following programmed I/O transfer with software strobes (note that several controllers may NOT work with blind pseudo-dma transfers ALL the time). Most SCSI/SASI controllers are reasonably fast at data transfer but are very poorly optimized for all the other activities to handle a command. For each command setup to the SCSI bus most controllers take between 2-5ms to transfer and decode the comand packet BEFORE THEY START SEEKING OR LOOKING FOR DATA ON THE TRACK. After the last data byte most controllers take another 1 TO 3 MS to return the status and message bytes. Thus including the SCSI transfer time from above -- MOST SCSI CONTROLLER READ/WRITE COMMANDS take 4 to 8 ms to COMPLETE. A simple Test-Unit-Ready command is generally more than 3ms for nearly every controller. In addition, using a logic analyser, measured the Mac cpu service times per disk request seem to be between 2 and 8ms depending on the system call and task on filesystems less than 3mb. To optimize for filesystem traffic the hardware interleave should be set somewhere between 4 and 11 ms (that is between 4:1 and 11:1) depending on the controller, host adapter, driver, and the types of disk requests you wish to optimize for. Closer to 4:1 for application startup and closer to 11:1 for most disk intensive applications like data bases. Missing interleave is very damaging -- disk throughput drops as much as 93% depending on the miss and original interleave. Unless the customer can set the interleave during format, I recomend something between 6:1 and 8:1 for machines in the Mac's class. All of this is a drop in the bucket when you find that the Mac filesystem/OS or finder crawls into a hole with filesystem that have 10mb or more of allocated files -- above 18mb bootup to the finder desk top takes more than a minute on a system that took 7sec at 3mb. Any hard disk larger than 20mb has to be partioned into MUCH smaller chunks. The Apple filesystem/OS and finder are just not designed for hard disks. So buyer beware when the salesman touts HIS benchmark ... do YOUR HOMEWORK and shop for price/performance that meets YOUR goals and pocketbook. Have fun .... John Bass -- John Bass (DBA: Fastime, DBA:DMS Design) DMS Design (System Design, Performance and Arch Consultants) {dual,fortune,polyslo,hpda}!dmsd!bass (805) 546-9141