ruiu@dragos.uucp (dragos) (11/21/89)
After listening to an excellent talk by John Mashey at a local user group meeting, in which he gave some very interesting details of the new "enterprise" server box based on the ECL R6000, I was left wondering a few things. In the box, the chip runs at 67Mhz, and the backplane runs at 270MB/sec. The bus is 32 bits wide, and 270Mb (255.6 actually) is needed to supply one piece of data on every clock cycle. The question that comes to mind immediately: How many hoops did the designers have to jump through to achieve this ? Does anyone have any information on just how difficult it is to do a design at this speed ? (I'm told by those who have done it that even working at 40MHz is a pain because of capacitance problems.) Are bus connectors a problem at high speeds ? This 270MB/sec. figure was surprising to me - previous high speed busses I had seen were in the range of 80-128 MB/sec. Am I just behind the times ? As well, what kind of memory designs can steadily deliver hundreds of MB/sec. ? We were told that the MIPS box has 594K of cache (67Mhz, hmmm... 20ns I assume) so maybe all reads and writes are page mode and each memory board has a cache too ? I wonder how much cache misses cost at 67MHz. I wonder how often they happen in the MIPS design. -- Dragos Ruiu (ruiu@dragos.uucp) All system administrators should hand out alberta!dragos!ruiu a bottle of valium with every news-reader uunet!myrias!dr man page.
davecb@yunexus.UUCP (David Collier-Brown) (11/23/89)
ruiu@dragos.uucp (dragos) writes: [about bus speeds] >As well, what kind of memory designs can steadily deliver hundreds of MB/sec. ? >We were told that the MIPS box has 594K of cache (67Mhz, hmmm... 20ns I assume) >so maybe all reads and writes are page mode and each memory board has >a cache too ? I don't know about modern stuff, but my old HoneyBun[1] used to fetch eight 72-bit+ecc double-words at a time from several (I think 4) physical boards. This gave lots of bandwidth, with the usual problems of latency. The bus wasn't (well, it was about 4" of dedicated wire), so it could happily run fast. --dave (history is fun: so is monomania) c-b [1] a Honeywell, now Bull, three-processor DPS-8/70M -- David Collier-Brown, | davecb@yunexus, ...!yunexus!davecb or 72 Abitibi Ave., | {toronto area...}lethe!dave Willowdale, Ontario, | Joyce C-B: CANADA. 416-223-8968 | He's so smart he's dumb.
hui@joplin.mpr.ca (Michael Hui) (11/24/89)
Be careful: find out how wide the data bus is first. In Northern Telecom's SONET fiber transmission product, the card connectors (which happen to be pin and socket type) are the result of a multi-million dollar research program. The backplane speeds are also "up there" too.
mslater@cup.portal.com (Michael Z Slater) (11/24/89)
Dragos Ruiu asks the following questions about the MIPS ECL system: >In the box, the chip runs at 67Mhz, and the backplane runs at 270MB/sec. >The bus is 32 bits wide, and 270Mb (255.6 actually) is needed to supply >one piece of data on every clock cycle. The question that comes to mind >immediately: How many hoops did the designers have to jump through to >achieve this ? Does anyone have any information on just how difficult >it is to do a design at this speed ? (I'm told by those who have done it >that even working at 40MHz is a pain because of capacitance problems.) >Are bus connectors a problem at high speeds ? > >This 270MB/sec. figure was surprising to me - previous high speed busses I >had seen were in the range of 80-128 MB/sec. Am I just behind the times ? Unlike all standard buses I'm aware of, the MIPS system bus uses ECL levels, with a differential pair for each signal. The bandwidth is simply derived by one transfer per clock times four bytes per transfer. To help make long bursts practical, the R6020 bus interface chip includes a two 36-entry FIFOs. The capacitance problems at 40 MHz you refer to are very real, but are due to the use of CMOS drivers and CMOS voltage levels. Having bipolar drivers and ECL levels makes things much easier at high clock rates. One tradeoff, of course, is power. The R6000 spec projects 12-15 watts worst case at 80 MHz, but specs supply current at 4 amps max. That's just the CPU chip! Add the FPC, FP multiplier, cache rams, and a bunch of gate arrays and PALs, and you have a small room heater. Note that this is not a general-purpose I/O bus; the MIPS system also has multiple VME buses for I/O. >As well, what kind of memory designs can steadily deliver hundreds of MB/sec. >We were told that the MIPS box has 594K of cache (67Mhz, hmmm... 20ns I assume >so maybe all reads and writes are page mode and each memory board has >a cache too ? The cache is two-level, with 8 ns RAMs for the first level (16K data and 64K instruction) and 15 ns for the second level (512K combined I+D). I assume that virtually all bus transactions are bursts, and that page mode is used on the memory boards. >I wonder how much cache misses cost at 67MHz. I wonder how often they happen >in the MIPS design. Miss cost is about 65 clock cycles (!). MIPS literature say that "observed hit rate on a wide range of programs is 99.5 percent." Michael Slater, Microprocessor Report mslater@cup.portal.com 550 California Ave., Suite 320, Palo Alto, CA 94306 415/494-2677 fax: 415/494-3718 ** Our next issue will have a detailed article on the R6000. If you'd like a sample copy, send a note to deena@cup.portal.com requesting a sample copy of the December issue of Microprocessor Report.
gcaw@otter.hpl.hp.com (Greg Watson) (11/25/89)
I would love to see some data on this bus, such as: o What technology? - ECL I guess? o How many boards can I plug in? o Does it support arbitrary bus-masters? If so what is the arbitration time? o What sort of connectors? How many signal/grounds on the bus? o Is it synchronous or asynchronous? I guess that will do for now.... **************************************************************************** Greg Watson gcaw@hplb.hpl.hp.com Hewlett-Packard Laboratories TEL:+44 272 799910 ext 24016 Filton Rd, Stoke Gifford FAX:+44 272 790554 Bristol BS12 6QZ. UK *****************************************************************************
patrick@convexc.uucp (Patrick F. McGehearty) (11/27/89)
In article <1989Nov21.032355.5779@dragos.uucp> ruiu@dragos.UUCP (dragos) writes: >In the box, the chip runs at 67Mhz, and the backplane runs at 270MB/sec. >The bus is 32 bits wide, and 270Mb (255.6 actually) is needed to supply >one piece of data on every clock cycle. >This 270MB/sec. figure was surprising to me - previous high speed busses I >had seen were in the range of 80-128 MB/sec. Am I just behind the times ? > High speed is a relative term (maybe a marketing word :-) The Convex C2 series supports simultaneous transfers between 4 processors and memory at 800 Mbytes per second, nominal. That works out to 200 Mbytes/second per processor. And that is a 2 year old product. I am sure that Cray processors have even higher speed buses. All of which is not to minimize accomplishment of the MIPS R6000 design team. Buses > 40 MHz are still pretty rare, so involve unusual design problems.