butcher@g.gp.cs.cmu.edu (Lawrence Butcher) (08/22/89)
Recently there has been a discussion about what separates mainframes from ordinary computers. It isn't cycles and it isn't memory. Generally it seems that mainframes have huge I/O capacity, and also memory bandwidth to support lots of concurrent activity. Non-mainframes are weak in I/O. Some microprocessor peripheral chips are very fancy. Ethernet, token ring, and some other communications chips are complete DMA systems, able to run for quite a while without CPU intervention. I always wanted to ask this newsgroup why SCSI support chips look like a simple uart. Boring. NCR seems to have made a chip which responds to both points above. There is an article in Aug 10 89 Electronic Design describing the NCR53C700 SCSI chip. The chip includes a complete 32 bit DMA controller (capable of burst transfers to 50 mByte/s), and some sort of programmable processor on chip. This chip seems quite fierce. Big time I/O seems simple and cheap. A fast RISC with 8 of these should be able to keep many disks busy. Check it out. The biggest, fastest business computers seem to run 1960's operating systems without protection, and allow user programs to do I/O without OS assistance. My new question. Is fast I/O all that micros need to bury mainframes? Or is user level I/O needed? If needed, how can simple hardware be built which allows direct user level DMA I/O? Lawrence
kurt@ibmarc.uucp (Kurt Shoens) (08/23/89)
In article <5932@pt.cs.cmu.edu> butcher@g.gp.cs.cmu.edu (Lawrence Butcher) writes about the new NCR SCSI chip and stipulates the following about business systems: >The biggest, fastest business computers seem to run 1960's operating systems >without protection, and allow user programs to do I/O without OS assistance. >My new question. Is fast I/O all that micros need to bury mainframes? Or >is user level I/O needed? If needed, how can simple hardware be built which >allows direct user level DMA I/O? I was wondering what operating system this refers to. IBM sells "TPF" (actually it's the latest version of the old Airline Control Program) to airlines, etc. that comes close to this description. I think TPF turns off the paging hardware (makes the machine run faster, apparently), but applications do not perform their own I/O. I think it would be messy trying to get applications to cooperate on the interrupt handlers. My stab at answering the question: I don't think that direct user level DMA I/O is necessary or desirable. User level I/O would take fewer instructions than having the operating system do it, but there should be plenty of mips anyway. Hardware-wise, you'd need lots of memory bandwidth and the ability to connect to many disks, tapes (or insert your favorite large number of bits per cubic inch archive medium here), and networks. Software-wise, I think you need a system that can credibly manage lots of disks and "terminals" (anything at the other end of the wire that people use), accommodates the kinds of applications businesses run (databases, transaction stuff, batch, and timesharing (if still appropriate)), has error recovery so that the system stays up mostly if pieces of it fail or experience transient problems, and has error reporting so that when software or hardware breaks the finger is pointed at the broken part or program. When you've got this complicated beast running a machine room floor full of disks, etc. it becomes hard to believe that it is still a micro. -- Kurt Shoens, IBM Almaden Research Center, ...!uunet!ibmarc!kurt
wayne@dsndata.uucp (Wayne Schlitt) (08/24/89)
In article <1026@ks.UUCP> kurt@ibmarc.uucp (Kurt Shoens) writes: > In article <5932@pt.cs.cmu.edu> butcher@g.gp.cs.cmu.edu (Lawrence Butcher) > writes about the new NCR SCSI chip and stipulates the following about > business systems: > > >The biggest, fastest business computers seem to run 1960's operating systems > >without protection, and allow user programs to do I/O without OS assistance. > >My new question. Is fast I/O all that micros need to bury mainframes? Or > >is user level I/O needed? If needed, how can simple hardware be built which > >allows direct user level DMA I/O? > > I was wondering what operating system this refers to. IBM sells "TPF" > [ ... ] i was wondering about this too and the only thing i could come up with is the "channel programs" that you can "execute" on most IBM mainframe OS's. of course the operating system has to go through and verify that these channel programs are correct and dont do anything the user shouldnt be able to do. the operating system has to add a bunch of additional CCW's and it does the real SIO instruction so it isnt really "user level I/O". for those of you who havent worked with this stuff, "Channel Command Words" can be thought of as very simple instructions to a very simple computer. you can do a few things really well as a "channel program" such as "skip until you find a tape mark, then skip until you find a block with a certain key" all without the main cpu having to do any work. the "channel cpu" is the thing executing the channel program. you can also do scatter/gather type i/o directly in a channel program, no block moves needed. one of the main limitations of the channel cpu is that it doesnt know about virtual memory so any address must be real addresses and you have to make sure the pages are locked in memory. i really dont know if "channel programs" are that much better than a good scsi board. both systems offload the main cpu from all the little hand holding needed to do most i/o. -wayne
butcher@g.gp.cs.cmu.edu (Lawrence Butcher) (08/24/89)
A number of people have pointed out that mainframe software isn't as moldy as I assumed. The recently publicized American Airlines Sabre system with more than 1200 disks online is probably not typical of business mainframes. A 19 inch rack holding 30 700 MByte 5.25 inch disks would seem like a lot of storage to me. A 64 bit DRAM system giving 40 mBytes per second of data to the I/O system while giving the processor another 40 would seem fast to me. A computer doing 8 concurrent disk accesses would impress me. I am however only a personal computer user. I have no idea of what a typical mainframe consists of. I would like to understanding what it would take to build an I/O and memory system which can deliver mainframe-like performance. For my education, could a couple of readers give me info about systems you are familiar with? Mainframes or supercomputers ok. Here are some questions whose answers I do not know: How many disk interfaces does a typical mainframe have? How many drives per interface? How big is the average drive? What is the max speed of the connection from disk interface to memory? How many transfers can happen at once? What is a typical short term I/O rate sustained over a period of a 30 seconds? What is a typical long term I/O rate sustained over a period of several hours? What percentage of real memory bandwidth does the I/O system get? What is an estimate of processor mips, to give MByte/sec / Mips? What are better questions? Thanks: Lawrence Butcher
aglew@urbana.mcd.mot.com (Andy-Krazy-Glew) (08/25/89)
>The biggest, fastest business computers seem to run 1960's operating systems >without protection, and allow user programs to do I/O without OS assistance. >My new question. Is fast I/O all that micros need to bury mainframes? Or >is user level I/O needed? If needed, how can simple hardware be built which >allows direct user level DMA I/O? Gould NPL's UIOM (Universal I/O Module) had a feature whereby it understood page tables, so could be told to do I/O into such-and-such a virtual map. Unfortunately, virtual I/O was seldom used because the UIOM had no TLB, and so virtual I/O was slow. From virtual I/O, it is a small step to direct user level I/O: the user needs some way of saying "Start my I/O now please, using the command packet that I've placed at address so-and-so". Note that memory mapping device control registers into user memory is unacceptable, because you don't want the user intefering with other users' I/O, unless you have one user per device (conceivable for displays and ttys; not acceptable for disks in a UNIX environment). One of the Japanese super-IBM-compatibles (I think that it was Hitachi) had a paper a while back on direct emulation of I/O instructions in a virtual machine --- required the virtual machine to be in contiguous physical memory, and a base and bounds register loaded into the channel controller. -- Andy "Krazy" Glew, Motorola MCD, aglew@urbana.mcd.mot.com 1101 E. University, Urbana, IL 61801, USA. {uunet!,}uiucuxc!udc!aglew My opinions are my own; I indicate my company only so that the reader may account for any possible bias I may have towards our products.
seanf@sco.COM (Sean Fagan) (08/25/89)
In article <5932@pt.cs.cmu.edu> butcher@g.gp.cs.cmu.edu (Lawrence Butcher) writes: >The biggest, fastest business computers seem to run 1960's operating systems >without protection, and allow user programs to do I/O without OS assistance. >My new question. Is fast I/O all that micros need to bury mainframes? Or >is user level I/O needed? If needed, how can simple hardware be built which >allows direct user level DMA I/O? CDC Cyber's don't; they do have protection (in both senses that I got from the above paragraph). Like other Seymour Cray machines, it doesn't have virtual memory; instead, you have a "base address" and a "limit register" (ba+limit is the highest memory location you can reference, and, of course, ba is added to all address references). However, you cannot do I/O randomly; the peripheral processors (which hold most of the OS), won't let you. Fast I/O is necessary, yes, for micros to "bury mainframes"; however, you also need a way to *use* that, which, hardware-wise, will require better memory subsystems (so that you can do I/O to at least one memory location and still run code at full speed). Also, of course, you need an OS that will take advantage of it, etc. It will be a few years until micros can catch up; I wouldn't hold my breath. -- Sean Eric Fagan | "[Space] is not for the timid." seanf@sco.UUCP | -- Q (John deLancie), "Star Trek: TNG" (408) 458-1422 | Any opinions expressed are my own, not my employers'.
sbf10@uts.amdahl.com (Samuel Fuller) (08/25/89)
In article <5962@pt.cs.cmu.edu> butcher@g.gp.cs.cmu.edu (Lawrence Butcher) writes: >For my education, could a couple of readers give me info about >--anyone who is interested >systems you are familiar with? Mainframes or supercomputers ok. >Here are some questions whose answers I do not know: All numbers are for Amdahl 5990-1400 >How many disk interfaces does a typical mainframe have? 128 Maximum (Channels) >How many drives per interface? 4 Maximum (I/O Interface per channel) >How big is the average drive? 3380 class drives hold 1.8GBytes >What is the max speed of the connection from disk interface to memory? 4.5 MB/sec >How many transfers can happen at once? 128 (All channels can simultaneously transfer data at their max rate) >What is a typical short term I/O rate sustained over a period of a 30 seconds? No data >What is a typical long term I/O rate sustained over a period of several hours? No data >What percentage of real memory bandwidth does the I/O system get? No data >What is an estimate of processor mips, to give MByte/sec / Mips? 105 CISC Mips for 4 processor model >What are better questions? How much DASD does a mainframe support? Average sites will have several hundred 3380 devices, meaning the average mainframe DASD farm will be in the several hundred gigabyte range. Additional Comments: Although mainframes haven't made the startling performance gains that microshave over the past few years. They are still advancing in both technology and performance. I was just looking over some of the sales literature for our processors and was surprised by the speed of the devices. Expanded Storage is 2GBytes of 120-ns DRAM Main memory is 512MBytes of 55-ns SRAM Cache is implemented with 2.8ns ECL Cycle time is 10ns The CPU is built from 336 ECL chips on a double-sided 42-layer water-cooled printed circuit board.(It's pipelined too!) No wonder the machines cost $10 Million. -- --------------------------------------------------------------------------- Sam Fuller / Amdahl System Performance Architecture I speak for myself, from the brown hills of San Jose. UUCP: {ames,decwrl,uunet}!amdahl!sbf10 | USPS: 1250 E. Arques Ave (M/S 139) INTERNET: sbf10@amdahl.com | P.O. Box 3470 PHONE: (408) 746-8927 | Sunnyvale, CA 94088-3470 ---------------------------------------------------------------------------
mproicou@blackbird.afit.af.mil (Michael C. Proicou) (08/25/89)
In article <56FO02qB4e0k01@amdahl.uts.amdahl.com> sbf10@amdahl.uts.amdahl.com (Samuel Fuller) writes: > >The CPU is built from 336 ECL chips on a double-sided 42-layer >water-cooled printed circuit board.(It's pipelined too!) ^ --------- | | ----------------------------------------- Of course, that's how they get the water in! :-) I remember looking in the Cyber 205 at Purdue, one side is miles of grey wiring, and the other is miles of copper tubing! Mike -- Mike Proicou mproicou@galaxy.afit.af.mil <- Preferred(?) Form mproicou@afit-ab.arpa <- Most Likely to Work? Go Figure!
davec@cayman.amd.com (Dave Christie) (08/26/89)
In article <5932@pt.cs.cmu.edu> butcher@g.gp.cs.cmu.edu (Lawrence Butcher) writes: >My new question. Is fast I/O all that micros need to bury mainframes? Or >is user level I/O needed? In article <?????> Sean Fagan writes: >CDC Cyber's don't; they do have protection (in both senses that I got from >the above paragraph). Like other Seymour Cray machines, it doesn't have >virtual memory; instead, you have a "base address" and a "limit register" Being an ex-CDC'er, I have to clear up a glaring omission: the above applies to the old Cyber 170's - the newer 180's do have virtual memory - quite Multics-like. (Except much bigger segments. "For God's sake," said the Multics folks, "make the segments as big as you can!") Having seen other postings of Sean's, I'm sure he's aware of this - just a slip. Don't let it happen again ;-). As for user-level I/O, do you (LB) mean for controlling special-purpose devices (not common in the mainframe world) or for dealing with weird and wonderful data structures out on disk? For the latter, the 180 architecture, along with the NOS/VE OS, provides what I think is the ultimate(*) in user-level i/o: none! What I really mean is: implicit i/o via memory mapped files. Simply associate a file(s) with a data structure(s) in the virtual address space of your software, and access it anyway you like. Actual physical i/o occurs simply as part of normal paging. No fuss, no muss. (*) but then, I'm not up on what people generally mean by user-level i/o. This works really well with large memories (like where your database starts to look memory-resident). Especially if you can safely leave modified data in memory without writing it to disk until the pages simply get aged out. I believe this is one place where current micros, risc or cisc, fall down on the job: it's difficult, if not impossible, to recover central memory when the system crashes. (I'm sure people who know otherwise will enlighten me as to how easily it can be done!) The segments and rings of 180 provide firewalling - errors, be they hardware faults, user, application, or os bugs, are well identified and well contained. Only if something goes wrong in the innermost ring, in which the system spends a minimum of time, do things get tricky - and that can usually be handled with a consistency check. This sort of recovery capability is crucial for database systems - one of the primary domains of mainframes. This is starting to look more like an advertisement for fault-tolerance rather than efficient user-level i/o. Just another aspect of the world of difference between micros and mainframes. Lest this also look like an advertisement for Cyber 180s, let me say that other mainframes likely have similar capabilities, and probably some micro- based systems can do some of it (like memory mapped files), but like I said, there's still a world of difference between micros and mainframes. >Fast I/O is necessary, yes, for micros to "bury mainframes"; however, you >also need a way to *use* that, which, hardware-wise, will require better >memory subsystems (so that you can do I/O to at least one memory location and >still run code at full speed). Also, of course, you need an OS that will >take advantage of it, etc. >Sean Eric Fagan | "[Space] is not for the timid." Hear, hear. And effective cache coherency - no mean feat in this sort of system. ---------------- Dave Christie An ex-CDCer who knows just enough to form dangerous opinons like these...
hascall@atanasoff.cs.iastate.edu (John Hascall) (08/26/89)
In article <26970> davec@cayman.amd.com (Dave Christie) writes: }In article <5932> butcher@g.gp.cs.cmu.edu (Lawrence Butcher) writes: ... }and wonderful data structures out on disk? For the latter, the 180 }architecture, along with the NOS/VE OS, provides what I think is the }ultimate(*) in user-level i/o: none! What I really mean is: implicit i/o }via memory mapped files. Simply associate a file(s) with a data }structure(s) in the virtual address space of your software, and }access it anyway you like. Actual physical i/o occurs simply as }part of normal paging. No fuss, no muss. But since you can also do this in VMS, it is by definition bad. ;-) I find it a very useful feature, too bad hardly anyone ever takes advantage of it (and its so easy, just two system calls: open and crmpsc). Almost makes you appreciate the AS/400 architecture (*almost*!). John Hascall ISU Comp Center
ingoldsb@ctycal.COM (Terry Ingoldsby) (08/30/89)
In article <1383@atanasoff.cs.iastate.edu>, hascall@atanasoff.cs.iastate.edu (John Hascall) writes: > In article <26970> davec@cayman.amd.com (Dave Christie) writes: > }In article <5932> butcher@g.gp.cs.cmu.edu (Lawrence Butcher) writes: ... > }and wonderful data structures out on disk? For the latter, the 180 > }architecture, along with the NOS/VE OS, provides what I think is the > }ultimate(*) in user-level i/o: none! What I really mean is: implicit i/o > }via memory mapped files. Simply associate a file(s) with a data > }structure(s) in the virtual address space of your software, and > }access it anyway you like. Actual physical i/o occurs simply as > }part of normal paging. No fuss, no muss. > > But since you can also do this in VMS, it is by definition bad. ;-) I can't give a reference, but I thought that memory mapped file I/O was going to be included in one of the future versions of UNIX. I can't remember if it will be in OSF or System V. It is convenient and can give quite a performance improvement over manual code. If I'm not mistaken moving data around manually is usually a lot slower than using something like the SWAPPER process do it for you. Since this group seems to have side-tracked itself into a comparison of the features of various OSes, I will continue the tangent. We use both VMS and UNIX in our environment. Although VMS is alleged to have more sophisticated scheduling algorithms, I can't say that I have really noticed a big difference. I like the UNIX automatic priority decreasing facility. In my meager understanding of the algorithms UNIX makes sure that everybody gets some time, no matter how little. VMS doesn't seem to do this. Jobs at priority 3 can sometimes be stalled completely if there are lots of priority 4 jobs around. I would prefer that priority 3 jobs got a little bit of time. I also note that most OSes seem preoccupied with managing memory and CPU load. This was important in the days of expensive memory and slow CPUs. What is often a bottleneck speed is I/O. For instance, if you want to take over a VMS machines it is quite easy. Do a *whole* bunch of small QIOs (queue an I/O request to a disk drive), but don't wait for the result (eg. do a bunch of writes but don't block). Even if the process doing this is at low priority, it eventually takes over the system. What happens is that higher priority jobs will typically need to do occasional input. They place their I/O request and the end of the queue, and block until the I/O completes. If there are 10000 requests in the queue, they block for a long time. Eventually everybody ends up blocked most of the time except the lower priority (anti-social) job or jobs which do no blocking I/O. I speak from experience; this has happened (accidentally) on our system. Moving back to comp.arch, I think the point about mainframes and minis being different from micros and workstations because of the I/O capacity is entirely correct. Our Intergraph VAX has special disk controllers. Each disk controller is worth (costs?) more than a workstation, and I haven't even added in the bus to house the controllers or the disk drives themselves. For that reason I think that small single user systems are ideally suited to simple operating systems like UNIX. UNIX (and for that matter VMS) don't effectively manage the I/O subsystems of larger machines. IMHO. -- Terry Ingoldsby ctycal!ingoldsb@calgary.UUCP Land Information Systems or The City of Calgary ...{alberta,ubc-cs,utai}!calgary!ctycal!ingoldsb
littauer@uts.amdahl.com (Tom Littauer) (08/30/89)
In article <452@ctycal.UUCP> ingoldsb@ctycal.COM (Terry Ingoldsby) writes: > ... >I think that small single user systems are ideally suited to >simple operating systems like UNIX. UNIX (and for that matter >VMS) don't effectively manage the I/O subsystems of larger >machines. IMHO. Most don't - it is, as you point out, alien to most UNIX system developers, since they live in a small-box world. It's cost us a lot to scale up UNIX systems to run on our big machines... -- UUCP: littauer@amdahl.amdahl.com or: {sun,decwrl,hplabs,pyramid,ames,uunet}!amdahl!littauer DDD: (408) 737-5056 USPS: Amdahl Corp. M/S 278, 1250 E. Arques Av, Sunnyvale, CA 94086 I'll tell you when I'm giving you the party line. The rest of the time it's my very own ravings (accept no substitutes).
hascall@atanasoff.cs.iastate.edu (John Hascall) (08/30/89)
In article <452> ingoldsb@ctycal.COM (Terry Ingoldsby) writes: }In article <1383>, hascall@atanasoff.cs.iastate.edu (John Hascall) writes: }> In article <26970> davec@cayman.amd.com (Dave Christie) writes: }> }In article <5932> butcher@g.gp.cs.cmu.edu (Lawrence Butcher) writes: [original thread on memory mapped I/O long since gone] [oops, that was the second thread, the first was SCSI] ... } .... I also note that most OSes seem preoccupied with managing }memory and CPU load. This was important in the days of expensive }memory and slow CPUs. What is often a bottleneck speed is I/O. }For instance, if you want to take over a VMS machines it is quite }easy. Do a *whole* bunch of small QIOs (queue an I/O request to }a disk drive), but don't wait for the result (eg. do a bunch of }writes but don't block). That's what the quotas BIOLM and DIOLM are for, to limit the number of Buffered and Direct I/O's that a process can have active at any one time. John Hascall ISU Comp Center
acockcroft@pitstop.West.Sun.COM (Adrian Cockcroft) (08/31/89)
In article <452@ctycal.UUCP>, ingoldsb@ctycal.COM (Terry Ingoldsby) writes: > In article <1383@atanasoff.cs.iastate.edu>, hascall@atanasoff.cs.iastate.edu (John Hascall) writes: > > In article <26970> davec@cayman.amd.com (Dave Christie) writes: > > }In article <5932> butcher@g.gp.cs.cmu.edu (Lawrence Butcher) writes: > ... > > }and wonderful data structures out on disk? For the latter, the 180 > > }architecture, along with the NOS/VE OS, provides what I think is the > > }ultimate(*) in user-level i/o: none! What I really mean is: implicit i/o > > }via memory mapped files. Simply associate a file(s) with a data > > }structure(s) in the virtual address space of your software, and > > }access it anyway you like. Actual physical i/o occurs simply as > > }part of normal paging. No fuss, no muss. > > > I can't give a reference, but I thought that memory mapped file I/O was > going to be included in one of the future versions of UNIX. I can't > remember if it will be in OSF or System V. It is convenient and can > give quite a performance improvement over manual code. > Memory mapped file I/O was introduced in SunOS 4.0 which has been shipping since around May 1988. It is one of the main parts of SunOS which has been incorporated into SVR4. Some of the features are page-by-page sharing and copy-on-write mechanisms. The whole of RAM is effectively used as a cache into the filesystem and if a file is being read by two processes using mmap then the file will be paged in and the pages shared. Writes to the file can be seen by the other process if you want or a copy-on-write can be invoked. I think the cp program was modified to use mmap. A related change was that SunOS 4.0 can page to regular files as well as disk partitions. Increasing the swap space on a 4.0 system is done online using two commands: % mkfile 16m /home/swap % swapon /home/swap Since the disk controller on a Sun uses DVMA to write dirctly to the process address space (no copying or buffering for most pages) the I/O is very efficient. Adrian -- Adrian Cockcroft Sun Cambridge UK TSE sun!sunuk!acockcroft Disclaimer: These are my own opinions
bruce@blender.UUCP (Bruce Thompson) (09/02/89)
Just a quick (I hope :->) note about file mapping. Apollo workstations provide a technique called Mapped-Segments which allow a process to link a file into it's virtual addressing space thus reducing I/O times to paging times. The interesting thing is that the documentation for the `Open System Toolkit', mentions that the `uasc' file manager uses mapped segments to implement unstructured ascii files (like regular UNIX files). Some quick background: The Apollo file system is object-oriented. Each file-system entry has a type, like `uasc'. Each type has a manager which is a set of functions which know how to manipulate objects of that type. When a file is opened, the appropriate type manager is called to perform the operations on the file. You can write managers which call other managers, so if you were to create a `structured ascii' file type you could use the `uasc' manager to handle operations which you do not want to directly deal with. An interesting side note, mapped segments can (and are) be used to implement shared memory between processes. ------------------------------------------------------------------------ Bruce Thompson CAE Engineer, NovAtel Communications Ltd. 1020 64 Ave, N.E., Calgary, Alberta, CANADA Disclaimer: I do not represent in any way the views/policies of my employer, and often not even my own. "Never underestimate the power of human stupidity" - Lazarus Long