watts (01/05/83)
CRLT's Codata Part ONE: the SUN system -- dream and genesis About two years ago, some people from Stanford university sat down and designed what, to them, would be the ideal computer set up in a few years. Being imaginitive people, they weren't content to use 1980 technology, so they decided to try to "look ahead" at the future. This was their dream: a VAX workstation with Unix for each user. Impossible in 1980? But techonology wasn't standing still. About this time, a fantastic new chip called the 68000 came out. It was pratically the speed and power of a main-frame processor, but came in a single silicon chip. This chip looked like it would be well suited to run Unix on, and showed signs of being the best "16-bit" microprocessor out. In addition, they took note of some of the things coming out of Xerox, most particularly, the Ethernet interface and the some of the innovative hardware features of smalltalk. Out of these was born the "sun board". It's major features were to be: (1) CPU board with 68000 processor MMU 256k bytes onboard memory (256k is the pratical lower limit for Unix and about right for one person w/ 68000) 2 UARTs (2) Multibus BUS structure. (easy to get the latest gizmos) (3) hi resolution graphics board w/ mouse. Straight from small talk! (4) Ethernet interface. The whole system was to go into one box, and each user would have his own Sun workstation all to himself. Notice there's no disk. The idea was that instead of the disk, you'd have the ethernet interface (about as fast as most cheap disk drives anyway) and it would all go off to a central file server which would have several mamouth [sp?] disk drives. This would allow file sharing, backup, etc. and be vastly cheaper than having each workstation have its own disk drive. They sat down, drew up hardware specs, and then came the purely practical problem -- how are they going to make this system, and make it cheaply enough to be useable? Well, they certainly didn't want to make the system themselves, and in order to be truely cheap, it would have to be made and sold in large enough quantities to keep prices down. So they found 5 companies and licensed each to produce the system. Now, each company had its own ideas of what it wanted to produce, and they weren't quite as "blue sky" as the original sun-board people. They wanted to produce multi-user computers in the image of the VAX, with one processor, memory, etc. So they took the original sun system design and modified it. Each company picked its own disk drive and manufactorer, and made other modifications. The SUN people at stanford weren't too pleased with this developement, so some of them formed their own company to put together their idea of the sun system should really look like. As far as Unix goes, it turns out that there is only ONE company that really has a good 68000 unix [at this time] and that company is Unisoft. A Berkeley outfit, they have bought a source license from Bell, and for a (relatively) modest fee, will port their 68000 version of Unix over to your machine, so that you don't have to buy a source license from Bell and do your own port. And you can save even more if you use the same devices that someone else used, so that the same device driver code can be used. Microsoft has a 68000 version of Xenix, when I saw it on the forum at the NCC, it appeared to be rather sick. I don't believe it's been commercially released yet. I don't know of anyone else whose as far as Microsoft is, let alone Unisoft. What this means in practical terms is that ALL of the sun boards out today share the same Unix and utilities, and often most of the same hardware peripherals, so as to save on developement costs. I've never seen a list of the 5 companies, but as near as I can reproduce them, they are: (1) Codata multi-user CTW-300; comes with liberty bay controller, 5 meg Seagate ST-506s, Central data dumb octal card. (2) Cyb (this may be an offshoot of some sort from Codata) Similar to Codata except uses MCS winchester controller (slow), comes in cute smoked-plastic case. (3) Pacific microsystems (or Pacific data?) (4) Callan data systems "work station"; built-in to a terminal; MCS controller. (5) forward systems (forum) [?] and of course now, (6) SUN systems "work station" with REALLY nice graphics; 84 meg SMD interface; etc. NOTE: This list is by no means accurate or complete. If have a better list, I'd love to see it. In particular, Cyb was at apparently at one point a distributer for Codata, which means I may be missing one of the original five... Part TWO, Detailed description of the Sunboard The SunBoard consists of the following major subsystems: (1) 68000 MPU (clock varies; 8Mhz for Codata; 10Mhz for Sun system) (2) MMU (3) UART chip (Intel 8274 or NEC 7201) -- dual UART chip. Used for ports A (the Console on the Codata) and B, either configured as a DTE (terminal) or DCE (computer/modem). (4) AMD 9513 timer chip. 5 channels, usage: 1 - watchdog 2 - operating system timer 3 - refresh 4 - Uart A 5 - Uart B (5) 16 bit input port. This is intended to be used as the mouse interface and the console keyboard (when using graphics board as console output) (6) Multibus (IEEE 976) interface (7) 256k bytes of onboard RAM; not accessible to multibus. (8) Up to 32K of ROM (but only 4K bootstrap is present on the Codata) The Sunboard was intended for a low-cost single-user system, so certain "short-cuts" were taken. The most important of these are software memory refresh and DMA. The 256k of onboard memory is dynamic RAM; however there is NO hardware refresh circutry for this memory. This is done by the 68000 instead, using software. The CPU is supposed to set up timer channel 3 to interrupt every 2 ms. and vector to special "refresh" code, which simply consists of 128 "no-op" instructions. This takes up something like 3.2 % of the CPU's time, but more importantly, only takes up channels on the timer chip and some ROM room, both of which would be taken up anyway. Perhaps more importantly, as far as limiting its multi-user performance goes, is the "software DMA". Since Multibus components, such as a Winchester drive controller, can't access onboard memory, it has to use a multi-bus board instead, and then the CPU has to transfer the data into the real intended destination on the Sunboard. Since this is a function of the operating system (Unix), this double transfer of data is pretty much transparant to user programs -- it just means slower data transfer rates. This is, again, reasonable for a single-user system where you aren't paying $100,000, but also limits the system performance. The reason for this limit of system performance is somewhat interesting. Since the 68000 has a substantial fraction of the performance of a VAX, which is supposed to be able to support 20 users, it would seem, on the surface, quite reasonable to have the 68000 do the transfer into memory -- after all, it is just using time it would otherwise be wasting. However, keep in mind that the disk drive still has to have memory to put stuff in. Since we can't guarrantee that the 68000 will be able to uninterruptably read memory out of the disk drive (it has to do that refresh or else!), there has to be a buffer around for the disk drive to put stuff in. With the typical controller, this will probably be DMA into a multi-bus memory buffer. After the transfer is complete, the 68000 then has to do the actual transfer into the onboard RAM. Therefore, the real delay is this double transfer, and this is the fundemental limit of the performance of the Sunboard for most applications. The other important feature of the Sunboard to talk about is the memory management. Motorola has been quite slow in bringing out their MMU for the 68000, (perhaps because of the economy), so the Sunboard people had to design their own. Unfortunately, they made some somewhat unfortunate decisions. This in part relates to the design of the 68000. The 68000 does 90% + of what it should do but has a few critical ommisions that complicate system design. Firstly, the 68000 doesn't have an instruction to let the kernel look directly at user memory (Move From User Space), which makes looking at parameters, etc., in user space complicated. Secondly, there is no way to distinguish between interrupt vector fetches and normal kernel memory, so the first page of kernel memory HAS to be interrupt vectors. Therefore, most 68000 systems have both the kernel and user memory mapped together, so that, when the system is in kernel mode, it can look at both user and system memory. This makes it easy for the system to look at user memory, but means that the user memory can't start at location 0, because that's where the interrupt vectors are. The sunboard is no exception to this, but this causes a few problems, especially for Unix. Most non-68000 Unix systems have user memory start at location 0. This makes the exchange of runnable copies easy because there are no references to any addresses except inside the runnable copy, and all calls to the system are via some sort of trap which doesn't contain any references to system memory. As long as you have enough memory to load the copy, you can run it. However, since the SUN board has to have at least SOME kernel memory at location 0, the kernel has to start at some other address. Unfortunately, this is usually at location 0x1000, which means that user memory has to start AFTER then end of the kernel. On our Codata, this happens to be at location 0x00020000 but other SUN board systems have a different load address for user programs. Since the header doesn't have the load address in it (it has the start address to load into the program counter, but this isn't the load address), there is no really good way to resolve this. This could cause problems with exchanging software for 68000 based Unix systems, as relocatable object will have to be supplied so that the final link can be done on the customer's machine. It appears that even 0x00020000 wasn't big enough to allow enough multi-bus disk buffers, so it seems likely that future releases of Unix are likely to have a higher load address... Part THREE: Our experience with the Codata We originally started looking for a system in the fall of 1981, and were looking for the best Unix system for the lowest price we could find. We had a grant extension to buy a Unix system for aprox. $15,000 (a bit optimistic in retrospect!) in order to develop a real computer conferencing system for schools. We were especially interested in 68000 systems because they offered the most promise of being powerful yet cheap. After some looking, we came across a firm called "Wicat" which looked especially promising -- they claimed that they had Unix V7. Some calls produced the fact that it was "running in their labs, except under their own kernel, MCS, which was supposed to be "completely" compatible with Unix. To make a long story short, by the next spring, we realized that it was NOT unix, would not do all the things we needed (dial-in access, etc.) and would not do so in the near future. We then learned about Codata and after more careful testing (including a dial-in session to assure ourselves that they really had Unix), were almost all set to get their system. We put things off until after the June NCC for 2 reasons, firstly, our final system cost was still a little high -- $18,000 including terminals, and secondly, we wanted to be sure that none of the "competition" had a better system, but after the NCC, it was clear that Codata was by far the best choice we could make. We finalized the deal, and the Codata arrived at the end of August, 11 months after we started looking for Unix systems. Current System Configuration of our Codata The CTW-300 is a multi-user Sunboard computer. It includes the following cards plugged into the multibus: Sunboard CPU with 68000, 256k onboard RAM, MMU, 2 serial ports, timer. Sister board, adds extra 256k to onboard CPU RAM without waitstates. Plessey 512k multibus memory card. Liberty Bay Winchester Disk Controller with SMS300 bipolar processor FLoppy disk controller Octal Serial card with 8 Signetics 2650 Usart chips. (only 4 ports seem reliable, however...) In addition, the computer also has hooked up to it: 3 Seagate ST-506 5 megabyte winchesters. Slow, but cheap. One seems to be sick (lb2a) and will be sent back. 1 Tandem Floppy drive. 360 k bytes on double-sided diskettes, capable of reading/writing IBM-PC floppies with proper software, which we don't have. 1 DC Hayes smartmodem, 300-only baud, unfortunately. 1 Racal-Vadic 3451 auto-answer triple-threat modem. 2 DEC VT-101 terminals. The Codata has Unisoft's Unix port for the 68000. It's apparently something of a mongrel internally, having parts of Berkeley, MIT and standard V7 Unix all mixed in together. Not too suprisingly, not too many of the Berkeley Kernel features work -- the new tty driver, job control and multiplexed pipes all don't work, although some code is present. However, a locking() call is present for file locking... We do have an early version of the C shell, which is missing a lot of the features later added to it, and also Ex/Vi, a visual editor. In addition, we have C (the portable C compiler) Yacc, and most of the other standard Unix utilities. We also have an old version of Xanadu's `fred' visual editor, which only works with DEC VT-101 terminals with auto-xon/off disabled. (more on that later) Like any relatively new system, we've had our share of problems, and the following is a brief summary of the major hasles that we've encountered. (a) Our system came with a relative dumb serial port card which had some fairly nasty problems. Firstly, it appears that half the UART chips are defective in a rather insidious fashion -- incoming character while output is taking place cause the framing error bit to be set. It took us a while to track that down because the device driver had some bugs in it, which limited its speed to effectively 4800 baud maximun. It also didn't properly support the modem control lines, partially due to some design problems in the card itself. Terminals look like modems without a carrier to the octal card, which means that in order to detect the carrier, it is necessary to tell the driver which ports have a modem. Theirs didn't do this, but fortunately, we were able to get them to send us a copy of the source to the drivers. (b) The terminals we selected, DEC VT101s, were disappointing. They came with a white phosophor instead of green, were missing such "smart" functions as delecting arbitrary characters and lines on the screen. Also they couldn't keep up at 9600 baud without handshaking, which caused problems with an early version of a local visual editor (Xanadu's Fred) which we were able to get from some friends which used ^Q/^S from the keyboard for some functions. The cursor keys sends off a 3-character sequence which fills up the input buffer suprising quickly. (c) The disk drives have perhaps been the worse problem we've had. The disk drives are Seagate ST-506s, which are cheap, slow drives. Unfortunately, besides being slow, they are also unreliable, and of the 4 drives we've gone through, 3 of them have crashed at least once, in a rather nasty fashion. They've come down with unrecoverable read errors on at least 8 sectors in the first 100 sectors. Unfortunately, this is right where the inodes (which point to the sectors in a file) live, which means that at least 64 files are lost, and the disk has to be reformatted. The controller, has its own problems. We were planning on adding additional storage to the 2 ST-506s that we got with the system, and were hoping that the ST-412, a related drive with an identical interface, would work. Unfortunately, it appears that the stepper motor has different timing charasterics, and the firmware on the controller can't be altered to support a mix of the two drives. So, it appears that we're stuck with ST-506s for the time being. The controller also doesn't support seeks on multiple drives at the same time, which is unfortunate. The ability to perform multiple seeks would improve system performance immensely. Most DEC disk drive systems can do this, but I don't know if the Seagate interface is capable of supporting this. However, this is the best performing disk drive system that we were able to find last summer, for a reasonable price. SMD disk systems started at about $10,000, rather above our means. (d) We've come across some other other rough edges: Kernel sometimes comes up with suprising messages, for example, tracing a TRAP #0 (the system call) results in a 'trace trap out of kernel' message on the console. An illegal instruction results in a huge dump of the MMU registers on the console. The message 'SUREG BOTCH' occasionally appears when large programs are run. Perhaps one of the most important factors in a system is the support it receives. This is particularly important with large, new and complicated systems such as Unix. One of the major problems we had with Wicat is that they just didn't care about us, the customer. They never managed to meet all of our specifications, as we were never able to hook up a parallel printer and 4 terminals. Indeed, it was a long time before we were able to have two people developing at once, as there wasn't enough memory for the C compiler to run when two people were logged in at once. They also never bothered to fix our tape drive (for backup), and it was very rare that we were able to talk to anyone at the other end who could answer any of our questions. On the other hand, Codata and Information Nexus have been extremely cooperative and helpful. They've been very interested in making sure that we have a working system, for example rushing a boot diskette to us after our root drive died, when it turned out that ours had the wrong version of 'restor' on it. It has been very easy to get ahold of knowledgeable people who are usually able to answer our questions, and it has also been a pleasure working with them. It would be difficult to come up with a more complete contrast than between the behavior of Wicat and Codata. One interesting problem which might be much more important to a small business is in learning how to setup and manage a Unix system. Like any other large system, there are lots of things that have to be taken care of, such as setting up accounts, backing up the system, and manageing problems, and there is very little information on this. It is, of course, scattered throughout the Unix documentation, but in general, Unix seems to be set up on the assumption that the people who run the system have first gone out and learned a lot about Unix. This is a reasonable assumption for an on-going University shop, but offers problems for the business that has just gone out and bought a new Unix system. Probably they will have to engage the services of a consultant to set up the system initially, and will have to engage someone for the long term to set up accounts and handle backup. Hopefully, better documentation in the future will ease this task considerably. There seem to be other SunBoard systems for as little as $10,000 and it seems likely that, as the manufactorers get more experience and costs drop (already the ST-506 have dropped from $1800 to $900) that it won't be too long before a system of roughly the performance of our Codata (perhaps with better graphics) comes out for under 2 thousand dollars. I expect that this will be within two years, and these systems will undoubtly be marked for the same market that the IBM-PC holds today. All in all, we've been well pleased with our Codata, and it is performing satisfactorily, despite the unexpected problems. But it wouldn't be as much fun without the problems!