bernhold@qtp.ufl.edu (David E. Bernholdt) (06/13/90)
Two weeks ago, I asked about using SSDs under UNICOS. Since there seemed to be some interest, I thought I'd summarize the half-dozen responses I got for the network. First of all, my thanks to those who responded: mcintosh@crnvma.cern.ch, shan@hall.cray.com (Sharan Kalwani), brooking@mcnc.org (Jim Brooking), djh@osc.edu (David Heisterberg), donlee@asylum.cray.com (Don Lee), and eugene@wilbur.nas.nasa.gov (Eugene N. Miya). There are three basic ways to use an SSD: 1) LDCACHE -- use the SSD as a cache for the (rotating) file systems. The LDCACHE can be configured with different size caches for different files systems, etc. to fit the site's needs. This option is totally controlled by the operators/management, of course. 2) file system -- the SSD itself can be configured as a file system, although I don't think anyone actually does this. It gives a faster transfer rate than using it as LDCACHE, but is rather small for a file system (SSD come in sizes up to 1 GW, I think) and one bad apple can fill up the disk & keep everone else on it from working. 3) SDS -- use the SSD as a "secondary data segment". By my understanding (no flames, please) this allows the SSD to be treated as a slower extension to main memory. It can be accessed using system calls like ssbreak, ssread, and sswrite, but apparently the OS is setup to use SDS to emulate a special disk device in which files can be explicitly assign(1)'ed to SDS. According to the man page for assign, files placed on SDS must also specify a size, and the file open fails if there is not enough space in SDS. It looks to me like you get the benefits of (2) with enough restrictions on the user that it becomes workable. It is also work noting that SDS can be managed on whole or in part by NQS as a resource called "quickfile" space. So that's how they *can* be used. How are they used? Management likes using it as ldcache and/or for swapping, because it gives the best overall performance. It doesn't seem like anyone would actually use it as a straight file system unless it was only for swapping. Many sites (of a small sampling -- those who responded to my query) run with some mixture of SDS and ldcache. Of course SDS is only useful for those who know about it and explicitly use it, but I guess many who can benefit from it know about it. How much programming do you have to do to use the SSD? Obviously if its all ldcache, its out of your hands. If its SDS and you want to just use assign(1), you also get away without programming. If you want to use SDS as extra memory, you'll have to program. The Cray training manual (TR-USC: UNICOS System Calls) and your local Cray analyst were mentioned as good references for how to explicitly code for the SSD. A couple of final notes (cautions) are in order, however. Just assigning a file to SDS or putting it on the SSD configured as a file system doesn't necessarily mean you'll get all of the wonderful benefits (speed) of the SSD. You have to use large block size, etc. to get good throughput -- so although there is technically no programming required, you may end up consuming a scarce resource and get no benefit from it if you aren't willing to do a little more work for it. Further, configurations vary widely from machine to machine, so what works well on one may not on another. It seems like a good idea to preapre a simple benchmark to determine how much work must be done on the CPU vs. SSD access to avoid thrashing the SSD. And someone, who shall remain nameless, suggests that if your strapped for memory & don't want to do the work programming for an SSD, just upgrade to a Cray-2. -- David Bernholdt bernhold@qtp.ufl.edu Quantum Theory Project bernhold@ufpine.bitnet University of Florida Gainesville, FL 32611 904/392 6365