bernhold@qtp.ufl.edu (David E. Bernholdt) (06/14/90)
Thanks to all who responded to my query about striping disks. Here is the promised summary... Machines mentioned include a preponderance of Convexen, also Alliant, Apollo, and Cray. Most experiences were positive. Problems mentioned include: 1) Disk crashes/bad disks. These are handled like any other file system for the most part -- you back them up regularly and they're fine. One person said they had a string of bad disks which was bad enough to put them off of striping. 2) Don't put multiple partitions in the same disk in a striped partition -- the disk will have to seek all over the place! 3) Due to limitations of unix file systems, it seems that on many implementations, striped partitions are limited to 2GB total size. Presumably some vendors have ways around this, but I don't claim to know who does & who doesn't -- ask the sales people. 4) One person mentioned that on the Alliant, the file system manager requires an extra 10-12 Mb to manage striped file systems. Apparently, this is not well publicized/documented, and obviously it could be a problem on a relatively small-memory machine. This was not mentioned by anyone else, so I don't know how many other vendors may have similar "problems". 5) Having read through the FPS documentation (what little there is) about striping, I found the interesting restriction that the number of disks striped into a given partition must be a power of two. No one mentioned this kind of restriction for other machines -- and Convex owners mentioned 3-way and 6-way striping, so they obviously don't have any such limitation. Performance issues: No one reported any kind of before/after benchmarks, but people seemed uniformly pleased by the performance improvement. One person says they got nearly a 4x improvement on I/O rate for a 4-way stripe. Obviously, striping is only going to help for systems which are I/O bound. Note that that's systems, and not necessarily just particular applications -- if a potentially I/O-bound application has to compete against enough low-I/O work already, faster I/O may not matter). Also, striping really only helps for large files -- small ones do just as well on regular file systems. One Convex owner said they had a setup with 5 disks on two controllers -- three regular and two striped together (on separate controllers). They were warned that having the non-striped disks served by the same controllers would create a bottleneck, however, they have found it not to be so. They are running XY451D controllers and Fujitsu Eagle disks. You definitely need to have the striped disks on different controllers, though. In conclusion: One person said explictly that the general fear of striping was pretty much unfounded, as long as you have reliable hardware and treat the striped file systems as you would others -- regular backups of important data, etc. From what I've seen so far, I'm inclined to agree. Hopefully, after we stripe our file systems, I'll still agree... Thanks again to everyone who responded. -- David Bernholdt bernhold@qtp.ufl.edu Quantum Theory Project bernhold@ufpine.bitnet University of Florida Gainesville, FL 32611 904/392 6365
datri@convex.com (Anthony A. Datri) (06/14/90)
>Convex owners mentioned 3-way and 6-way striping, so they obviously don't >have any such limitation. We've got an 11-way /tmp in house. Gotta do *something* with those pesky a partitions... >One Convex owner said they had a setup with 5 disks on two controllers ... >to be so. They are running XY451D controllers and Fujitsu Eagle >disks. You definitely need to have the striped disks on different >controllers, though. It helps to have multiple controllers (isn't that *always* true?), but in reality, ConvexOS can stripe together (if you reeeeealy want to) any assortment of partitions, as long as they're on the same *kind* of controller -- typically the old xy451, newer iphase 4200, or our own IDC. You're warned, however, if you specify two partitions on the same drive. This must be an ooooold customer -- we haven't done Eagles in a good while, and we sell very few of those crufty xy451's any more. Personally, I've never really been convinced that striping is always a performance win on a real-world machine with a large number (like 80) of simultaneous users. If you've got n MB/sec coming through your I/O system, does it really matter how you slice it? (I'm just a sysadm, not a salescreature, and I try very hard to not sound like the latter). -- -- vi belongs in errno.h: editor too big