rgoguen@BBN.COM (Robert J. Goguen) (10/04/89)
Does anyone have any info regarding Disk Striping on a UNIX Operating System. I don't have hardware to support bit/byte level striping. I am going to attempt to do sector level striping accross N disks. I'm also going to allow a single filesystem span N disks. I'd be interested in attaining papers on this subject and general feelings. Robert Goguen rgoguen@bbn.com phone (617) 873-3667
ghfeil@white.toronto.edu (Georg Feil) (10/10/89)
Here are my few references in the area. None of these pertain specifically
to UNIX.
\bibitem[Patt88]{raid}
D. A. Patterson, G. Gibson, and R. Katz,
``A Case for Redundant Arrays of Inexpensive Disks (RAID)'',
{\em Proceedings of ACM SIGMOD}, Chicago, Illinois, June 1988.
\bibitem[Livn87]{multidisk}
M. Livny, S. Khoshafian, and H. Boral,
``Multi-Disk Management Algorithms'',
{\em Proceedings of ACM SIGMETRICS}, Banff, Alberta, May 1987.
\bibitem[Kim86]{synchdisk}
M. Y. Kim,
``Synchronized Disk Interleaving'',
{\em IEEE Transactions on Computers},
Vol.\ C-35, No.\ 11, Nov.\ 1986.
\bibitem[Sale86]{diskstripe}
K. Salem and H. Garcia-Molina,
``Disk Striping'',
{\em IEEE International Conference on Data Engineering},
1986.
Georg.
--
Georg Feil
ghfeil@white.toronto.edu
...if that doesn't work try one of:
{uunet,pyramid,decvax,allegra,watmath,utzoo}!utcsri!white!ghfeil
ghfeil%white.toronto.edu@relay.cs.net (ARPA)danny@uunet.UU.NET (Danny Wilson) (10/12/89)
In article <9330@saturn.ucsc.edu>, rgoguen@BBN.COM (Robert J. Goguen) writes: > > Does anyone have any info regarding Disk Striping on a UNIX Operating System. > > I don't have hardware to support bit/byte level striping. I am going to > attempt to do sector level striping accross N disks. I'm also going to allow > a single filesystem span N disks. I'd be interested in attaining papers > on this subject and general feelings. > > Robert Goguen rgoguen@bbn.com phone (617) 873-3667 This kind of disk system was mentioned in the promotional literature for Apollo's quad-processor, mini-supercomputer (super-minicomputer ?), Unix workstation, the DN10000. While they claimed that this technique provided great power advantages etc. they never really explained what 'striping' is. Can anyone comment on what this is? -- Danny Wilson IDACOM Electronics danny@idacom.uucp Edmonton, Alberta alberta!idacom!danny C A N A D A X.400 danny@idacom.cs.ubc.cdn
scott@cs.rochester.edu (Michael Scott) (11/08/89)
In article <9647@saturn.ucsc.edu> yaronf%shum.Huji.AC.IL@CUNYVM.CUNY.EDU (Yaron Farber) writes: | | Two further pointers: ... Here's the full ref: %T Bridge: A High-Performance File System for Parallel Processors %A P. C. Dibble %A M. L. Scott %A C. S. Ellis %K dcs %J Proceedings of the Eighth International Conference on Distributed Computing Systems %P 154-161 %D 13-17 June 1988 %C San Jose, CA See also: %A P. C. Dibble %A M. L. Scott %T Beyond Striping: The Bridge Multiprocessor File System %J Computer Architecture News %D September 1989 %O Special issue on architectural support for I/O %A P. C. Dibble %A M. L. Scott %T External Sorting on a Parallel Interleaved File System %J University of Rochester 1989-90 Computer Science and Engineering Research Review %D 1989 -- Michael L. Scott University of Rochester (716) 275-7745 scott@cs.rochester.edu scott%rochester@CSNET-RELAY {decvax, allegra, cmcl2}!rochester!scott
dfk@grad13.cs.duke.edu (David F. Kotz) (11/09/89)
Since it seems to be of interest again, here is my collection of Disk
Striping and Parallel I/O references. I would be grateful for any
suggested additions. I would also like a paper copy of anything marked
"unverified" below, as I have't been able to find some references.
--------- cut here ----------
% BIBTEX Format bibliography
% Disk Striping and Parallel I/O
% From David Kotz, dfk@cs.duke.edu, November 1989
% You may use this bibliography for reference and citation but you
% can't publish it as a whole; i.e., the compilation is mine.
% First, some macros
@string{hyper88 = "Third Conference on Hypercube Concurrent Computers
and Applications"
@string{ieeetc = "IEEE Transactions on Computers"}
@string{metrics87 = "Proceedings of the 1987 ACM Sigmetrics Conference
on Measurement and Modeling of Computer Systems"}
@string{vldb12 = "14th International Conference on Very Large Data
Bases"}
@string{compcon = "Proceedings of IEEE Compcon"}
@string{icpp88 = "Proceedings of the 1988 International Conference on
Parallel Processing"}
@string{hyper89 = "Fourth Conference on Hypercube Concurrent Computers
and Applications"}
@string{ieeetor = "IEEE Transactions on Reliability"}
@string{vldb14 = "14th International Conference on Very Large Data
Bases"}
@string{ctr = "Computer Technology Review"}
@techreport{abu-safah:speedup,
author = "Walid Abu-Safah and Harlan Husmann and David Kuck",
title = "On {Input/Output} Speed-up in Tightly-coupled
Multiprocessors",
institution = "Department of Computer Science, Univ. of
Illinois at Urbana-Champaign",
year = 1984,
number = "UIUCDCS-R-84-1182",
keyword = "parallel I/O, I/O"
}
@inproceedings{asbury:fortranio,
author = "Raymond K. Asbury and David S. Scott",
title = "{FORTRAN} {I/O} on the {iPSC/2}: Is there read after write?",
booktitle = hyper89,
year = 1989,
note = "To appear",
keyword = "unverified, parallel I/O, hypercube, IPSC2, file access pattern"
}
@inproceedings{bitton:shadow,
author = "D. Bitton and J. Gray",
title = "Disk Shadowing",
booktitle = vldb14,
year = 1988,
pages = "331--338",
keyword = "parallel I/O, disk shadowing, reliability"
}
@inproceedings{boral:critique,
author = "H. Boral and D. {DeWitt}",
title = "Database machines: an idea whose time has passed?",
booktitle = "Proceedings of the 1983 International Workshop on
Database Machines",
year = "1983",
publisher = "Springer-Verlag",
pages = "166--187",
keyword = "file access pattern, parallel I/O, I/O, database machine"
}
@techreport{brandwijn:dasd,
author = "Alexandre Brandwajn",
title = "Performance Benefits of Parallelism in Cached {DASD}
Controllers",
number = "UCSC-CRL-88-30",
institution = "Computer Research Laboratory, UC Santa Cruz",
month = nov,
year = 1988,
keyword = "parallel I/O, disk caching, disk hardware"
}
@inproceedings{browne:io-arch,
author = "J. C. Browne and A. G. Dale and C. Leung and R. Jenevein",
title = "A Parallel Multi-Stage {I/O} Architecture with
Self-managing Disk Cache for Database Management Applications",
booktitle = "Database Machines: Proceedings of the Fourth
International Workshop on Database Machines",
publisher = "Springer-Verlag",
year = 1985,
month = mar,
keyword = "parallel I/O, disk caching, database"
}
@techreport{chen:raid,
author = "Peter Chen and Garth Gibson and Randy Katz and David
Patterson and Martin Schulze",
title = "Two papers on {RAIDs}",
number = "UCB/CSD 88/479",
month = dec,
year = 1988,
institution = "UC Berkeley",
keyword = "parallel I/O, RAID, disk array"
}
@inproceedings{copeland:bubba,
author = "George Copeland and William Alexander and Ellen
Boughter and Tom Keller",
title = "Data Placement in {Bubba}",
booktitle = "ACM SIGMOD Conference '88",
month = jun,
year = 1988,
pages = "99--108",
keyword = "parallel I/O, database, disk caching"
}
@unpublished{crockett:manual,
author = "Thomas W. Crockett",
title = "Specification of the Operating System Interface for
Parallel File Organizations",
year = 1988,
note = "Publication status unknown",
keyword = "parallel I/O, parallel file system"
}
@techreport{crockett:par-files,
author = "Thomas W. Crockett",
title = "File Concepts for Parallel {I/O}",
year = 1988,
institution = "ICASE, NASA-Langley",
number = "",
keyword = "parallel I/O, file access pattern, parallel file
system"
}
@article{csa-io,
author = "T. J. M.",
title = "Now: Parallel storage to match parallel {CPU} power",
journal = "Electronics",
year = 1988,
month = dec,
volume = 61,
number = 12,
pages = 112,
keyword = "parallel I/O, disk array"
}
@techreport{dewitt:gamma,
author = "David J. {DeWitt} and Robert H. Gerber and Goetz Graefe and
Michael L. Heytens and Krishna B. Kumar and M. Muralikrishna",
title = "{GAMMA}: A High Performance Dataflow Database Machine ",
institution = "Dept. of Computer Science, Univ. of Wisconsin-Madison",
year = 1986,
number = "TR-635",
month = mar,
keyword = "parallel I/O, database, GAMMA"
}
@inproceedings{dewitt:gamma-dbm,
author = "David J. DeWitt and Shahram Ghandeharizadeh and
Donovan Schneider",
title = "A Performance Analysis of the {GAMMA} Database Machine",
booktitle = "ACM SIGMOD Conference '88",
month = jun,
year = 1988,
pages = "350--360",
keyword = "parallel I/O, database, performance analysis, Teradata, GAMMA"
}
@inproceedings{dewitt:gamma2,
author = "David J. DeWitt and Robert H. Gerber and Goetz Graefe
and Michael L. Heytens and Krishna B. Kumar and M. Muralikrishna",
title = "{GAMMA} --- {A} High Performance Dataflow Database Machine",
booktitle = vldb12,
year = 1986,
pages = "228--237",
keyword = "parallel I/O, database, GAMMA"
}
@inproceedings{ellis:prefetch,
author = "Carla Schlatter Ellis and David Kotz",
title = "Prefetching in File Systems for {MIMD} Multiprocessors",
booktitle = icpp89,
year = 1989,
month = aug,
pages = "I:306--314",
keyword = "dfk, parallel file system, prefetching, disk caching,
MIMD, parallel I/O"
}
@inproceedings{flynn:hyper-fs,
author = "Robert J. Flynn and Haldun Hadimioglu",
title = "A Distributed {Hypercube} File System",
booktitle = hyper88,
year = 1988,
pages = "1375--1381",
keyword = "parallel I/O, hypercube, parallel file system"
}
@article{garcia:striping-reliability,
author = "Hector Garcia-Molina and Kenneth Salem",
title = "The Impact of Disk Striping on Reliability",
journal = "{IEEE} Database Engineering Bulletin",
month = mar,
year = 1988,
volume = 11,
number = 1,
pages = "26--39",
keyword = "parallel I/O, disk striping, reliability, disk array"
}
@inproceedings{gibson:failcorrect,
author = "Garth A. Gibson and Lisa Hellerstein and Richard M.
Karp and Randy H. Katz and David A. Patterson",
title = "Failure Correction Techniques for Large Disk Arrays",
booktitle = "Third International Conference on Architectural
Support for Programming Languages and Operating
Systems (ASPLOS III)",
month = apr,
year = 1989,
pages = "123--132",
keyword = "parallel I/O, disk array, RAID, reliability"
}
@techreport{gibson:raid,
author = "Garth Gibson and Lisa Hellerstein and Richard Karp
and Randy Katz and David Patterson",
title = "Coding techniques for handling failures in large disk arrays",
number = "UCB/CSD 88/477",
month = dec,
year = 1988,
institution = "UC Berkeley",
keyword = "parallel I/O, RAID, reliability, disk array"
}
@mastersthesis{husmann:format,
author = "Harlan Edward Husmann",
title = "High-Speed Format Conversion and Parallel {I/O} in
Numerical Programs",
year = 1984,
month = jan,
school = "Department of Computer Science, Univ. of Illinois at
Urbana-Champaign",
note = "Available as TR number UIUCDCS-R-84-1152.",
keyword = "parallel I/O, I/O"
}
@booklet{intel:examples,
key = "Intel",
howpublished = "Intel Corporation Background Information",
title = "Concurrent {I/O} Application Examples",
year = 1989,
keyword = "file access pattern, parallel I/O, IPSC2, hypercube"
}
@booklet{intel:ipsc2io,
key = "Intel",
howpublished = "Intel Corporation",
title = "{iPSC/2} {I/O} Facilities",
year = 1988,
note = "Order number 280120-001",
keyword = "parallel I/O, hypercube, IPSC2"
}
@misc{intelio,
key = "Intel",
title = "Intel beefs up its {iPSC/2} supercomputer's {I/O} and
memory capabilities",
howpublished = "Electronics",
year = 1988,
month = nov,
volume = 61,
number = 11,
pages = 24,
keyword = "parallel I/O, hypercube, IPSC2"
}
@article{katz:io-subsys,
author = "Randy H. Katz and John K. Ousterhout and David A.
Patterson and Michael R. Stonebraker",
title = "A Project on High Performance {I/O} Subsystems",
journal = "{IEEE} Database Engineering Bulletin",
month = mar,
year = 1988,
volume = 11,
number = 1,
pages = "40--47",
keyword = "parallel I/O, RAID, Sprite, reliability, disk striping,
disk array"
}
@phdthesis{kim:interleave,
author = "Michelle Y. Kim",
title = "Synchronously Interleaved Disk Systems with their
Application to the Very Large {FFT}",
school = "IBM Thomas J. Watson Research Center",
address = "Yorktown Heights, New York 10598",
year = 1986,
note = "IBM Report number RC12372",
keyword = "parallel I/O, disk striping, file access pattern,
disk array"
}
@article{kim:interleaving,
author = "Michelle Y. Kim",
title = "Synchronized Disk Interleaving",
journal = ieeetc,
year = 1986,
volume = "C-35",
number = 11,
pages = "978--988",
month = nov,
keyword = "parallel I/O, disk striping, disk array"
}
@article{kotz:prefetch,
author = "David Kotz and Carla Schlatter Ellis",
title = "Prefetching in File Systems for {MIMD} Multiprocessors",
journal = ieeetpds,
year = 1990,
month = jan,
volume = 1,
number = 1,
pages = "?",
note = "To appear.",
keyword = "dfk, parallel file system, prefetching, MIMD, disk
caching, parallel I/O"
}
@inproceedings{livny:stripe,
author = "M. Livny and S. Khoshafian and H. Boral",
title = "Multi-Disk Management Algorithms",
booktitle = metrics87,
year = "1987",
pages = "69--77",
month = may,
keyword = "parallel I/O, disk striping, disk array"
}
@techreport{lo:disks,
author = "Raymond Lo and Norman Matloff",
title = "A Probabilistic Limit on the Virtual Size of
Replicated File Systems",
institution = "Department of EE and CS, UC Davis",
year = 1989,
keyword = "parallel I/O, replication, file system, disk shadowing"
}
@article{manuel:logjam,
author = "Tom Manuel",
title = "Breaking the Data-rate Logjam with arrays of small
disk drives",
journal = "Electronics",
year = 1989,
month = feb,
volume = 62,
number = 2,
pages = "97--100",
keyword = "parallel I/O, disk array, I/O bottleneck"
}
@article{masters:pario,
author = "Masters",
title = "Improve Disk Subsystem Performance with Multiple Serial
Drives in Parallel",
journal = ctr,
volume = 7,
number = 9,
month = jul,
year = 1987,
keyword = "unverified, parallel I/O"
}
@article{matloff:multidisk,
author = "Norman S. Matloff",
title = "A Multiple-Disk System for both Fault Tolerance and
Improved Performance",
journal = ieeetor,
volume = "R-36",
number = 2,
year = 1987,
month = jun,
pages = "199--201",
keyword = "parallel I/O, reliability, disk shadowing"
}
@article{mokhoff:pario,
author = "Nicholas Mokhoff",
title = "Parallel Disk Assembly Packs 1.5 {GBytes}, runs at 4
{MBytes/s}",
journal = "Electronic Design",
month = nov,
year = 1987,
pages = "45--46",
keyword = "parallel I/O, I/O, disk hardware, disk striping,
reliability"
}
@article{moren:controllers,
author = "Moren",
title = "Design of Controllers is Key Element in Disk Subsystem Throughput",
journal = ctr,
month = "Spring",
year = 1988,
keyword = "unverified, parallel I/O, disk hardware"
}
@inproceedings{ng:diskarray,
author = "Spencer Ng",
title = "Some Design Issues of Disk Arrays",
booktitle = compcon,
note = "San Francisco, CA",
month = "Spring",
year = 1989,
pages = "137--142",
keyword = "parallel I/O, disk array"
}
@inproceedings{ng:interleave,
author = "S. Ng and D. Lang and R. Selinger",
title = "Trade-offs Between Devices and Paths in Achieving
Disk Interleaving",
booktitle = "IEEE Symposium on Computer Architecture",
year = 1988,
pages = "196--201",
keyword = "parallel I/O, disk hardware, disk caching, I/O bottleneck"
}
@techreport{park:pario,
author = "Arvin Park and K. Balasubramanian",
title = "Providing Fault Tolerance in Parallel Secondary Storage
Systems",
number = "CS-TR-057-86",
institution = "Department of Computer Science, Princeton University",
month = nov,
year = 1986,
keyword = "parallel I/O, reliability"
}
@inproceedings{patterson:raid,
author = "David Patterson and Garth Gibson and Randy Katz",
title = "A case for redundant arrays of inexpensive disks {(RAID)}",
booktitle = "ACM SIGMOD Conference '88",
month = jun,
year = 1988,
pages = "109--116",
keyword = "parallel I/O, RAID, reliability, cost analysis, I/O
bottleneck, disk array"
}
@inproceedings{patterson:raid2,
author = "David Patterson and Peter Chen and Garth Gibson and
Randy H. Katz",
title = "Introduction to Redundant Arrays of Inexpensive Disks {(RAID)}",
booktitle = compcon,
month = "Spring",
year = 1989,
pages = "112--117",
keyword = "parallel I/O, RAID, reliability, cost analysis, I/O
bottleneck, disk array"
}
@inproceedings{pierce:pario,
author = "Paul Pierce",
title = "A Concurrent File System for a Highly Parallel Mass
Storage System",
booktitle = hyper89,
year = 1989,
keyword = "unverified, parallel I/O, hypercube, IPSC2,
parallel file system"
}
@inproceedings{reddy:hyperio1,
author = "A. L. Reddy and P. Banerjee and Santosh G. Abraham",
title = "{I/O} Embedding in Hypercubes",
booktitle = icpp88,
year = 1988,
volume = 1,
pages = "331--338",
keyword = "parallel I/O, hypercube"
}
@inproceedings{reddy:hyperio2,
author = "A. L. Reddy and P. Banerjee",
title = "{I/O} issues for hypercubes",
booktitle = "International Conference on Supercomputing",
year = 1989,
note = "To appear",
keyword = "unverified, parallel I/O, hypercube"
}
@inproceedings{reddy:pario,
author = "A. Reddy and P. Banerjee",
title = "An Evaluation of multiple-disk {I/O} systems",
booktitle = icpp89,
year = 1989,
pages = "I:315--322",
keyword = "parallel I/O, disk array, disk striping"
}
@unpublished{reddy:pario2,
author = "A. Reddy and P. Banerjee",
title = "Evaluation of multiple-disk {I/O} systems",
journal = ieeetc,
note = "submitted",
keyword = "unverified, parallel I/O, disk array, disk striping"
}
@inproceedings{salem:diskstripe,
author = "Kenneth Salem and Hector Garcia-Molina",
title = "Disk Striping",
booktitle = "IEEE 1986 Conference on Data Engineering",
year = 1986,
pages = "336--342",
keyword = "parallel I/O, disk striping, disk array"
}
@techreport{salem:striping,
author = "Kenneth Salem and Hector Garcia-Molina",
title = "Disk Striping",
institution = "EECS Dept. Princeton Univ.",
number = 332,
year = 1984,
month = dec,
keyword = "parallel I/O, disk striping, disk array"
}
@inproceedings{schneider:pjoin,
author = "Donovan A. Schneider and David J. DeWitt",
title = "A Performance Evaluation of Four Parallel Join Algorithms
in a Shared-Nothing Multiprocessor Environment",
booktitle = "ACM SIGMOD Record",
month = jun,
year = 1989,
pages = "110--121",
keyword = "parallel I/O, database, performance analysis"
}
@techreport{schulze:raid,
author = "Martin Schulze",
title = "Considerations in the Design of a {RAID} Prototype",
number = "UCB/CSD 88/448",
month = aug,
year = 1988,
institution = "UC Berkeley",
keyword = "parallel I/O, RAID, disk array, disk hardware"
}
@inproceedings{schulze:raid2,
author = "Martin Schulze and Garth Gibson and Randy Katz and David
Patterson",
title = "How Reliable is a {RAID}?",
booktitle = compcon,
year = 1989,
month = "Spring",
keyword = "parallel I/O, reliability, RAID, disk array, disk hardware"
}
@mastersthesis{stabile:disks,
author = "James Joseph Stabile",
title = "Disk Scheduling Algorithms for a Multiple Disk
System",
school = "UC Davis",
year = 1988,
keyword = "parallel I/O, parallel file system, mirrored disk, disk
scheduling"
}
@article{stone:query,
author = "Harold S. Stone",
title = "Parallel Querying of Large Databases: {A} Case
Study",
journal = "IEEE Computer",
year = 1987,
month = oct,
volume = 20,
number = 10,
pages = "11--21",
keyword = "parallel I/O, database, SIMD, connection machine"
}
@techreport{stonebraker:xprs,
author = "Michael Stonebraker and Randy Katz and David
Patterson and John Ousterhout",
title = "The Design of {XPRS}",
number = "UCB/ERL M88/19",
month = mar,
year = 1988,
institution = "UC Berkeley",
keyword = "parallel I/O, disk array, RAID, Sprite, disk hardware,
database"
}
@manual{symult:guide,
title = "Programmer's Guide to the Series 2010 System",
organization = "Symult Systems",
address = "Monrovia, CA",
edition = "First",
month = "March 24",
year = 1989,
keyword = "parallel I/O, parallel file system"
}
@booklet{teradata:dbc,
key = "Teradata",
howpublished = "Teradata Corporation Booklet",
title = "{DBC/1012}",
year = 1988,
keyword = "parallel I/O, database machine, Teradata"
}
@techreport{think:cm-2,
key = "TM",
title = "Connection Machine Model {CM-2} Technical Summary",
institution = "Thinking Machines",
number = "HA87-4",
month = apr,
year = 1987,
keyword = "parallel I/O, connection machine, disk hardware, SIMD"
}
@inproceedings{witkowski:hyper-fs,
author = "Andrew Witkowski and Kumar Chandrakumar and Greg
Macchio",
title = "Concurrent {I/O} System for the {Hypercube} Multiprocessor",
booktitle = hyper88,
year = 1988,
pages = "1398--1407",
keyword = "parallel I/O, hypercube, parallel file system"
}
------------
David Kotz
Department of Computer Science, Duke University, Durham, NC 27706 USA
ARPA: dfk@cs.duke.edu
CSNET: dfk@duke
UUCP: decvax!duke!dfkwilkes@hplabsz.HPL.HP.COM (John Wilkes) (11/15/89)
In article <9647@saturn.ucsc.edu>, yaronf%shum.Huji.AC.IL@CUNYVM.CUNY.EDU (Yaron Farber) writes: > > Two further pointers: (though for more sophisticated schemes...) > ... > 2. \bibitem{wilkes} Wilkes, J. DataMesh --- scope and objectives: > a commentary. TR HPL-DSD-89-44, Hewlett-Packard Laboratories, > Palo Alto, CA, July 1989. Since I've encountered at least one poor soul who has tried to track this down the hard way ;-}, here's a simpler way to get a copy: send e-mail to wilkes@hplabs.hp.com, referencing this news article and the TR number. In case anybody cares, the document is a proposal (and partial rationale) for a research project to develop the technology needed for a storage server in the 1995 timeframe. If you are interested in discussing possible research collaboration on such a thing, please send mail ... john wilkes +1 (415) 857-3568 wilkes@hplabs.hp.com