ok@quintus (06/16/88)
Amongst the goodies recently posted to comp.sources.unix was one whose README file said that it was a rewrite of a public domain version, and that one of the changes that had been made was to use read(2)/write(2)/ lseek(2)/open(2)/close(2) for I/O instead of using stdio. THAT WAS A BAD IDEA. If you want to make your program's I/O more efficient, simply replacing stdio calls by low-level UNIX calls is not a good idea. If you use stdio, that package will do I/O a buffer at a time, so you only get a system call (and a disc access) when a buffer is filled or emptied. The package I am talking about was reading and writing 56 byte chunks, so it was doing a system call and a disc access for every 56 bytes. (And since 56 does not divide 8192 evenly, some of these transfers could cost two disc accesses.) To make your disc I/O more efficient, do as few transfers as possible, and do as much useful work as you can in each transfer. For example, in this balanced tree package, it would have been a good idea to keep the tree as a tree of 8k "pages", each page containing a set of keys (128 64-byte nodes would fit into a page, reducing the number of disc accesses required by a factor of 7). [This isn't a question, but it is the answer to a question that _should_ have been asked before the package was written.]
sbc@sp7040.UUCP (Stephen Carroll) (06/18/88)
In article <122@quintus.UUCP>, ok@quintus writes: > > stuff deleted.... > > THAT WAS A BAD IDEA. > > If you want to make your program's I/O more efficient, simply replacing > stdio calls by low-level UNIX calls is not a good idea. If you use > stdio, that package will do I/O a buffer at a time, so you only get a > system call (and a disc access) when a buffer is filled or emptied. > > The package I am talking about was reading and writing 56 byte chunks, > so it was doing a system call and a disc access for every 56 bytes. > (And since 56 does not divide 8192 evenly, some of these transfers > could cost two disc accesses.) > WRONG!!!! You could read and write 1 byte at a time, but this does not create a disk access for each system call. It is true that read/write causes a system call on each invocation, as compared to stdio, but Unix DOES NOT read and write the disk on each of those system calls. The kernel will buffer at least on block of data at a time, and more likely two blocks with sequential reads and disk read-aheads. It is true that the greater number of system calls and context switches will tend to slow the program down, but they will not cause more disk accesses. The only case where this would be true would be on a loaded system where kernel buffers are at a premium and have to be flushed very often. Of course if you were accessing the disk via the raw interface (character special device), this is then true. But of course the stdio library would not work in this case, would it! MAIL: Steve B. Carroll PHONE: (801) 594 5623 Unisys Corporation, MS B2F12 FAX: (801) 594 5660/4750/5174 322 North 2200 West Salt Lake City, UT 84116-2979 TELEX: (310) 3789522 UNIVAC SLC UUCP: ...!{ihnp4 | hpda | sun }!sp7040!sbc
ok@quintus.uucp (Richard A. O'Keefe) (06/20/88)
In article <425@sp7040.UUCP> sbc@sp7040.UUCP (Stephen Carroll) writes: >In article <122@quintus.UUCP>, ok@quintus writes: >> The package I am talking about was reading and writing 56 byte chunks, >> so it was doing a system call and a disc access for every 56 bytes. >> (And since 56 does not divide 8192 evenly, some of these transfers >> could cost two disc accesses.) > >WRONG!!!! You could read and write 1 byte at a time, but this does not create >a disk access for each system call. It is true that read/write causes a >system call on each invocation, as compared to stdio, but Unix DOES NOT read >and write the disk on each of those system calls. Oh drat, insufficient context. I was confusing two things in my comments. (1) the program in question was doing read(2) and write(2) instead of stdio calls, so it was getting a lot more system calls than it needed (2) it was ALSO doing a lot of seeks (which I didn't mention) so each of those reads and writes tended to turn into an actual disc access because they weren't adjacent. FOR THIS PARTICULAR PROGRAM, it is the case that UNIX *does* end up doing one disc access per read(2) or write(2). [You can use the BSD "time" command to find out how many accesses are happening.] Any program which does nearly as many seeks as transfers is likely to see the same thing. There are two points which I was trying to make: (1) if you are doing sequential I/O, using read(2) and write(2) is _not_ a win over stdio, because you do more system calls. (2) if you are doing random I/O into a disc file, the way to better performance is to try to do a few large transfers, with as much useful work per transfer as possible. At this level of hacking, read(2) and write(2) may be appropriate, but they aren't _enough_. Caution to wizards: don't jump to the conclusion that someone misunderstands UNIX. He may be quoting quite accurately what time(1) says that particular program is doing. (He may also be bad at explaining things.)