brown@noao.arizona.edu (Mike Brown) (02/01/88)
Why is the transfer rate when a process writes to a remote NFS file 3-4 times
smaller than the transfer rate when reading a remote NFS file?
- Is this asymmetry a characteristic of NFS?
- Do Ultrix(2.0) and 4.3BSD/NFS from Mt. Xinu both have brain damaged
NFS implementations?
- Is my test brain damages?
I am surprized at the difference in the tranfer rate for writing a file
compared to reading a file. I understand that creating/writing a file in
Unix is slower because of extra overhead involved during file allocation.
The difference I see is far greater than the difference in performance for
reads/writes on a local file system in Unix. The disks I use have effective
transfer rates of about 280-300 Kilobytes/sec.
The test I ran was:
writing:
time /bin/cp /local/2_megabytes /remote/2_megabytes
time /bin/cp /local/2_megabytes /remote/2_megabytes
time /bin/cp /local/2_megabytes /remote/2_megabytes
reading:
time /bin/cp /remote/2_megabytes /local/2_megabytes
time /bin/cp /remote/2_megabytes /local/2_megabytes
time /bin/cp /remote/2_megabytes /local/2_megabytes
The test was run between microvaxes running 4.3BSD from Mt. Xinu
and it was also run between microvaxes running Ultrix 2.0. The systems
were running multiuser with no other user processes active.
The transfer rates were: reading (110-90 Kbytes/sec)
writing (20-25 Kbytes/sec)
Regards,
Mike Brown Biomedical Computer Lab.
Washington University
700 S. Euclid Ave.
St. Louis, MO 63110
(314) 362-2135
uucp: uunet!wucs1!brown or {...}!noao!brown
internet: brown@noao.arizona.edu
( Please excuse my posting this from arizona. )
( News out of Wash. Univ. is broken. )hedrick@athos.rutgers.edu (Charles Hedrick) (02/01/88)
You note that reading is faster than writing via NFS. We see this on Suns also, though the difference we see is not quite as drastic as yours. Some tests I just did showed writing a big file to be about a factor of 3 slower than reading. I believe this is because reading can do readahead, but it is hard to predict what the next byte a user is going to write is going to be. (Note that the machine doing the reading has a Ciprico controller, so the controller is doing a lot of readahead also. Reading across the network was actually twice as fast as reading locally, because the local machine had a normal Xylogics controller.) There are some other things that can affect NFS performance though. try nfsstat before and after your test. See if you are getting retransmissions. We have run into systems that can send data faster than they can receive them (Sun 4's). In that case, it turned out that not running biod actually improved throughput by a factor of 3. For more normal cases, biod does help. You can sometimes increase performancy by running more copies, e.g. /etc/biod 8 instead of /etc/biod 4.
sxm@philabs.Philips.Com (Sandeep Mehta) (02/01/88)
In article <663@noao.UUCP> brown@noao.arizona.edu (Mike Brown) writes: > >Why is the transfer rate when a process writes to a remote NFS file 3-4 times >smaller than the transfer rate when reading a remote NFS file? > > ...... File reading is probably faster because of the read ahead, but when you write back blocks to a remote NFS, the server probably waits for acknowlegement of a successful write, i.e. the server waits until it receives the ack, before it trashes the cached block. I would like to know if there is detailed answer to this. sandeep -- Sandeep Mehta TEL : (914)-945-6478 Robotics & Flexible Automation UUCP: uunet!philabs!sxm Philips Laboratories ARPA: sxm@philabs.philips.com
sas@pyrps5 (Scott Schoenthal) (02/02/88)
In article <663@noao.UUCP> brown@noao.arizona.edu (Mike Brown) writes: > >Why is the transfer rate when a process writes to a remote NFS file 3-4 times >smaller than the transfer rate when reading a remote NFS file? Due to the stateless nature of NFS, all write requests received by a server must be written synchronously to the disk before they can be acknowledged to the client. If this was not true, the following could happen: Client A sends a write request (block 'n') to Server B. Server B acknowledges the write but does not write the block. Server B crashes. Server B then comes back up. Client A will then send block 'n+1' thinking that 'n' has been written out. In a UNIX NFS server implmenetation, the inode and block maps must also be updated during each synchronous block write. sas ---- Scott Schoenthal sas@pyrps5.pyramid.com Pyramid Technology Corp.
sxn%ingersoll@Sun.COM (Stephen X. Nahm) (02/02/88)
In article <663@noao.UUCP> brown@noao.arizona.edu (Mike Brown) writes: >Why is the transfer rate when a process writes to a remote NFS file 3-4 times >smaller than the transfer rate when reading a remote NFS file? > > - Is this asymmetry a characteristic of NFS? Yes. Here's a quote from ``The Sun Network Filesystem: Design, Implementation and Experience'' by Russel Sandberg: ``Much of the development time of NFS has been spent in improving performance. Our goal was to make NFS comparable in speed to a small local disk. The speed we were interested in is not raw throughput, but how long it takes to do normal work.'' There is then a discussion of performance tweaks that were made, concluding with: ``With these improvements, a diskless Sun-3 (68020 at 16.67 MHz.) using a Sun-3 server with a Fujitsu Eagle disk, runs the benchmarks faster than the same Sun-3 with a local Fujitsu 2243AS 84 Megabyte disk on a SCSI interface.'' Then: ``The 'write' [NFS operation] is slow because it is synchronous on the server. Fortunately, the number of write calls in normal use is very small (about 5% of all calls to the server) so it is not noticeable unless the client writes a large remote file.'' Finally: ``Since many people base performance estimates on raw transfer speed we also measured those. The current numbers on raw transfer speed are: 250 kilobytes/second for read and 60 kilobytes/second for write on a Sun-3 with a Sun-3 server.'' The factors you should be aware of are: NFS implementation, CPU bandwidth (both server and client), and type of disk involved (both server and client). >The transfer rates were: reading (110-90 Kbytes/sec) > writing (20-25 Kbytes/sec) Your ratios match well with what Rusty reported in his paper (his was about 4.1 read-to-write, and yours is about 4.4). Perhaps the CPU bandwidth of your client and server is the limiting factor. > - Do Ultrix(2.0) and 4.3BSD/NFS from Mt. Xinu both have brain damaged > NFS implementations? I believe both are based on Sun's reference port. We port SunOS NFS to vanilla 4.2 and 4.3BSD. The licensees use that as a reference in building their own product. > - Is my test brain damages? Not if this represents the typical kind of work you do. However a fairer benchmark would be real work, like doing compiles, nroffs, whatever. Steve Nahm sxn@sun.COM or sun!sxn Portable ONC/NFS
ps@diab.UUCP (Per Erik Sundberg) (02/02/88)
In article <663@noao.UUCP> brown@noao.arizona.edu (Mike Brown) writes: > >Why is the transfer rate when a process writes to a remote NFS file 3-4 times >smaller than the transfer rate when reading a remote NFS file? > > - Is this asymmetry a characteristic of NFS? Yes it is. Due to the statelessnes of the NFS protocol, writes are SYNCRONOUS. This is to make sure that the data written is on stable storage when the client call returns. Local file writes are normally asyncronous, which will make it possible to optimize the writes to disk using delayed writes. BTW, early Lachman System V NFS implementations have a bug in them, that will use async writes on the server side. Will improve performance a bit though :-) -- Per-Erik Sundberg, Diab Data AB SNAIL: Box 2029, S-183 02 Taby, Sweden ANALOG: +46 8-7680660 UUCP: mcvax!enea!diab!ps
grandi@noao.arizona.edu (Steve Grandi) (02/02/88)
In article <14163@pyramid.pyramid.com> sas@pyrps5.pyramid.com (Scott Schoenthal) writes: >Due to the stateless nature of NFS, all write requests received by a >server must be written synchronously to the disk before they can be >acknowledged to the client. If this was not true, the following could happen: > >In a UNIX NFS server implmenetation, the inode and block maps must >also be updated during each synchronous block write. At the Sun Users Group meeting I heard about one of "hacks" added to Sun OS and NFS to make NFS performance "good enough" to replace the unloved ND protocol for booting, paging and swapping. Consider what happens when an NFS server makes a write on behalf of a client. Even the simplest case of a "replacement" (a write that does not cause the file to be extended) requires TWO synchronous writes: one to write the data block and one to update time fields in the inode. Thus one of the hacks added to Sun OS 4.0 is to eliminate updating the inode atime and mtime fields for a "non-extending" write. Since client swap space will be contained in pre-allocated files on the server, you have just eliminated half of this source of overhead for client paging and swapping in NFS. -- Steve Grandi, National Optical Astronomy Observatories, Tucson AZ, 602-325-9228 UUCP: {arizona,decvax,hao,ihnp4}!noao!grandi or uunet!noao.arizona.edu!grandi Internet: grandi@noao.arizona.edu SPAN/HEPNET: 5356::GRANDI or DRACO::GRANDI
dougs@sequent.UUCP (Doug Schwartz) (02/03/88)
In article <663@noao.UUCP>, brown@noao.arizona.edu (Mike Brown) writes: > > Why is the transfer rate when a process writes to a remote NFS file 3-4 times > smaller than the transfer rate when reading a remote NFS file? I may not be able to explain why you are getting what you are getting, but this seems to be in line with Sun's published statistics. From the Summer 1985 USENIX prodeedings, page 129, "The current numbers on raw transfer speed are: 120 kilobytes/second for read (cp bigfile /dev/null) and 40 kilobytes/second for write." -- Doug Schwartz Sequent Computer Beaverton, Oregon tektronix!ogcvax!sequent!dougs