cowan@spanky.sps.mot.com (12/21/89)
Any one got a few minutes to answer a few questions? For UUCP, FTP, and NFS, in what way (if any) do each of these programs perform data integrity checks and data correction? I'm not talking about how a packet reaches the destination, but how, if by sheer magic a bit in the data being transferred is munged up, it is detected and corrected. At what levels (in the protocol stack) is the data integrity checking done. What I'm really interested in is knowing is: Which one of these three methods is most reliable (in a data integrity sense) for transferring data, and why? (Opinions, without sensible backing arguments, will be directed to /dev/null.) Any takers? Andy Cowan cowan@soleil.sps.mot.com (602)821-4942 Andy Cowan cowan@soleil.sps.mot.com (602)821-4942
meissner@osf.osf.org (Michael Meissner) (12/21/89)
In article <1972@dover.sps.mot.com> cowan@spanky.sps.mot.com () writes: | |Any one got a few minutes to answer a few questions? | |For UUCP, FTP, and NFS, in what way (if any) do each of these programs |perform data integrity checks and data correction? I'm not talking |about how a packet reaches the destination, but how, if by sheer magic |a bit in the data being transferred is munged up, it is detected and |corrected. At what levels (in the protocol stack) is the data |integrity checking done. When I was at Data General, I had some particularly bad experiences with NFS. Our ethernet(s) at the time had a variety of hardware hooked up to it (Data General MV's running AOS/VS communicating with X.25 and some TCP/IP over ethernet, MV's running DG/UX communicating with TCP/IP, MSDOS-based PC's speaking X.25, etc.). We added some Sun workstations to bootstrap to the AViiON (88k based) system. The Sun's were running 3.5 SunOS. We started noticing random file corruption problems when compiling stuff on the Sun's on NFS mounted partitions from MV's running DG/UX. At first we thought it was some bug in our server code, but the network people tracked it down to random packets getting a bit or two trashed. In order to get higher throughput, SunOS 3.5 NFS uses UDP connections. Unlike TCP, UPD connections can come out of order, dupiclated, trashed, etc. and it is assumed that higher layers of software will deal with this. However there is an option to enable the checksums on UDP packets, and not deliver the packets that had a bad checksum specified. The way we dealt with the problem was to take the NFS reference source from Sun, recompile the one module that spit out the UDP packets to turn on UDP checksums, ar the module into the appropriate kernel library, and regen the systems. Viola, it fixed our problem. By the way, we did attempt to report the bug to Sun, and where told it was a feature, since checksumming slows down the machine. In practice, we never noticed the slowdown, and we could go back to building the software, without some object file randoming having it's bits changed. |What I'm really interested in is knowing is: Which one of these three |methods is most reliable (in a data integrity sense) for transferring |data, and why? (Opinions, without sensible backing arguments, will |be directed to /dev/null.) | |Any takers? See above. The normal NFS over UDP depends on your link layers of the software not to trash the bits. Both FTP (via the reliable TCP streams), and UUCP (via the low level packet handler) are protected by checksums or the like. How well those checksums behave under stress, (ie, whether they can fixup multiple bit errors or not) I don't know. I have heard about random FTP corruption problems, but most of those are either not using binary/image mode, or ftp'ing a file from a NFS mounted disk.
barmar@Think.COM (12/21/89)
FTP and NFS do no integrity checking of their own. FTP is intended to be
used over a reliable byte-stream, normally TCP, which does perform
integrity checks (using a checksum by default). NFS uses RPC, which
normally runs over UDP, which has an optional checksum (it's the sender's
option -- the receiver is required to check it if it's included).
Since UUCP is normally run over phone lines, which don't have high
integrity, it does its own integrity checks.
Barry Margolin, Thinking Machines Corp.
barmar@think.com
{uunet,harvard}!think!barmarjohnl@esegue.segue.boston.ma.us (John R. Levine) (12/21/89)
In article <1972@dover.sps.mot.com> cowan@spanky.sps.mot.com () writes: >For UUCP, FTP, and NFS, in what way (if any) do each of these programs >perform data integrity checks and data correction? Uucp passes 128 byte packets each protected by a CRC of, I believe, 8 bits. If a packet is bad the receiver ignores it and the sender times out and retransmits it. The most common place I've seen uucp lose data is that older versions didn't notice if the disk filled up. FTP and NFS both depend on the underlying network to do the error correction. Ethernets and dedicated lines with the usual synchronous protocols both do CRCs and again the strategy is to ignore the bad block and wait until it's retransmitted. The ignoring happens at the IP level; retransmission happens at the virtual circuit level for TCP and is up to the application for UDP. There are optional checksums in TCP (used by ftp) and UDP (used by nfs) headers, but they are often turned off on the assumption that link level error correction will be adequate. SLIP (IP over RS-232 async) does no error detection at all, so except for the aforementioned checksums, you're naked. -- John R. Levine, Segue Software, POB 349, Cambridge MA 02238, +1 617 864 9650 johnl@esegue.segue.boston.ma.us, {ima|lotus|spdcc}!esegue!johnl "Now, we are all jelly doughnuts."
art@salt.acc.com (Art Berggreen) (12/22/89)
In article <1989Dec21.025229.2837@esegue.segue.boston.ma.us> johnl@esegue.segue.boston.ma.us (John R. Levine) writes: >There are optional checksums in TCP (used by ftp) and UDP (used by nfs) >headers, but they are often turned off on the assumption that link level error >correction will be adequate. NO! Only UDP checksums are optional (a dubious option at that), TCP checksums are ALWAYS required. -- + Art Berggreen Advanced Computer Communications + | <art@salt.acc.com> Santa Barbara Street | + (805)963-9431 Santa Barbara, CA 93101 +