fouts@orville.nas.nasa.gov (Marty Fouts) (02/07/88)
In article <1244@homxc.UUCP> dwc@homxc.UUCP (Malaclypse the Elder) writes: > >. . . for one, i have been >thinking of the amount of copying that is done in passing >data through the various layers of network protocols. perhaps >with a very LARGE kernel address space and suitable alignment >of data structures, this data can be memory-mapped through the >different layers instead of copied. > >i haven't really thought much about it. what do people think? > The only times that data copies are necessary in protocol implementations are 1) Crossing security/consistency boundries, such as from the user address space to the kernel. and 2) When a layer adds cruft such as headers to unaligned packets, such as when an IP header gets shoved in front of a TCP packet. passing the packet from one layer to another can easily be done by passing pointers to the start of the packet. There are work arounds which can limit the amount of copying necessary in both cases, and they buy various amounts of performance on various architectures. They can almost all be implemented with no more kernel memory than the amount necessary to buffer all of the packets you want buffered + one maximum packet worth for crufty things, so the amount of kernel address space only has to be large if you intend to hold a lot of data. An example (hack) workaround for boundry crossing is to require applications to page align buffers and use copy on write. (You can also set the system up so only data is in the user buffer and not care until transmission time what the application does with it, but if the application unmaps the page, cleanup is a bother.) A workaround for packet forming is to use scatter/gather i/o in the hardware controllers and build pointer structures like: Some_Packet -> Layer_1_header -> (body of header) Layer_2_header -> (not necessarily contiguous body) . . . Layer_N_header -> (somewhere other altogether) Body -> (Data in user address space) And then pass &Some_Packet around to the various layers and let them build all of the cruft they want by filling in pointers (or not, if this layer doesn't need to do anything for this packet.) If the controller can handle the scatter/gather, than there is a big performance win; even if it can't, much copying can be avoided at a fairly small cost in the device drivers. This doesn't add (much) to the amount of kernel address space you use for buffers, which you may want to make large for other reasons.