5432gws@homxb.UUCP (G.SWIFT) (05/07/85)
Recently, some controversy has arisen in my group regarding whether interprocess pipes use primary (main) or secondary (disc) memory for buffering. Our best guess is that buffering is done internally in the operating system, however, there are some that believe that since a file descriptor is returned by the pipe sytem call that this indicates that a disc file is being used. I would appreciate a response via mail from anyone who knows the definitive answer to this question. Thank You George Swift AT&T Bell Labs H0 3E-334 x6105
chris@umcp-cs.UUCP (Chris Torek) (05/09/85)
> Recently, some controversy has arisen in my group regarding whether > interprocess pipes use primary (main) or secondary (disc) memory for > buffering. The answer is "yes" and "yes". System (III, V, Vr2, Vr2v2 [standard!]) and 4.1BSD (and 4.0 and 3.0 and 2.9 and 2.8BSD) implement pipes as disk files. 4.2BSD implements them as sockets (which never go to disk). However, in most cases the data in pipes never gets out of the block file system buffer cache (main memory). (Performance analysis? What performance analysis?) -- In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 454 4251) UUCP: {seismo,allegra,brl-bmd}!umcp-cs!chris CSNet: chris@umcp-cs ARPA: chris@maryland
mwherman@watcgl.UUCP (Michael W. Herman) (05/13/85)
> (Performance analysis? What performance analysis?)
I have done some experimenting in an attempt to determine what the
effective/sustained data transfer rates are for pipes in 4.2BSD on
a 780. The experiment consisted of a simple producer process writing
blocks of data to a consumer process. A total of 1M bytes of data
was transferred in each case. The statistics I gathered consisted
of the block size (i.e. arg 3 to *write(2)*) and elapsed time for each
of the producer and consumer processes. I varied the block size from
128 bytes to 4096 bytes in steps of 128 bytes (In another version of
the experiment, I used an upper bound of 16K bytes.) The pipe was created
by calling the *pipe(2)* function. The experiments were run on an almost
empty machine (i.e. load average < 1.0).
With only one pair of producer/consumer processes running on the 780,
a maximum sustained transfer rate of about 330,000 bytes/second was realized
using a block size of 3K bytes. There were other local maximums at 1K bytes
(200,000 bytes/second), 2K bytes (280,000 bytes/second) and 4K bytes (326,000
bytes/second). For comparison, the transfer rate for a block size of 128 bytes
was about 34,000 bytes/second.
With 4 pairs of producer/consumer processes running simultaneously, the
corresponding figures were:
block size (bytes) approx. transfer rate (bytes/second)
4K 47,000
3K 70,000
2K 40,000
1K 35,000
128 6,300
If anyone would like to run these programs on their machines, I'll be
happy to send them to you. Warning: they're pretty "bare bones" and
you will probably want to change them to suit your own purposes.
Michael Herman
Computer Graphics Laboratory
Department of Computer Science
University of Waterloo
Waterloo, Ontario, Canada N2L 3G1
UUCP: {allegra,ihnp4,watmath}!watcgl!mwherman
-or- {cbosgd,clyde,decvax,linus,tektronix,utcsrgv}!watmath!watcgl!mwherman
CSNET: mwherman%watcgl@waterloo.CSNET
ARPA: mwherman%watcgl%waterloo.CSNET@csnet-relay.ARPA
p.s. I didn't do any statistical tests of significance with my results.jf@sal.UUCP (Johan Finnved) (05/21/85)
In article <5566@umcp-cs.UUCP> chris@umcp-cs.UUCP (Chris Torek) writes: >> .. whether interprocess pipes use primary (main) or secondary (disc) memory > >The answer is "yes" and "yes". >... However, in most cases the data in pipes never gets out of the block >file system buffer cache (main memory). > Version 7 does a poor job at avoiding disc writes when using pipes: Whenever a write operation fills up a buffer a write is started even though the odds are high that the data will soon be read and no longer needed. Moreover when pipe reads have consumed a buffer it does not clear the B_DELWRI bit so delayed writes of no longer referenceable data happen. So using a pipe usually creates writes but seldom reads. Are later versions different ? Johan Finnved, Objecta, Sweden <jf@sal.UUCP>