dwh@twg-ap.UUCP (Dave Hamaker) (05/17/88)
I find myself often needing/wanting to put filters before and/or after raw devices and network connections. Often, the volume of data involved exceeds the ulimit file size controls. If I don't raise the ulimit, the pipes fail. This doesn't make sense to me, as pipes do not accumulate resources in the same kind of way as disk files: an out-of-control disk file can use up all the disk space; an out-of-control pipe consumes only renewable resources. The ulimit documentation is inclusive on the matter; in fact, it sounds confused. If you ask me, I smell a bug of omission; the pipe type just rode along with the file type during ulimit implementation, without much thought about it. -Dave Hamaker The Wollongong Group ...!ucbvax!ucsfcgl!hoptoad!sun!amdahl!twg-ap!dwh
gwyn@brl-smoke.ARPA (Doug Gwyn ) (05/17/88)
In article <242@twg-ap.UUCP> dwh@twg-ap.UUCP (Dave Hamaker) writes: >If I don't raise the ulimit, the pipes fail. If this is true, then it's a horrible bug. Please fix it!
levy@ttrdc.UUCP (Daniel R. Levy) (05/18/88)
In article <242@twg-ap.UUCP>, dwh@twg-ap.UUCP (Dave Hamaker) writes: # I find myself often needing/wanting to put filters before and/or after raw # devices and network connections. Often, the volume of data involved exceeds # the ulimit file size controls. If I don't raise the ulimit, the pipes fail. # This doesn't make sense to me, as pipes do not accumulate resources in the # same kind of way as disk files: an out-of-control disk file can use up all # the disk space; an out-of-control pipe consumes only renewable resources. # The ulimit documentation is inclusive on the matter; in fact, it sounds # confused. If you ask me, I smell a bug of omission; the pipe type just rode # along with the file type during ulimit implementation, without much thought # about it. # The Wollongong Group What VERSION of System V are you using, on what machine? The only pipe-related ulimit problem I've noticed here (ttrdc ttrdc 2.0v3 1208 3B-20S) is if I make the ulimit so ridiculously small (like 9) that the ulimit is smaller then the pipe's buffering capacity (5120 bytes). In this event a write larger than the ulimit indeed fails with EFBIG. I suppose you could call that a bug, albeit a rather unlikely bug. If I make the ulimit larger than the pipe's buffering capacity (even 11) then I can call write() with as much as I please (1 megabyte? no problem, it works) to a pipe. (I just tried it again on a 3B2, SVR3.1. Same results.) -- |------------Dan Levy------------| Path: ihnp4,<most AT&T machines>!ttrdc!levy | AT&T | Weinberg's Principle: An expert is a | Data Systems Group | person who avoids the small errors while |--------Skokie, Illinois--------| sweeping on to the grand fallacy.
kent@happym.UUCP (Kent Forschmiedt) (05/19/88)
[ In article <242@twg-ap.UUCP> dwh@twg-ap.UUCP (Dave Hamaker)
notes that his ulimit is applied to pipes as well as regular
files, and that he feels that it is very stupid. ]
I think that ulimit, at least as implemented, and especially as
configured in most distributions, is extremely stupid. Fortunately
for me, Motorola made it a tunable parameter, so I can make it VERY
large and seldom worry about it. For others, it is not so easy.
I have told a few people how to use a fake init:
/* init.fake.c make a bootable floppy before you try this */
main(argc,argv) char **argv;
{ ulimit(2,BIGNUM); execv("/etc/init.real", argv); }
I have never heard anyone say anything nice about ulimit, though -
only, "How do I get around it?" To me, that is a sure indication
of a misfeature.
--
--
Kent Forschmiedt -- kent@happym.UUCP, tikal!camco!happym!kent
Happy Man Corporation 206-282-9598dwh@twg-ap.UUCP (Dave Hamaker) (05/19/88)
In article <242@twg-ap.UUCP>, dwh@twg-ap.UUCP I write: > ... > If I don't raise the ulimit, the pipes fail. > ... Oh embarrassment! I had come to this conclusion because of past experiences and when I try it now, it doesn't fail. The circumstantial evidence was overwhelming... but apparently quite wrong. Something else may have been at work; I have no reason to suspect the operating system now. My apologies for wasting everyone's time and the net bandwidth. -Dave Hamaker The Wollongong Group ...!ucbvax!ucsfcgl!hoptoad!sun!amdahl!twg-ap!dwh
gwyn@brl-smoke.ARPA (Doug Gwyn ) (05/19/88)
In article <244@twg-ap.UUCP> dwh@twg-ap.UUCP (Dave Hamaker) writes: >> If I don't raise the ulimit, the pipes fail. >Oh embarrassment! I had come to this conclusion because of past experiences >and when I try it now, it doesn't fail. The circumstantial evidence was >overwhelming... but apparently quite wrong. Something else may have been at >work; I have no reason to suspect the operating system now. Don't be so quick to retract this. I have it on very good authority that several important things break that shouldn't, once the user has lowered ulimit to (say) 0. I'm not sure that pipe writes are in this category but it wouldn't surprise me. I tried an experiment on our Cray-2 and was rather startled at the form taken by the system failure modes that this triggered. I was afraid to try the experiment that I really wanted to... I urge all System V vendors to completely DISABLE ulimit, not just because of this problem but because it is not a good solution to whatever problem it was perceived as "fixing". Its effects are far too drastic for whatever valid use it might have had.
chris@mimsy.UUCP (Chris Torek) (05/19/88)
>In article <244@twg-ap.UUCP> dwh@twg-ap.UUCP (Dave Hamaker) writes: >>> If I don't raise the ulimit, the pipes fail. >>Oh embarrassment! I had come to this conclusion because of past experiences >>and when I try it now, it doesn't fail. In article <7928@brl-smoke.ARPA> gwyn@brl-smoke.ARPA (Doug Gwyn) writes: >Don't be so quick to retract this. Indeed not. As it happens, in older Unixes, the `apparent' size of a pipe was reset only when the reader caught up with the writer (or something along those lines---I never looked at that code, and now my pipes are sockets, but a friend says this). It may be that if the reader is perenially 1 block behind the writer, that the pipe eventually reaches the ulimit. I would have to look at the code to be sure, but it should not be too hard to test. -- In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 454 7163) Domain: chris@mimsy.umd.edu Path: uunet!mimsy!chris
thorinn@diku.dk (Lars Henrik Mathiesen) (05/21/88)
In article <11575@mimsy.UUCP> chris@mimsy.UUCP (Chris Torek) writes: >As it happens, in older Unixes, the `apparent' size of a pipe was reset >only when the reader caught up with the writer (or something along >those lines---I never looked at that code, and now my pipes are >sockets, but a friend says this). It may be that if the reader is >perenially 1 block behind the writer, that the pipe eventually reaches >the ulimit. I just looked at Edition 6 (Lions): Yes, the apparent size is reset to zero only when the reader catches up to the writer, but the writer is blocked when the apparent size reaches 4kB (the `small file' limit). Now if someone changed the latter behaviour without changing the former, and then put in ulimit, the described symptoms might occur -- but think of it: A pipe allocating and deallocating disk blocks (on the root device!) somewhere in a doubly indirected block. Stupefying. The socket implementation has an advantage here, as it is not bound to a single linear sequence. -- Lars Mathiesen, DIKU, U of Copenhagen, Denmark [uunet!]mcvax!diku!thorinn Institute of Datalogy -- we're scientists, not engineers.