jpo@cs.nott.ac.uk (Julian Onions) (07/28/86)
I try to use stdio as often as possible for several reasons (such as
portability, efficiency, laziness ...) but I feel it lacks one or two calls
to achieve some of the less every-day options.
Most of these can be got around if you `know' the internal stdio
structure - but that negates the portability win. Anyway, anyone care to
a) comment on these
b) mention their own pet stdio hates
c) show portable ways to implement these
d) tell me they're already implemented ==> RTFM.
-------------------------
Addition: fpending(fp)
Synopsis: returns the number of characters that are buffered up
on the stream pointed to by fp.
Use: Has several uses, in particular useful when you are
applying select(2) to a FILE, select may say there is
nothing to read when in actual fact there is stuff
buffered up. The only other way around this is to
set buffering to 0 which is rather inefficient.
It's trivial to add this as another macro.
-------------------------
Addition: finvbuf(fp)
Synopsis: invalidate in-core stdio buffer for FILE *fp.
Use: If you know by other information that a file you
currently have open has changed then you need to
force a reread of the file ensuring that all buffered
information is junked. The only safe/portable alternative
is to fclose/fopen the file descriptor again which is
reasonably expensive. It should probably not guarantee
anything about the offset or else do a rewind(3).
-------------------------
Addition: fslam(fp)
Synopsis: shut the given descriptor in a hurry without flushing it.
Use: If you know that flushing a descriptor will cause a block.
e.g. you have a timeout writing to a terminal cos someones
^S'd it, so you decide to ignore that terminal & carry on.
How can you shut it, fclose will try and flush the data
and will hang the program again. If finvbuf() is in place
this could be used instead. I suppose you could get around
it with
close(fileno(fp));
fclose(fp);
but it seems a bit messy and may cause indigestion for
the fclose routine.
-------------------------
Addition: fbufsiz(fp), fbuftype(fp)
Synopsis: fbuzsiz returns size of buffer in use, fbuftype indicates
what sort of buffering is in use (none, line or full).
Use: To determine whats going on dynamically. E.g.
if( fbufsiz(stderr) == 0 || fbuftype(stderr) == _IONBF)
{
printf("stderr non buffered - fixing\n");
fclose(stderr);
stderr = fdopen(2, "w")
setbuf(stderr, buffer);
}
basically, you can set lots of interesting buffering modes
but you can't discover whats in use.
-------------------------
An inconsistency - why is there no ungetchar(c) defined??
Julian.
jpo@cs.nott.ac.uk mcvax!ukc!nott-cs!jpo
--
Julian Onionshenry@utzoo.UUCP (Henry Spencer) (07/31/86)
> Addition: fpending(fp) > ... > Use: Has several uses, in particular useful when you are > applying select(2) to a FILE... Clearly the right solution to that is fselect(), not fpending(). I'm surprised that there isn't an fselect(), in fact. The other suggestions seem plausible at first glance. -- EDEC: Stupidly non-standard brain-damaged incompatible Henry Spencer @ U of Toronto Zoology proprietary protocol used. {allegra,ihnp4,decvax,pyramid}!utzoo!henry
david@ism780 (08/04/86)
My wish list for stdio would include the following function: flushall (); Flushall would be used to make sure that all output done to stdio files was actually written to the file system. Flushall would fflush all stdio files that are currently in output mode, similar to what _cleanup does but without closing the files. It would not affect files that are in input mode. This allows one to not be concerned with having things flushed until it is required, for example if one uses the functions fork, exec, sync, ustat, quota, or abort.
mouse@mcgill-vision.UUCP (08/09/86)
> I try to use stdio as often as possible for several reasons [...] but > I feel it lacks one or two calls [...] > a) comment on these > b) mention their own pet stdio hates > c) show portable ways to implement these > d) tell me they're already implemented ==> RTFM. > Addition: fpending(fp) (Henry Spencer comments that "the right solution to that is fselect(), not fpending().". I disagree; suppose you want to select on a FILE* and on a file descriptor both? How do you tell the difference? > Addition: finvbuf(fp) I wanted this one, so I wrote it. But I called mine fdumpbuf(). > Addition: fslam(fp) Interesting. Never wanted it, though. > Addition: fbufsiz(fp), fbuftype(fp) Never wanted 'em. > An inconsistency - why is there no ungetchar(c) defined?? I have wondered. Here are the things I have wanted badly enough to add to stdio: fdumpbuf(FILE *f) Mentioned above, throws away the in-core buffer. FILE *fopenfxn(int (*fxn)(), char *mode) Function-stream I/O. Returns a FILE*, open for read or write (not both, "r+" is treated as "r"). When a character is written to (resp. read from) the stream, the function is called. On write, the character written is passed as an argument; on read, the returned value is returned to the user (or to scanf etc). FILE *fopenstr(char *str, int len, char *mode) An extension to sprintf() and sscanf(). This returns a stream which performs I/O to a string (this makes sprintf() and sscanf() unnecessay, though they are still convenient). unfdopen(FILE *f) Undoes fdopen(), that is, closes the FILE* without close()ing the underlying file descriptor. Unfdopen is trivial; fopenfxn and fopenstr are implemented by making the stream appear unbuffered to putc and getc and then changing _flsbuf and _filbuf to add the functionality. Comments anyone? -- der Mouse USA: {ihnp4,decvax,akgua,utzoo,etc}!utcsri!mcgill-vision!mouse think!mosart!mcgill-vision!mouse Europe: mcvax!decvax!utcsri!mcgill-vision!mouse ARPAnet: utcsri!mcgill-vision!mouse@uw-beaver.arpa "Come with me a few minutes, mortal, and we shall talk." - Thanatos (Piers Anthony's Bearing an Hourglass)
argv@sri-spam.ARPA (AAAARRRRGGGGv) (08/12/86)
> Here are the things I have wanted badly enough to add to stdio: > > > FILE *fopenfxn(int (*fxn)(), char *mode) > Function-stream I/O. Returns a FILE*, open for read or write > (not both, "r+" is treated as "r"). When a character is > written to (resp. read from) the stream, the function is > called. On write, the character written is passed as an > argument; on read, the returned value is returned to the user > (or to scanf etc). This shouldn't be that hard to do: first, take the fildes of the open stream, set it to be asynch IO, then loop. When data is there to be read, you'll get a SIGIO. For some strange reason, I seem to remember that this is can only be done on pseudo-tty's or other types of devices (I use it frequently for windows on the SUN); so I don't know if this will work well on normal files or sockets. /* (no error checking here -- don't try this at home without mom's help! */ #include <fcntl.h> ... int fd = open ("/dev/tty", 0); /* read a tty for example */ signal (SIGIO, io_handler); /* this proc will get a sigio if data on a stream is ready to be read */ ioctl(fd, F_SETFL, fcntl(fd, F_GETFL, 0) | FASYNC)); /* set async IO */ for (;;) looping_proc(); ... io_handler() { char buf[BUFSIZ]l puts("Hey, there's data to be read on the fildes!"); read(fd, buf, BUFSIZ); do_something_with_data(buf); } > FILE *fopenstr(char *str, int len, char *mode) > An extension to sprintf() and sscanf(). This returns a stream > which performs I/O to a string (this makes sprintf() and > sscanf() unnecessay, though they are still convenient). I request you more fully describe this functionality. I don't see how adding the routines of an IO library is going to speed up what sprintf and sscanf already do. Perhaps to do something like this would be "interesting", but that's about it. Again, please elaborate. > unfdopen(FILE *f) > Undoes fdopen(), that is, closes the FILE* without close()ing > the underlying file descriptor. This would be nice. > Unfdopen is trivial; fopenfxn and fopenstr are implemented by making > the stream appear unbuffered to putc and getc and then changing _flsbuf > and _filbuf to add the functionality. Looking at the source to puts (fputs) and so forth, I've found that it is much faster to never unbuffer anything and to simply macro that which you'd like to have "unbuffered": #define unbuffered_puts(s,f) fputs(s,f), fflush(f) The reason for this is that if the data is unbuffered, fputs merely sets the buffer to a buffer within the routine, sets a few flags, goes thru a while loop, using the macro in stdio to putc to the stream, then fflushes again, finally resettting all the flags it started with. Finallly (not that this has anything to do with it), it returns the last character put to the stream (undocumented). In any event, what's preventing you from free()ing the structure and leaving the fd open? The biggest problem with routines that people write that do popen like read_write_popen(command, &Stdin, &Stdout, &Stderr); /* open a process and be able to read and write to/from it */ is that they pipe for each fildes, fdopen, and in case of error, return without closing all open fildes's, unallocating the FILE*'s and closing the process if necessary. A more elegant routine to unfdopen (as you described) would be a matter of convenience. dan <argv@sri-spam.arpa> "Wouldn't ya like to be a pointer, too... (be a pointer, be a double pointer, [repeat and fade])"
jpo@cs.nott.ac.uk (Julian Onions) (08/18/86)
In article <479@mcgill-vision.UUCP> mouse@mcgill-vision.UUCP writes: >> Addition: fpending(fp) >(Henry Spencer comments that "the right solution to that is fselect(), >not fpending().". I disagree; suppose you want to select on a FILE* >and on a file descriptor both? How do you tell the difference? OK, I'm greedy, I want both. fpending(fp) I wanted after some code that checked the input for pending characters using FIONREAD. fselect is rather involved requiring several lines of code, fpending comes out as something like #define fpending(fp) ((fp)->_cnt) Give either fpending or fselect the other can be built and fpending seems simplest, but maybe not the answer to the right question. All in all, fselect is the right way to go (stdio features tracking system calls), but fpending is so trivial it might as well be thrown in. >> Addition: fbufsiz(fp), fbuftype(fp) >Never wanted 'em. Neither have I, but for the sake of consistency it'd be nice to tell what buffering is set as well as being able to set buffering. >Here are the things I have wanted badly enough to add to stdio: >FILE *fopenfxn(int (*fxn)(), char *mode) > Function-stream I/O. Can't say I've ever thought about this one. I'm not sure when you'd use it either. >FILE *fopenstr(char *str, int len, char *mode) > An extension to sprintf() and sscanf(). This returns a stream > which performs I/O to a string (this makes sprintf() and > sscanf() unnecessay, though they are still convenient). I like this one, C++ has something like this in its streams stuff, and I liked the idea there. Being able to treat incore strings and files the same seems a good idea. e.g. if(argc>1) if(( fp = fopen(argv[1], "r")) == NULL) process(fp = fopenstr(argv[1], strlen(argv[1]), "r")); else process(fp); else process(fp = stdin); fclose(fp); >unfdopen(FILE *f) > Undoes fdopen(), that is, closes the FILE* without close()ing > the underlying file descriptor. Yes, I'd thought of that, however as you say its trivial and can be done reasonably portably e.g. unfdopen(fp) FILE *fp; { int oldf, tmpf; tmpf = dup(oldf = fileno(fp)); fclose(fp); return dup2(tmpf, oldf); } Seems like a good idea. When you've finished playing around with fprintf's etc. and just want the file descriptor for locking or some other nefarious process and not all the expensive buffering space. I'd prefer fdclose as the name though. Julian. jpo@cs.nott.ac.uk jpo%cs.nott.ac.uk@ucl-cs.arpa mcvax!ukc!nott-cs!jpo -- Julian Onions
guy@sun.uucp (Guy Harris) (08/21/86)
> >Here are the things I have wanted badly enough to add to stdio: > >FILE *fopenfxn(int (*fxn)(), char *mode) > > Function-stream I/O. > Can't say I've ever thought about this one. I'm not sure when you'd > use it either. It might be useful for I/O on "streams" that aren't implemented by a file or a string. I suspect somebody out there has wanted such a beast (for instance, one might use it with the 4.2BSD "syslog" facility, so that each line written to the stream was logged. > >FILE *fopenstr(char *str, int len, char *mode) > > An extension to sprintf() and sscanf(). This returns a stream > > which performs I/O to a string (this makes sprintf() and > > sscanf() unnecessay, though they are still convenient). It looks straightforward to implement IF you have the 4BSD version of standard I/O, where there's a "this is a string" flag for each FILE structure. The S5 version, unfortunately, has only a "char" for the flags, not a "short", so it says that the "last" FILE entry, and only that entry, is a stream. If AT&T-IS is willing to put up with the terrifying possibility of forcing people to delete *object* files and recompile them for S5R4 (not *executable images*, they'll work fine as long as they don't use shared libraries; if you can control which version of a shared library you bind to, you can even make images built against shared libraries work), they can do this too. (Which way did V7 do this?) Given "fopenstr" you can actually provide a version of "sprintf" that performs *bounds checking* on the string it's printing into. (Wow, what a unique concept, software that doesn't scribble its data space when a user types in a string longer than it expected!) > Seems like a good idea. When you've finished playing around with > fprintf's etc. and just want the file descriptor for locking or > some other nefarious process and not all the expensive buffering space. > I'd prefer fdclose as the name though. That name might imply that the "fd" gets "close"d. "unfdopen" is, admittedly, a bit awkward. -- Guy Harris {ihnp4, decvax, seismo, decwrl, ...}!sun!guy guy@sun.com (or guy@sun.arpa)
karl@haddock (09/03/86)
jpo@cs.nott.ac.uk (Julian Onions) writes: >All in all, fselect is the right way to go (stdio features tracking >system calls), ... Hasn't anyone noticed that at least part of the purpose of stdio is to make code portable? Some systems (even some UNIXes) don't have any way to write a function such as fselect(). >>FILE *fopenfxn(int (*fxn)(), char *mode) >> Function-stream I/O. >Can't say I've ever thought about this one. I'm not sure when you'd >use it either. Potentially useful, though kind of slow if it has to pass one character at a time through the function. But (ignoring error conditions) the first arg should be "char (*)(void)" if the second is "r", or "void (*)(char)" if the second is "w"; this could cause problems with type-checking. (Best solution is to drop the second arg and use two functions, I think.) Actually, the function argument should probably be analogous to read/write rather than getc/putc. But there should be one more argument to fopenfxn(), viz. a (void *) argument to be passed to fxn() to distinguish streams. >>FILE *fopenstr(char *str, int len, char *mode) >> An extension to sprintf() and sscanf(). This returns a stream >> which performs I/O to a string (this makes sprintf() and >> sscanf() unnecessay, though they are still convenient). >I like this one, C++ has something like this in its streams stuff, >and I liked the idea there. Being able to treat incore strings and >files the same seems a good idea. I like it too, but maybe it should be called "stropen" (cf. "fdopen"). It's a good idea to require the len argument even for string reading, so '\0' can be treated as an ordinary character if desired. Karl W. Z. Heuer (ima!haddock!karl; karl@haddock.isc.com), The Walking Lint
karl@haddock (09/03/86)
sun!guy (Guy Harris) writes: > >FILE *fopenstr(char *str, int len, char *mode) >It looks straightforward to implement IF you have the 4BSD version of >standard I/O, where there's a "this is a string" flag for each FILE >structure. The S5 version, unfortunately, has only a "char" for the flags, >not a "short", so it says that the "last" FILE entry, and only that entry, >is a stream. Making the last FILE special was indeed a botch, but it isn't necessary to expand the _flag field to fix it. Just use (iop->_file == -1) on string files. >Given "fopenstr" you can actually provide a version of "sprintf" that >performs *bounds checking* on the string it's printing into. (Wow, what a >unique concept, software that doesn't scribble its data space when a user >types in a string longer than it expected!) And given "fopenfxn", you could provide a version of "sprintf" that not only detects the overflow, but *fixes* it with a realloc()! Karl W. Z. Heuer (ima!haddock!karl; karl@haddock.isc.com), The Walking Lint
mouse@mcgill-vision.UUCP (der Mouse) (09/04/86)
In article <6224@sri-spam.ARPA>, argv@sri-spam.ARPA (AAAARRRRGGGGv) writes: [>> is AAAARRRRGGGGv quoting me] >> Here are the things I have wanted badly enough to add to stdio: >> FILE *fopenfxn(int (*fxn)(), char *mode) >> Function-stream I/O. Returns a FILE*, open for read or write >> (not both, "r+" is treated as "r"). When a character is >> written to (resp. read from) the stream, the function is >> called. > This shouldn't be that hard to do: > first, take the fildes of the open stream, set it to be asynch IO, > then loop. When data is there to be read, you'll get a SIGIO. For > some strange reason, I seem to remember that this is can only be done > on pseudo-tty's or other types of devices (I use it frequently for > windows on the SUN); so I don't know if this will work well on normal > files or sockets. (By the way, the "strange reason" is that the code isn't in the kernel for it!) [excerpted example follows:] > int fd = open ("/dev/tty", 0); > signal (SIGIO, io_handler); > ioctl(fd, F_SETFL, fcntl(fd, F_GETFL, 0) | FASYNC)); > io_handler() > { > puts("Hey, there's data to be read on the fildes!"); > } You apparently misunderstood (ok, let's be nice, I didn't make myself clear). What fopenfxn does is not what the above does (or even what it appears to be trying to do). There is no file descriptor associated with a function stream! For example, if we do f = fopenfxn(foo_fxn,"w"); then putc(c,f); and foo_fxn(c,f); are equivalent. The advantage is that functions like printf() can be used on the resulting stream, and call the function once for each character generated. Reading works analogously: f = fopenfxn(foo_fxn,"r"); c = getc(f); is equivalent to c = foo_fxn(f); but makes scanf() and friends work as well. >> FILE *fopenstr(char *str, int len, char *mode) >> An extension to sprintf() and sscanf(). This returns a stream >> which performs I/O to a string (this makes sprintf() and >> sscanf() unnecessay, though they are still convenient). > > I request you more fully describe this functionality. I don't see how > adding the routines of an IO library is going to speed up what > sprintf and sscanf already do. Perhaps to do something like this > would be "interesting", but that's about it. Again, please > elaborate. Sure. f = fopenstr(buf,"w"); fprintf(f,"Current square is: "); print_square(f,current_square); fprintf(f,"and no other!"); fclose(f); Or f = fopenstr(buf,"w"); fprintf(f,"List is:"); for (element=list;element;element=element->link) { fprintf(f," %d",element->value); } fprintf(f,"."); fclose(f); Using this instead of sprintf or sscanf should produce no speed difference (well, actually, it should be slightly slower because of the extra function calls involved....). >> unfdopen(FILE *f) >> Undoes fdopen(), that is, closes the FILE* without close()ing >> the underlying file descriptor. > This would be nice. Yeah. To get it with vanilla stdio requires playing games with dup, and will cause lots more pain if (as for stdin/out/err) you want the original descriptor number. Even then, you gotta have a descriptor free. >> Unfdopen is trivial; fopenfxn and fopenstr are implemented by making >> the stream appear unbuffered to putc and getc and then changing _flsbuf >> and _filbuf to add the functionality. > > Looking at the source to puts (fputs) and so forth, I've found that > it is much faster to never unbuffer anything and to simply macro that > which you'd like to have "unbuffered": > #define unbuffered_puts(s,f) fputs(s,f), fflush(f) The point of telling stdio that the stream is unbuffered is that then other stdio functions such as printf and scanf will call _flsbuf (resp. _filbuf) for every character. > The reason for this is that if the data is unbuffered, fputs merely > sets the buffer to a buffer within the routine, sets a few flags, > goes thru a while loop, using the macro in stdio to putc to the > stream, then fflushes again, finally resettting all the flags it > started with. Finallly (not that this has anything to do with it), > it returns the last character put to the stream (undocumented). You are running 4.3, right? It was fun making these work when we shifted to 4.3....4.2 puts (and similar functions) didn't do this "let's pretend it's buffered for a little while...." trick. > In any event, what's preventing you from free()ing the structure and > leaving the fd open? Huh? What does this have to do with any of the above?
chris@umcp-cs.UUCP (Chris Torek) (09/04/86)
In article <86900034@haddock> karl@haddock writes: >>>FILE *fopenfxn(int (*fxn)(), char *mode) >Actually, the function argument should probably be analogous to read/write >rather than getc/putc. But there should be one more argument to fopenfxn(), >viz. a (void *) argument to be passed to fxn() to distinguish streams. Indeed, there would be a certain symmetry to the whole thing if one could write reader(f, buf, len) FILE *f; { return (read(fileno(f), buf, len)); } ... FILE *f = fopenrf(reader, "r"); fileno(f) = fd; instead of FILE *f = fdopen(fd, "r"); -- In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 454 1516) UUCP: seismo!umcp-cs!chris CSNet: chris@umcp-cs ARPA: chris@mimsy.umd.edu
chris@umcp-cs.UUCP (Chris Torek) (09/04/86)
In article <3265@umcp-cs.UUCP>, I left out a declaration: > reader(f, buf, len) > FILE *f; char *buf; int len; > { ... (`len' would have defaulted properly, of course.) -- In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 454 1516) UUCP: seismo!umcp-cs!chris CSNet: chris@umcp-cs ARPA: chris@mimsy.umd.edu
karl@haddock (09/08/86)
umcp-cs!chris (Chris Torek) writes: >In article <86900034@haddock> karl@haddock writes: >>>>FILE *fopenfxn(int (*fxn)(), char *mode) >>Actually, the function argument should probably be analogous to read/write >>rather than getc/putc. But there should be one more argument to fopenfxn(), >>viz. a (void *) argument to be passed to fxn() to distinguish streams. > >Indeed, there would be a certain symmetry to the whole thing if one >could write > reader(f, buf, len) FILE *f; { return (read(fileno(f), buf, len)); } > FILE *f = fopenrf(reader, "r"); > fileno(f) = fd; >instead of > FILE *f = fdopen(fd, "r"); As I mentioned, I think it has to be (void *) in general, thus reader(void *v, char *buf, int len) { return (read(*(int *)v, buf, len)); } FILE *f = fopenrf(&fd, reader, "r"); so that the more general cases could be supported, e.g. read from string: typedef struct { char *t_start, *t_end; } tbuf; reader(void *v, char *buf, int len) { register tbuf *t = (tbuf *)v; len = min(len, t->t_end - t->t_start); memcpy(buf, t->t_start, len); t->start += len; return (len); } FILE *f = fopenrf(&t, reader, "r"); You could get away with having the first arg to reader() be "FILE *", but in any case "void *_id" needs to replace or supplement the existing "int _file" in the FILE structure. If there are separate functions fopenrf() and fopenwf(), is there any need for the third argument? Karl W. Z. Heuer (ima!haddock!karl; karl@haddock.isc.com), The Walking Lint
chris@umcp-cs.UUCP (Chris Torek) (09/14/86)
I wrote: >>Indeed, there would be a certain symmetry to the whole thing if one >>could write >> reader(f, buf, len) FILE *f; { return (read(fileno(f), buf, len)); } >> FILE *f = fopenrf(reader, "r"); >> fileno(f) = fd; >>instead of >> FILE *f = fdopen(fd, "r"); In article <86900044@haddock> karl@haddock writes: >As I mentioned, I think it has to be (void *) in general, thus > reader(void *v, char *buf, int len) { > return (read(*(int *)v, buf, len)); > } > FILE *f = fopenrf(&fd, reader, "r"); I agree. Indeed, I had a more voluminous example that used generic pointers, but I thought that the idea would be hidden by all the implementation details, so I trimmed it down to the one in >> above. >You could get away with having the first arg to reader() be "FILE *", but in >any case "void *_id" needs to replace or supplement the existing "int _file" >in the FILE structure. I think the generic pointer is better. The reading or writing routine should need no access to the stdio structures, just as read() and write() need none. >If there are separate functions fopenrf() and fopenwf(), is there any need >for the third argument? I had a reason for including it at the time, but now I cannot recall it. Ah well. There should be some provision for read-and-write modes, though: typedef int (*iofunc)(void *cookie, char *buf, int len); FILE *fopenrwf(void *cookie, iofunc r, iofunc w); (to specify types everywhere). -- In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 454 1516) UUCP: seismo!umcp-cs!chris CSNet: chris@umcp-cs ARPA: chris@mimsy.umd.edu