anand@cmx.npac.syr.edu (Rangachari Anand) (08/02/88)
This is a problem that others have surely faced on multiprocessors
such as the Multimax. Consider a program which forks a number of
child processes each of which need to do I/O from a given file. If the
fscanf and fprintf functions are used, the buffering action seems to
load a block of the file into a buffer which is in the private memory
of a process. Then when other processes attempt to read from the
same file, they load blocks of the file into their respective buffers.
Thus they cant read the file in the correct sequence.
I have managed to write my programs in such a way that they always use the
same process to do all the I/O which is a nuisance. I tried to use
the setbuff call to use a shared buffer but that did not seem to work.
Any suggestions?
Thanks
R. Anand
Internet: anand@amax.npac.syr.edu
Bitnet: ranand@sunrisejeff@Alliant.COM (Jeff Collins) (08/03/88)
In article <585@cmx.npac.syr.edu> anand@cmx.npac.syr.edu (Rangachari Anand) writes: > > This is a problem that others have surely faced on multiprocessors > such as the Multimax. Consider a program which forks a number of > child processes each of which need to do I/O from a given file. If the > fscanf and fprintf functions are used, the buffering action seems to > load a block of the file into a buffer which is in the private memory > of a process. Then when other processes attempt to read from the > same file, they load blocks of the file into their respective buffers. > Thus they cant read the file in the correct sequence. Yes this is definately a problem. Given the buffering syntax of stdio, there is really no way to do what you want. The only suggestions that I would have would be to use read(2)/write(2) directly (ie. eliminate the buffering). This is fairly painfull as you don't get the functionality of fscanf and fprintf. Perhaps you can do an sprintf(3) into a string, then write(2) the string to the file. Reading may be more difficult because you probably don't know the length to read until after you have read it. The other possible alternative is to change the stdio libraries so that the buffers use shared memory. I wouldn't think that this would be a trivial solution, and is probably not something that you would want to do yourself - but it is an idea. Jeff Collins
jb@CS.BROWN.EDU (08/04/88)
I'm not exactly sure how things are behaving, but I'd guess that the problem is that while the buffer was shared, the file pointer was not shared. To fix this problem, after you do the fopen(), copy the entire file pointer (FILE *) into a new copy in a block of shared memory. Set the buffer by patching the copied structure to be in shared memory as well. (Copying after the setbuf call would probably work as well.) Use this new file pointer instead of the one returned by fopen for all subsequent i/o. Don't forget that this is a shared data structure and that mutual exclusion is needed around calls to fprintf and fscanf (or any other routines in stdio). This hack should work, but is totally nonstandard. It assumes some knowledge of how stdio works and is outside what is docummented. Use at your own risk. Jim
paradis@encore.UUCP (Jim Paradis) (08/05/88)
In article <2185@alliant.Alliant.COM> jeff@alliant.UUCP (Jeff Collins) writes: >Given the buffering syntax of stdio, there is really no way to do >what you want. The only suggestions that I would have would be to >use read(2)/write(2) directly (ie. eliminate the buffering). > > The other possible alternative is to change the stdio >libraries so that the buffers use shared memory. I wouldn't think >that this would be a trivial solution, and is probably not something >that you would want to do yourself - but it is an idea. Hi, Jeff! There IS a third way... it's not pretty, but it gets the job done without having to change the I/O libraries: (1) Create a region of shared memory that's at least BUFSIZ + sizeof(FILE) long. (2) After opening the file, copy the FILE object into the shared memory region, and refer to it ONLY by the shared-memory address from now on. (3) Use setbuf(3) to place the buffer associated with the FILE object into shared memory. (4) Put spinlocks around all of your stdio calls so that you don't accidentally step all over yourself. Note, of course, that this scheme has the {feature|bug} that this file object is COMPLETELY shared among all processes; therefore, if one process changes the state of the object (e.g. moves the file pointer), that change is seen by all other processes. Standard parallel programming practices apply; if you change the state of the object and a future operation depends on the state you produced (e.g. fseek followed by fread), then you must lock the ENTIRE section of code until such time as you no longer depend on the object having a particular state. -- Jim Paradis, Encore Computer (.signature undergoing renovations. Please pardon our appearance.)