augustss@cs.chalmers.se (Lennart Augustsson) (11/06/90)
In a language implementation that I am doing I've got the following problem: I'm using multiple processes (processors) to run a program, during the execution of the program it may be necessary to open files (and sockets). After a file has been opened it would be desirable if all processes (and not just the one doing the open) could read from that file (the reason for this is to keep all processes "equal"). I can see no clean solution to this problem. The descriptor that the opening process gets back needs to be distributed to the others somehow, but as far as I know there is now mechanism for that. Or is there? 4.3 BSD Unix contains a way to distribute file descriptors through the sendmsg and recvmsg system calls, but as usual Sequent are years behind when it comes to the operating system (sendmsg&recvmsg can send file descriptors in the UNIX domain on vanilla 4.3 BSD, Ultrix, and SunOS). Has anyone else had this problem? Have you got any solutions? Hello Sequent, are you listening? Are you ever going to catch up with your competitors? (I could mention other stuff that are still ridden with 4.2 bugs, but enough Sequent-bashing for one posting.) (I have another solution using signals in mind, but I don't like it.) -- Lennart Augustsson -- Lennart Augustsson Email: augustss@cs.chalmers.se
keyvan@pyra.co.uk (Keyvan Shirnia Pre Sales) (11/07/90)
In article <1990Nov6.015433.10642@mathrt0.math.chalmers.se> augustss@cs.chalmers.se (Lennart Augustsson) writes: >In a language implementation that I am doing I've got the following >problem: I'm using multiple processes (processors) to run a program, >during the execution of the program it may be necessary to open files >(and sockets). After a file has been opened it would be desirable >if all processes (and not just the one doing the open) could read from >that file (the reason for this is to keep all processes "equal"). >I can see no clean solution to this problem. The descriptor that the >opening process gets back needs to be distributed to the others >somehow, but as far as I know there is now mechanism for that. Or is >there? Well, if all your processes are running on the same machine, you do not need to use sockets. I think a better solution would be to use pipes. (Although pipes are built using socketpair() system call!! ;-) So the idea is something like this: (!) When spawning its processes, the parent can also open up pipes between itself and its children. Then by simply using read() and write() system calls you can transfer data between your processes. If you don't want the children to hang around the read() system call (block for data to be written to the pipe) then I'm afraid you have to use signals. (ie. send a signal to the child process when you know you are about to send the data to the child) Not a very pretty sight!! ;-) >4.3 BSD Unix contains a way to distribute file descriptors through the >sendmsg and recvmsg system calls, but as usual Sequent are years >behind when it comes to the operating system (sendmsg&recvmsg can send >file descriptors in the UNIX domain on vanilla 4.3 BSD, Ultrix, and >SunOS). Can I just say something here! You mentioned you would like to distribute the file descriptors between your processes. I think it is worth mentioning the fact that you have to send the filename to the children. It's the children's job to open files and get appropriate file descriptors. The child as a process has its own user space, which is totally seperate from its parent. So the child is totally unaware of the fact that the parent has just opened a file, and updated its user space file table. However, if the parent opened up the file before spawning any processes, the children will inherit the parents file table including the file descriptors. sendmsg and recvmsg are just equivalent to write() and read() on pipes, respectively. >(I have another solution using signals in mind, but I don't like it.) You have to be very wary when using signals. Its very easy to get mixed up with all the processes talking to each other using signals. Just for your information, this task under ATT's inter process communication is very much simpler. All you need to do, is to setup a shared memory segment, and lock the segment using semaphores. This way the parent has control over the children and can indicate when they should read data from the shared memory. NOTE: you would still have to use signals, if the children are unable to block on the semaphore lock. I hope this helps. Best regards, Keyvan P.S. If you have any more questions regarding this, please Do ask!! ;-) -m------- Keyvan Shirnia Tel : +44 252 373035 ---mmm----- Pre Sales Support -----mmmmm--- Pyramid Technology Ltd. keyvan@pyra.co.uk -------mmmmmmm- Farnborough GU14 7NA, England.
kevinc@crg8.sequent.com (Kevin Closson) (11/08/90)
In article <1990Nov6.015433.10642@mathrt0.math.chalmers.se> augustss@cs.chalmers.se (Lennart Augustsson) writes: >In a language implementation that I am doing I've got the following >problem: I'm using multiple processes (processors) to run a program, >during the execution of the program it may be necessary to open files >(and sockets). After a file has been opened it would be desirable >if all processes (and not just the one doing the open) could read from >that file (the reason for this is to keep all processes "equal"). >I can see no clean solution to this problem. The descriptor that the >opening process gets back needs to be distributed to the others >somehow, but as far as I know there is now mechanism for that. Or is >there? >4.3 BSD Unix contains a way to distribute file descriptors through the >sendmsg and recvmsg system calls, but as usual Sequent are years >behind when it comes to the operating system (sendmsg&recvmsg can send There is no mention here of the Dynix rev currently on your machine. Under Dynix 3.0.17, the sendmsg() and recvmsg() calls are available. It could be as simple as an upgrade of the OS for you to be fble to draw on this functionality. >file descriptors in the UNIX domain on vanilla 4.3 BSD, Ultrix, and >SunOS). >Has anyone else had this problem? Have you got any solutions? >Hello Sequent, are you listening? Are you ever going to catch up with ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ We are indeed listening! Perhaps this issue could have been address by Sequent Customer Service at the 800 number provided you by your sales representative. Please e-mail me if I can be of further assistance! -- Kevin Closson Sequent Technical Support Database/Layered Products x83072
augustss@cs.chalmers.se (Lennart Augustsson) (11/09/90)
In article <657916216.AA4508@flaccid> keyvan@pyra.co.uk (Keyvan Shirnia) writes: >In article <1990Nov6.015433.10642@mathrt0.math.chalmers.se> augustss@cs.chalmers.se (Lennart Augustsson) writes: > >> [stuff deleted] > >Well, if all your processes are running on the same machine, you do not >need to use sockets. I think a better solution would be to use pipes. >(Although pipes are built using socketpair() system call!! ;-) > >So the idea is something like this: (!) > >When spawning its processes, the parent can also open up pipes between >itself and its children. Then by simply using read() and write() system calls >you can transfer data between your processes. If you don't want the children >to hang around the read() system call (block for data to be written to the >pipe) then I'm afraid you have to use signals. (ie. send a signal to the >child process when you know you are about to send the data to the child) > >Not a very pretty sight!! ;-) Among my parallel process there is no master (there is of course a parent, but this is forgotten after the startup phase), and I would like to keep it that way. As you say you could do something with pipes, but since I absolutely don't want any processes hanging anywhere in the normal case this would involve signals and would be messy. >>4.3 BSD Unix contains a way to distribute file descriptors through the >>sendmsg and recvmsg system calls, but as usual Sequent are years >>behind when it comes to the operating system (sendmsg&recvmsg can send >>file descriptors in the UNIX domain on vanilla 4.3 BSD, Ultrix, and >>SunOS). > >Can I just say something here! You mentioned you would like to distribute >the file descriptors between your processes. I think it is worth mentioning >the fact that you have to send the filename to the children. It's the children's >job to open files and get appropriate file descriptors. Not so!! You don't have to distribute filenames. One process can open a file (or create it with socket, or any other way you can get a file descriptor) and then send this to any other process using a UNIX domain socket with sendmsg and having the socket in the accright field. When the sender closes that file descriptor (which may be dup:ed if you want to keep the original) the receiver can use the number it got in the accright field as a file descriptor and it will refer to whatever it refered to in the sender. >The child as a process has its own user space, which is totally seperate from >its parent. So the child is totally unaware of the fact that the parent has >just opened a file, and updated its user space file table. However, if the >parent opened up the file before spawning any processes, the children will >inherit the parents file table including the file descriptors. Well, that's exactly the problem. >sendmsg and recvmsg are just equivalent to write() and read() on pipes, >respectively. On pipes yes, but not on UNIX domain sockets. What I had in mind using sendmsg/recvmsg would also involve signals, but in a rather safe way. It would work like this. Whenever a process has opened a new file it will (for each of its coworkers) dup it, send it with sendmsg, close it and send a signal to the receiver. The receiver will in the signal routine do a recvmsg (and maybe store the descriptor in a table) and then return from the signal routine. Any future attemt to read can then be done via this descriptor. >>(I have another solution using signals in mind, but I don't like it.) > >You have to be very wary when using signals. Its very easy to get mixed up >with all the processes talking to each other using signals. I know! >Just for your information, this task under ATT's inter process communication >is very much simpler. All you need to do, is to setup a shared memory segment, >and lock the segment using semaphores. This way the parent has control over the >children and can indicate when they should read data from the shared memory. >NOTE: you would still have to use signals, if the children are unable to block >on the semaphore lock. The processes are already running in shared memory (with locks and all). The problem is (as I said) that there is no master process, all proceses should be considered equal so that any one of them can perform any of the many tasks in that makes up my program. This means that as soon as one of them opens a file and gets a new descriptor all the others should be able to read/write on that descriptor to. >I hope this helps. I'm not sure it does, but thanks anyway. I have some alternative solutions, but I don't like any of them (I don't like the sendmsg/recvmsg either, but I think it's the best so far): 1) Have a designated I/O process. All I/O is performed by this process and the workers have to send some kind of "requests" to it, results will be returned by having the I/O process putting it in shared memory and signalling via some semaphore. 2) Whenever a file is opened a table (in shared memory, btw. there already is one since all processes share I/O buffers, buffer pointers, counters etc.) is updated with the process id of the opener. When someone else needs I/O on this descriptor it will place its request somewhere in shared memory and signal the opener. It's signal routine will then perform the request, signal back somehow and return. I hate all this signalling and semaphores, it's inefficient and error prone. > >Best regards, >Keyvan > >P.S. If you have any more questions regarding this, please Do ask!! ;-) > -- Lennart Augustsson Email: augustss@cs.chalmers.se
augustss@cs.chalmers.se (Lennart Augustsson) (11/10/90)
In article <20383@crg5.UUCP> kevinc@crg8.UUCP (Kevin Closson) writes: >In article <1990Nov6.015433.10642@mathrt0.math.chalmers.se> augustss@cs.chalmers.se (Lennart Augustsson) writes: >> [ my previous posting deleted] > There is no mention here of the Dynix rev currently on your machine. Under >Dynix 3.0.17, the sendmsg() and recvmsg() calls are available. It could >be as simple as an upgrade of the OS for you to be fble to draw on this >functionality. > Sorry! I forgot to say that we are running 3.0.17. That does not help. I quote from the manual page for recvmsg: ... The msg_accrights and msg_accrightslen fields are ignored, but present in the structure to retain compatibility with other versions of UNIX. > > We are indeed listening! Perhaps this issue could have been address >by Sequent Customer Service at the 800 number provided you by your sales >representative. Well, 800 numbers don't work from Sweden :-) I will make a more formal request for an upgrade to sendmsg/recvmsg. -- Lennart Augustsson Email: augustss@cs.chalmers.se