brian@ucsd.Edu (Brian Kantor) (12/28/89)
I don't have time to write this, but if I had them, I'd use them, so here it is the idea raw and unfettered.... A programmatic device (pdev) is a program (an executable binary) living in the /dev directory. When a user program tries to open it for reading or writing, a process is started that runs the program, and ties its stdin and stdout to the user program open. All I/O to and from the pdev appears to the user as though it is simply reading and writing a device; the pdev can open real or pseudo-devices and do real I/O as appropriate. Ordinarily, the last close will cause the pdev to stop running and exit. The permissions bits would control access just as they do to real files and devices, SUID and SGID would work, with some redefinitions: The owner 'execute' bit means that the pdev is exclusive access; as long as one process has the pdev open all others attempting to open it will block (or get EWOULDBLOCK). The group and other execute bits are ignored. The read or write bits are used as in normal Unix file system semantics. If neither is on for the process requesting the open, the pdev process is not started and EACCES is returned. An attempt to read or write an open pdev in violation of the permissions granted also returns EACCES. The 'sticky' bit means that the pdev starts running on the first open and does not exit on the last close; the second and subsequent opens of the pdev will talk to the same process for as long as it stays running. 'sticky' probably implies exclusive access, since there is no multiplexing [i.e., all I/O between the pdev and the user program is on the pdev's stdin/out]. The exclusive-access bit in the open system call causes the pdev to grant exclusive access (i.e., to behave as though it had the owner execute bit on) but only temporarily; when the process requesting exclusive access closes the pdev it reverts to whatever access the owner execute bit grants. (I don't see a need nor a way for a process to grant multiple access to a pdev that the system manager has set up as exclusive.) I don't know what IOCTLs should do with pdevs. The applications for this range from expanded security (where the pdev enforces/allows access to real devices), more complete device independence (now you can make every modem look like a Hayes), special interfaces to bizarre devices, etc, etc. In a way, they're simple-minded streams without all the streams claptrap. As I say, I haven't got time to write these. But it might make a nice senior project for some real up-and-coming Unix student wizard. Just be sure to give me credit in the paper. Oh, and if this isn't an original idea, pardon me. I honestly can't remember having heard of this before in the Unix world. (It is vaguely reminiscent of the Class-B security procedures in the Burroughs B-6500 et seq, but is much much more general than that was.) Brian Kantor UCSD Network Operations UCSD C-024, La Jolla, CA 92093-0124 USA brian@ucsd.edu ucsd!brian BRIAN@UCSD
jsloan@handies.ucar.edu (John Sloan,8292,X1243,ML44E) (12/28/89)
From article <10641@ucsd.Edu>, by brian@ucsd.Edu (Brian Kantor): > A programmatic device (pdev) is a program (an executable binary) living > in the /dev directory. When a user program tries to open it for > reading or writing, a process is started that runs the program, and > ties its stdin and stdout to the user program open. : > Oh, and if this isn't an original idea, pardon me. I honestly can't > remember having heard of this before in the Unix world. I agree, if I had these I'd use them. And the idea can't be original, because I've been carrying it along in my head for a long time, and I know that I'm not smart enough to have thought of it myself. Anyway, this is a pretty object oriented approach, where you send and receive messages (read/write) to an object (which may be a process or a file). Thus, when you receive data, it may be coming raw from a file, it may be preprocessed, it may be a network link to another process, or it may be generated internally by the object. I hadn't considered restricting the object to /dev, but given the history of UNIX, what with pseudo-terminals and UNIX-domain sockets often being placed there, its the logical place. It strikes me that once the Mach-model is actually realized, that is, most of the BSD kernel functions are moved into user-level processes, Mach is the logical platform on which to base your pdevs. The Mach message passing may be an appropriate mechanism to use. If you carry this to its logical conclusion, ALL files and devices would be such objects, and all "I/O" would be message passing. It just occurred to me (which means this is probably really half baked) that this is another software paradigm that really got its start in the hardware arena. Back when I was doing real-time kernels, device drivers, and embedded systems for a living, I realized that when I read from a controller, I really had no idea where the data was coming from. It was just a convenient convention that it was coming from some disk. What with the use of solid state devices to mimic disks, and with demand paged memory using disks to mimic memory, and with generic controller interfaces such as SCSI servicing tapes, disks, and even network interfaces, most of our I/O is based on these convenient conventions. There is no reason why this couldn't be done completely in the software arena, with object oriented languages and client-server processes. I dimly recall that some facility like you describe may well have existed in some incarnation of OS/360 (MFT, MVT, SVS, MVS), from my time in the hazy past as a systems jock on those beasts. But (rig for overgeneralization!) the UNIX byte-stream I/O model would make this easier to implement, IMHO, than in the IBM block-oriented I/O model. Neat idea. John Sloan NCAR/SCD NSFnet: jsloan@ncar.ucar.edu P.O. Box 27588 P.O. Box 3000 +1 303 497 1243 AMA#515306 Lakewood CO 80227 Boulder CO 80307 +1 303 232 8678 DoD#000011 Logical Disclaimer: belong(opinions,jsloan).belong(opinions,_):-!,fail.
avr@cbnewsj.ATT.COM (adam.v.reed) (12/29/89)
In article <10641@ucsd.Edu>, brian@ucsd.Edu (Brian Kantor) writes: > A programmatic device (pdev) is a program (an executable binary) living > in the /dev directory. When a user program tries to open it for > reading or writing, a process is started that runs the program, and .... > Oh, and if this isn't an original idea, pardon me. I honestly can't > remember having heard of this before in the Unix world. (It is vaguely This looks (except for some ad-hoc details) like a trivial instance of a stream device with a single stream module. Why write if you can buy? Adam_V_Reed@ATT.com
andrew@alice.UUCP (Andrew Hume) (12/29/89)
a very similar scheme has existed for at least a couple of years in research unix; it was first done by dave presotto and dennis ritchie under the (rough) label 'connection server'. most of what you describe can be done more simply by a mounted stream, which is part of SVR4. the generalization of this, implementing files as processes, has a longer history: greg rose investigated this for a phd thesis around 1979 at UNSW in australia. i am sure others have also looked at this before.
bill@twwells.com (T. William Wells) (12/29/89)
In article <10641@ucsd.Edu> brian@ucsd.Edu (Brian Kantor) writes:
: I don't know what IOCTLs should do with pdevs.
Have the execute bit determine if ioctls are permitted.
Of course, given that the protocol between the kernel and the
program is adequate, the program could do all access control.
---
Bill { uunet | novavax | ankh | sunvice } !twwells!bill
bill@twwells.comjfh@rpp386.cactus.org (John F. Haugh II) (12/29/89)
In article <5804@ncar.ucar.edu> jsloan@handies.ucar.edu (John Sloan,8292,X1243,ML44E) writes: >From article <10641@ucsd.Edu>, by brian@ucsd.Edu (Brian Kantor): >> A programmatic device (pdev) is a program (an executable binary) living >> in the /dev directory. When a user program tries to open it for >> reading or writing, a process is started that runs the program, and >> ties its stdin and stdout to the user program open. > : >> Oh, and if this isn't an original idea, pardon me. I honestly can't >> remember having heard of this before in the Unix world. > >I agree, if I had these I'd use them. And the idea can't be original, >because I've been carrying it along in my head for a long time, and >I know that I'm not smart enough to have thought of it myself. I implemented a `programatic device' at a former job to give me a C/A/T typesetter for an old-style troff and an HP 7475 plotter for the 20/20 spreadsheet the company used. The implementation was almost too simple to mention ... while true do 7475toplot < /dev/plot | tplot -Thplj | lp -og done where 7475toplot converted 7475 commands to tplot, tplot -Thplj converted tplot to HP LaserJet commands, and lp -og sent raw characters to the system laser printer. /dev/plot was a named pipe. The commands would sit there waiting for another command to open the pipe for writing. That would happen, the input would be read and the writers side of the pipe closed, etc. Some what more clever would have been a multiplexed pipe driver where each open was paired up to the next open of the opposite flavor. Like PTYs, except there are multiple channels per device and successive opens create new channels. I never saw a need for a device which permitted read-write operations. PTYs would work if the TTY line discipline code was trashed. More seriously, Brian gave a number of wonderful new ideas. But missed one I thought of as a neat basis for a user-level remote or virtual file system. Block devices. Character devices are a dime-a-dozen, but you can't mount a character device. The device open and close routines would build up an ioctl packet with just open or close. The underlying program would get around to performing an ioctl on the program side of the device and use the open or close packet to set up or tear down the underlying virtual device. The strategy routine works similiarly, but in two steps. First, an ioctl packet is built giving the same information a strategy routine always gives ;-). The driver then waits for a responding ioctl. The responding ioctl packet contains the data to satisfy the read or contains the address of the programs buffer to accept the data for a write. There are a few problems with this idea ... [ And I promise I won't suggest putting sticky bits on directories like I did the last time ;-) ] -- John F. Haugh II UUCP: ...!cs.utexas.edu!rpp386!jfh Ma Bell: (512) 832-8832 Domain: jfh@rpp386.cactus.org
rec@dg.dg.com (Robert Cousins) (12/29/89)
In article <17507@rpp386.cactus.org> jfh@rpp386.cactus.org (John F. Haugh II) writes: >In article <5804@ncar.ucar.edu> jsloan@handies.ucar.edu (John Sloan,8292,X1243,ML44E) writes: >>From article <10641@ucsd.Edu>, by brian@ucsd.Edu (Brian Kantor): >>> A programmatic device (pdev) is a program (an executable binary) living >>> in the /dev directory. When a user program tries to open it for >>> reading or writing, a process is started that runs the program, and >>> ties its stdin and stdout to the user program open. >> : >>> Oh, and if this isn't an original idea, pardon me. I honestly can't >>> remember having heard of this before in the Unix world. >> >>I agree, if I had these I'd use them. And the idea can't be original, >>because I've been carrying it along in my head for a long time, and >>I know that I'm not smart enough to have thought of it myself. > >I implemented a `programatic device' at a former job to give me a C/A/T >typesetter for an old-style troff and an HP 7475 plotter for the 20/20 >spreadsheet the company used. > >More seriously, Brian gave a number of wonderful new ideas. But >missed one I thought of as a neat basis for a user-level remote >or virtual file system. Block devices. Character devices are a >dime-a-dozen, but you can't mount a character device. > >John F. Haugh II UUCP: ...!cs.utexas.edu!rpp386!jfh >Ma Bell: (512) 832-8832 Domain: jfh@rpp386.cactus.org Actually, there exists a way to do all of this under most UNIXs today. Using the mount(2) call (or its logical equivalent) for NFS mounts, it is possible to register a user level program as an NFS server. The server would recieve V-node based operations and would appear as a file system to the host. The `pdev' operations could be performed by simply piping data through a defined resource program (many of which are already written). Special device I/O such as reading foreign file systems (MS-DOS, CD-ROM (hsg, iso 9660), Appleshare, etc.) could be built in this way. A very useful fact is that this server would be able to implement "virtual directories" which could contain all manner of things from various services to virtual devices (NFS doesn't support devices directly, but they can be fudged somewhat) and configuration information. Since systems can support multiple servers, there is no reason why any one server must be omnipotent. Use one for Pdevs, one for file systems, one for Comms, one for streams operations which are too bizarre to worry with otherwise. Once the base code for installing oneself and mounting oneself is done, the remainder should be simple. (Sounds like a good thing to put into a library, too.) Lastly, the interpretation of security is done by the server. This translates into extreme flexability for some touch applications. Picture, for example, a special "database filesystem" which magically keeps all information in its files in the proper order. Different files in the same directory are actually the same table with different views as defined by the administrator. However, when some users attempt to access some views, they succeed, while others fail.... The possibilities here are many. Just more fuel for the fire. Robert Cousins Dept. Mgr, Workstation Dev't. Data General Corp. Speaking for myself alone.
pcg@aber-cs.UUCP (Piercarlo Grandi) (12/30/89)
In article <3062@cbnewsj.ATT.COM> avr@cbnewsj.ATT.COM (adam.v.reed) writes: In article <10641@ucsd.Edu>, brian@ucsd.Edu (Brian Kantor) writes: > A programmatic device (pdev) is a program (an executable binary) living > in the /dev directory. When a user program tries to open it for > reading or writing, a process is started that runs the program, and .... > Oh, and if this isn't an original idea, pardon me. I honestly can't > remember having heard of this before in the Unix world. (It is vaguely This looks (except for some ad-hoc details) like a trivial instance of a stream device with a single stream module. Why write if you can buy? But the module, except when using some tricks, has to be linked in with the kernel. On the other hand you can find a nice description of portals in the 4.2BSD system manual. Alongside with wrappers and user process domains they have never been implemented. It has become fashionable is instead to implement fake file systems, instead of devices, and mount them here and there. This is particularly easy with NFS; you create a server that accepts the NFS procotol but does all sort of things instead of actually operating on files. Under Mach, or the INET domain in 4.2BSD TCP/IP you ask the service server to give you a handle (socket, port) to a named service. If the server is not active, the service server will start it. Using the Unix path walker as service server is attractive, but has its limitations, even if as I understand it Plan 9 does precisely that. -- Piercarlo "Peter" Grandi | ARPA: pcg%cs.aber.ac.uk@nsfnet-relay.ac.uk Dept of CS, UCW Aberystwyth | UUCP: ...!mcvax!ukc!aber-cs!pcg Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@cs.aber.ac.uk
wade@wd.msc.umn.edu (Wade Stebbings) (12/30/89)
In article <10641@ucsd.Edu> brian@ucsd.Edu (Brian Kantor) writes: >A programmatic device (pdev) is a program (an executable binary) living >in the /dev directory. When a user program tries to open it for >reading or writing, a process is started that runs the program, and >ties its stdin and stdout to the user program open. All I/O to and >from the pdev appears to the user as though it is simply reading and >writing a device; the pdev can open real or pseudo-devices and do real >I/O as appropriate. Ordinarily, the last close will cause the pdev to >stop running and exit. . . . SPRITE pseudo-devices are very similar to this idea; one main difference is the way open/close are handled. SPRITE is a Berkeley research project. I believe the paper "Pseudo Devices: User-Level Extensions to the Sprite File System" is in the USENIX proceedings from a couple years back (in San Francisco). This stuff also runs on BSD, if I remember the paper correctly. wade