dave@vdelta.UUCP (Dave Arnold) (01/20/88)
I am new to TCP/IP, so please excuse me if this is elementary, but... Could someone please tell me what approach is taken for applications programs to use a TCP/IP network? More specifically, what does a C program look like that is doing network I/O on a TCP/IP network? Is the source code for the FTP client portable across all systems on the ARPA Internet? Are there efforts underway to standardize a C language interface for TCP/IP? I have asked a particular vendor about this for VAX/VMS, and all he said was "You get a QIO interface". Great, so when we migrate to OSI, we will have to rewrite all our software. How does BSD sockets and streams fit into this? We would like to create a "Virtual network runtime library", that would allow application programs to interface to any kind of network. Has such a model already been suggested? Does anybody think that this idea is not feasable? Thanks in advance. Dave Arnold dave@arnold.UUCP
jmg@cernvax.UUCP (jmg) (01/21/88)
In article <25@vdelta.UUCP> dave@vdelta.UUCP (Dave Arnold) writes: >Are there efforts underway to standardize a C language interface for TCP/IP? >I have asked a particular vendor about this for VAX/VMS, and all he said was >"You get a QIO interface". Great, so when we migrate to OSI, we will have >to rewrite all our software. How does BSD sockets and streams fit into this? We decided to define a set of routines for using a transport service, called Common Applications Transport Service (CATS). It was oriented towards OSI, but has been implemented on top of DECNET also. If I have understood it correctly it is more oriented towards streams than sockets, although we think that it could go onto both. The routines have been put onto various computers and operating systems, and there is quite a lot of interest: physicists also want to avoid changing their programs more than necessary. Of course, the OSI TP4 is not really meant to be used directly by programmers, and there are thus no defined programming interfaces. However, in the real world people chase after efficiency, speed etc., which they feel is better by avoiding such things as presentation and session level. (if that does not start a religious war then nothing will!)
gwyn@brl-smoke.ARPA (Doug Gwyn ) (01/21/88)
In article <25@vdelta.UUCP> dave@vdelta.UUCP (Dave Arnold) writes: >Are there efforts underway to standardize a C language interface for TCP/IP? Many vendors provide 4BSD-compatible socket-oriented library routines, which may consist of a mixture of system call interfaces and user-mode algorithms, or some emulation that accomplishes the same thing. AT&T recently defined a Transport Level Interface standard, which is somewhat more general and is used by STREAMS-based network code. Without having looked into it in detail, I suspect that either of these libraries could for the most part be emulated in terms of the other. >We would like to create a "Virtual network runtime library", that would allow >application programs to interface to any kind of network. Good luck; there are too many oddball networks in existence. One thing you might consider is to make provisions to use whatever user-level facilities each operating system has. For example, in a current project, in order to run a particular application that massages a data file, I map the name of the data object to an information file that tells me what command to use to fire up a remotely-executing slave process, then I just pipe stuff to and fro independently of the underlying network implementation. Works great.
jimp@cognos.uucp (Jim Patterson) (01/26/88)
In article <25@vdelta.UUCP> dave@vdelta.UUCP (Dave Arnold) writes: >Are there efforts underway to standardize a C language interface for TCP/IP? Sun has developed a definition called RPC (Remote Procedure Call) which is becoming a defacto standard in some environments. RPC is designed to support the function-call paradigm across TCP/IP networks. Sun has published the protocol to encourage its implementation on other systems. In conjunction with RPC, Sun has an excellent protocol called XDR (eXternal Data Representation) to deal with data conversions between different machine architectures in a flexible but structured manner. While I've studied these protocols a bit, I don't have any direct experience with them. I also don't know if anyone has implemented them on VAX/VMS systems (but I suspect that this has been done). You might contact Sun Microsystems to get more information about RPC and XDR. -- Jim Patterson Cognos Incorporated UUCP:decvax!utzoo!dciem!nrcaer!cognos!jimp P.O. BOX 9707 PHONE:(613)738-1440 3755 Riverside Drive Ottawa, Ont K1G 3Z4
guy@gorodish.Sun.COM (Guy Harris) (01/29/88)
> Sun has published the protocol to encourage its implementation on other > systems. The source code to an implementation of RPC has also been made available. It has been posted to "comp.sources.unix" and is included in the User Contributed Software portion of the 4.3BSD tape; I think we also make it available for the cost of a tape. This implementation is basically a C implementation (RPC need not be implemented in C; some people at The Johns Hopkins Hospital implemented it for MVS and MUMPS as part of an application they developed, with bindings in COBOL and MUMPS) for systems providing various 4.2BSD functions such as the socket calls, "select", "gettimeofday", etc.. It can probably be reimplemented on UNIX and non-UNIX systems providing equivalent functionality. It also includes transport interfaces for TCP and UDP. It is not tied to the Internet protocols; the Johns Hopkins Hospital version uses XNS Sequenced Packet Protocol to talk to the IBM mainframe and a homebrew asynchronous serial line protocol to talk to various other machines. An client implementation of RPC for the PC under MS-DOS is provided with the PC-NFS 2.0 product. > I also don't know if anyone has implemented them on VAX/VMS systems (but I > suspect that this has been done). They have been implemented to the extent that at least one NFS server implementation for VAX/VMS exists; I don't know if an RPC or XDR library is provided for VAX/VMS that applications can use. Guy Harris {ihnp4, decvax, seismo, decwrl, ...}!sun!guy guy@sun.com
schwartz@gondor.cs.psu.edu (Scott E. Schwartz) (01/29/88)
In article <2168@cognos.UUCP> jimp@cognos.UUCP (Jim Patterson) writes: >In article <25@vdelta.UUCP> dave@vdelta.UUCP (Dave Arnold) writes: >>Are there efforts underway to standardize a C language interface for TCP/IP? > >Sun has developed a definition called RPC (Remote Procedure Call) >which is becoming a defacto standard in some environments. >Sun has published the protocol to encourage its >implementation on other systems. In conjunction with RPC, Sun has >an excellent protocol called XDR (eXternal Data Representation) >to deal with data conversions between different machine >architectures in a flexible but structured manner. >You might contact Sun Microsystems to get more information about >RPC and XDR. Sun has done more than just publish the protocol definition. The source for the entire thing (at least the 4.3BSD version) is available from titan.rice.edu (the sun-spots-digest people) by anonymous ftp. I recently compiled it all on a VAX/750 with no trouble at all. Sun encloses a fun demo called "rstat_svc" which suntools perfmeters can interrogate to find out load, %cpu, etc. It all comes with about 100 pages of documentation, too. By the way, Apollo recently announced some other RPC protocol that they would like to become the standard, but I don't know anything about it. -- Scott Schwartz schwartz@gondor.cs.psu.edu
gwyn@brl-smoke.ARPA (Doug Gwyn ) (01/30/88)
In article <2168@cognos.UUCP> jimp@cognos.UUCP (Jim Patterson) writes: >In conjunction with RPC, Sun has >an excellent protocol called XDR (eXternal Data Representation) ... I wouldn't call it "excellent". A few months back, when I needed to design and implement a package for inter-system data exchange as part of a large project we're developing, the first thing I thought of was to adopt XDR for my machine-independent data transfer. So, I looked it up in the Sun reference manual and found that it had FAR too much overhead for my purposes. I ended up implementing my own scheme that runs much faster.
mishkin@apollo.uucp (Nathaniel Mishkin) (01/30/88)
In article <3256@psuvax1.psu.edu> schwartz@gondor.cs.psu.edu (Scott E. Schwartz) writes: >In article <2168@cognos.UUCP> jimp@cognos.UUCP (Jim Patterson) writes: >>In article <25@vdelta.UUCP> dave@vdelta.UUCP (Dave Arnold) writes: >>>Are there efforts underway to standardize a C language interface for TCP/IP? >> >>Sun has developed a definition called RPC (Remote Procedure Call) >>which is becoming a defacto standard in some environments. > >>Sun has published the protocol to encourage its >>implementation on other systems. > >By the way, Apollo recently announced some other RPC protocol that >they would like to become the standard, but I don't know anything >about it. Apollo's RPC system is called the Network Computing Architecture (NCA). Apollo's portable implementation of NCA is called the Network Computing System (NCS). NCS runs on a BSD and System V Unix variants, VMS, and MS/DOS. NCA defines a special-purpose lightweight request/response protocol that can (and does) run over a variety of datagram (unreliable) services (e.g. UDP/IP), yet maintains the "at-most-once" call semantic. (Calls to procedures declared to be idempotent are supported at slightly lower cost.) The protocol supports reflecting server exceptions back to clients and asynchronous interrupts (quits) at the client being passed to a server process running on the client's behalf. Fragmentation and reassembly are performed if the input or output parameters don't fit in a single datagram. The reason we went with a special-purpose protocol was to minimize message exchange overhead and load on server and client. (We expect it to not be uncommon for a single server to be contacted by 100s of clients over, say, a one minute period. We expect it to not be uncommon for a single client to want to contact 100 servers over a similar period.) Also, not all virtual circuit protocols are created equally, making it hard to write a single package that will run equally well over a variety of them. NCA defines a data presentation protocol, the Network Data Representation (NDR), which operates in what we call a "quasi receiver makes it right" fashion. I.e. it supports a fairly rich set of different canonical scalar data representations which a sender can chose from and a receiver must convert from (if necessary) upon receiving the data. Our goal is that when like machines communicate, no data conversion need occur. NCA defines a language, the Network Interface Definition Language (NIDL), that looks like (say) a Modula-2 interface. (The language also has a C-like variant, which tries to be in the spirit of ANSI C and C++.) NCS includes a compiler for NIDL. The compiler produces client and server stubs that interface with the network runtime and handle data conversion issues. The NIDL compiler can be used in conjunction with applications written in Unix "f77" (no small task, I'm afraid), as well as C. The NIDL compiler and NCS runtime go to fairly great pains to avoid data copying and calls to dynamic storage allocation routines. NIDL has a mechanism for dealing with types not marshallable by the compiler (e.g. arbitrary data structures like trees, linked lists). For each such type, the interface defines a corresponding "transmissible type"; the interface writer is obliged to write routines (called from the automatically generated stubs) that convert between the transmissible and the abstract type. This scheme has the virtue of being declarative as far as the interface is concerned -- the interface (and not any user-written code) precisely specifies what appears "on the wire". NCA defines and NCS supports a rich, object-oriented binding model promoting a view of distributed computing that is based on what you want to do, not where you want it done. (I know that was buzzspeak, but I don't want this to go on forever.) This model is embedded in NIDL, making it possible to define interfaces and write applications that have no outward sign of being "remote" or having anything to do with networks, host names, or network addresses. Various Apollo projects and products have been implemented using NCS. These include a replicated object (service) location service, a replicated user registry (upon which the Unix "getpw" calls are implemented), a distributed software license checkout system (Network License Server). Outside Apollo, various projects based on NCS are underway, some at companies and some at universities. University of Iowa developed a simulation system that uses a Alliant FX series minisuper and an Apollo cooperatively. An independent implementation of the NCA/RPC protocol was done at the San Diego Supercomputer Center for their CTSS/Cray system. Since someone referred to Sun RPC as a "de facto standard", I feel free to introduce my (admitedly biased) comment that (a) I don't really see this as being the case, and (b) NCA/NCS has numerous technical advantages over Sun RPC. Sun RPC is widespread because NFS is based on it. Until recently, there wasn't an effective interface definition language or compiler for that language (and the current one has various limitations), thus making Sun RPC not a terribly attractive system for developing a variety of large distributed applications. Correct (i.e. at-most-once) call semantics require the programmer to know to choose (and to in fact choose) a proper (i.e. reliable, virtual circuit) underlying transport (e.g. TCP/IP). Use of UDP/IP even in the case of idempotent procedures can cause program failures if at runtime it turns out that the input or output parameters don't fit into a single UDP datagram. The Sun RPC interface definition language (RPCL) obliges all procedures to take one input parameter and return results via function value (essentially burdening the application program with the marshalling and unmarshalling process). Now I know there's someone who knows (and is going to tell me and the world) that you can get Sun RPC from Sun (or whoever) for free, and Apollo is not giving away NCS. We do have a draft specification of NCA which we will give away. (Who knows, we may have to charge the cost of materials and mailing if we get a ton of requests for it.) The NCS source code costs money. This is not the appropriate forum to go into details about cost and value, but I will at least say that universities can (under license) get it for a nominal amount of money. We hope that binary versions of NCS will become readily available at low price. The world will have to decide whether it wants to pay for it. We think it's worth it. There was a paper describing all this in the Summer 1987 Usenix. People can feel free to contact me for more information. -- -- Nat Mishkin Apollo Computer Inc. Chelmsford, MA {decvax,mit-eddie,umix}!apollo!mishkin
ok@quintus.UUCP (Richard A. O'Keefe) (01/30/88)
In article <7202@brl-smoke.ARPA>, gwyn@brl-smoke.ARPA (Doug Gwyn ) writes: > In article <2168@cognos.UUCP> jimp@cognos.UUCP (Jim Patterson) writes: > >In conjunction with RPC, Sun has > >an excellent protocol called XDR (eXternal Data Representation) ... > I wouldn't call it "excellent". A few months back, when I needed to > ... > it up in the Sun reference manual and found that it had FAR too much > overhead for my purposes. I ended up implementing my own scheme that > runs much faster. How can a program run much faster than a PROTOCOL? The protocol is one thing, the source-code implementation of it that SUN give away is quite another. With respect to the protocol, XDR is historically related to Courier, so there's a fair bit of experience behind the *protocol*. It may well be a good idea to write one's own implementation of the protocol, but it is not such a good idea to invent yet another data conversion protocol.
henry@utzoo.uucp (Henry Spencer) (01/31/88)
> Since someone referred to Sun RPC as a "de facto standard"... Apollo notwithstanding, Sun RPC *is* a de facto standard. Not because it's good, but because it was first, and is already widespread. > ... The world will have to decide whether it wants to pay for it... *This* part of the world most assuredly isn't going to pay for it until we see a manufacturer-independent standard, meaning in particular that Apollo or Sun can't change the rules any time they feel like it. (Please spare us the automatic "we would never do that" response. Having had experience with the SVID-Of-The-Month Club run by AT&T, we no longer believe it.) -- Those who do not understand Unix are | Henry Spencer @ U of Toronto Zoology condemned to reinvent it, poorly. | {allegra,ihnp4,decvax,utai}!utzoo!henry
david@elroy.Jpl.Nasa.Gov (David Robinson) (01/31/88)
In article <2168@cognos.UUCP>, jimp@cognos.uucp (Jim Patterson) writes: > While I've studied these protocols a bit, I don't have any direct > experience with them. I also don't know if anyone has implemented > them on VAX/VMS systems (but I suspect that this has been done). > You might contact Sun Microsystems to get more information about > RPC and XDR. I have ported them to work on VMS running Network Research's FUSION TCP/IP package. It was an easy port because the support a BSD style interface. I have ported some RPC based services from Suns with little difficulty to VMS, problems were mostly with VMS C. I know of someone who partially ported it to the VMS Excelan TCP/IP but with considerable difficulty and it didn't work well because Excelan only provides a VMS QIO interface that is based on 4.1BSD networking. -- David Robinson elroy!david@csvax.caltech.edu ARPA david@elroy.jpl.nasa.gov ARPA {cit-vax,ames}!elroy!david UUCP Disclaimer: No one listens to me anyway!
greg@vertical.oz (Greg Bond) (02/02/88)
In article <610@cresswell.quintus.UUCP>, ok@quintus.UUCP (Richard A. O'Keefe) writes: > In article <7202@brl-smoke.ARPA>, gwyn@brl-smoke.ARPA (Doug Gwyn ) writes: > > [talks about Sun XDR protocol] > > it up in the Sun reference manual and found that it had FAR too much > > overhead for my purposes. I ended up implementing my own scheme that > > runs much faster. > > How can a program run much faster than a PROTOCOL? > The protocol is one thing, the source-code implementation of it that > SUN give away is quite another. Correct. If the PROTOCOL requires significant overhead of bytes on the wire, then the IMPLEMENTATION will be limited in the speed it can achieve, no matter how good the code implementing it is. We may say that "standards" are more important than efficiency. In this case, Doug seems to have made the judgement that the application required an efficiency un-obtainable with XDR spec, and invented his own, and assumed the support responsibility etc. Yes, the protocol is not the implementation. But great implementation may not save a poor or inappropriate protocol. [ I make this as a general point; I have no knowledge of XDR per se ] -- #define WHOAMI Gregory Bond, Vertical Software, Melbourne, Australia #define ADDRESS greg@vertical.oz, uunet!vertical.oz!greg #define JOKE Ain't no-one here but us chickens... #include <standard/disclaimer>