pae@athena.mit.edu (Philip Earnhardt) (07/05/90)
In <4b2772cd.20b6d@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: > I'm so glad that now there's yet another newsgroup (i.e., this one) I have > to read to make sure I don't miss something important :-) Pretty clever of > you-all to put the conversation someplace where I wouldn't look (as opposed > to, oh comp.sys.apollo or comp.protocols.misc or something). Not clever > enough though! > mishkin@apollo.hp.com > ^^^^^^^^^^^^^ Oh, no! The plot has been uncovered: keep Mishkin from seeing the postings by putting them on comp.sys.hp. The mind boggles at the diabolical nature of this conspiracy! Never mind that Mishkin works for HP, or that this list has gotten many Apollo mailings since the takeover, or even that markl@hpbbi4.HP.COM (Mark Lubfkin) was the one who started the thread. Conspiracy theory aside, you're probably right. This discussion does not belong on any manufacturer-specific mailing list. I'm cross-posting to comp.protocols.misc and suggest that we move the discussion there. > While there is certainly some truth to your comments about the problems > with the approach of building a reliable transport in user space, I think > most of them are tractable and tolerable in light of the benefits that > you get from using our approach. Many of these are described in my paper > you cited in your message. For example, the connection-based approach > imposes some penalties in connection setup and teardown. Obviously these > costs are amortized in case you make a lot of calls with that connection. > Unfortunately, we expect to see lots of clients that need to make a ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > relatively few calls to a relatively large number of servers in turn. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > In this situation, the connection management overhead can be high. This is a scenario that I don't understand in NCS. What if, say, 100 different clients each make a single non-idempotent call to a server (and then go off and talk to different servers). Say that each of the calls returns 2K of data to the client. Since the calls are non-idempotent, the server is required to hold each client's response until it is acknowledged. How long does the server keep hold of the 200K of data? BSD implementations usually specify a worst-case round-trip delay of 1-2 minutes for the network, but a private network may have worse worst-case delay on some of its remote machines. Does the client send out an explicit ACK after getting a response to its RPC call? Does it do this on only non-idempotent requests? Can the server's queued data potentially grow without bound, or will it start refusing new RPC requests at some point? What is the worst-case delay that NCS assumes for a datagram? Could there be other assumptions about transport behavior that are wired into the NCS implementation? > However, it's really pretty hard for > me to believe that your datagram-based RPC is significantly (if at all) > cheaper than NCS's, assuming you specify (in NCS/NIDL) that the procedure > you're defining is "idempotent" (to make the comparison fair). All we > (and presumably you) do is send one message out and one message back. > The overhead of calling the OS to do the message I/O and the cost of > the message I/O dominate. The messages on the wire are going to look pretty similar to each other. However, my understanding of the NCS implementation is that you have a separate thread that processes the receipt of data/data reassembly/request for missing data fragments. At the very least, NCS will have the added overhead of an extra context switch on receipt of the client and server messages to get the message to the appropriate thread. This may be picking nits, but I believe that the Netwise RPC will be the faster datagram technology. >>Mike: what will NCS2.0 do WRT broadcast? Will it be available with both >>types of transport? If broadcast is not available under connection-oriented >>transports, won't this constitute non-uniform transport behavior? >Geez, cut me some slack. I will arrange that the manual has a skull >and crossbones around the section that describes the broadcast feature >of RPC. I'm not so facist as to refuse to let people use broadcast in >the datagram RPC system just because the people who define connection-based >transports aren't clever or ambitious enough to figure out how to support >broadcast in their protocols. In case you missed the point, it was the OSF DCE Rationale that says that uniform transport behavior was one of its criteria for selection. If you're saying that's a silly concept, I'll agree with you. I would be interested in hearing your ideas on how broadcast should work in, say, TCP/IP. A couple of general questions: 1. Why is NCS2.0 being offered on top of connection-oriented transports? Under what conditions will you recommend that programmers specify using connection-oriented transports? 2. Are there any documents which describe the wire-level behavior of NCS on top of datagram-oriented transports: o What type of packets NCS is sending (UDP or IP). o When acknowledgments are sent. o How retransmission timers work--when are they used, what values are they set to, how the application can change the values. Also, there were several unanswered questions from the earlier posting: 1. What does NCS2.0 do in environments that don't support multiple threads? 2. Will OSF be using OSI protocols? If so, which ones? 3. What about availability: have any dates been announced? What platforms will be supported? Will the OSF DCE be available on non-UNIX systems? What about DOS PCs? Phil Earnhardt Netwise, Inc. 2477 55th St. Boulder, CO 80301 Phone:303-442-8280 UUCP: onecom!wldrdg!pae My opinions do not reflect any official position of Netwise.
pae@athena.mit.edu (Philip Earnhardt) (07/05/90)
In article <4b278ff2.20b6d@apollo.HP.COM>, mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: > In article <jwright.645863199@quonset>, jwright@cfht.hawaii.edu (Jim > Wright) writes: |> Could either/both of you address the issue of "server selection". |> As I understand it now, RPCs have to be directed at a specific |> server, while NCS provides some sort of "broker" to arbitrate calls |> at runtime. Might this be a significant differentiation? > I think there's probably some bit of confusion here. (It's a little > hard to tell since I never got a copy of the innermost article above.) > The "LB" consists of two pieces: The Global Location Broker (GLB) and > the Local Location Broker (LLB). > > An LLB runs on every machine and does the forwarding job (i.e., > transparently forwards a request sent to the MACHINE to the appropriate > server PROCESS on the machine). We believe that the facts that this > scheme results in only one real network message exchange and that it's > all transparent to the application are important. > > The GLB is a place a client can look to find the address of a particular > machine. Clients make an RPC to the GLB and then use the results to > make a call to another machine. While these two steps can be pretty much > concealed (via NIDL's auto-binding mechanism), from the point of view > of the wire, two message exchanges occur, as opposed to the one that > occurs when using (just) the LLB (i.e., because you know the machine > you want to talk to a priori). ...and presumably the GLB is what is doing the "rendezvous". Under TCP/IP, the analogue to the LLB is Yellow Pages, or rpcbind, or perhaps lookup in a local database. Are we in agreement on the terminology? Netwise uses the native naming(s) that is provided in the environment. Some environments (e.g., NetBIOS) provide something similar to rendezvous by via a "locator". Contrary to earlier postings, Sun's native rpcbind naming provides a rendezvous-like service via datagram broadcast: 1. client broadcasts to rpcbind servers for PROG #, VERS #, REV # 2. if a machine's rpcbind has had a worker register with PROG #, VERS #, the request is forwarded to the worker. Some naming environments do not provide anything like rendezvous. Rendezvous is a useful thing. More generally, dynamic X.500-style naming/directory services seem to be the wave of the future. There appear to be several problems in the current LLB/GLB strategy: 1. From what we've heard, the LLB/GLB is an add-on application. Normally, an RPC developer pays for the development toolkit, develops his distributed application, then produces his distributed application without paying a royalty to the toolkit manufacturer (except for the Sun "toolkit", which is free). However, the LLB/GLB must be available on the machine running the distributed application. If the LLB/GLB isn't on the target machine, the provider of the application must get it installed there in order to use it. This is, in effect, a royalty payment to HP for use of the NCS RPC. 2. Until recently, the GLB was apparently available on very few platforms (e.g. Apollo workstations). Unless you were using those machines, or were willing to have your customers buy one for their network, the GLB was unavailable. 3. In a complex network, there may be serveral subnets. Customers need to have control of the granularity of their subnet broadcasts--they may want to talk to a GLB that's available on their local subnet, or they may want to talk to GLBs on other subnets. Does the LLB/GLB scheme provide control of the granularity of broadcasts? Phil Earnhardt Netwise, Inc. 2477 55th St. Boulder, CO 80301 Phone:303-442-8280 UUCP: onecom!wldrdg!pae My opinions do not reflect any official position of Netwise.
pae@athena.mit.edu (Philip Earnhardt) (07/05/90)
In <4b2772cd.20b6d@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) says: > To cast the argument in the most extreme terms (always a good way to > have a rational conversation :-), many people probably think the "asm" > feature of C compilers is a good idea. That doesn't mean it should be > included in ANSI C. I have no doubt that people have found ways to use > Netwise's customization feature to do things that they've found useful. > This doesn't mean it's a good idea. It's simply too unconstrained. It appears to me that what you are saying is that either NCS allows all possible mechanisms for client-server interaction or that if NCS doesn't include a particular mechanism then a distributed systems designer shouldn't need it and (more distressingly) shouldn't be able to use it. From my perspective, this is simply too constrained. We constantly get calls from customers who have come up with perfectly valid ways to construct a client server interaction that are not directly handled by the mechanisms that we provide (and I suspect are not handled directly by the mechanisms available with NCS). Whereas you might claim that they are using the RPC mechanism in a way that 'breaks' the RPC paradigm, I would show them how to customize their RPC spec to work in the way that they need it to. It seems to me that C language programmers (even ANSI C programmers :-) will be unwilling to allow any organization to completely limit their flexibility based on arbitrary 'standards' of client-server computing. Of course, there is danger inherent in the customization features of the RPC Tool, but these dangers are offset by the benefits gained ... there are also dangers in using pointers in C and their use is largely unconstrained, this doesn't mean it's a good idea to remove them from the language ... > Further, I suspect it's frequently used to get around deficiencies in > the base system. Just out of curiousity, have you ever used the Netwise RPC Tool? I have not used NCS so anything that I have said here is mostly from hearsay. It seems to me that any 'base system' will have its own set of deficiencies based on the biases of the designers of the system and that the only real way to get around these deficiencies is to allow for some kind of customization by the end user. It is clear that programmers are going to customize their application and that the best solution is to provide a well thought out mechanism that can be used to customize their code in a well defined way. Some questions: - Does NCS 1.x provide mechanisms for asynchronous RPC calls? callback routines? user-specified encryption algorithms (i.e. for someone who doesn't trust that the NSA hasn't broken the DES)? auditing of arbitrary user-specified parameters/statistics? - Does NCS 2.0 add any of these features? [ RPC protocol description deleted ] > As I understand it, the Netwise customization feature allows both the > "RPC protocol" and the protocol specified by a particular interface > definition to be manipulated in fairly arbitrary ways. First, I think > that being able to manipulate the RPC protocol itself is a bad idea (for > the reasons I stated above). Once again, unless the 'base system' provides for all possible models of client-server computation, you are going to have some unhappy customers. These customers may have to go through wild gyrations to get around arbitrary restrictions placed on them by the base system, but in the end they will customize their application to work with the model of their choosing. jake edge L & E Technologies, Inc. (303) 440-1142 ...!onecom!wldrdg!jake Disclaimer: For all of the usual reasons, what I say does not neccesarily reflect the views of my employer ... in addition, my views do not neccesarily reflect those of my client (Netwise).
mishkin@apollo.HP.COM (Nathaniel Mishkin) (07/06/90)
In article <1990Jul5.040529.16093@athena.mit.edu>, pae@athena.mit.edu (Philip Earnhardt) writes: |> In <4b2772cd.20b6d@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: |> Oh, no! The plot has been uncovered: keep Mishkin from seeing the postings by |> putting them on comp.sys.hp. The mind boggles at the diabolical nature of this |> conspiracy! Never mind that Mishkin works for HP, or that this list has gotten |> many Apollo mailings since the takeover, or even that markl@hpbbi4.HP.COM |> (Mark Lubfkin) was the one who started the thread. OK, OK. I could have seen this coming. It's just that keeping up with the multi-100 articles/day on rec.arts.startrek has gotten me down. I promise to be a good boy and read all the mail about the vices and virtues of HP-UX from now on :-) |> This is a scenario that I don't understand in NCS. What if, say, 100 |> different clients each make a single non-idempotent call to a server |> (and then go off and talk to different servers). Say that each of the |> calls returns 2K of data to the client. Since the calls are |> non-idempotent, the server is required to hold each client's response |> until it is acknowledged. The basic idea is that the client ack's the response either implicitly (by making another call to that server) or explicitly (by sending an explicit "ack" message). The implementation defers sending an explicit ack for a few seconds (in case the client is in fact making multiple back-to-back calls to the server). The server retransmits the response periodically until the response is acknowledged. Upon receipt of a duplicate response, the client retransmits an explicit ack (assuming that a previous transmission of the explicit ack got lost). (It was a bug in 1.5 that this last step didn't happen.) If all else fails and the server never hears an ack from the client, the server will discard the response after some period of time (currently 30 seconds). Maybe I'm wrong, but I don't think any of this can be fundamentally better for TCP/IP. If I have 100 connections open and I've written 2K of data to each of them, my buffer usage is tied to each of the client's ack'ing in a timely fashion the data I've sent it. In practice, I'd think, TCP/IP is going to be worse in dealing with this scenario. First, I'm consuming valuable kernel resources in the form of these 100 connections (TCBs). (I wouldn't be surprised if most Unix kernels were configured in a way so that if a single server consumed 100 TCP connections, it would be using a large fraction of all the TCP connections allowed for the whole system.) Second, all the unack'd data has to live in wired kernel mbuf's, whereas NCS's buffers are in (page-able) user address space pages. (I suspect that what's really going to happen if you use TCP/IP is that one of the "send" system calls is just going to block awaiting the availability of an mbuf. Needless to say, this would be disasterous from the RPC library's point of view.) |> The messages on the wire are going to look pretty similar to each other. |> However, my understanding of the NCS implementation is that you have a |> separate thread that processes the receipt of data/data reassembly/request for |> missing data fragments. At the very least, NCS will have the added overhead of |> an extra context switch on receipt of the client and server messages to get |> the message to the appropriate thread. The cost of the context switches is in the noise compared to the cost of the message I/O and making calls to things like "select" and "recvfrom". When someone gives me an environment where the local OS doesn't consume unseemly amounts of time doing fundamentally nothing, I'll worry about things like context switch overhead. |> >Geez, cut me some slack. I will arrange that the manual has a skull |> >and crossbones around the section that describes the broadcast feature |> >of RPC. |> In case you missed the point, it was the OSF DCE Rationale that says that |> uniform transport behavior was one of its criteria for selection. If you're |> saying that's a silly concept, I'll agree with you. I would be interested in |> hearing your ideas on how broadcast should work in, say, TCP/IP. Everything is a matter of degree. Broadcast is a feature that a relatively few (and hopefully almost no) applications need. At-most-once call semantics is a feature that a relatively large (possibly almost all) applications need. Thus, making the former a function of transport choice can be considered tolerable. Making the latter such a function seems unwise. |> A couple of general questions: |> |> 1. Why is NCS2.0 being offered on top of connection-oriented transports? |> Under what conditions will you recommend that programmers specify using |> connection-oriented transports? Some environments have only COTPs. (E.g., our ISO/OSI friends have not yet seen the wisdom of making a priority defining and implementing CLTPs.) We want NCS to run in these environments. Hence we must make NCS run over COTPs. Time will tell about what the right recommendations to make are. I would like to believe that things will turn out so that I can say "Use COTP only when there is no CLTP available". An RPC expert made the argument to me that I should say "Use CLTPs in LANs and COTPs in WANs" because getting an RPC/CLTP to work right in a LAN environment is easier than getting it to work right in a WAN environment. Maybe he's right. However, I'd like to think that there's not black magic here that would prevent me from doing what the COTP does over WANs. But maybe it'd turn out to not be worth the effort. Like I said, time will tell. |> 2. Are there any documents which describe the wire-level behavior of NCS |> on top of datagram-oriented transports: See "Network Computing Architecture" (Prentice-Hall, ISBN 0-13-611674-4). Let me know if it doesn't answer all your questions. We tried to be complete, but I wouldn't be surprised if we failed in some areas. There's always the next edition :-) |> Also, there were several unanswered questions from the earlier posting: |> |> 1. What does NCS2.0 do in environments that don't support multiple threads? The NCS 2.0 implementation currently depends on threads. We made this choice with our eyes open. It was simply a tradeoff between what we felt we could get done with what set of tools. I believe that this choice will not be problematic for Unix environments, as POSIX is standardizing on a multi-thread interface (PThreads), and OSF chose a portable implementation of PThreads (CMA from DEC) as part of the DCE selection. I don't believe OS/2 should be a serious problem since it already supports multi-threading. That leaves MS/DOS. I believe getting this one taken care of is only a matter of time/money and expertise. |> 2. Will OSF be using OSI protocols? If so, which ones? I don't know what OSF's plans are for delivering support for the OSI protocols. I'm sure they will be acquiring OSI implementations though. |> 3. What about availability: have any dates been announced? I'm afraid I don't recall the content of OSF's public statements. I believe they said words to the effect that stuff will be available in '91. |> What platforms will be supported? Will the OSF DCE be available on |> non-UNIX systems? What about DOS PCs? OSF sells source. The source they sell should be widely portable to Unix systems, as that's OSF's target market. I imagine the source will be largely portable to VMS because of DEC's involvement in the project. I don't know what OSF's commitment is to DOS. I'm sure they understand the wisdom of supporting that environment. -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
mishkin@apollo.HP.COM (Nathaniel Mishkin) (07/06/90)
In article <1990Jul5.041344.16511@athena.mit.edu>, pae@athena.mit.edu (Philip Earnhardt) writes: > ...and presumably the GLB is what is doing the "rendezvous". Under > TCP/IP, the analogue to the LLB is Yellow Pages, or rpcbind, or > perhaps lookup in a local database. Are we in agreement on the > terminology? I think not. By TCP/IP, I assume you mean Sun RPC on TCP/IP. If you're comparing NCS and Sun RPC, then the LLB is somewhat analogous to the Sun RPC Portmapper not to YP. (Note that unlike the Portmapper, the LLB does forwarding/portmapping transparently to the client.) I supposed you could draw an analogy between the GLB and YP, in that they both maintain a database for network-wide (as opposed to host-wide) consumption. However, YP doesn't specify a way to make dynamic updates. Updates happen by having an administrator edit some text file and then "push" the data to slave YP servers. The GLB supports dynamic update of the database directly from application processes. > 1. From what we've heard, the LLB/GLB is an add-on application. > Normally, an RPC developer pays for the development toolkit, > develops his distributed application, then produces his distributed > application without paying a royalty to the toolkit manufacturer > (except for the Sun "toolkit", which is free). However, the LLB/GLB > must be available on the machine running the distributed > application. If the LLB/GLB isn't on the target machine, the > provider of the application must get it installed there in order to > use it. This is, in effect, a royalty payment to HP for use of the > NCS RPC. To be able to do rendezvous -- i.e., to support clients that don't essentially hardwire a server host name ("hardwire" == accept on the command line or take from a local config file) -- you need something like YP or the GLB. Sun does not give away the source to YP nor do they support it in binary form on non-Sun platforms. Some (all right, many) vendors have licensed YP from Sun. We don't give away the GLB. At NCS 1.5, we sell it in binary form for certain non-HP systems. We also sell the source for the entire NCS runtime, including the GLB and LLB, for somewhere in the small thousands of dollars. There are no royalties due from a source licensee who ships binaries derived from licensed source. To support the hardwired case, yes, you need the LLB on the server system (as you need the Portmapper for Sun RPC). The LLB is available in binary for for various systems (it comes with the NCS runtime binary product). We should simply allow 3rd parties to re-ship the LLB binary with their products. I don't know if we do. That would be easy enough to fix. At worst, the 3rd party could buy the NCS runtime source and build and re-ship the LLB themselves. (They wouldn't really wouldn't even have to build it. After they bought the source, they could just ship our binaries. No one could tell the difference after all. Thus, we should simply tell people they can re-ship the LLB binary without buying the source. We're not making big bucks off of selling the NCS runtime source.) > 2. Until recently, the GLB was apparently available on very few > platforms (e.g. Apollo workstations). Unless you were using those > machines, or were willing to have your customers buy one for their > network, the GLB was unavailable. See above. > 3. In a complex network, there may be serveral subnets. Customers need > to have control of the granularity of their subnet broadcasts--they > may want to talk to a GLB that's available on their local subnet, > or they may want to talk to GLBs on other subnets. Does the LLB/GLB > scheme provide control of the granularity of broadcasts? First of all, it IS possible to have multiple disjoint GLB database in a single environment. But I really don't want to argue the virtues of the GLB. For all I care, people can use YP to do rendezvous for their NCS applications. Nothing precludes it as far as I can tell. You'd simply call the appropriate YP functions (yp_bind, yp_match) supplying the database name and key value and get back a network address or host name or whatever you've decided to put in your YP map. You then make your NCS RPC binding using the returned data. What I'm saying is that NCS can use "the native naming(s) that is provided in the environment" too. We just so happen to supply a "naming system" of our own too (GLB). We think it has certain virtues. We think other systems have virtues too. The OSF DCE will have a version of DEC's naming server, which supports a hierarchical name space based on replicated database. Names in the name space can have arbitrary attribute/value pairs stored with them. You can use this data to do RPC rendezvous. -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
mishkin@apollo.HP.COM (Nathaniel Mishkin) (07/06/90)
> Of course, there is > danger inherent in the customization features of the RPC Tool, but these > dangers are offset by the benefits gained ... there are also dangers in > using pointers in C and their use is largely unconstrained, this doesn't > mean it's a good idea to remove them from the language ... I understand your position and I don't know that there's much point arguing about the relative merits of customization. We simply disagree. > Just out of curiousity, have you ever used the Netwise RPC Tool? I have not > used NCS so anything that I have said here is mostly from hearsay. I've read the documentation and seen bits of code (samples, examples, etc.) written to use it. > - Does NCS 1.x provide mechanisms for asynchronous RPC calls? callback > routines? user-specified encryption algorithms (i.e. for someone > who doesn't trust that the NSA hasn't broken the DES)? auditing of > arbitrary user-specified parameters/statistics? > - Does NCS 2.0 add any of these features? Async RPC is another one of those religious issues. My religion says that they're a bad idea because they're hard to understand, represent an unnecessary new aspect in the computing model, and that one should use existing primitives for asynchrony (process) and communications (procedure call) together to achieve the same effect. And in anticipation of the "but not all systems support cheap multi-threading" objection, I'll say (a) let's fix that and not introduce new features, and (b) it can't be that much harder to bring up a simple threading package than it is to bring up the async part of an RPC system that support async RPC and the return is much higher for doing the former than the latter. NCS 2.0 supports callbacks to known procedures (i.e., not callbacks through passed procedure pointers). Our intention to to support callbacks in general as part of the base system. Authentication and privacy features are part of NCS 2.0. -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
pae@athena.mit.edu (Philip Earnhardt) (07/11/90)
In <4b6999c3.20b6d@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: > Maybe I'm wrong, but I don't think any of this can be fundamentally better > for TCP/IP. If I have 100 connections open and I've written 2K of data > to each of them, my buffer usage is tied to each of the client's > ack'ing in a timely fashion the data I've sent it. In practice, I'd > think, TCP/IP is going to be worse in dealing with this scenario. First, > I'm consuming valuable kernel resources in the form of these 100 > connections (TCBs). (I wouldn't be surprised if most Unix kernels were > configured in a way so that if a single server consumed 100 TCP > connections, it would be using a large fraction of all the TCP connections > allowed for the whole system.) Second, all the unack'd data has to live > in wired kernel mbuf's, whereas NCS's buffers are in (page-able) user > address space pages. (I suspect that what's really going to happen if > you use TCP/IP is that one of the "send" system calls is just going to > block awaiting the availability of an mbuf. Needless to say, this would > be disasterous from the RPC library's point of view.) Actually, things should work pretty smoothly under TCP. Since the connection is being maintained by the kernel, the acknowledgment can be sent immediately after the packet is processed by the kernel. NCS would have the ack sent out 3 seconds after the user process has processed the data. Also, acknowledgments can be sent back to the server for fragments of the message. It's probably not necessary to understand all the nuances of the NCS implementation in order to understand the fundamental problem with a connection-oriented protocol implemented outside the kernel on top of a datagram transport. The kernel has access to real-time resources that are not available to user processes. There may be multiple network interfaces for a transport on the same system that can support different datagram sizes--the user process doesn't need to understand these differences to efficiently use a transport. There are undoubtedly some implementations of NCS that will work reasonably well for a given networking configuration, but will not work for the general case. If the connection-oriented scheme at the user level were the way to go, why isn't everyone else jumping on the bandwagon? Why not implement X that way? Are there any RFCs that discuss the NCS technology? Have any of the Internet gurus endorsed it? > Everything is a matter of degree. Broadcast is a feature that a relatively > few (and hopefully almost no) applications need. ^^^^^^^^^ ??? In order for your GLB to work, you need broadcast. |> 2. Are there any documents which describe the wire-level behavior of NCS |> on top of datagram-oriented transports: > See "Network Computing Architecture" (Prentice-Hall, ISBN 0-13-611674-4). > Let me know if it doesn't answer all your questions. We tried to be > complete, but I wouldn't be surprised if we failed in some areas. There's > always the next edition :-) I had already looked through this book. I hadn't found answers to the specific questions I had asked: o What type of packets NCS is sending (UDP or IP). o When acknowledgments are sent. (there's some information about the timers, but there's no timeline charts about what goes down the wire.) o How retransmission timers work--when are they used, what values are they set to, how the application can change the values. > I don't believe OS/2 should be a serious problem since it already supports > multi-threading. That leaves MS/DOS. I believe getting this one taken ^^^^^^^^^^^^^^^^^^ > care of is only a matter of time/money and expertise. There's the Mac, Atari, and other types of personal computers. Some of the mainframe environments don't have anything like threads. Many of the older UNIX systems don't support multiple threads. There are lots of other types of workstations, minicomputers, and proprietary turnkey systems that fall through the cracks. Finally, the NCA book mentions "small, embedded microprocessors used in real-time data acquisition environments" as a potential platforms for NCS. For these systems, there will have to be modifications to the OS to support multiple threads (I assume non-preemptive application-provided multithreading won't work). There's the added expense of a new OS and the hassles of configuring/installing the system software. Some of these computer systems may be at the end of their product cycle; the manufacturer may be unwilling to make any changes to their OS. There may be architectural reasons that mutlithreading can not be implemented in some environments. Should Distributed Computing not be available in these environments just because NCS implementations require multiple threads? In <4b69b513.20b6d@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: >> 3. In a complex network, there may be serveral subnets. Customers need >> to have control of the granularity of their subnet broadcasts--they >> may want to talk to a GLB that's available on their local subnet, >> or they may want to talk to GLBs on other subnets. Does the LLB/GLB >> scheme provide control of the granularity of broadcasts? > First of all, it IS possible to have multiple disjoint GLB database in > a single environment. > [discussion of NCS naming flexibility deleted] The response didn't seem to address the issue raised. How exactly does one specify the granularity of the broadcast? In <4b69c9e1.20b6d@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: > Async RPC is another one of those religious issues. My religion says > that they're a bad idea because they're hard to understand, represent > an unnecessary new aspect in the computing model, and that one should > use existing primitives for asynchrony (process) and communications > (procedure call) together to achieve the same effect. > [Discussion of the merits of implementating multi-threading ommitted.] The comparison between (preemptive) multiple threads and asynchronous calls is interesting. I find asynchronous calls fairly straightforward--you place the call, do something else for a while, check for a result, and get the reply. On the other hand, I think multiple threads are much more complex. All procedures must be reentrant--all non-stack data (globals and static variables) must either be: o read-only or o written only by one thread (and typically can't even be read by other threads unless the datum is simple) or o protected by environment-specific mechanisms that lock out other threads while the data is being modified. This is nasty stuff. There is no automatic mechanism (e.g. an option to lint) to determine if a given module is reentrant. If the code is non-reentrant, the kinds of problems that crop up are timing-related--they will be non-repeatable and probably undebuggable. For UNIX systems, the thing that makes more sense is fork(). Data segments are shared between the processes *until* one of them decides to write data. That page is then copied and the process that wrote to it keeps the copy. If data really needs to be shared between the processes, one of the UNIX-suplied inter-process communication mechanisms (or maybe even RPC ;-) could be used. It does cost a bit more than multiple threads, but you will wind up with lots more hair in your head. I'm not suggesting that NCS would be better off using multi-tasking for implementation of its pseudo connection-oriented protocols. Implementing fork() and copy on write would be tricky on a lot of today's platforms. I do feel that asynchronous calls are far easier to understand than multiple threads. Phil Earnhardt Netwise, Inc. 2477 55th St. Boulder, CO 80301 Phone:303-442-8280 UUCP: onecom!wldrdg!pae My opinions do not reflect any official position of Netwise.
pae@athena.mit.edu (Philip Earnhardt) (07/11/90)
In <4b69c9e1.20b6d@apollo.HP.COM>, mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: > I understand your position and I don't know that there's much point > arguing about the relative merits of customization. We simply disagree. This seems to be a bit of a 'back-off' from your previous postings ... are we calling it a draw (i.e. yes there are dangers in customization and yes there are instances where they are neccesary)? > Async RPC is another one of those religious issues. My religion says > that they're a bad idea because they're hard to understand, represent > an unnecessary new aspect in the computing model, and that one should > use existing primitives for asynchrony (process) and communications > (procedure call) together to achieve the same effect. And in anticipation > of the "but not all systems support cheap multi-threading" objection, > I'll say (a) let's fix that and not introduce new features, and (b) it > can't be that much harder to bring up a simple threading package than ^^^^^^^^^^^ well in the RPC Tool case, it is infinitely harder as no modifications to the base system were neccesary to support asynchronous RPCs. The customizations that you find so objectionable make it fairly trivial to support asynchronous RPCs. > it is to bring up the async part of an RPC system that support async > RPC and the return is much higher for doing the former than the latter. I think that it is only hard to allow asynchronous calls when the RPC system does not allow customization of its state transitions. The RPC Tool needed no specific modifications to allow asynchronous RPCs, just some customizations to the RPC spec to modify the state transitions. This allows the customer to decide which method of asynchrony is best for their application. If asynchronous RPCs are 'bad', why expend the effort to add cheap multi-threading to operating systems that don't currently support it? Also, multi-threading support may or may not be easy to add to an existing system and it is certainly not easy for application developers to add into existing applications. For this reason, it seems to me that offering asynchronous RPC within the framework of the existing RPC system (and allowing multi-threading or multi- tasking in environments that support it) is a must. In the Network Computing Architectures book, embedded microprocessors are specifically mentioned (page 18 if memory serves) ... are you suggesting that multi-threading be added to these systems? > Authentication and privacy features are part of NCS 2.0. Maybe I am misunderstanding here, but it sounds like you are saying that NCS 2.0 will provide methods for authentication and for privacy that are 'hardcoded' into the system. If (for some reason) a customer has their own method that they wish to use (i.e. their own encryption scheme) they will not be able to use it and that instead they must use the method provided by NCS 2.0 ... am I jumping to conclusions? BTW, in some hallway discussions that we have had around here, the concept of uniform transport behavior (sp?) has come up and it leaves me a bit puzzled. From my (perhaps somewhat limited) understanding of the current thinking in the networking community, there appear to be two camps, connection-oriented and connection-less (thus the CO/CL split in the OSI) and what uniform transport behavior in effect does is to come down clearly in the CO camp. (i.e. whether the transport is CO or CL is unimportant (except in the case of broadcast) as NCS establishes a CO protocol in either case). Is this the intent? The Netwise approach is to have uniform CO transport behavior and uniform CL transport behavior and to therefore recognize this fundamental split ... I am curious as to your thinking on this. ------------------------------------------------------------------------------ The verbiage above does not neccesarily reflect the views of my employer or client (Netwise). jake edge - L & E Technologies, Inc. 2888 Bluff St. Suite 142 Boulder, CO 80302 Ma Bell: (303) 440-1142 UUCP: ...!onecom!wldrdg!jake
mishkin@apollo.HP.COM (Nathaniel Mishkin) (07/11/90)
In <1990Jul10.224543.20314@athena.mit.edu> pae@athena.mit.edu (Philip Earnhardt) writes: >Actually, things should work pretty smoothly under TCP. Since the connection >is being maintained by the kernel, the acknowledgment can be sent immediately >after the packet is processed by the kernel. NCS would have the ack sent out 3 >seconds after the user process has processed the data. Also, acknowledgments >can be sent back to the server for fragments of the message. I suspect things WON'T work smoothly under TCP, for the reasons I said. First, your scenario (100 different clients each make a single non-idem call to a server) implies that the server is able to open (and keep open without being considered anti-social) 100 TCP connections. (BTW, whether your system lets you have more than 32 file descriptors is only the first issue. How many simultaneous TCP connections your kernel allows and how much these "cost" in kernel resources are others.) Second, with TCP, any data sent on a connection that is not yet ack'd is consuming wired kernel buffers. There is nothing that REQUIRES a receiving TCP to acknowledge a TCP segment immediately upon its receipt. TCP implementations should (lord knows if most do) have the goal of piggy-backing ack's on data flow in the reverse direction to avoid the sending of gratuitous empty TCP segments (IP packets) that simply contain ack info. In any case, TCP having the "connection maintained by the kernel" really has pretty little to do with anything. NCS could ack every response packet immediately after it arrived (as opposed to postponing the way it actually does). The delay induced by switching out from the kernel to user mode upon packet receipt isn't the problem. The problem is that the sender is stuck with whatever ack policy the receiver is following. Sure, you can solve the sender's problem by ack'ing all the time. However, if you know the sender has a moderate to large amount of buffering capabilities (because the buffers are page-able and in user space), you can ack at a more reasonable rate and save network bandwidth. >It's probably not necessary to understand all the nuances of the NCS >implementation in order to understand the fundamental problem with a >connection-oriented protocol implemented outside the kernel on top of a >datagram transport. There is certainly some reason to be concerned that it might be the case that the kernels we have around don't provide enough information in a timely fashion to user mode to make the implementing of a connection oriented protocol (like NCS over UDP is) as good as one would like. Our (and other's) experience has shown that one can live with the limitations. (And as you can see from my earlier comments, the alternate approach of using TCP is not without limitations either.) >If the connection-oriented scheme at the user level were the way to go, why >isn't everyone else jumping on the bandwagon? Beats me. OSF seems interested. >Why not implement X that way? Ask the X folks. Presumably they decided that they were more interested in doing window system stuff than network protocols. >Are there any RFCs that discuss the NCS technology? At the time we set out to write the NCA book, we considered whether it could/should be an RFC. For better or worse, the book was written with a WYSIWIG editor and (at least until recently) RFC's had to be in flat ASCII form. Apparently Postscript RFCs are now accepted. We could produce the book in Postscript and submit it as an RFC. Somehow I think it's probably simply for people who are interested to just get the book. (The royalties to HP are nothing to write home about, I assure you :-) >Have any of the Internet gurus endorsed it? As I understand it, the topic of RPC is one that surfaces regularly at some IETF working group meetings. I don't think they've endorsed anything at this time. > >> Everything is a matter of degree. Broadcast is a feature that a relatively >> few (and hopefully almost no) applications need. > ^^^^^^^^^ >??? In order for your GLB to work, you need broadcast. OK, substitute "close to one" for "almost no". The idea is that most applications will leverage off of a single broadcast (to the GLB) so that those applications themselves don't have to use broadcast for anything else. Also, the GLB can be be accessed without broadcast (i.e., via specifying its location in a local config file). >> See "Network Computing Architecture" (Prentice-Hall, ISBN 0-13-611674-4). >> Let me know if it doesn't answer all your questions. We tried to be >> complete, but I wouldn't be surprised if we failed in some areas. There's >> always the next edition :-) > >I had already looked through this book. I hadn't found answers to the >specific questions I had asked: > > o What type of packets NCS is sending (UDP or IP). UDP. I had intended there to be an appendix to the book titled "NCA/RPC in the IP environment". It fell through the cracks. The mapping is straigtforward. One NCA/RPC datagram is one UDP packet. > o When acknowledgments are sent. (there's some information about the timers, > but there's no timeline charts about what goes down the wire.) Sounds like a good area for improvement. Presumably you understand the picture based on my postings. > o How retransmission timers work--when are they used, what values are > they set to, how the application can change the values. Actually, I thought this was covered, but perhaps not clearly enough. >For these systems, there will have to be modifications to the OS to support >multiple threads (I assume non-preemptive application-provided multithreading >won't work). There's the added expense of a new OS and the hassles of >configuring/installing the system software. Well it's a surprise to me that the OS needs to be modified. A number of people have written useful multi-threading packages that don't require OS mods. Presumably some OS's would require mods. Presumably some OS's don't have C compilers either. My goal is not for me to fix every problem in the world. >Some of these computer systems may be at the end of their product cycle; the >manufacturer may be unwilling to make any changes to their OS. There may be >architectural reasons that mutlithreading can not be implemented in some >environments. Should Distributed Computing not be available in these >environments just because NCS implementations require multiple threads? Clearly someone should figure out how to build implementations that don't require multiple threads. Like I said, my goals is not for me to fix every problem in the world. The portability of a single implementation of a system is only one measure of a system's usefulness. NCS is highly portable. No, it is not infinitely portable. Sue me. (Be prepared to sue lots of people who produce lots of software that lots of people use and consider to be portable.) >The response didn't seem to address the issue raised. How exactly does one >specify the granularity of the broadcast? Sorry. I didn't understand this to be your question. There is no control over the granularity (by which I assume you mean "scope"). This is one of the reasons people dislike broadcast. (Some people, including me, like MULTIcast, but unfortunately, access to such facilities is pretty hit-or-miss these days.) >The comparison between (preemptive) multiple threads and asynchronous calls is >interesting. I find asynchronous calls fairly straightforward--you place the >call, do something else for a while, check for a result, and get the reply. Another religious issue for sure. My problem is that I don't know what "a while" and "check for a result" mean: when and how do I do it? Multi-threading systems will be implemented and become widespread. Such systems have primitives for doing synchronization (signalling and waiting for events). Why do you want to invent and make me use duplicate primitves (in support of async RPC)? -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
mishkin@apollo.HP.COM (Nathaniel Mishkin) (07/11/90)
In <1990Jul10.225103.20551@athena.mit.edu> pae@athena.mit.edu (Philip Earnhardt) writes: >> I understand your position and I don't know that there's much point >> arguing about the relative merits of customization. We simply disagree. > >This seems to be a bit of a 'back-off' from your previous postings ... are >we calling it a draw (i.e. yes there are dangers in customization and yes >there are instances where they are neccesary)? No. I am simply not interested in trying to convince you any more that customization is a bad idea. I believe (a) that you strongly believe that it's a good idea, and (b) that I'm not going to change your mind. >> Authentication and privacy features are part of NCS 2.0. > >Maybe I am misunderstanding here, but it sounds like you are saying that NCS 2.0 >will provide methods for authentication and for privacy that are 'hardcoded' >into the system. If (for some reason) a customer has their own method that they >wish to use (i.e. their own encryption scheme) they will not be able to use it >and that instead they must use the method provided by NCS 2.0 ... am I jumping >to conclusions? The authentication facilities of NCS 2.0 are modularized in a way that permits their replacement. I'm sure someone will jump up and down and say "So you do have customization!" so I guess I might as well address it now. While there seems to be a strong movement to MIT Project Athena's Kerberos system (which is what NCS 2.0 works with), we (the NCS developers) are not empowered by anyone (not even ourselves :-) to declare that this is the one that everyone will eventually use. In case Kerberos doesn't become a standard (based on the OSF DCE RFT decision, it *is* now an OSF standard) or in case someone really wants to use something else, we arranged that the Kerberos piece can be removed and another one plugged in. We DO NOT consider this something that would be done by application developers. We consider our design as a safety net in case things don't go in the way we have high confidence they will go. It's not very useful if everyone in a single environment is not using the same authentication mechanism (protocol, key distribution center, etc.). For these reasons, we think of the design of NCS's authentication component as a good engineering decision, not as the creation of a general customization facility. >BTW, in some hallway discussions that we have had around here, the concept of >uniform transport behavior (sp?) has come up and it leaves me a bit puzzled. >From my (perhaps somewhat limited) understanding of the current thinking in the >networking community, there appear to be two camps, connection-oriented and >connection-less (thus the CO/CL split in the OSI) and what uniform transport >behavior in effect does is to come down clearly in the CO camp. (i.e. whether >the transport is CO or CL is unimportant (except in the case of broadcast) as >NCS establishes a CO protocol in either case). Is this the intent? The Netwise >approach is to have uniform CO transport behavior and uniform CL transport >behavior and to therefore recognize this fundamental split ... I am curious >as to your thinking on this. I don't know about your "two camp" notion. If two camps are present anywhere, it's at the network layer: some people think it's important to have CL network protocols (like IP) and some people don't think it's so important (and are happy to have [only] a CO network protocol). No one disagrees about the need for CO transport protocols. No one thinks that the behavior of a particular transport protocol (e.g., TCP) should vary as a function of what (allowed) network protocol it runs on top of. I.e., if someone were to make a TCP that ran over OSI CLNP (a not unthinkable scenario), I think people would be pretty annoyed if you (a) didn't make it reliable (in the way people generally think of TCP/IP as being reliable), and (b) you advertised your software as being TCP. Similarly, our feeling is that people would be annoyed at someone's advertising something as "being RPC" yet making its behavior a function of some lower level network protocol choices. -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
pae@athena.mit.edu (Philip Earnhardt) (07/20/90)
In <4b8742f6.20b6d@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: >>This seems to be a bit of a 'back-off' from your previous postings ... are >>we calling it a draw (i.e. yes there are dangers in customization and yes >>there are instances where they are neccesary)? > >No. I am simply not interested in trying to convince you any more that >customization is a bad idea. I believe (a) that you strongly believe that >it's a good idea, and (b) that I'm not going to change your mind. I agree, however there may be others following this dialogue who still hang in the balance. In my previous posting (and I do realize that it is more than a bit confusing as to who is posting -- we have a bit of a problem with our news feed) there were a number of issues that the Netwise RPC Tool solves through customization (for example asynchronous RPC) that you have done a fair amount of hand waving about. As you have stated, threads will provide a way to do asynchronous RPC (for environments that support threads), but you also seem to believe that asynchronous RPC is blasphemous (i.e. your religion says that it is a bad thing). You really can't have it both ways. And though threads will provide a method to accomplish async, it is clearly a much less straightforward mechanism for an application programmer to use. This is one of the reasons that I believe customization is by and large a good thing. The Netwise tool does not inflict its users with the biases of the tool designers. >>Maybe I am misunderstanding here, but it sounds like you are saying that NCS 2.0 >>will provide methods for authentication and for privacy that are 'hardcoded' >>into the system. If (for some reason) a customer has their own method that they >>wish to use (i.e. their own encryption scheme) they will not be able to use it >>and that instead they must use the method provided by NCS 2.0 ... am I jumping >>to conclusions? > >The authentication facilities of NCS 2.0 are modularized in a way that >permits their replacement. I'm sure someone will jump up and down and >say "So you do have customization!" so I guess I might as well address >it now. While there seems to be a strong movement to MIT Project >Athena's Kerberos system (which is what NCS 2.0 works with), we (the >NCS developers) are not empowered by anyone (not even ourselves :-) to >declare that this is the one that everyone will eventually use. In case >Kerberos doesn't become a standard (based on the OSF DCE RFT decision, >it *is* now an OSF standard) or in case someone really wants to use >something else, we arranged that the Kerberos piece can be removed and >another one plugged in. We DO NOT consider this something that would ^^^^^^ >be done by application developers. We consider our design as a safety ^^^^^^^^^^^^^^^^^^^^^^^^^ >net in case things don't go in the way we have high confidence they will >go. It's not very useful if everyone in a single environment is >not using the same authentication mechanism (protocol, key distribution >center, etc.). For these reasons, we think of the design of NCS's >authentication component as a good engineering decision, not as the ^^^^^^^^^^^^^^^^^^^^^^^^^ >creation of a general customization facility. So, to paraphrase, the authentication and privacy features of NCS 2.0 are hardcoded. They could be changed by you folks if Kerberos doesn't make it as a standard, but application developers (i.e. customers who buy NCS) are stuck with it. I would not try to claim that there is something inherently wrong with Kerberos, but if some developer has a legitimate problem with it (perhaps because it uses the DES), and they are working under NCS 2.0, they are stuck ... true? I suppose if an NCS user really had to change it for their own reasons, they could buy the NCS source and replace the Kerberos module... In addition, I think it would be more fair to say that modularizing the Kerberos portion of NCS is a standard engineering decision. I think that most folks in this day and age would be flabbergasted that a conceptually seperate portion of the system was not put into a seperate module. >>BTW, in some hallway discussions that we have had around here, the concept of >>uniform transport behavior (sp?) has come up and it leaves me a bit puzzled. >>From my (perhaps somewhat limited) understanding of the current thinking in the >>networking community, there appear to be two camps, connection-oriented and >>connection-less (thus the CO/CL split in the OSI) and what uniform transport >>behavior in effect does is to come down clearly in the CO camp. (i.e. whether >>the transport is CO or CL is unimportant (except in the case of broadcast) as >>NCS establishes a CO protocol in either case). Is this the intent? The Netwise >>approach is to have uniform CO transport behavior and uniform CL transport >>behavior and to therefore recognize this fundamental split ... I am curious >>as to your thinking on this. > >I don't know about your "two camp" notion. If two camps are present >anywhere, it's at the network layer: some people think it's important >to have CL network protocols (like IP) and some people don't think it's >so important (and are happy to have [only] a CO network protocol). No >one disagrees about the need for CO transport protocols. Am I missing something here? Isn't UDP a CL transport protocol that is used by a fair number of current applications (including NFS which is likely the most widely used RPC application in the world today)? Aren't there provisions for a CLTP in the OSI reference model? I think that there are those who would disagree about _only_ having a COTP (which may be stretching your statement a bit). >No one thinks that the behavior of a particular transport protocol (e.g., >TCP) should vary as a function of what (allowed) network protocol it >runs on top of. I.e., if someone were to make a TCP that ran over OSI >CLNP (a not unthinkable scenario), I think people would be pretty annoyed >if you (a) didn't make it reliable (in the way people generally think >of TCP/IP as being reliable), and (b) you advertised your software as >being TCP. Similarly, our feeling is that people would be annoyed at >someone's advertising something as "being RPC" yet making its behavior >a function of some lower level network protocol choices. But again, you folks are constraining the choices of your customers. There are distinct sets of advantages and disadvantages to CO vs. CL transports and the NCS product allows only the set for CO transports. BTW, if the home-brew CO transport over UDP is so much better than TCP, why didn't you folks stay with that for NCS 2.0 rather than adding TCP support? With the Netwise approach, you get (in effect) _both_ CL and CO RPC and allow user of the tool to decide for themselves which of these two options is 'best' for their application. If a user wishes the advantages of a CL RPC (largely reduced overhead) and is willing to pay the price that implies (lost messages, etc.), they should have the opportunity to make that decision. Clearly there exist applications that will benefit from this. My overall impression of NCS is that it is largely a 'do it our way' system. The NCS designers made decisions about what RPC should be and how it should operate and that these are the only allowable ways to do RPC within NCS. In my experience in dealing with customers, this simply will not work. These folks have their own ideas about how RPC should work and they intend to use it that way. Trying to convince them that certain things are not religiously correct is a futile exercise. A much better alternative is to give users the flexibility to decide for themselves and (as much as possible) to warn them about potential problems with their approach. In the end, C programmers (as opposed to Pascal programmers) are used to doing things their way even though there is some (perhaps great) danger associated with it and IMHO this is as it should be. ------------------------------------------------------------------------------ The verbiage above does not neccesarily reflect the views of my employer or client (Netwise). jake edge - L & E Technologies, Inc. 2888 Bluff St. Suite 142 Boulder, CO 80302 Ma Bell: (303) 440-1142 UUCP: ...!onecom!wldrdg!jake
pae@athena.mit.edu (Philip Earnhardt) (07/20/90)
In 4b8742f6.20b6d@apollo.HP.COM mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: > The authentication facilities of NCS 2.0 are modularized in a way that > permits their replacement. I'm sure someone will jump up and down and > say "So you do have customization!" so I guess I might as well address > it now. [Explanation ommitted.] Since you provided no details about how this modularization works, it's difficult to comment. Presumably you provide a mechanism for the application to specify the security routine(s). Is your scheme general enough to accomodate any conceivable security approach? For instance, does the escape mechanism allow for the size of the data to change during the encryption process? If you allow security to happen via Netwise-style customization, this would not be an issue. As an aside, computer security is starting to branch out from DES-based systems (which Kerberos is currently using). Lotus is using a public key-based system for its new groupware product, and I heard Novell has signed up with RSA Associates for using public-key system. Developers using RPC may not be happy if they're stuck with DES. > It's not very useful if everyone in a single environment is > not using the same authentication mechanism (protocol, key distribution > center, etc.). I don't see why this is true. You need to have agreement on encryption for a SINGLE distributed application--it is not necessary to impose the same mechanism for ALL applications. Maybe someone wants to encrypt only the sensitive parts of their data and doesn't want to incur the overhead for all the data. Programmers may want to not use encryption during initial development of their distributed application and turn on security later. > I don't know about your "two camp" notion. If two camps are present > anywhere, it's at the network layer: some people think it's important > to have CL network protocols (like IP) and some people don't think it's > so important (and are happy to have [only] a CO network protocol). No > one disagrees about the need for CO transport protocols. Actually, the "two camp" notion comes from ISO. In case you are unfamiliar with ISO standards, ISO 7498 describes the Basic Reference Model for networking. Addendum 1 to this document describes connectionless mode transmission of data. The CO/CL distinction at the Network layer is the one that has gotten the most attention, but different CO/CL services are also available at the Transport, Session, Presentation, and Applications layers. The basic structure of Connection-Oriented ISO protocols is that there are 3 phases to a connection: the connection handshake, the data transfers, and the disconnection handshake. The basic structure of the Connectionless ISO protocols is a single message--all the layers' data are put in a single message on the wire. In an earlier posting, Mishkin had said: > I'm sure [OSF] will be acquiring OSI implementations though. They may be acquiring OSI implementations, but it's unclear what they'll be using them for. Adding OSI protocols to NCS would involve fundamental changes to its underlying architecture. NCS fails to fit the OSI connection-oriented model because it doesn't perform connect and disconnect handshakes. NCS also fails to fit the connectionless model--there are ACKs and REPs during the data transfer. If NCS developers have a committment to international standards, they should be using the OSI protocols for Session Management, Data Representation, and Application Control. None of these are present in the NCS2.0 product. > No one thinks that the behavior of a particular transport protocol (e.g., > TCP) should vary as a function of what (allowed) network protocol it > runs on top of. I.e., if someone were to make a TCP that ran over OSI > CLNP (a not unthinkable scenario), I think people would be pretty annoyed > if you (a) didn't make it reliable (in the way people generally think > of TCP/IP as being reliable), and (b) you advertised your software as > being TCP. Similarly, our feeling is that people would be annoyed at > someone's advertising something as "being RPC" yet making its behavior > a function of some lower level network protocol choices. In the words of the New Yorker: Block that Metaphor! The analogy does not work well. In case you hadn't noticed, Sun's RPCGEN has provided datagram-based RPCs for quite a few years. Could you produce some (one) of these annoyed programmers? Perhaps you've heard of NFS? Programmers need to understand about idempotent semantics to know when they can use a true datagram RPC; they also have to understand idempotent semantics when they're writing their NCS NIDL specification. Our RPC Tool Products on top of CL protocols obey the OSI Reference Model (Connectionless); our RPC Tool Products on top of CO protocols obey the OSI Reference Model (Connection Oriented). If you want NCS to be OSI compliant, you can either change your products or change the OSI Reference Model. I can't gauge the relative difficulty of these two tasks. Phil Earnhardt Netwise, Inc. 2477 55th St. Boulder, CO 80301 Phone:303-442-8280 UUCP: onecom!wldrdg!pae My opinions do not reflect any official position of Netwise.
pae@athena.mit.edu (Philip Earnhardt) (07/20/90)
In <4b872c9b.20b6d@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: >> Are there any RFCs that discuss the NCS technology? > At the time we set out to write the NCA book, we considered whether it > could/should be an RFC. For better or worse, the book was written with > a WYSIWIG editor and (at least until recently) RFC's had to be in flat > ASCII form. Apparently Postscript RFCs are now accepted. We could produce > the book in Postscript and submit it as an RFC. Somehow I think it's > probably simply for people who are interested to just get the book. > (The royalties to HP are nothing to write home about, I assure you :-) (I'm amused by the reason :-). It may be helpful, though, to consider a real, live RFC--comments you get from a Request for Comments may give you some hints on how to make NCS work efficiently on the Internet. One thing I found ambiguous in the Architecture book was the use of Fragment Acknowledgment. Page 30 of the Architecure book says that the fragment acknowledgment may be set in a packet to request acknowledgments of the individual fragments. However, I found no mention of how a programmer would require or prohibit fragment acknowledgment. Under what conditions would someone want fragment acknowledgment? Why is this something an applications developer should worry about? On page 43, the "Suggested Time" for fragment acknowledgment is 1 second (it's my understanding that all of these timers are hardwired in a given NCS implementation). Isn't a 1-second timeout far too short for fragment acknowledgment? If the mean round-trip time for 2 Internet hosts is 5 seconds, won't this mean that fragments will get needless multiple retransmissions? >> Have any of the Internet gurus endorsed it? > As I understand it, the topic of RPC is one that surfaces regularly at > some IETF working group meetings. I don't think they've endorsed anything > at this time. I'm talking about NCS's connection-oriented protocol on top of UDP. If you truly think that you've invented a better mousetrap, these are the folks that I'll believe. Speaking of TCP/IP, from an earlier posting: |> A couple of general questions: |> |> 1. Why is NCS2.0 being offered on top of connection-oriented transports? |> Under what conditions will you recommend that programmers specify using |> connection-oriented transports? > Some environments have only COTPs. (E.g., our ISO/OSI friends have not > yet seen the wisdom of making a priority defining and implementing CLTPs.) > We want NCS to run in these environments. Hence we must make NCS run > over COTPs. It is true that some environments only have COTPs. However, I have never seen a TCP/IP environment that did not have access to UDP/IP and vice versa. (Does anyone know of any exceptions?) Why bother putting NCS2.0 on top of TCP/IP? If you do provide NCS on top of TCP/IP, when do you recommend that it be used? >> Some of these computer systems may be at the end of their product cycle; the >> manufacturer may be unwilling to make any changes to their OS. There may be >> architectural reasons that mutlithreading can not be implemented in some >> environments. Should Distributed Computing not be available in these >> environments just because NCS implementations require multiple threads? > Clearly someone should figure out how to build implementations that don't > require multiple threads. Either multi-threading is a keystone of NCS, or it isn't. If multithreading isn't available in all environments, then your proposed mechanism of doing asynchronrous RPC via forking a thread won't work. > Like I said, my goals is not for me to fix every problem in the world. The > portability of a single implementation of a system is only one measure of a > system's usefulness. NCS is highly portable. No, it is not infinitely ^^^^^^^^^^^^^^^^^^^^^^^ > portable. Sue me. (Be prepared to sue lots of people who produce lots of > software that lots of people use and consider to be portable.) How portable is it? How does the availability of NCS compare with RPCGEN? How about Netwise's RPC Tool? How big is the NCS compared with these other products? > Sorry. I didn't understand this to be your question. There is no control ^^^^^^^^^^^^^^^^^^^ > over the granularity (by which I assume you mean "scope"). This is one ^^^^^^^^^^^^^^^^^^^^ > of the reasons people dislike broadcast. (Some people, including me, > like MULTIcast, but unfortunately, access to such facilities is pretty > hit-or-miss these days.) I believe you mean that NCS provides no control of the granularity. One can control the range of a broadcast by changing the address mask. >> The comparison between (preemptive) multiple threads and asynchronous calls is >> interesting. I find asynchronous calls fairly straightforward--you place the >> call, do something else for a while, check for a result, and get the reply. > Another religious issue for sure. My problem is that I don't know what > "a while" and "check for a result" mean: when and how do I do it? If you're doing asynchronous calls, you have some point when you have enough information to perform the call, but don't need the result immediately. A simple strategy is to go back to what you're doing until you need the results, then block waiting for them. If you don't understand that, then I don't understand what you meant in an earlier posting: > My religion says that they're a bad idea because they're hard to understand, > represent an unnecessary new aspect in the computing model, and that one > should use existing primitives for asynchrony (process) and communications > (procedure call) together to achieve the same effect. Do you mean that a process should fork a new thread to do the RPC call? In your example, isn't the parent thread going to do stuff for "a while" then "check for a result" of its child? Sounds at least as hard to me. For other readers: it's useful to remember one other thing about async operations. Async is pretty simple to understand if there is only one outstanding operation. It is possible to have multiple async calls over a connection, but the rules get much more complicated. I would not recommend multiple outstanding asynchronous calls over the same connection. > Multi-threading systems will be implemented and become widespread. Such > systems have primitives for doing synchronization (signalling and waiting > for events). Why do you want to invent and make me use duplicate primitves > (in support of async RPC)? Yeah, it looks like multi-threaded systems will become more widespread. It will be interesting to see what sort of problems that this unleashes on the world. I'm not tremendously enthusiastic about locking out all other threads every time I want to modify a global. The one that really frightens me, though, is the process of software maintenance. Some junior programmer will not understand THE RULES for changing a multi-threaded application and will introduce bugs that will not surface for months later.... I had claimed that there was no automatic system for detection of data access collisions; this was incorrect. It turns out that the Space Shuttle computers are multiprocessing but not memory-mapped--different processes are accessing the same global data. Unfortunately, the computers provided facilities for protecting only 256 variables; a typical shuttle mission has over 9,000 shared variables. The computer designers came up with a novel solution: since are points of synchronization between the processes, the relative time that each process is accessing each variable is known. From this data, the designers were able to create an "alabi file" for each variable that exhaustively examined each access and *proved* that no reads and writes overlapped. Needless to say, computation of these proofs take a rather astonishing number of computrons and had a rather negative impact on code flexibility in the shuttle. Phil Earnhardt Netwise, Inc. 2477 55th St. Boulder, CO 80301 Phone:303-442-8280 UUCP: onecom!wldrdg!pae My opinions do not reflect any official position of Netwise.
mishkin@apollo.HP.COM (Nathaniel Mishkin) (07/24/90)
In <1990Jul20.032431.7497@athena.mit.edu> Jake Edge (onecom!wldrdg!jake), appearing this night only as pae@athena.mit.edu (Philip Earnhardt) writes: >In <4b8742f6.20b6d@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) >writes: >As you have stated, threads will provide a way to do asynchronous RPC >(for environments that support threads), but you also seem to believe >that asynchronous RPC is blasphemous (i.e. your religion says that it >is a bad thing). You really can't have it both ways. And though threads >will provide a method to accomplish async, it is clearly a much less >straightforward mechanism for an application programmer to use. This >is one of the reasons that I believe customization is by and large a > good thing. I'm not sure what you think the contradiction is. Asynchrony is a fact of the world. I'm neither for nor against it. I merely want to take advantage of it with a single, extant abstraction: the process model ("threads"). You appear to think abstraction is not straightforward and invent another one: asynchronous procedure call. You're entitled. I'll stick with the existing orthogonal abstractions of process and procedure call. >The Netwise tool does not inflict its users with the biases of the tool >designers. I'm happy to let the market judge my biases. >So, to paraphrase, the authentication and privacy features of NCS 2.0 >are hardcoded. They could be changed by you folks if Kerberos doesn't >make it as a standard, but application developers (i.e. customers who >buy NCS) are stuck with it. I would not try to claim that there is >something inherently wrong with Kerberos, but if some developer has >a legitimate problem with it (perhaps because it uses the DES), and >they are working under NCS 2.0, they are stuck ... true? I suppose if >an NCS user really had to change it for their own reasons, they could >buy the NCS source and replace the Kerberos module... At worst. If the demand appeared to exist, I suppose we could document the interface between the authentication module and the rest of NCS. This would allow people to write authentication modules without having the source to NCS. >In addition, I think it would be more fair to say that modularizing >the Kerberos portion of NCS is a standard engineering decision. Ah, we agree on at least one thing! >>I don't know about your "two camp" notion. If two camps are present >>anywhere, it's at the network layer: some people think it's important >>to have CL network protocols (like IP) and some people don't think it's >>so important (and are happy to have [only] a CO network protocol). No >>one disagrees about the need for CO transport protocols. > >Am I missing something here? Isn't UDP a CL transport protocol that is used >by a fair number of current applications (including NFS which is likely the >most widely used RPC application in the world today)? I guess I'm not sure what point you're trying to make. You initially made some comments about there being two "camps" and that our choice of uniform RPC behavior over various transports put us in the CO "camp". The only "camp" we're in is the one that thinks that the behavior of a remote call should be constant. >Aren't there provisions for a CLTP in the OSI reference model? Provisions: yes. Widespread implementations: unclear. >I think that there are those who would disagree about _only_ having >a COTP (which may be stretching your statement a bit). I said: "No one disagrees about the need for CO transport protocols." Maybe that was an obscure phrasing. I think everyone thinks we should have COTPs. I didn't say "everyone thinks we should have ONLY COTPs". >But again, you folks are constraining the choices of your customers. As does the C compiler writer who chooses to make his compiler save registers on ALL procedure calls, not just on those generated by a compiler targeted for a particular hardware architecture. >There are distinct sets of advantages and disadvantages to CO vs. CL >transports and the NCS product allows only the set for CO transports. What are you talking about? Stop talking about transports. Talk about RPC. Sun RPC has the "advantage" that it allows you to make procedure calls whose semantics are ill-defined (or defined to be something like "at LEAST once") and, as a result, optimizations can be made to reduce the cost of doing the RPC. It turns out that this "advantage" is selected by telling the programmer which transport to choose. In NCS, we achieve the analogous optimization by telling people to declare a procedure to be "idempotent". Idempotence is a property of a procedure, not a network transport protocol. Since we're dealing in remote PROCEDURE calls, this seems like the more appropriate abstraction. >BTW, if the home-brew CO transport over UDP is so much better than TCP, >why didn't you folks stay with that for NCS 2.0 rather than adding TCP >support? One reason NCS 2.0 runs over TCP because TCP is about the only practical, widespread COTP around and we want to demonstrate that NCS can run over COTPs. Also, it is not inconceivable that TCP will behave better than NCS's own protocol in certain environments. People in such environments may choose to use NCS/TCP. We want to give our customers a choice too, you know :-) >With the Netwise approach, you get (in effect) _both_ CL and CO RPC >and allow user of the tool to decide for themselves which of these two >options is 'best' for their application. If a user wishes the advantages >of a CL RPC (largely reduced overhead) and is willing to pay the price >that implies (lost messages, etc.), they should have the opportunity >to make that decision. Clearly there exist applications that will benefit >from this. We've been over this again and again. People don't want "CL and CO RPC". They want RPC. They want to make certain performance tradeoffs made available to them. NCS addresses this desire in an appropriate way. --------------------- In <1990Jul20.033536.7794@athena.mit.edu> pae@athena.mit.edu (Philip Earnhardt) writes: >Since you provided no details about how this modularization works, it's >difficult to comment. Presumably you provide a mechanism for the >application to specify the security routine(s). We believe the authentication modularization has sufficient generality. >Actually, the "two camp" notion comes from ISO. In case you are unfamiliar >with ISO standards, ISO 7498 describes the Basic Reference Model for >networking. ... Thanks for Networking 101. I'm fully aware that the OSI model defines CO and CL transport and network protocols. We're talking about RPC, right? >In an earlier posting, Mishkin had said: >> I'm sure [OSF] will be acquiring OSI implementations though. > >They may be acquiring OSI implementations, but it's unclear what they'll be >using them for. Adding OSI protocols to NCS would involve fundamental changes >to its underlying architecture. NCS fails to fit the OSI connection-oriented >model because it doesn't perform connect and disconnect handshakes. NCS also >fails to fit the connectionless model--there are ACKs and REPs during the data >transfer. > >If NCS developers have a committment to international standards, they should >be using the OSI protocols for Session Management, Data Representation, and >Application Control. None of these are present in the NCS2.0 product. Get ALL your facts straight. First of all, there's currently no ISO standard for RPC. Second, the protocol NCS 2.0 defines for running over COTPs adheres as closely as possible to how we imagine such an ISO standard will appear and fit into the OSI model. This includes presentation negotiation and the rest of the ball of mud. Believe me, I've had T-CONNECT through A-CONNECT up to my ears. It is of course true that NCS does not support ASN.1/BER. (For those blissfully unaware of the ISO process, those are ISO standards for data representations.) In the opinion of the NCS architects and developers, in the absence of any ISO standard (or even draft) covering RPC, it didn't seem to make a whole lot of sense to be "standard" in the data rep front when the rest of the protocol stack would be non-standard anyway. You don't make ISO people happy (REALLY happy, that is) by saying "Hey look, I used ONE of your protocols here". They want the kind of interoperability that comes from having the whole protocol stack -- top to bottom -- defined and implemented everywhere. The Netwise RPC that includes ASN.1/BER support is still (gasp!) proprietary in the eyes of the ISO world, in spite of that support. And since I suspect most of you out there (still reading anyone?) are actually interested in reality (in addition to international standards), bear in mind that it seems to be fairly universally recognized that the performance of ASN.1/BER is pretty horrible in both time and space. >In the words of the New Yorker: Block that Metaphor! The analogy does >not work well. In case you hadn't noticed, Sun's RPCGEN has provided >datagram-based RPCs for quite a few years. Could you produce some (one) >of these annoyed programmers? Could you produce some (one) programmer :-) Given the widespread availability of Sun RPC on a variety of vendors' platforms, why aren't there tons of Sun RPC based applications? >Perhaps you've heard of NFS? The classic example of how a single, non-representative distributed application has skewed the minds of thousands about what the appropriate characteristics of a distributed application toolkit should be. Yes, millions USE NFS every day. But NFS is just ONE APPLICATION. Thus, despite its obvious utility, it is still a sample of one. We have no reason to believe that the needs of the majority of the distributed applications to come have much in common with the needs of NFS. >Programmers need to understand about idempotent semantics to know when >they can use a true datagram RPC; they also have to understand idempotent >semantics when they're writing their NCS NIDL specification. You've got it exactly right. With NCS, programmers need to understand things about RPC semantics. With Sun RPC, they need to understand things about network protocols. I know which approach I'd pick :-) ----------------- In <1990Jul20.034330.8152@athena.mit.edu> Phil Earnhardt (onecom!wldrdg!pae) writes: >One thing I found ambiguous in the Architecture book was the use of Fragment >Acknowledgment. Page 30 of the Architecure book says that the fragment >acknowledgment may be set in a packet to request acknowledgments of the >individual fragments. However, I found no mention of how a programmer would >require or prohibit fragment acknowledgment. Let's get this straight. The NCA book is a PROTOCOL SPEC, not something application programmers use. Users of TCP don't read RFC 793. The setting of the frag ack bit has the same relevant to application programmers as the TCP ACK flag. >On page 43, the "Suggested Time" for fragment acknowledgment is 1 second (it's >my understanding that all of these timers are hardwired in a given NCS >implementation). Isn't a 1-second timeout far too short for fragment >acknowledgment? If the mean round-trip time for 2 Internet hosts is 5 seconds, >won't this mean that fragments will get needless multiple retransmissions? Ugh. Suffice it to say that the spec should be taken with a grain of salt when it comes to parameters such as the one you're pointing to. We focused most of our energy on specifying the non-parametric aspects of the protocol. A correct implementation (e.g., NCS 2.0) would do round trip time estimations. >I'm talking about NCS's connection-oriented protocol on top of UDP. If you >truly think that you've invented a better mousetrap, these are the folks that >I'll believe. You've found me out. I'm actively hiding all my work from exactly the people qualified to judge it. Give me a break. If everyone listened to the right people at the right moment, Sun would never have gotten away with sending 8K UDP datagrams (generally considered to be highly poor form) in Sun RPC. I'll put the NCA book on the Christmas mailing lists for everyone in the IETF. I have but one brain and 24 or so hours a day. >It is true that some environments only have COTPs. However, I have never >seen a TCP/IP environment that did not have access to UDP/IP and vice >versa. (Does anyone know of any exceptions?) Why bother putting NCS2.0 >on top of TCP/IP? If you do provide NCS on top of TCP/IP, when do you >recommend that it be used? See above. >Either multi-threading is a keystone of NCS, or it isn't. If >multithreading isn't available in all environments, then your proposed >mechanism of doing asynchronrous RPC via forking a thread won't work. Yes. You won't be able to make concurrent calls in environments that don't support threading. I also can't port my Unix programs that call "fork" to MS/DOS. >How portable is it? How does the availability of NCS compare with RPCGEN? >How about Netwise's RPC Tool? How big is the NCS compared with these >other products? I'll concede the "we give away source code for mediocre software" award to Sun for RPCGEN. I'll concede the "we managed to ship binaries of our product on a fantastically large number of systems" award to Netwise for RPC Tool. I'll reserve the right to claim the "superior distributed application development toolkit" award for NCS. (I can't comment on the relative sizes of the systems since I don't know.) If installed base were everything, we'd all still be typing things like "//sysin DD *" on punch cards. >I believe you mean that NCS provides no control of the granularity. One can >control the range of a broadcast by changing the address mask. I guess I'm not sure what you mean, but if it involves doing a kind of broadcast that goes through either MAC or network level bridges, my understanding is that many (most?) people consider that to be anti-social. In any case, I think broadcast is a dangerous tool which should be used carefully (i.e., not often). >If you're doing asynchronous calls, you have some point when you have >enough information to perform the call, but don't need the result >immediately. A simple strategy is to go back to what you're doing until >you need the results, then block waiting for them. How do I get the results? In a synchronous call, the results are in the output parameters defined in the procedure signature. Do you create some other random stub procedure that just has output params or what? What if want to have more than one call executing concurrently? How do I coordinate and collect the results? With the multi-thread / synchronous call approach, the answer is create more threads and just get the result via output parameters. >Do you mean that a process should fork a new thread to do the RPC call? >In your example, isn't the parent thread going to do stuff for "a while" >then "check for a result" of its child? Sounds at least as hard to me. Not necessarily. I might be collecting statistics about some remote system. Each thread makes a call to a different system and when the result appear, the thread inserts the info (using mutex locks, of course) into a common data structure. No one is "checking for results". >For other readers: it's useful to remember one other thing about async >operations. Async is pretty simple to understand if there is only one >outstanding operation. It is possible to have multiple async calls over a >connection, but the rules get much more complicated. I would not recommend >multiple outstanding asynchronous calls over the same connection. Exactly. See my comments above. Are you telling me async RPC is OK unless I want to do more than one remote thing at a time? Seems sort of a limiting model. >Yeah, it looks like multi-threaded systems will become more widespread. It >will be interesting to see what sort of problems that this unleashes on the >world. I'm not tremendously enthusiastic about locking out all other threads >every time I want to modify a global. The one that really frightens me, >though, is the process of software maintenance. Some junior programmer will >not understand THE RULES for changing a multi-threaded application and will >introduce bugs that will not surface for months later.... Asynchrony is hard to understand. All the more reason to have a single abstraction -- the process/thread -- to deal with it. With a little language support (especially for stack-based exception handling and mutex locks) it's pretty tractable. I recommend Andrew Birrell's DEC SRC tech report on experience programming in such an environment (SRC Report 35, "An Introduction to Programming with Threads"). -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
dieter@york.cs.ucla.edu (Dieter Rothmeier) (07/25/90)
Did anybody save the discussion on RPC technologies? I only got the tail end of it, and earlier articles have expired. Or is this group archived somewhere? Thanks, Dieter
cmcmanis@stpeter.Eng.Sun.COM (Chuck McManis) (07/28/90)
This discussion is great fun guys, keep it up. I'd like you to consider
a paradigm for a moment and see if both of you are talking about the same
thing. It goes something like this :
There are two distinct ways in which Remote Procedure Call technologies
are used. The first (and currently most common) is in the development
of network services. The second is in the development of distributed
applications. Distinguishing between what type of program you are
writing can make a whole bunch of issues clear. First, lets look at
network services.
A network service is an application that usually runs on a single
machine. The application exports to the world "functions" (procedures
for you PASCAL types) which can be called by client applications when
they want to use the service. From the user's perspective there is a
well documented protocol and a bunch of "stubs" that they can call to
access the various exported procedures of the service. These services
are generally things like name servers or remote file servers. In this
environment developing applications is fairly straight forward. The
programmer looks at the "man" page for the functions they need to call
(let's say our name service exports the function "Lookup") and writes
in the code to call that function. The application programmer doesn't
care about "semantics" or "data encoding styles" or whatever. They
just want to call the function and get a result. Once the application
is written the code is linked with the "stubs" and the programmers
work is done. From the network service developer's perspective, he
(forgive the sexist language) wants to be able to create a service
make it available but doesn't want to become some wizard sockets
programmer. Here an environment such as ONC/RPC or NCS helps out because
they present a paradigm that looks like procedure calls as opposed to
networking calls and the grunt work is handled by a bunch of library
routines. (Compared to something like 'sendmail' which is very
difficult to maintain and the protocol somewhat difficult to document)
Here the service designer can decide "how close to the metal" he
wants to write his code because only he knows the true requirements
being made of the underlying network. So to summarize network services,
they are easy to use by application programmers because they just link
to the service's library of client stubs. The service developers
want to develop services that are reliable and easy to maintain and
the tools provided help them do that.
A distributed application on the other hand is a whole different
beast. Here we have a process (single execution context) which
has pieces of itself operating on different machines (discontiguous
address spaces). Here the requirements of the underlying distribution
technology become very strict since ideally the system works identically
whether or not any piece of it is distributed. This is where such
overly marketed things like "at most once" semantics (or more commonly
"exactly once") semantics are required. The key differences between
these sorts of applications and network services are that while several
clients may access the functions of a network service, exactly one
client accesses a piece of itself across the network.[ As an aside,
I haven't found any research yet that suggests distributing an
application has any positive effect on that application, much like
the parallel processor challenge. ] It is still unclear what the
ideal environment for developing distributing applications is, much like it
is unclear about how to write parallel applications in a general
way. (I know, lots of great work but no clear conclusions yet)
Bottom line, you decide. I'm always a bit suspicious when the snowmobile
salesman says "But this is clearly the superior snowmobile! Not very
many people own one so it's distinctive and it is the only one with
air conditioning!"
--
--Chuck McManis Sun Microsystems
uucp: {anywhere}!sun!cmcmanis BIX: <none> Internet: cmcmanis@Eng.Sun.COM
These opinions are my own and no one elses, but you knew that didn't you.
"I tell you this parrot is bleeding deceased!"pae@athena.mit.edu (Philip Earnhardt) (08/10/90)
In <4bc8788d.20b6d@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: >>So, to paraphrase, the authentication and privacy features of NCS 2.0 >>are hardcoded. They could be changed by you folks if Kerberos doesn't >>make it as a standard, but application developers (i.e. customers who >>buy NCS) are stuck with it. I would not try to claim that there is >>something inherently wrong with Kerberos, but if some developer has >>a legitimate problem with it (perhaps because it uses the DES), and >>they are working under NCS 2.0, they are stuck ... true? I suppose if >>an NCS user really had to change it for their own reasons, they could >>buy the NCS source and replace the Kerberos module... > >At worst. If the demand appeared to exist, I suppose we could document >the interface between the authentication module and the rest of NCS. >This would allow people to write authentication modules without having >the source to NCS. I'd rather not beat a dead horse, but ... this would then allow customization by application developers ... clearly a limited form of customization (as the 'RPC protocol' would presumably be left intact). Which leaves open the question of what other sections of NCS may need to be 'customized' by the end user. I do not believe that you would claim that you have thought of all possible applications of NCS and therefore you could not have included all things that the users may find neccesary. For this reason, Netwise-style customizations, while not allowing all possible customizations, certainly allows the users a great deal more flexibility in building their applications. >>But again, you folks are constraining the choices of your customers. > >As does the C compiler writer who chooses to make his compiler save >registers on ALL procedure calls, not just on those generated by a >compiler targeted for a particular hardware architecture. This is a very odd defense. To defend constraining your customers by pointing the finger at other folks who have done so in the past is ludicrous. >In NCS, we achieve >the analogous optimization by telling people to declare a procedure to >be "idempotent". Idempotence is a property of a procedure, not a network >transport protocol. Since we're dealing in remote PROCEDURE calls, this >seems like the more appropriate abstraction. And if one of your customers chooses to use NCS/TCP, what will declaring a procedure as idempotent do for them? Specifically what optimizations will be applied? >>With the Netwise approach, you get (in effect) _both_ CL and CO RPC >>and allow user of the tool to decide for themselves which of these two >>options is 'best' for their application. If a user wishes the advantages >>of a CL RPC (largely reduced overhead) and is willing to pay the price >>that implies (lost messages, etc.), they should have the opportunity >>to make that decision. Clearly there exist applications that will benefit >>from this. > >We've been over this again and again. People don't want "CL and CO RPC". >They want RPC. They want to make certain performance tradeoffs made >available to them. NCS addresses this desire in an appropriate way. I am not sure how you can make such a sweeping generalization. Is this the result of some HP marketing survey or do you believe that just by saying it, you can make it true? Nat, in going over the most recent postings from both sides, the signal to noise ratio appears to be decreasing dramatically (Chuck McManis' encouragement notwithstanding) ... shall we try to keep it to a more technical level? (This posting was done with that intent) ------------------------------------------------------------------------------ The verbiage above does not neccesarily reflect the views of my employer or client (Netwise). jake edge - L & E Technologies, Inc. 2888 Bluff St. Suite 142 Boulder, CO 80302 Ma Bell: (303) 440-1142 UUCP: ...!onecom!wldrdg!jake
mishkin@apollo.HP.COM (Nathaniel Mishkin) (08/13/90)
In article <1990Aug9.180750.5568@athena.mit.edu>, pae@athena.mit.edu (Philip Earnhardt) writes: |> In <4bc8788d.20b6d@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) |> writes: |> >At worst. If the demand appeared to exist, I suppose we could document |> >the interface between the authentication module and the rest of NCS. |> >This would allow people to write authentication modules without having |> >the source to NCS. |> |> I'd rather not beat a dead horse, but ... Me neither. |> this would then allow customization by application developers ... |> clearly a limited form of customization (as the 'RPC protocol' would |> presumably be left intact). I think the word "limited" is key. |> Which leaves open the question of what other sections of NCS may need |> to be 'customized' by the end user. I do not believe that you would |> claim that you have thought of all possible applications of NCS and |> therefore you could not have included all things that the users may |> find neccesary. For this reason, Netwise-style customizations, while |> not allowing all possible customizations, certainly allows the users |> a great deal more flexibility in building their applications. I won't deny that customization gives programmers more options. We obviously have philosophically different opinions about whether increasing the programmer's options in this way is ultimately a good thing, i.e., whether it yields a system that makes it easier to write and maintain correct distributed applications or whether it would be better to learn (admittedly, over time) what the programmer's real needs are an address them in a coherent way in future versions of the product. |> >>But again, you folks are constraining the choices of your customers. |> > |> >As does the C compiler writer who chooses to make his compiler save |> >registers on ALL procedure calls, not just on those generated by a |> >compiler targeted for a particular hardware architecture. |> |> This is a very odd defense. To defend constraining your customers |> by pointing the finger at other folks who have done so in the past |> is ludicrous. I guess I don't see why it's an odd defense. I'm not pointing my finger at any one else. I thought I was making a humorous analogy. I guess I mixed up my complaints about Netwise's lack of (what we call) "transport transparency" (i.e., ensuring constant call semantics regardless of what transport you happen to be running RPC over) with my complaints about customization. The analogy was that building a system that sometimes yields one call semantics and sometimes another (depending on what transport the application was running over) was like having a C compiler sometimes preserve registers (that contains useful values) across calls and sometimes not (depending on what hardware architecture the C compiler was targetted for), yielding different program behavior. I'll try to re-draw the analogy to apply to customization. If someone said to a C compiler writer: "Hey look, I know what I'm doing and I don't want you to save registers when I make this call; give me a way of telling the compiler to not save registers", I think people would argue that that kind of customization is a bad idea. |> And if one of your customers chooses to use NCS/TCP, what will declaring |> a procedure as idempotent do for them? Specifically what optimizations |> will be applied? I'm aware of none that can be applied. Is this a problem? The programmer states an invariant fact about his procedure. Seems appropriate. The underlying system may or may not be able to take advantage of this fact. |> >We've been over this again and again. People don't want "CL and CO RPC". |> >They want RPC. They want to make certain performance tradeoffs made |> >available to them. NCS addresses this desire in an appropriate way. |> |> I am not sure how you can make such a sweeping generalization. Is |> this the result of some HP marketing survey or do you believe that |> just by saying it, you can make it true? The day marketing can get me correct answers to questions like this will be a happy one for me, believe me. Unfortunately, my experience has been that it's hard or impossible to answer these questions by survey. Sometimes one just applies one's own engineering sense and more limited interactions with application builders. All I know is that it has always been my impression (reinforced, BTW, by things that marketing people tell me) that there's a large body of programmers to whom "network programming" is a daunting prospect. I have always taken the RPC model as a way to help those programmers by hiding the network from them -- to relieve them from having to understand things that you and I understand only too well. I am trying to develop a system where, as the system evolves, programmers have to know less and less about the network. In service of this goal, I try to make it the case that the system doesn't require me to talk about "CL vs. CO" in describing the system to its users. -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
cmcmanis@stpeter.Eng.Sun.COM (Chuck McManis) (08/14/90)
In article <4c2d2946.20b6d@apollo.HP.COM> (Nathaniel Mishkin) writes: >I guess I don't see why it's an odd defense. I'm not pointing my finger >at any one else. I thought I was making a humorous analogy. I guess >I mixed up my complaints about Netwise's lack of (what we call) "transport >transparency" (i.e., ensuring constant call semantics regardless of what >transport you happen to be running RPC over) with my complaints about >customization. The analogy was that building a system that sometimes >yields one call semantics and sometimes another (depending on what >transport the application was running over) was like having a C compiler >sometimes preserve registers (that contains useful values) across calls >and sometimes not (depending on what hardware architecture the C compiler >was targetted for), yielding different program behavior. Fortunately for most programmers they specify "connection-oriented" and get the exact same behaviour as NCS's connection oriented dressing over most other transports. You are absolutely correct that if the programmer say's "I'll take _any_ transport." The one they get will always work but may not maintain strict procedure call semantics. And yet, if they as for connection oriented transports, they will always get reliable at most once semantics, unlimited argument size, and all those other nice things that one may want at the expense of keeping some state of the connection around. I've heard the argument that it is "more efficient" for NCS to maintain that state than it is for the kernel to maintain that state but state is state. Even the ECMA people recognize this is a better way to implement these semantics and specify TP4 as the underlying transport for ECMA 127. >I'll try to re-draw the analogy to apply to customization. If someone >said to a C compiler writer: "Hey look, I know what I'm doing and I >don't want you to save registers when I make this call; give me a way >of telling the compiler to not save registers", I think people would >argue that that kind of customization is a bad idea. And your entire argument goes up in smoke. You see, someone _did_ say that to C compiler writers, and they _did_ say *I must have this* because some people _do_ know better and they _don't_ want to have your choices forced down their programmatic throat. In the C compiler arena these are called "#pragma" statements and at least one C compiler I know of lets you completely redefine the way in which calls are made by the compiler! And guess what that's in the standard too! Your belief that your customers don't want to be able to customize their network applications is unfounded. There are people in both camps. There always will be, NetWISE and Sun have always tried to cater to both because they are all customers. >All I know is that it has always been my impression (reinforced, BTW, >by things that marketing people tell me) that there's a large body of >programmers to whom "network programming" is a daunting prospect. This is a tautology. You could just as easily say that there are a large body of _people_ to whom "programming" is a daunting prospect. But that doesn't justify making BASIC the only programming language. >I have always taken the RPC model as a way to help those programmers by >hiding the network from them -- to relieve them from having to understand >things that you and I understand only too well. I am trying to develop >a system where, as the system evolves, programmers have to know less >and less about the network. In service of this goal, I try to make it >the case that the system doesn't require me to talk about "CL vs. CO" >in describing the system to its users. I like it, a shade on the "I know better than you" side but it shows a strong conviction. Would you feel better if we put using "connectionless" transports in the advanced section of the manual? I think everyone can agree on the goal, but most programmers disagree with the philosophy that "we will decide what is best for you." That reeks of proprietaryness. Think of it like a car manual. The first few pages start off with how to start the car, shift gears and turn on the headlights. By the time you get to the end of the NetWISE manual you can swap out carburetors and reset the timing for maximum performance. Halfway through your manual it says "no serviceable parts inside, go see your dealer." Sorry but that doesn't cut it. -- --Chuck McManis Sun Microsystems uucp: {anywhere}!sun!cmcmanis BIX: <none> Internet: cmcmanis@Eng.Sun.COM These opinions are my own and no one elses, but you knew that didn't you. "I tell you this parrot is bleeding deceased!"
pae@athena.mit.edu (Philip Earnhardt) (08/14/90)
In <4bc8788d.20b6d@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: >>On page 43, the "Suggested Time" for fragment acknowledgment is 1 >>second (it's my understanding that all of these timers are hardwired >>in a given NCS implementation). Isn't a 1-second timeout far too >>short for fragment acknowledgment? If the mean round-trip time for 2 >>Internet hosts is 5 seconds, won't this mean that fragments will get >>needless multiple retransmissions? > Ugh. Suffice it to say that the spec should be taken with a grain of > salt when it comes to parameters such as the one you're pointing to. > We focused most of our energy on specifying the non-parametric aspects > of the protocol. A correct implementation (e.g., NCS 2.0) would do round > trip time estimations. The Network Computer Architecture Book (describing NCS 1.x implementations) gives "suggested times" for timer implementations. For your current products, how do retransmission timers work? Specifically, do NCS 1.x implementations on top of UDP/IP have a fixed retransmission timer or do they employ some sort of adaptive algorighm? Phil Earnhardt Netwise, Inc. 2477 55th St. Boulder, CO 80301 Phone:303-442-8280 UUCP: onecom!wldrdg!pae My opinions do not reflect any official position of Netwise.
pae@athena.mit.edu (Philip Earnhardt) (08/14/90)
There have been three ideas about RPC, asynchronous processing, and multi-threaded systems that have been somewhat lost in the recent discussion: 1. Asynchronous processing is a separate issue from RPC. Non-distributed applications have and will continue to use asynchronous processing. 2. One very powerful way to use RPC technology is to take an existing non-distributed application and use an RPC Toolkit to make it a distributed application. If you want to take an existing standalone application that uses asynchronous processing and make your asynchronous call be a remote procedure call, we feel the most straightforward thing to do is an Asynchronous RPC call. Our asynchronous customization provides the three things you need to do that: a routine to invoke the remote procedure, a routine to check the status of the procedure, and a routine to get the results of the remote procedure. There will be existing analogues to these routines in your non-distributed application. Alternatively, if you want to create a multi-threaded application for doing the remote procedure call, fine. The Netwise RPC Tool will work correctly in multi-threaded environments. My personal opinion is that multi-threading is the wrong tool for the job (for a variety of reasons--see below). However, it shouldn't matter what I think--the decision should be left to the designer: 3. Asynchronous processing and multiple threads are both tools that are available to the developer of distributed application. Whenever possible, both of these methods should be available to the designer of the distributed application. Give the designer the choice. My main objection to the NCS philosophy is demonstrated in Mishkin's last message <4bc8788d.20b6d@apollo.HP.COM> : > Asynchrony is hard to understand. All the more reason to have a single > abstraction -- the process/thread -- to deal with it. Mishkin thinks Asynchrony is hard to understand; it is not available in NCS. Why not give the designer the choice? The July 9, 1990 issue of UNIX Today! contains the OSF DCE Rationale and Netwise's response to it. Hopefully, these discussions about Asynchronous RPC will give some insight into the customization issue discussed there. ---------------------------------------------------------------- In <4bc8788d.20b6d@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: >> If you're doing asynchronous calls, you have some point when you have >> enough information to perform the call, but don't need the result >> immediately. A simple strategy is to go back to what you're doing until you >> need the results, then block waiting for them. > How do I get the results? In a synchronous call, the results are in > the output parameters defined in the procedure signature. Do you create > some other random stub procedure that just has output params or what? > What if want to have more than one call executing concurrently? How > do I coordinate and collect the results? With the multi-thread / > synchronous call approach, the answer is create more threads and just ^^^^^^^^^^ > get the result via output parameters. The exact way that our asynchrouous customization works is that the application performs the asynchronous call twice. The first call sends the calling parameters and any globals needed for the call. The second call blocks until the call is complete, updates globals and return parameters and sets the return value. A second routine lets them check the status of the call. As noted above, this is pretty darn similar to what someone is doing for asynchronous I/O in a standalone application. If someone prefers "the answer" for his distributed application, he could change the customization to use our select-like mechanism to check the status of the concurrent calls. He would have changed the interface specification to update the globals directly instead of having return parameters. They could even have put each call in a separate thread, provided threads are available in all environments. I'm not too enthuiastic about "the answer" you've described, because it's drifting away from Procedure Call semantics--the return vales for each of the calls must also be stored in globals, too. An alternative would be to change the customization to use the select-like mechanism to see when results are available for each the concurrent calls, then get the results. Whatever. >>> My religion says that they're a bad idea because they're hard to >>> understand, represent an unnecessary new aspect in the computing model, >>> and that one should use existing primitives for asynchrony (process) and >>> communications (procedure call) together to achieve the same effect. >> Do you mean that a process should fork a new thread to do the RPC call? >> In your example, isn't the parent thread going to do stuff for "a while" >> then "check for a result" of its child? Sounds at least as hard to me. > Not necessarily. I might be collecting statistics about some remote system. > Each thread makes a call to a different system and when the result appear, > the thread inserts the info (using mutex locks, of course) into a common > data structure. No one is "checking for results". ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ I don't understand. Someone is waiting for all of those results to come in. It really doesn't matter if it's looking for return values or looking at a global data structure. What you are proposing is still an asynchronous call! >> For other readers: it's useful to remember one other thing about async >> operations. Async is pretty simple to understand if there is only one >> outstanding operation. It is possible to have multiple async calls over a >> connection, but the rules get much more complicated. I would not recommend >> multiple outstanding asynchronous calls over the same connection. > Exactly. See my comments above. Are you telling me async RPC is OK > unless I want to do more than one remote thing at a time? Seems sort > of a limiting model. What I'm saying is that having multiple outstanding asynchronous calls on one id involves extra management. This is a function of having multiple asynchronous calls over one channel--it really has nothing to do with RPC. If you're doing a non-distributed application with asynchronous calls, it also involves more management to support multiple outstanding asynchronous calls. I'd imagine it's pretty complicated (or maybe impossible) in NCS to manage multiple outstanding asynchronous calls in a single thread. You would probably create multiple threads. In the Netwise RPC sytsem, if you wanted multiple async calls to the same server, you would have multiple connection ids. This would be extra overhead on the Netwise system. You have your choice of separate ids for each of the asynchronous calls or non-trivial customization to manage the calls over a separate id. Or you could examine your application and see if it's really necessary to have multiple outstanding asynchronous calls to the same server. The choice would rest with the designer of the distributed system. >> Yeah, it looks like multi-threaded systems will become more widespread. It >> will be interesting to see what sort of problems that this unleashes on the >> world. I'm not tremendously enthusiastic about locking out all other threads >> every time I want to modify a global. The one that really frightens me, >> though, is the process of software maintenance. Some junior programmer will >> not understand THE RULES for changing a multi-threaded application and will >> introduce bugs that will not surface for months later.... > Asynchrony is hard to understand. All the more reason to have a single > abstraction -- the process/thread -- to deal with it. With a little > language support (especially for stack-based exception handling and mutex > locks) it's pretty tractable. I recommend Andrew Birrell's DEC SRC tech > report on experience programming in such an environment (SRC Report 35, > "An Introduction to Programming with Threads"). Assuming that this package does solve the problem, what is its availability? Is it shipping on DEC platforms? Is the technique proprietary, or is DEC willing to give it away to everyone? Will all vendors agree to use it, or will they invent different schemes? In short, are you willing to guarantee interoperability for this scheme on all multi-threaded platforms? If so, when? As noted above, what you are doing is still an asynchronous call. It's just hidden under the threads. If Asychrony is hard to understand, Asychrony Obfuscated by Threads should be even harder to understand. In <4b872c9b.20b6d@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: > Why do you want to invent and make me use duplicate primitves > (in support of async RPC)? We support Asynchronous RPC via customization. Would NCS need to add primitives to support Asynchronous RPC? In <4bc8788d.20b6d@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: >> Either multi-threading is a keystone of NCS, or it isn't. If >> multithreading isn't available in all environments, then your proposed >> mechanism of doing asynchronrous RPC via forking a thread won't work. > Yes. You won't be able to make concurrent calls in environments that > don't support threading. I also can't port my Unix programs that call > "fork" to MS/DOS. This hasn't really addressed the issue. Either multi-threading is a keystone of NCS, or it isn't. If multi-threading is not available in all NCS environments, then your proposed mechanism of doing asynchronous calls is not portable. The presence or absence of "fork" in MS/DOS has no bearing on the workability of Asynchronous RPC in that environment. Phil Earnhardt Netwise, Inc. 2477 55th St. Boulder, CO 80301 Phone:303-442-8280 UUCP: onecom!wldrdg!pae My opinions do not reflect any official position of Netwise.
burdick@hpspdra.HP.COM (Matt Burdick) (08/16/90)
> Mishkin thinks Asynchrony is hard to understand; it is not available > in NCS. Why not give the designer the choice? This seems pretty lame: some people think that threads are easier or cleaner to use - yet Netwise doesn't offer that ability. Does this mean that NetWise should "give the designer the choice" by offering threads? -matt -- Matt Burdick | Hewlett-Packard burdick@hpspd.spd.hp.com | Intelligent Networks Operation
pae@athena.mit.edu (Philip Earnhardt) (08/16/90)
In article <13710001@hpspdra.HP.COM> burdick@hpspdra.HP.COM (Matt Burdick) writes: >> Mishkin thinks Asynchrony is hard to understand; it is not available >> in NCS. Why not give the designer the choice? > >This seems pretty lame: some people think that threads are easier or >cleaner to use - yet Netwise doesn't offer that ability. Does this >mean that NetWise should "give the designer the choice" by offering >threads? > > -matt >-- >Matt Burdick | Hewlett-Packard >burdick@hpspd.spd.hp.com | Intelligent Networks Operation Please reread my earlier posting--you have a couple of factual errors. Netwise does support multiple threads. Also, any multi-threaded approach must still use asynchronous semantics. You would have to be using both threads and asynchronous semantics. How could this be cleaner and easier? Phil Earnhardt Netwise, Inc. 2477 55th St. Boulder, CO 80301 Phone:303-442-8280 UUCP: onecom!wldrdg!pae My opinions do not reflect any official position of Netwise.
burdick@hpspdra.HP.COM (Matt Burdick) (08/17/90)
> Please reread my earlier posting--you have a couple of factual errors. > Netwise does support multiple threads. I thought one of the bad things about NCS was the difficulty of porting it to platforms that didn't easily support multiple lightweight task or thread emulation. Your previous posting doesn't make it clear whether it is RPC Tool that provides multiple thread capability or whether it will merely continue to function in an environment (such as Mach) that already provides it. > Also, any multi-threaded approach must still use asynchronous > semantics. You would have to be using both threads and asynchronous > semantics. How could this be cleaner and easier? The threads *are* the asynchronous semantics. Picture two threads: one is reading from a device and inserting new data in a global database, the other is providing the user interface to the database. Except for watching for mutual exclusion, neither thread needs to know about the other's existance. A third thread could easily be added that tweaks the database in a different way. The code for the first two threads would not need to be modified in any way. -matt -- Matt Burdick | Hewlett-Packard burdick@hpspd.spd.hp.com | Intelligent Networks Operation
mishkin@apollo.HP.COM (Nathaniel Mishkin) (08/21/90)
In article <140558@sun.Eng.Sun.COM>, cmcmanis@stpeter.Eng.Sun.COM (Chuck McManis) writes: |> I've heard the argument that it is "more efficient" for NCS to maintain |> that state than it is for the kernel to maintain that state but state |> is state. Even the ECMA people recognize this is a better way to |> implement these semantics and specify TP4 as the underlying transport |> for ECMA 127. The day I start basing performance decision decisions on statements of international standards organizations... Well, you get the idea. This whole "conversation" has gone on so long I can't remember whether I made this point or not yet: The issue has nothing to do with NCS's implementation or the fact that it's "based on datagrams" or whatever. The right way to think about what we did was simply that we designed a virtual circuit protocol with the knowledge in hand that RPC was going to be layered on top of it. Doing so lets you better "piggyback" information. For example, the initial "data" for the "connection" can be passed in the connection "open" message. |> And your entire argument goes up in smoke. You see, someone _did_ say |> that to C compiler writers, and they _did_ say *I must have this* because |> some people _do_ know better and they _don't_ want to have your choices |> forced down their programmatic throat. In the C compiler arena these |> are called "#pragma" statements and at least one C compiler I know of |> lets you completely redefine the way in which calls are made by the |> compiler! And guess what that's in the standard too! "#pragma" may be in the standard, but the particular things you can SAY in a pragma are not in the standard. Good luck if you use a pragma to express some semantically relevant intent and you try to take your code to another system/compiler that doesn't know about that particular pragma option. |> >I have always taken the RPC model as a way to help those programmers by |> >hiding the network from them -- to relieve them from having to understand |> >things that you and I understand only too well. ... |> |> I like it, a shade on the "I know better than you" side but it shows |> a strong conviction. Would you feel better if we put using "connectionless" |> transports in the advanced section of the manual? I think everyone can |> agree on the goal, but most programmers disagree with the philosophy |> that "we will decide what is best for you." That reeks of proprietaryness. Well, what can I say? I clearly am not going to disagree with the point that it's bad to have a "we know best" attitude all the time. It's a matter of degree. When I call read(2), I don't get to control how long the data stays in the buffer cache and I don't get to control how many blocks are read ahead. Does this reek of proprietary-ness or is it just "doing the right thing"? -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
mishkin@apollo.HP.COM (Nathaniel Mishkin) (08/21/90)
In article <1990Aug13.203329.13458@athena.mit.edu>, pae@athena.mit.edu (Philip Earnhardt) writes: |> The Network Computer Architecture Book (describing NCS 1.x implementations) |> gives "suggested times" for timer implementations. For your current products, |> how do retransmission timers work? Specifically, do NCS 1.x implementations |> on top of UDP/IP have a fixed retransmission timer or do they employ some sort |> of adaptive algorighm? The former. As I said, this is a deficiency that's being corrected. -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
mishkin@apollo.HP.COM (Nathaniel Mishkin) (08/21/90)
In article <1990Aug13.231646.17415@athena.mit.edu> pae@athena.mit.edu (Philip Earnhardt) writes: >My main objection to the NCS philosophy is demonstrated in Mishkin's last >message <4bc8788d.20b6d@apollo.HP.COM> : > >> Asynchrony is hard to understand. All the more reason to have a single >> abstraction -- the process/thread -- to deal with it. > >Mishkin thinks Asynchrony is hard to understand; it is not available in NCS. >Why not give the designer the choice? You are misrepresenting what I think to be my fairly clearly stated position. Asynchrony *is* available when using NCS -- you get it by using threads. When I said "asynchrony", I meant it in the large -- asynch coordination whether it's based on threads or based on some sort of async RPC mechanism. Either way, it's hard to understand. >The exact way that our asynchrouous customization works is that the >application performs the asynchronous call twice. The first call sends the >calling parameters and any globals needed for the call. The second call blocks >until the call is complete, updates globals and return parameters and sets the >return value. A second routine lets them check the status of the call. As >noted above, this is pretty darn similar to what someone is doing for >asynchronous I/O in a standalone application. Does the first call return some "magic cookie" that I can pass to the second call to corelate the two calls? If not, then can I not make two async calls at the same time (i.e., in a row) without waiting for the result in between? Can I call a call like Unix "select(2)" or "semop(2)" in a way that lets me block waiting on a file descriptor or semaphore AND the async call's completion? More customization required? With threads these questions simply don't arise. If you need to do two calls at the same time, you make two threads. If you need to do an N-way wait, you create N threads and then synchronize (if necessary) using threading primitives. >I don't understand. Someone is waiting for all of those results to come in. It >really doesn't matter if it's looking for return values or looking at a global >data structure. Like I said, I'm collecting statistics and maintaining a database. No one is "looking" for anything. I'm making a remote call and storing the result in the database. Simple. >What you are proposing is still an asynchronous call! What I'm still supporting is asynchrony. Asynchrony isn't an option. It's a reality. The only question is how best to support it. >Assuming that this package does solve the problem, what is its availability? >Is it shipping on DEC platforms? Is the technique proprietary, or is DEC >willing to give it away to everyone? Will all vendors agree to use it, or will >they invent different schemes? In short, are you willing to guarantee >interoperability for this scheme on all multi-threaded platforms? If so, when? DEC has an implementation of the POSIX draft multi-threading interface (Pthreads). This implementation will be part of the DCE product available from OSF. The implementation has already been ported to several Unix systems. An completely implementation of the PThreads interface is running on top of Mach kernel threads. Presumably, when POSIX finalizes the Pthread spec, it will become even more widely implemented. >This hasn't really addressed the issue. Either multi-threading is a keystone >of NCS, or it isn't. If multi-threading is not available in all NCS >environments, then your proposed mechanism of doing asynchronous calls is not >portable. The presence or absence of "fork" in MS/DOS has no bearing on the >workability of Asynchronous RPC in that environment. I don't know what you mean by "keystone". I tried to address the issue by analogy, but I guess I failed. Any application that uses multi-threading (as some NCS-based applications will, as that is the only way to achieve asynchrony) will be portable only to systems that support multi-threading. My analogy was only trying to point out that I could also say "Any application that uses ``fork'' to achieve asynchrony (as some Unix-based applications will, as that is the only way to achieve asynchrony) will be portable only to systems that support ``fork''". Hence, applications that use "fork" will not be portable to MS/DOS. I haven't seen tons of mail excoriating people for using "fork". -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
pae@athena.mit.edu (Philip Earnhardt) (08/27/90)
In <4c555a34.20b6d@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: >> This hasn't really addressed the issue. Either multi-threading is a >> keystone of NCS, or it isn't. If multi-threading is not available >> in all NCS environments, then your proposed mechanism of doing >> asynchronous calls is not portable. The presence or absence of >> "fork" in MS/DOS has no bearing on the workability of Asynchronous >> RPC in that environment. > I don't know what you mean by "keystone". I tried to address the issue > by analogy, but I guess I failed. Any application that uses > multi-threading (as some NCS-based applications will, as that is the > only way to achieve asynchrony) will be portable only to systems that > support multi-threading. Well, there are several different potential meanings for "keystone". I've looked over your earlier messages, but never found a definitive answer to the following questions: 1. Does the implementation of NCS2.0 in an environment require that thread support be available in the environment (or that a thread support package be ported to that environment)? 2. If an environment has native thread support, is that what's used? Does the NCS system have the concept of providing a standardized threads API in all environments? 3. If the answer to #1 is no, what is the set of functionality that will not be available to developers in non-threaded environments? From your above meeeage, it appears as if Asynchronous RPC will not be available in a single-threaded environment. Will you be able to have server code run in a single-threaded environment? Are there other NCS features that won't be available? Phil Earnhardt Netwise, Inc. 2477 55th St. Boulder, CO 80301 Phone:303-442-8280 UUCP: onecom!wldrdg!pae My opinions do not reflect any official position of Netwise.
mishkin@apollo.HP.COM (Nathaniel Mishkin) (08/27/90)
In article <1990Aug27.015014.8881@athena.mit.edu>, pae@athena.mit.edu (Philip Earnhardt) writes: |> 1. Does the implementation of NCS2.0 in an environment require that thread |> support be available in the environment (or that a thread support package |> be ported to that environment)? |> 2. If an environment has native thread support, is that what's used? Does the |> NCS system have the concept of providing a standardized threads API in all |> environments? |> 3. If the answer to #1 is no, what is the set of functionality that will not |> be available to developers in non-threaded environments? From your above |> meeeage, it appears as if Asynchronous RPC will not be available in a |> single-threaded environment. Will you be able to have server code run in a |> single-threaded environment? Are there other NCS features that |> won't be available? Gee, I really wasn't trying to hedge. I guess I'll just have to be very clear this time. The NCS 2.0 implementation is multi-threaded (hence it requires a threads package to be usable). We found this the most natural and simple way to do the implementation. I believe other implementations of the same network protocols could be constructed in a way that does not require a threads package. (NCS 1.5, which is compatible with NCS 2.0, does not require a threads package.) NCS 2.0 does not contain its own threads package. The NCS 2.0 implementation uses the PThreads API. PThreads is a draft POSIX standard that seems to have high probability of being formally standardized. However, NCS's use of PThreads is not so deep that this use couldn't be replaced by use of another threads API and package. We have no plans to do this work though. (It would probably be easier simply to implement a subset PThread veneer over any "native" thread support.) NCS does not support "async RPC" in the sense that Netwise (and others) might use that term. When using NCS, to achieve concurrency (what we've sometimes been calling "asynchrony"), you need a threads API and package available to the application programmer. For any given implementation ("port") of NCS, an application programmer wanting to achieve concurrency will likely have to use the same threads package that that NCS implementation uses, it being unlikely that two threads packages will happily co-exists within the same address space. If someone builds an NCS implementation that does NOT require a threads package, and the environment does not otherwise contain a threads package, application programmers will not be able to achieve concurrency. Like I've been saying, we believe in multi-threading and are actively working to foster a standard threads API and the availability of portable threads implementations. We think this view is shared by many people. -- -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
pae@athena.mit.edu (Philip Earnhardt) (08/29/90)
In article <1990Aug27.094625@apollo.HP.COM>, mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: > The NCS 2.0 implementation is multi-threaded (hence it requires a > threads package to be usable). We found this the most natural and > simple way to do the implementation. I believe other > implementations of the same network protocols could be constructed > in a way that does not require a threads package. (NCS 1.5, which > is compatible with NCS 2.0, does not require a threads package.) > If someone builds an NCS implementation that does NOT require a > threads package, and the environment does not otherwise contain a > threads package, application programmers will not be able to achieve > concurrency. Just to make sure I understand you clearly, let me re-state what I think you've said: While current NCS2.0 implementations are multi-threaded via the POSIX PThreads package, there is no architectural restriction that all ports of NCS2.0 must either use PThreads for their implementation or even that all ports use multiple threads. Finally, are the multiple threads of a NCS2.0 implementation cooperative -- do current implementations need a preemptive multi-threading package? Phil Earnhardt Netwise, Inc. 2477 55th St. Boulder, CO 80301 Phone:303-442-8280 UUCP: onecom!wldrdg!pae My opinions do not reflect any official position of Netwise.
mishkin@apollo.HP.COM (Nathaniel Mishkin) (09/01/90)
In article <1990Aug29.012657.1305@athena.mit.edu>, pae@athena.mit.edu (Philip Earnhardt) writes: >Just to make sure I understand you clearly, let me re-state what >I think you've said: > > While current NCS2.0 implementations are multi-threaded via the > POSIX PThreads package, there is no architectural restriction that > all ports of NCS2.0 must either use PThreads for their > implementation or even that all ports use multiple threads. Correct. >Finally, are the multiple threads of a NCS2.0 implementation cooperative -- do >current implementations need a preemptive multi-threading package? Preemption is required if servers are going to have threads (including threads running some client's call) that are not going to voluntarily yield (or terminate) within a relatively short period of time, e.g., 30 seconds. This is because the server has a thread created by the NCS implementation that listens for "ping" messages from the client (which is trying to determine the server's liveness). If this thread doesn't get a chance to run for long enough, the client will think the server has died. (Note that the client automatically reduces the ping frequency during the duration of a single call.) -- -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
pae@athena.mit.edu (Philip Earnhardt) (09/06/90)
>> Finally, are the multiple threads of a NCS2.0 implementation cooperative >> -- do current implementations need a preemptive multi-threading package? > Preemption is required if servers are going to have threads (including > threads running some client's call) that are not going to voluntarily > yield (or terminate) within a relatively short period of time, e.g., > 30 seconds. This is because the server has a thread created by the NCS > implementation that listens for "ping" messages from the client (which > is trying to determine the server's liveness). If this thread doesn't > get a chance to run for long enough, the client will think the server > has died. (Note that the client automatically reduces the ping frequency > during the duration of a single call.) Again, let me paraphrase to see if I understand what you're saying here: "A multi-threaded implementation of NCS must use preemptive threads." The alternative to this is to impose the semantic limitation that the user's appliction code must not run for "too long". This would qualify as a pretty arbitrary limitation to put on a distributed applications programmer. Phil Earnhardt Netwise, Inc. 2477 55th St. Boulder, CO 80301 Phone:303-442-8280 UUCP: onecom!wldrdg!pae My opinions do not reflect any official position of Netwise.
pae@athena.mit.edu (Philip Earnhardt) (09/07/90)
In <5370@ptsfa.PacBell.COM> jag@PacBell.COM (Jim Goncher) writes: > [questions about the status of the ISO/OSI RPC Standard] The number of the beast is ISO/IEC JTC 1/SC 21 N 4927. This is a working draft--the first of 4 steps in becoming a standard. I'm not sure if such standards are available from the traditional sources for ISO documents--the copy I have has the phone number for ANSI on the cover -- (212) 642-4932. ROSE is used in an RPC-like fashion in X.500 and X.400 applications. I have never directly worked with ROSE; my understanding is that it contains the protocol units for doing RPC. However, there are other issues that must be addressed for a standalone RPC--handling of binding and external language interfaces. Proponents of ROSE may argue that ROSE adequately deals with such issues--these debates are part of what's going on in the standards bodies meetings. The proposed RPC standard is a "Layer 7 and Beyond" protocol--the "and Beyond" is addressing things like the language interface to RPC. ROSE is certainly a [mechanism for creating a] layer 7 protocol. In earlier discussions on this group Mishkin has tried to discount the importance of Netwise's using ISO 8327 (Session Layer Protocol) and ISO 8822 (Presentation Layer Protocol) in its new products: <4bc8788d.20b6d@apollo.HP.COM> Mishkin says: > Get ALL your facts straight. First of all, there's currently no ISO > standard for RPC. Second, the protocol NCS 2.0 defines for running over > COTPs adheres as closely as possible to how we imagine such an ISO standard > will appear and fit into the OSI model. This includes presentation > negotiation and the rest of the ball of mud. Believe me, I've had > T-CONNECT through A-CONNECT up to my ears. If RPC is a Layer 7 protocol, the *exact* way that it would fit into a Profiled OSI Stack is on top if the standard OSI Presentation and Session layers. Several quesitons: 1. Do the NCS developers think that OSI RPC is going to be a layer 7 standard? If not, why not? 2. If the answer to #1 is yes, what protocols do they think will be used at Layer 6 and Layer 5? If not ISO 8327 and ISO 8824, what other protocls will be used? 3. Your second sentence above talks about NCS on top of COTPs. What about NCS on top of CLTPs? Will these be ISO compliant? Will they use the ISO connection-oriented or connectionless upper layer protocols? Phil Earnhardt Netwise, Inc. 2477 55th St. Boulder, CO 80301 Phone:303-442-8280 UUCP: onecom!wldrdg!pae My opinions do not reflect any official position of Netwise.
mishkin@apollo.HP.COM (Nathaniel Mishkin) (09/08/90)
In article <1990Sep5.194621.11656@athena.mit.edu>, pae@athena.mit.edu (Philip Earnhardt) writes: >The alternative to this is to impose the semantic limitation that the user's >appliction code must not run for "too long". This would qualify as a pretty >arbitrary limitation to put on a distributed applications programmer. I agree. That's why I am (and OSF is) an advocate for a standard threading interface with a portable implementation. I might say: Of course, Sun RPC has gotten by (or maybe not gotten by :-) for years with limitations of this sort (and more). E.g., if you use Sun RPC over UDP/IP, you're limited to 8K worth of data and the call semantics are unpredicatble. "Use Sun RPC over TCP/IP", you might say. Oh well, then the number of concurrent calls I'm allowed to have is limited by the number of TCP connection I can (effectively) have open at a time. Seems like a pretty arbitrary limitation to put on a distributed applications programmer. The load of having, oh, 100 clients doing lightweight things may be low. It's just the poorly chosen requirement the RPC package made on the system that's preventing me from doing what I want. But I wouldn't want to compare NCS to Sun RPC :-) -- -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
mishkin@apollo.HP.COM (Nathaniel Mishkin) (09/08/90)
In article <1990Sep7.004206.17225@athena.mit.edu>, pae@athena.mit.edu (Philip Earnhardt) writes: >In earlier discussions on this group Mishkin has tried to discount the >importance of Netwise's using ISO 8327 (Session Layer Protocol) and >ISO 8822 (Presentation Layer Protocol) in its new products: > >If RPC is a Layer 7 protocol, the *exact* way that it would fit into a >Profiled OSI Stack is on top if the standard OSI Presentation and >Session layers. > >Several quesitons: > >1. Do the NCS developers think that OSI RPC is going to be a layer 7 > standard? If not, why not? To the extent I understand all this, yes. >2. If the answer to #1 is yes, what protocols do they think will be used > at Layer 6 and Layer 5? If not ISO 8327 and ISO 8824, what other > protocls will be used? I don't get it. The fact that you use ISO standard Session and Presentation layer protocols is merely interesting, not important in achieving an RPC system that meets some standard, because there IS no RPC standard at the higher levels. Whatever you (or we) do at those layers does not yield a "standard RPC", regardless of whether we happen to use standard protocols at lower levels. To the extent that you use standard protocols at lower levels and to the extent that it's hard to produce (or find existing) systems that implement or use those protocols, you may have some advantage in the race to achieve a standard RPC system, when the higher-level protocols are standardized. My supposition is that implementation of the session layer protocols is the business of the platform upon which I'm building an RPC system and that there are a variety of existing implementations of ASN.1/BER that I could freely obtain or buy to use as the basis for the presentation level implementation of my RPC system. It's just not that big a deal, and the requirement is not going to appear tomorrow (because the RPC standard is not going to appear tomorrow). >3. Your second sentence above talks about NCS on top of COTPs. What > about NCS on top of CLTPs? Will these be ISO compliant? Will they > use the ISO connection-oriented or connectionless upper layer protocols? As I understand things, whether I can build an NCS that is an RPC system that is ISO compliant and runs over CLTP's is a function of whether ISO defines upper level RPC protocols that are defined to work on top of CLTP's. I think ROSE is defined to use COTP's. I suspect that any ISO standard RPC will be similarly defined (although maybe other ISO standard RPC protocols will be defined later to run over CLTP's). When a CLTP-oriented ISO RPC protocol is defined, I imagine we will implement it. It's hard to be concrete about things that depend on things that don't exist. -- -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
pae@athena.mit.edu (Philip Earnhardt) (09/11/90)
In <1990Sep7.153710@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: >> Again, let me paraphrase to see if I understand what you're saying here: >> >> "A multi-threaded implementation of NCS must use preemptive threads." >> >>The alternative to this is to impose the semantic limitation that the user's >>appliction code must not run for "too long". This would qualify as a pretty >>arbitrary limitation to put on a distributed applications programmer. > I agree. That's why I am (and OSF is) an advocate for a standard threading > interface with a portable implementation. > [...] Your response didn't address the question at hand. What are you going to do until PThreads (or some multi-threaded standard) is available in all environments? Specifically, are ports of NCS permitted to use non-preemptive threads? Phil Earnhardt Netwise, Inc. 2477 55th St. Boulder, CO 80301 Phone:303-442-8280 UUCP: onecom!wldrdg!pae My opinions do not reflect any official position of Netwise.
vipin@samsun.Sun.COM (Vipin Samar) (09/11/90)
In comp.protocols.misc mishkin@apollo.HP.COM writes: > Of course, Sun RPC has gotten by (or maybe not gotten by :-) for > years with limitations of this sort (and more). E.g., if you use > Sun RPC over UDP/IP, you're limited to 8K worth of data and the call > semantics are unpredicatble. "Use Sun RPC over TCP/IP", you might > say. Oh well, then the number of concurrent calls I'm allowed to > have is limited by the number of TCP connection I can (effectively) > have open at a time. Seems like a pretty arbitrary limitation to > put on a distributed applications programmer. The load of having, > oh, 100 clients doing lightweight things may be low. It's just the > poorly chosen requirement the RPC package made on the system that's > preventing me from doing what I want. I have been reading with interest your discussions on RPCs. I do not have much documentation on NCS2.0 and hence have some difficulty in understanding lots of NCS issues and policies. For example, - how does the NCS library decide which transport to use. Say, I have both TCP and UDP on both client and the server machine. Are there any heuristics in place upon which you decide which transport has to be used? Do you prefer one over the other? - Using UDP for hundreds of connections is a good idea, but I wonder whether to support streaming on top of that is a good idea because it can lead to all sorts of rate-flow problems. TCP is great at backing off in stormy and overloaded networks, while UDP just exacerbates the problem. Any comments? Vipin Samar. Disclaimer: I speak for none but myself. vipin@Eng.sun.com || sun!Eng!vipin
mishkin@apollo.HP.COM (Nathaniel Mishkin) (09/11/90)
In article <1990Sep10.182341.11165@athena.mit.edu>, pae@athena.mit.edu (Philip Earnhardt) writes: >In <1990Sep7.153710@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: >> I agree. That's why I am (and OSF is) an advocate for a standard threading >> interface with a portable implementation. >> [...] > >Your response didn't address the question at hand. What are you going >to do until PThreads (or some multi-threaded standard) is available in >all environments? Specifically, are ports of NCS permitted to use >non-preemptive threads? What are you looking for? I said "I agree". It's a limitation. I'm going to do nothing to make up for the lack of a preemptive threading package. As far as what is "permitted" -- I don't say what's permitted and what's not. (I suppose we could go off and discuss esoterica about who will be able to use the "NCS" or "DCE RPC" trademarks and under what terms, but that hardly seems worthwhile.) A port of NCS that doesn't have a preemptive threads package available to it will be less useful (by some degree that's a function of what applications are doing) than a port that does have such a package available. -- -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
mishkin@apollo.HP.COM (Nathaniel Mishkin) (09/12/90)
In article <142133@sun.Eng.Sun.COM>, vipin@samsun.Sun.COM (Vipin Samar) writes: > - how does the NCS library decide which transport to use. Say, I have > both TCP and UDP on both client and the server machine. Are there any > heuristics in place upon which you decide which transport has to be used? > Do you prefer one over the other? The model (which is more or less the same for all versions of NCS) is that when servers start up, they create some set of communication endpoints with certain characteristics (in BSD-speak: sockets of some socket type and protocol in some protocol family with either a well-known or dynamically assigned port). The choice of characteristics can be controlled by the server writer, or (at NCS 2.0) the server can say (in effect) "create sockets of all protocol families and socket types that both the RPC library and the underlying system support. (You could actually do this yourself in NCS 1.5, but we've made a simple function to do it for you in 2.0.) Once the server has created the communication endpoints, it advertises the locations (essentially sockaddrs) of these endpoints in some network accessible database. NCS 1.5 supports the Global Location Broker for doing this. NCS 2.0 (OSF DCE RPC) works in conjunction with a distributed, hierarchical name service. Both the GLB and the name service can store locations. Now for the client side of things. Presumably, a client has some abstract identification for the server it's trying to reach. E.g., it may have an "interface ID" (what Sun RPC calls a program number, I think), or it may have a pathname in the distributed name service, or it might have some sort of "object ID" for the object (e.g., database, printer, file) that it wants to operate on. Whatever this abstract ID is, the client contacts some 3rd party service (like the GLB or the name service) (which typically it uses broadcast to locate) to translate the abstract ID into a set of locations of servers. Once the client has the locations, it picks out the ones that it can use. (Like sockaddrs, the stored locations hold information about the network protocols you need to use to talk to the server at that location.) This "picking out" step happens transparently to the client. (More transparently in 2.0 than in 1.5.) The client application (or client stub, in case you're using one of the fancier automatic binding features) then presents a location to the RPC runtime which returns a "binding handle" to the client. The client then makes remote calls using the binding handle. (BTW, NCS currently has no way to pick among an apparently equivalent set of locations. This is an area for future work. E.g., you might want to favor TCP over X.25, if both are available; you might favor servers on the same network as you, if they exist. Some of these schemes are hard to formalize and/or implement given existing underlying technology.) At no point does the client express anything like "use TCP", or "use connection-based RPC", or the like. All that information is encapsulated in the location structure and used transparently by the RPC runtime to dispatch to the appropriate RPC runtime code to run the appropriate network protocols (from RPC down through transport). > - Using UDP for hundreds of connections is a good idea, but I wonder whether > to support streaming on top of that is a good idea because it can lead > to all sorts of rate-flow problems. TCP is great at backing off in > stormy and overloaded networks, while UDP just exacerbates the problem. I guess I'm not sure what your point is. NCS uses a flow control protocol on top of UDP. I.e., NCS uses UDP in much the way that TCP uses IP. Thus, virtually all the tricks that TCP uses are candidates for being used (and many are used) by NCS. -- -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
pae@athena.mit.edu (Philip Earnhardt) (09/12/90)
In <1990Sep11.120119@apollo.HP.COM> mishkin@apollo.HP.COM (Nathaniel Mishkin) writes:
pae> Again, let me paraphrase to see if I understand what you're saying here:
pae>
pae> "A multi-threaded implementation of NCS must use preemptive threads."
pae>
pae> The alternative to this is to impose the semantic limitation that the
pae> user's application code must not run for "too long". This would qualify
pae> as a pretty arbitrary limitation to put on a distributed applications
pae> programmer.
mish> I agree. That's why I am (and OSF is) an advocate for a standard
mish> threading interface with a portable implementation. [...]
pae> Your response didn't address the question at hand. What are you going
pae> to do until PThreads (or some multi-threaded standard) is available in
pae> all environments? Specifically, are ports of NCS permitted to use
pae> non-preemptive threads?
mish> What are you looking for? I said "I agree". It's a limitation. I'm
mish> going to do nothing to make up for the lack of a preemptive threading
mish> package. As far as what is "permitted" -- I don't say what's
mish> permitted and what's not. (I suppose we could go off and discuss
mish> esoterica about who will be able to use the "NCS" or "DCE RPC"
mish> trademarks and under what terms, but that hardly seems worthwhile.)
Gee, I thought the original question was pretty clear: Are non-preemptive
thread implementations permitted? "I agree" isn't an appropriate answer to
such a question.
Who does say what's permitted and what's not permitted in an NCS port?
What's to keep someone from making a port that doesn't interoperate with
another NCS port?
mish> A port of NCS that doesn't have a preemptive threads package available
mish> to it will be less useful (by some degree that's a function of what
mish> applications are doing) than a port that does have such a package
mish> available.
You could say something even stronger than this. Ask the question:
How does someone create a distributed application using NCS such that
his distributed code is portable to all environments?
or perhaps
What portability problems does NCS itself create?
Based on what you've said above, someone who wants a portable NCS application
had better make sure that his code doesn't run for "too long".
Do you think it's important to support users in creating portable NCS
applications?
Phil Earnhardt Netwise, Inc. 2477 55th St. Boulder, CO 80301
Phone:303-442-8280 UUCP: onecom!wldrdg!pae
My opinions do not reflect any official position of Netwise.mishkin@apollo.HP.COM (Nathaniel Mishkin) (09/12/90)
In article <1990Sep11.223905.13417@athena.mit.edu>, pae@athena.mit.edu (Philip Earnhardt) writes: >Who does say what's permitted and what's not permitted in an NCS port? A good question that many people have tried to address in the context of many existing protocols (and programming languages, and ...). I don't know that I have a good answer. The strictest answer is that what's permitted and required are exactly what some spec says is permitted and required. The creation of such a spec that is absolutely complete is a very tricky matter. It took a long time and a lot of experience to get, say, the TCP spec(s) (plus notes to implementors) into a state where multiple implementors who read the specs actually produce implementations that interoperate. Even though most people simply clone existing TCP implementations these days, one STILL sees the occasional interoperability problem. What's more, I'm not aware of any official "conformance test suite" for TCP, although such a existence suite would seem to be an extraordinarily useful adjunct to specs. We believe that in the current NCS spec we have gone a long way toward making a complete spec, but clearly it's not absolutely complete. Questions such as you're raising -- "Is an implementation conformant if it can deal only with calls of short duration" -- is not explicitly addressed in the spec. It should be. I am reluctant to strictly disallow such implementations, since allowing them does have some value. Let's say we'd put it in the "Minimal NCA/RPC" appendix to the spec. By the way, I don't know that I've ever seen a spec of the Netwise RPC protocol (i.e., the one that your current, non-SunRPC-based products use). Can you make one available? >What's to keep someone from making a port that doesn't interoperate with >another NCS port? In general, the only thing that stops someone from making a non-interoperating implementation (port or brand new) of some network protocol is market pressure. What's to keep someone from making an implementation of TCP that doesn't interoperate with another TCP implementation? (Some protocol definers try to leverage the legal system. As I recall, Sun doesn't let a vendor use the "NFS" trademark unless the vendor's implementation passes some verification suite. Not an unreasonable idea.) >Do you think it's important to support users in creating portable NCS >applications? No, and I don't like apple pie either :-) Of course it's important. But you seem to treat everything as black and white. ALL systems have aspects that limit portability to some extent. Like I said, if I use Sun RPC over TCP and I need to handle 100's of clients, my application is not going to be portable to systems that support only 20 file descriptors per process. Does that mean the system is useless? No. Does it mean it's less useful than one that requires only 1 file descriptor. To some degree, yes. How does this "degree" map onto the space of real world systems and applications? Time will tell. -- -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
vipin@samsun.Sun.COM (Vipin Samar) (09/13/90)
mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: >Once the client has the locations, it picks out the ones that it can >use. (Like sockaddrs, the stored locations hold information about the >network protocols you need to use to talk to the server at that location.) >This "picking out" step happens transparently to the client. (More ^^^^^^^^^^^^^ >transparently in 2.0 than in 1.5.) Yes, this is precisely what I am interested in knowing. Does transparently mean "random" here because any of the available protocols give the same functionality or is there some implicit ordering? >(BTW, NCS currently has no way to pick among an apparently equivalent >set of locations. This is an area for future work. E.g., you might >want to favor TCP over X.25, if both are available; you might favor servers >on the same network as you, if they exist. Some of these schemes are >hard to formalize and/or implement given existing underlying technology.) Yes, I agree this is a tough problem on formalizing these heuristics - perhaps after all it is not such a bad idea to give control to users in such cases. One can never really have all the bases covered. >> - Using UDP for hundreds of connections is a good idea, but I wonder whether >> to support streaming on top of that is a good idea because it can lead >> to all sorts of rate-flow problems. TCP is great at backing off in >> stormy and overloaded networks, while UDP just exacerbates the problem. >I guess I'm not sure what your point is. NCS uses a flow control protocol >on top of UDP. I.e., NCS uses UDP in much the way that TCP uses IP. >Thus, virtually all the tricks that TCP uses are candidates for being >used (and many are used) by NCS. O.K., so you did build the entire TCP layer on top of UDP and you added all the code for that in the NCS library. Has that step really solved all the problems or perhaps has just moved the problems from one place to another. If one cannot support huge (say 100) TCP connections because of resource problem, then you have said that NCS/UDP will be able to overcome the resource problem. I wonder if that is true, because building up of entire TCP functionality on top of UDP is going to require system resources as well. Multiplexing the file descriptors may solve this problem to some extent, but not completely, because one would still need to keep all those buffers hanging around for any decent congestion control and rate-flow problems. Also, retransmissions would be of the entire UDP packet size and not just of the size of the MTU of the network. Or have you worked around this problem by reducing the UDP packet size to the MTU size? I hope not. This retransmission problem will happen only during stormy weather, but whenever that happens, it sure will let hell break loose. TCP backs off very graciously in such circumstances. How does your protocol behave in these cases? Are all the TCP tricks on top of UDP in the kernel or in the user land? If it is in user land, I am not sure how gratifying performance you are going to get out of it. Atleast for +8K data, the performance will be abysmal. Most of the TCP magic is normally in the kernel (which provides for higher thru'put). Would you like to share some performance numbers for NCS/UDP and NCS/TCP? or perhaps just their ratio [if perf numbers are confidential]. Another problem with NCS approach is that the user cannot take advantages out of any of the new performance improvements of any of the transports. They will have to wait for the NCS team to hack those same fixes into their software. In ONC, they need not do anything. Any comments ? vipin Disclaimer: I speak for no one but myself. vipin@Eng.sun.com || sun!Eng!vipin
korsberg@abaa.uucp (Ed Korsberg) (09/14/90)
In article <142319@sun.Eng.Sun.COM> vipin@samsun.Sun.COM (Vipin Samar) writes: > >O.K., so you did build the entire TCP layer on top of UDP and >you added all the code for that in the NCS library. Has that step really >solved all the problems or perhaps has just moved the problems from one >place to another. > >Another problem with NCS approach is that the user cannot take advantages >out of any of the new performance improvements of any of the transports. >They will have to wait for the NCS team to hack those same fixes into their >software. In ONC, they need not do anything. > I am just joining this discussion so please excuse my innocence on some issues that may have be resolved before. It seems to me that implementing TCP functions in a layer 7 application or ROSE or RPC function is a duplication of effort. TCP (and TP4) has been in developement for years and should be fairly stable from one implementation to another. With this stability people should be able to replace existing software based TCP (or TP4) implementations with either better software solutions or hopefully silicon assisted solutions. In other words why recreate the wheel? Reuse what already is in place and works. -------------------------------------------------------------------------------- -- Ed Korsberg E-mail: korsberg@aa.ab.com Allen Bradley Inc. phone: 313-998-2470 555 Briarwood Circle Ann Arbor, Mich 48104
mishkin@apollo.HP.COM (Nathaniel Mishkin) (09/14/90)
In article <142319@sun.Eng.Sun.COM>, vipin@samsun.Sun.COM (Vipin Samar) writes: >Yes, this is precisely what I am interested in knowing. Does transparently >mean "random" here because any of the available protocols >give the same functionality or is there some implicit ordering? It means "random because any of the available protocols give the same functionality". Like I said, define an scheme for making a more intelligent choice among equivalent servers is an area for future work. >Yes, I agree this is a tough problem on formalizing these >heuristics - perhaps after all it is not such a bad idea to give control to >users in such cases. One can never really have all the bases covered. I definitely agree that there should be at least one mode that lets user policy choices come into play. >O.K., so you did build the entire TCP layer on top of UDP and >you added all the code for that in the NCS library. Has that step really >solved all the problems or perhaps has just moved the problems from one >place to another. Clearly there are SOME problems that just don't go away. We believe that many do. >If one cannot support huge (say 100) TCP connections because of resource >problem, then you have said that NCS/UDP will be able to >overcome the resource problem. I wonder if that is true, because >building up of entire TCP functionality on top of UDP is going to require >system resources as well. Multiplexing the file descriptors may >solve this problem to some extent, but not completely, because one would >still need to keep all those buffers hanging around for any decent >congestion control and rate-flow problems. Of course. However it is also of course true that NCS RPC's buffers (unlike those of the kernel implementation of TCP) aren't consuming wired physical memory, thus giving NCS some more room to play. And then there's the wired space in the kernel for TCP connection blocks and any other miscellaneous TCP overhead. Further, the nature of the NCS RPC protocol lets the server simply blow off its pseudo-connections to clients at any point that a call is not in progress; the pseudo-connection reestablishment happens automatically and transparently to the application (and the stubs). It is also true that even an RPC running on top of a COTP has buffer management problems and that they're arguably HARDER than RPC's over CLTP's. Note that when I say RPC/COTP I mean one that has somewhat more complicated problems than say, SunRPC/TCP. The problematic scenario is the case where for some reason, the stub is not consuming its input data stream for some period of time. In our model, we have a thread (or set of threads), distinct from the thread running the stub, that's responsible for actually reading from the connection. One reason for this is so that client-generated "cancels" (i.e., requests to abort calls) can be processed even if the stub/server thread isn't currently in the RPC runtime. Thus, there is a layer of buffering between the connection and the stub. There is finite space for this buffering. Since once you've read data from a connection, you can't discard the data (unless you want to oblige the client stubs to be smart enough to restart the call), you have the problem that you might have to be smart enough to figure that you have to stop reading from some of your open connections. But then you don't see the "cancel" message. Without boring people any more, you can more or less work around these problems, but it's by no means simple. >Also, retransmissions would be of the entire UDP packet size and not just >of the size of the MTU of the network. Or have you worked around this >problem by reducing the UDP packet size to the MTU size? I hope not. >This retransmission problem will happen only during stormy weather, but >whenever that happens, it sure will let hell break loose. >TCP backs off very graciously in such circumstances. How does your protocol >behave in these cases? We endeavor to send MTU-sized UDP packets and I don't know why you "hope not". The general wisdom as I see it seems to be that depending on IP-level fragmentation is a bad idea. Of course, it's currently hard to know the (what's called) "path MTU". (I refer people to the IETF "Path MTU Discovery" document for more details.) So we guess and try to be conservative and we eagerly await the widespread implementation of the protocol modifications required to do path MTU discovery. I think the IETF has now endorsed them. >Are all the TCP tricks on top of UDP in the kernel or in the user land? >If it is in user land, I am not sure how gratifying performance you are going >to get out of it. We have no special kernel support. (Note that there's a version of NCS that runs in the kernel, but it's only for the benefit of NCS applications that live in the kernel.) I can't give you numbers right now. (I don't have them and I don't know that I could make them public right now anyway.) (BTW, I would not be surprised to hear that NCS 1.5's bulk data throughput is lower than TCP's on some systems.) However, we believe that doing (what amounts to) a COTP in user space on top of UDP will be sufficiently rewarding. Even if it is only as fast or slight slower than TCP it is rewarding in that it has better scaling properties, which we care a lot about. Further, we're basing much our opinions on experiments that we've done and on the real-life experience the CMU ITC people with their Rx RPC system, which essentially does the same thing that NCS RPC does. In any case I take seriously my responsibility to produce a real evaluation of the wisdom of this approach once I'm in a position to do the evaluation. >Another problem with NCS approach is that the user cannot take advantages >out of any of the new performance improvements of any of the transports. >They will have to wait for the NCS team to hack those same fixes into their >software. In ONC, they need not do anything. Fair enough. On the other hand, to get the benefits of performance improvements in NCS, all you need to do is get a new NCS library and not get a whole new kernel from your vendor (as would likely be required to get any improved TCP). My experience (as both vendor and consumer) has been that getting updated user-mode software to customers is typically much easier than getting new kernels to them. -- -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com
vipin@samsun.Sun.COM (Vipin Samar) (09/18/90)
> Vipin Samar writes: >>Yes, I agree this is a tough problem on formalizing these >>heuristics - perhaps after all it is not such a bad idea to give control to >>users in such cases. One can never really have all the bases covered. > >I definitely agree that there should be at least one mode that lets >user policy choices come into play. So, we have agreed on one thing that it is a good idea to have an escape hatch for programmers/users to be able to choose a particular transport. Any ideas as to how will you extend NCS to do it? ONC (tirpc) allows it today by using NETPATH environment variable. One can either set it or use one of the default transports of the required type available on the system. >>If one cannot support huge (say 100) TCP connections because of resource >>problem, then you have said that NCS/UDP will be able to >>overcome the resource problem. I wonder if that is true, because >>building up of entire TCP functionality on top of UDP is going to require >>system resources as well. >Of course. However it is also of course true that NCS RPC's buffers >(unlike those of the kernel implementation of TCP) aren't consuming wired >physical memory, thus giving NCS some more room to play. And then there's >the wired space in the kernel for TCP connection blocks and any other >miscellaneous TCP overhead. Further, the nature of the NCS RPC protocol >lets the server simply blow off its pseudo-connections to clients at >any point that a call is not in progress; the pseudo-connection >reestablishment happens automatically and transparently to the application >(and the stubs). I am assuming that you blow away the connection state only when you have the guarantee that the data has reached the other side. TCP also frees up all the associated buffer space in such cases. The only difference would be in the number of mbuf's (2?)reqd to keep the state/sequencing. Streaming/UDP becomes a win, only if one knows beforehand that there will be hundreds of connections for that particular RPC service. Even when one is using TCP, the server can blow away the connection (if the call is not in progress) and the client will simply have to rebind (at some extra cost). The client RPC library should provide the required transparency. The question is which is better - pay huge cost at each call (discussed in earlier messages) OR pay rebinding cost only when it is required? In my opinion, second approach is better. I am sure that there are many TCP/IP experts on this newsgroup. Does any one want to share his/her opinion on building the TCP functionality on top of UDP? >It is also true that even an RPC running on top of a COTP has buffer >management problems and that they're arguably HARDER than RPC's over >CLTP's. Note that when I say RPC/COTP I mean one that has somewhat more >complicated problems than say, SunRPC/TCP. The problematic scenario >is the case where for some reason, the stub is not consuming its input >data stream for some period of time. In our model, we have a thread >(or set of threads), distinct from the thread running the stub, that's >responsible for actually reading from the connection. One reason for >this is so that client-generated "cancels" (i.e., requests to abort calls) >can be processed even if the stub/server thread isn't currently in the >RPC runtime. Thus, there is a layer of buffering between the connection >and the stub. There is finite space for this buffering. If I assume that both the threads are share the buffer space, then what do you mean by "finite space for this buffering"? Also, the problem described above seems more like a problem created by your implementation. Also, this problem should exist for CLTP too. How did you handle that case? >We endeavor to send MTU-sized UDP packets and I don't know why you "hope >not". Because I want good performance and that can happen only by minimizing transitions between kernel and user land. Choosing a large number is bad for stormy cases & retransmissions and leads to overrun for machines with fewer mbuf's or less memory. Choosing a small number for UDP packet size is bad for performance. Sun decided to go with a compromise - 8K UDP packet size. In NCS, is there a mechanism for size negotiation for faster/bigger machines (lots of memory) which can easily handle bigger size UDP packets? Just curious - what do you guys do for NFS? Do you use 1K packets only? Actually, you can easily rest all of my fears by posting some performance ratios for UDP/TCP (not numbers - I know they are confidential) > In any case I take seriously >my responsibility to produce a real evaluation of the wisdom of this >approach once I'm in a position to do the evaluation. I hope that performance will be one of the criterian. >-- > -- Nat Mishkin vipin Disclaimer: I speak for no one but myself. vipin@Eng.sun.com || sun!Eng!vipin
vipin@samsun.Sun.COM (Vipin Samar) (09/18/90)
In article <1990Sep14.093420@apollo.HP.COM>, mishkin@apollo.HP.COM (Nathaniel Mishkin) writes: > > >Another problem with NCS approach is that the user cannot take advantages > >out of any of the new performance improvements of any of the transports. > >They will have to wait for the NCS team to hack those same fixes into their > >software. In ONC, they need not do anything. > > Fair enough. On the other hand, to get the benefits of performance > improvements in NCS, all you need to do is get a new NCS library and > not get a whole new kernel from your vendor (as would likely be required > to get any improved TCP). My experience (as both vendor and consumer) > has been that getting updated user-mode software to customers is typically > much easier than getting new kernels to them. > > -- > -- Nat Mishkin Kernel would normally come with the machine. However, the RPC library may or may not come with the machine. So, to order NCS library, I will need to know which version/release of transports I am running. And the vendors (you as the technology supplier, others as distributors for different platforms) will have to keep different versions of the library depending upon the transport available on my machine. Unless this chain is short and things happen smoothly, the time lag between the day I get my machine and the day I have compatible/tuned NCS software could be substantial - perhaps time to buy a new machine or install the new OS and then the cycle would start all over again. And you as a vendor would certainly have a fun time maintaining all different versions of software for various transports! vipin Disclaimer: I speak for no one but myself. vipin@Eng.sun.com || sun!Eng!vipin
mishkin@apollo.HP.COM (Nathaniel Mishkin) (09/19/90)
In article <142630@sun.Eng.Sun.COM>, vipin@samsun.Sun.COM (Vipin Samar) writes: >So, we have agreed on one thing that it is a good idea to have an >escape hatch for programmers/users to be able to choose a >particular transport. Any ideas as to how will you extend NCS to do it? I don't have any concrete proposal. However, let me point out some things. There are two kinds of choices one might want to make: (1) choose which protocol stack (by which I mean a generalization of your scheme that lets you choose which transport you want) you want to use, and (2) choose which SERVER you want, among a set of equivalent servers. The first kind of choice is addressed in part by your environment variable. (I think it's more likely what you'd want to specify is a ranking of stacks; i.e., "I want to use RPC/UDP/IP if the server has it, otherwise I'll accept RPC/ROSE/.../TP4, otherwise ...".) The second choice is a little trickier. You might want to choose based on the number of hops to the server or the class of machine it is or how busy it is. Basically, NCS 2.0 has three functions: "import", "lookup", and "select". (These aren't the exact names of course.) "import" basically just calls "lookup" followed by "select". "lookup" returns a set of "compatible servers" -- i.e., ones that you have a prayer of being able to talk with (i.e., they've registered support for the interface you're interested in and they share all the necessary communications protocols with you). "select" picks one server out of a set returned by "lookup", currently at random. The idea is that most clients will just use "import" and trust that "select" does an OK thing and that in the future maybe it will do even more OK things. However, sophisticated clients are free to call "lookup" and select a server based on whatever criteria the application writer feels like applying, including looking at environment variables and reading configuration files. >I am assuming that you blow away the connection state only when you have the >guarantee that the data has reached the other side. Yes. >TCP also frees >up all the associated buffer space in such cases. The only difference >would be in the number of mbuf's (2?)reqd to keep the state/sequencing. >Streaming/UDP becomes a win, only if one knows beforehand that there will be >hundreds of connections for that particular RPC service. > Even when one is using TCP, the server can blow away the >connection (if the call is not in progress) and the client will simply >have to rebind (at some extra cost). The client RPC library should >provide the required transparency. This is in essence what NCS 2.0 does when running over a COTP. Note that there are some slightly tricky details having to do with what it looks like to the client in the case where it's launching a "request" over the connection at exactly the same time the server is closing the connection. The client might have a bit of a hard time distinguishing the network close from a server crash and can't know whether the call might have been executed and as a result not know whether it's safe -- i.e., meets the "at most once" rule -- to try to execute the call again at the same or another server. >The question is which is better - >pay huge cost at each call (discussed in earlier messages) OR pay >rebinding cost only when it is required? In my opinion, second approach >is better. What's the huge cost? Anyway, I wouldn't write off the rebinding cost. The TCP disconnect and connect are going to cost around 6 network messages. A busy server is going to have to close connections fairly regularly. (Of course NCS has some analogous overhead, but (a) it's lower because there are no close messages, and (b) the authentication setup is piggy-backed on the connection setup messages. Any authentication messages [challenge/response] would be yet more overhead messages in a TCP-based scheme.) Note also that I don't know how small you can reduce the "state/sequencing" info in TCP and I don't know how big your mbufs are, but in NCS/RPC we can keep literally just (the equivalent of) the state value, connection ID, and sequence number and we're using the generalized user space heap storage package that has few restrictions (e.g., in the granularity of the allocation size). We can keep a LOT of connections' state. >If I assume that both the threads are share the buffer space, then what do >you mean by "finite space for this buffering"? Also, the problem >described above seems more like a problem created by your implementation. It may seem that way, but I think it's not. It all falls out from the requirement of being able to cancel an in-progress call. I believe this makes it problematic to let the thread that's doing the unmarshalling actually be the thread that's reading from the connection. Thus, a layer of buffering is required. There is some fixed number of buffers you will be willing to assign to this task. Once this space is exhausted you have a problem. >Also, this problem should exist for CLTP too. How did you handle that case? RPC/CLTP can always discard data that it's read from the network because the sender never discards the data until the data has been ack'd by the receiver. When the receiver discards data, it doesn't ack it, inducing the sender to retransmit the data. (The details include the fact that the receiver will set his "offered window" value in the ack's it sends to zero.) >>We endeavor to send MTU-sized UDP packets and I don't know why you "hope >>not". > >Because I want good performance and that can happen only by minimizing >transitions between kernel and user land. Choosing a large number is >bad for stormy cases & retransmissions and leads to overrun for machines >with fewer mbuf's or less memory. Choosing a small number for UDP packet >size is bad for performance. Sun decided to go with a compromise - 8K >UDP packet size. I understand the penalties of kernel/user transitions, but I also understand the sending of 8K UDP datagrams to be a dangerous proposition since if any one IP fragment is lost, you are obliged to retransmit ALL the fragments since there's no mechanism for indicating which fragment was lost. >In NCS, is there a mechanism for size negotiation >for faster/bigger machines (lots of memory) which can easily handle >bigger size UDP packets? This is not a machine size/speed issue. It has to do with the network. If there are 5 gateways between the sender and the receiver and they're busy, the odds that one of the fragments will be lost could well be unacceptably high (esp. considering the cost of a loss will be the load of sending ALL the fragments again). I'll leave it for the Internet experts to make some "official" statement (if such a thing is possible), but I've gotten the strong feeling that depending on IP fragmentation is a bad idea. >Just curious - what do you guys do for NFS? Do you use 1K packets only? I'm the wrong person to ask about NFS. I don't have anything to do with it. >Actually, you can easily rest all of my fears by posting some performance >ratios for UDP/TCP (not numbers - I know they are confidential) I don't have numbers right now. >> In any case I take seriously >>my responsibility to produce a real evaluation of the wisdom of this >>approach once I'm in a position to do the evaluation. > >I hope that performance will be one of the criterian. Yes, but it's important to remember that it's only one of the criteria -- -- Nat Mishkin Cooperative Object Computing Operation Hewlett-Packard Company mishkin@apollo.hp.com