NELSON%MSU@PUCC.PRINCETON.EDU ("Doug.Nelson") (09/08/90)
A few weeks ago, there were a number of messages exchanged on these lists regarding the performance of IBM's VM TCP/IP. There were a couple inaccuracies in the statements made by the poster (Bill Simpson) which I pointed out privately to him. Since he has not chosen to post these corrections to the list, I will do so. In addition, I'll throw in a little defense for IBM, Spartacus, and Merit. This posting is primarily intended to set the record straight. Let me know if I have failed to do so. -------------- From Bill Simpson (Aug 23, 1990): > Well, I spent about 15 hours benchmarking NIS.NSF.NET against other > machines on the internet, carefully documenting the packet traces, > and passing my observations on to IBM. I did this at my own expense, > as a matter of curiousity, because I thought that the folks who were > sending the messages were "real programmers" who were really interested > in getting it right (a Jay Elinsky was also involved). I'm sure IBM has "real programmers" working on VM TCP/IP. My impressions of the product, in general, are very favorable. I would also assume that IBM would address Bill's reported problems, but it may be necessary to report them through the normal support channels, which Bill has no direct access to. In addition, his packet traces were obtained on a low speed dial-up line, and are likely to be less than complete without some parallel tracing on the Ethernet segment attached to the IBM system in question. > The NIS.NSF.NET hardware was contributed by IBM, and I understand > that the software is the latest IBM release. My benchmark was carefully > conducted for a direct comparison of the TCP/IP implementations, > under the worst possible circumstances to make implementation flaws > stand out clearly. The failures demonstrated are plainly of the > host, and not of the environment. Without seeing the Ethernet packet trace, I doubt if the environment (one-of-a-kind software supporting the Ethernet to async IP gateway, over a network with a maximum packet size of 240 bytes) can be ignored. > As to the "very satisfied" customers, I will point out that MSU has > not yet installed the VM TCP/IP product, even though it came "free" > with other products. And I occasionally read "IBMTCP-L" on bitnet. > 'Nuf said. Hardly. MSU *has* installed the VM TCP/IP product, and is using it as our production IP software on two of our IBM systems. We are planning to do so on our main 3090 system as well, but haven't, strictly due to a lack of staff time, and not to any dissatisfaction with the product. Also, we are running another TCP/IP package (KNET, from Spartacus), which handles our production load at the moment. ------------ On to a message from Bill giving specific throughput results: > CONCLUSION: > IBM-FAL for VM runs at 1/2 the speed of many of its competitors. > It fails to meet several host requirements, not just for TCP/IP > but for FTP and SMTP as well. Under a worst case scenario, Bill may have uncovered a problem with VM TCP/IP and its retransmission algorithms, unless they were doing the best that they can, and a router was discarding packets with a less than optimal algorithm. How many systems do you know that don't do something less than optimal under some worst case scenario? Bill never listed the shortcomings w/r/t host requirements (hopefully, he did for IBM). Maybe he can do that now for the list. > Unfortunately, the only product which I am aware of with worse performance > is Spartacus KNET, also for VM. To defend Spartacus, they have addressed these performance problems in their latest release, by implementing slow start and exponential backoff. This is another update that has been delayed due to lack of staff time. > For the good of the user community, I would recommend to Merit that > the RFC mirror be moved to a more capable machine. How many other users of the Merit anonymous service have experienced problems of this nature? If this is only affecting a single user, I'd hardly see that as a mandate for switching operating systems. -------------- On to a respondent's comments (Hank Nussbacher, Aug 25, 1990): > I think in all fairness, you first need to identify yourself. I have > looked at the Bitearn Nodes database ans was unable to find you listed > as a contact person at MSU. Can you please describe your position at > the MSU computer center. Bill works for an off-campus user of MSU Computer Lab services. I am not aware of any other affiliation between Bill and MSU. > >Let's keep the record straight: the folks at Merit *are* "real programmers > >... > > I cannot attest to how good the people at Merit are, but I can to the > quality of the people at IBM behind the Tcp/Ip package. I'll vouch for the Merit folks as well - they are a competent programming staff. ------------- And a response to Bill's response (dated Aug 27, 1990) to comments from Robert Craig (dated Aug 26, 1990): > > For the record, we run FAL 1.2.1 on a 4381 with a BTI ELC. > > We began with an 8232 and performance *was* abysmal. > > It is much better now... > > For the record, I don't know nor care about the specific *hardware* > which was being compared. (How was I to know, with no host DNS > record, and no announcement in the FTP welcome message.) > The hardware is whatever IBM chose to demonstrate their best to the > internet community. I merely noted the machines which were being > compared, in very little detail, so that others would understand > the basis for comparison (more hops, heavy load, etc). But yet, Bill is willing to make sweeping generalizations about IBM's software, without understanding anything about the hardware. This is one of the reasons I originally ignored his postings, as much as I was able to do so. I don't think IBM placed a 4381 with an 8232 in Ann Arbor to "demonstrate their best". I suspect most of the Internet community is paying much more attention to other IBM contributions to the NSFnet backbone project, such as router performance, network management tools, T1/T3 capabilities, etc., than on a small, ancillary system which maintains some useful network information. > I also have a number of messages saying that they have given up > entirely on NIS.NSF.NET, including one with a much more direct > connection than I have. Also, messages thanking me for the posting.... Have any of these other people gone public? Have they attempted to pass their concerns on to IBM or Merit, and if so, what response did they get? ---------------- Doug Nelson nelson@msu.edu Network Software Manager nelson@msu.bitnet Michigan State University