bobj@MCL.UNISYS.COM (Bob Jones) (02/08/88)
I recently saw a message from Hal Peterson at Cray (hrp%ishmael.cray.com @uc.msc.umn.edu) regarding the DCA protocol laboratory test tools which were being beta tested and Cray submitted to those tests. I have just one rebuttal to a comment made by Hal. The tests themselves do not take a significant amount of time to run. On the contrary, the complete protocol suite can be tested fully in one or two days - providing the remote drivers are working properly and the network is not giving us problems. We do have direct dial up to the facility also. The problem that Hal alluded to in terms of time is really that of trying to correct the bugs found and being retested. That is what took so long. If the drivers are working and net is ok, the following times are generally common for testing the protocols: TCP - 2 hours IP - 6 hours (this is being reduced to approx 2 hours) FTP - 1 hour Telnet - 1 hour SMTP - 1 hour I want to reiterate that this is the actual testing time and not the attempts to go off and correct problems and come back up for testing. bobj
Mills@UDEL.EDU (02/08/88)
Bob, Holy Cow. If the network is okay almost anything will work, even ISOspeak. What you want to know is how the implementations work when the net ISN'T okay, when the remote drivers are buggy and at arbitrary times during the day when packets are falling off the Earth and so forth. I would give the okay-network test only a Journeyman Certificate, hardly more worthy that an Ethernet Certificate. The heavy dudes claim Mailbridge Certificates and the real wowsers claim the most exclusive of all, the NSFNET Medal. Dave
bobj@MCL.UNISYS.COM (Bob Jones) (02/08/88)
Dave, thanks for comments. The test system is robust enough to recover from net-induced perturbations. Thats not a real problem. What has happened from our experience is that we spend a lot, (lots!) of time doing recoveries and not getting on with real testing. Also the test system has a rather extensive corruption capability to test the implementations without the network adding to the pot. I believe we can claim the Mailbridge Certificate. The NSFNET Medal? I refuse to die on my sword. bobj
kleonard@PRC.Unisys.COM (Ken Leonard --> kleonard@gvlv2@prc.unisys.com) (02/09/88)
In article <8802071346.aa05817@Huey.UDEL.EDU> Mills@UDEL.EDU writes: >Bob, > >Holy Cow. If the network is okay almost anything will work, even ISOspeak. >... What Bob meant by "network OK" was the simple ability to keep ANY connection open, for the test-control channel, for even 10 minutes or so! You ain't seen nuthin, in the way of netcrap, until you've been on the wrong side of an imp with experimental (pre-alpha grade) code working thru a gate with equivalent code. Otherwise, the point is correct, and the test tools in question not only work thru garbage--they can make garbage when it is needful. Regardz, Ken
Mills@UDEL.EDU (02/09/88)
Ken, Point taken, with the exception of your comment on the depth of excrement in the swamp. Having swept the alligators from the wrong end of the IMP, you are cordially invited to swat the lizards on the NSFNET Backbone and dependent tributaries. It's much like a municipal waterworks system going out to the users, but also like a municipal sewer system coming back. Soon you may apply for a Merit Badge Gongfermer Grade for demonstrated competence in reptile extermination, which would seem a lot more fun than creepy-crawling on the ARPANET and EIN. Dave
hrp@windsor.CRAY.COM (Hal Peterson) (02/12/88)
Bob, You are correct and I was being too flip with my wording: the tests are indeed quickly RUN. What I meant to say is that it takes a long time TO BE ABLE TO RUN the tests, because of the time necessary to develop a ``remote driver'' program to run on the machine under test. [The remote driver sits on top of the protocol under test and responds to commands and data from the central test site, opening and closing connections, echoing data, and so forth.] Our experience indicates that anyone who wants to run these tests should allocate several people and several months to the task. On a machine for which remote drivers are available, that time would be shortened to a couple of weeks for one person. We've been at this for a year and a half now and we have yet to run all of the tests. This has happened for a number of reasons: some are unique to Cray, some are political, and some are a result of working with a system in progress, whose specifications have been changing while we work. I am still convinced that our time is being well spent, that this is a worthy effort, and that users of the production version of the test suite will have a much easier time of it than we have. -- Hal Peterson / Cray Research / 1440 Northland Dr. / Mendota Hts, MN 55120 hrp%hall.CRAY.COM@umn-rei-uc.ARPA ihnp4!cray!hrp (612) 681-3145
CLYNN@G.BBN.COM (02/13/88)
Bob, Could you summarize the sorts of problems that the testing suite has found? I think that the list of "common" bugs, maybe sorted in order of decreasing frequency, would be helpful both to those who are doing implementations and to those who are trying to make the specs clearer. Charlie
JFISHER@USGSRESV.BITNET (James R. Fisher) (02/13/88)
My eyebrows recently raised on seeing reference to the NSFnet Medal for implementations. We are thinking of joining NSFnet and I would VERY much like to hear of known defects and perceived problems with NSFnet, AND of any implementations which are worthy of the Medal. Flames welcome.
bobj@MCL.UNISYS.COM (Bob Jones) (02/16/88)
Hal, thanks for comments. I am hoping some industrious laboratory will offer the remote drivers in some generic for that can be quickly adapted to any machine environ (with generic hooks?). If it will be of any comfort to you, we are at a point where all of the specs are firm now. Thanks to you, we have learned a lot and your persistence has helped us to debug our test system in the process. Cray deserves a hand! bobj