sob@harvisr.harvard.edu (Scott Bradner) (09/13/90)
I was more than a little surprised to hear the other day that I was on contract to Wellfleet. At least that was what some people were saying. Well it is not true. To do anything like that might put a bit of a taint on the testing results that I've been reporting. :-) Full disclosure time: I have done & continue to do some consulting, but I have not done and am not doing any with any of the vendors of devices that I'm now testing. (Hey, I even chose not to buy any cisco stock, dumb move?) Like with many tall tales there may be a slim germ of truth somewhere in it. Here is the story I tell the router vendors that call & offer to send me a router for testing. Back about 2 months ago I got a call from Proteon saying that they had fixed the problems that we had found last year & had improved the throughput by quite a bit, and could we give the new code a test? (This was on a voice mail message.) The same day I got a call from someone at Wellfleet asking, in effect, why was I not doing a new round of tests since it had been a while since the last batch. say I: "Well I don't have good enough equipment to do the tests required & there ain't no way to get the kind of funding required here at Harvard" said he: "Well we can donate/loan you a test setup we have built." say I: "Humm", (then to self, "I don't see any other way to do something before Interop but what would the other vendors think?") "Why don't I come out and talk about it?" so I did. What came of it was an informal agreement (nothing on paper) for Wellfleet to loan Harvard some equipment & agree to have the person who had done their software make "a few" changes/additions to what he had done to make the tests more like what the Benchmarking Methodology Working Group had been discussing. Wellfleet also agreed that they would work out a way that other organizations could get access to the equipment in the future if there was enough interest. Access to the source code was also offered. ( an aside, the test packet generator is a Wellfleet router with special software. It is setup with pairs of ethernet ports. One port of each pair is used to produce packets right fast, the second port is used to produce the "keep alive" traffic required to convince some protocols that there is a node out there.) Now, I fully expect that Wellfleet is not 100% altruistic in their offer, they expect to have their box do quite well in the tests & the results will look better coming from a independent source than they would from Wellfleet. It might & it might not (I've not bashed it yet, I'm starting in on it tomorrow.) but the tests will be fair and open. The same tests will be run on all routers. Also, Dan Lanciani & I (mostly Dan) have come up with a "simple" hardware design for a card to put in a PC along with a NI5210 to replicate the total functionally of the Wellfleet hardware. We will produce the card asap and redo some or all of the tests with the new hardware as a final check. Software will be written to automaticly run a series of tests and produce the results as the BMWG RFCs will describe. (The source to the test software and the artwork for the board will be placed for anonymous ftp.) I've told this story to all of the vendors when they call wanting to test a router, none of them have expressed reservations about the arrangement. I'm sorry that I did not mention it in the 1st posting but I was still working out the details. It was a mistake to not wait. But hindsite is easy... Scott PS - bashing update here to be tested or dates set: Proteon (P4200 & RIG) Wellfleet 3com cisco NSC Timeplex and (I think) BBN
kwe@buit13.bu.edu (Kent England) (09/14/90)
In article <4160@husc6.harvard.edu> sob@harvisr.UUCP (Scott Bradner) writes: >I was more than a little surprised to hear the other day that I was on >contract to Wellfleet. At least that was what some people were saying. > >Well it is not true... > >Like with many tall tales there may be a slim germ of truth somewhere >in it. > >Scott > Well, I read your story about Wellfleet donating some test equipment for your router bashing and I think you're wrong. There isn't even a slim germ of truth indicating any conflict of interest. I think what you have found is an unusual level of general excitement on the part of commercial people who smell a large new market and think Scott Bradner and his router bashing might be the key to their getting a piece of that market. You're in the spotlight, but I, for one, feel more comfortable about that than what we had before; which was a bunch of vendor engineers and marketing people pushing, shoving, yelling, and lying about their bridge and router performance. At this point, we are beginning to see some semblance of order, sort of like when the teacher walks into the classroom and the spitballs stop flying. --Kent