vjs@rhyolite.wpd.sgi.com (Vernon Schryver) (02/14/91)
In the discussion about UDP checksums, someone wrote to me: > I have no idea if [name censored] would reply to a bid that required RFC 1122 > conformance and claim conformance. They don't conform. As you might guess from my return address, I've no shortage of bids, RFQ's and RFP's that have either checklist items for "conforms to RFC 112[23]" or copies of the entire checklists from the RFCs. Not long ago, a room full of people concerned with such things sat down to determine the company's official answer. We started at the beginning, considering each SHOULD, MAY, CAN, WILL, WON'T, OUGHT, COULD, and ONLY-IDIOTS-WOULD-DO-OTHERWISE. We filled many hours with "gee, of course we do/don't/would/wouldn't do that, but let's check...see the code there ...but look...ok, add to the list for verification." After lots of this powerful fun, we paused to estimate when we'd be finished making the list and how long the list would be. The encouraging answer was that the list would be finished in only a few months and would be only a little bigger than the RFC's. We thought about the effort and considered the size of the existing bug lists for the areas covered by the RFCs. We individually compared the personal satisfaction to be gained from completing the list with other pursuits like making TCP/IP/FDDI run 30% faster. Then we considered the gain to our customers from having less fast, less feature-full, buggier but "conforming" implementations. The result is that we will make an honest effort to perpetually look through the RFCs for real and potential problems, and to add discovered or reported conflicts to the bug lists to be fixed with the other bugs. However, we won't claim conformance. We'll just provide the linage of our source. This decision did not make our sales people cry with joy. In other words, RFC-1122 and RFC-1123 are interesting reading, great guides for writing or fixing code, and wonderful tools for adjudicating blame among vendors. They are terrible conformance standards, at least at this non-military-standards sort of shop. I know the response many will have is that 27 gov. agencies and 63 industrial consortia are developing test facilities. Old and new experience with test suites such as SVVS and POSIX, to name only two, make me skeptical of that panacea. Are there different perspectives from others, outside the gov. agencies, consortia, etc who want to sell us and our customers their seals of approval? Vernon Schryver, vjs@sgi.com
barmar@think.com (Barry Margolin) (02/14/91)
In article <85285@sgi.sgi.com> vjs@rhyolite.wpd.sgi.com (Vernon Schryver) writes: >As you might guess from my return address, I've no shortage of bids, RFQ's >and RFP's that have either checklist items for "conforms to RFC 112[23]" or >copies of the entire checklists from the RFCs. From my minimal experience with RFQ's (one big project about seven years ago), I don't think such items are a critical factor. An RFQ typically contains quite a few requirements for a facility, and it's rare that any vendor can meet all of them. For instance, a TCP/IP RFQ might specify conditions on performance, usability, and conformance to a standard. Some respondents will do better jobs than others on various pieces of the requirements; implementation A may conform to the standard completely, implementation B may have a more complete API, while implementation C may perform better. The customer will have to decide which aspects are more important, and decide among the respondents on this basis. A vendor should look upon host requirements RFCs as additional guidelines, and add them to their to-do lists. They must still reconcile these with the problems and suggestions from users, marketing requirements, etc., and prioritize the list. When responding to the RFQ, all you can do is hope that your strengths in other areas outweigh your deficiencies in others. Of course, if there *is* a vendor that meets all the requirements, you may be out of luck, but that's the nature of competitive bidding and the free market. -- Barry Margolin, Thinking Machines Corp. barmar@think.com {uunet,harvard}!think!barmar