Jerome_V_Vollborn@cup.portal.com (10/21/89)
As the lead article in this discussion points out, the U.S. Government has finally discovered that much of the software it pays for doesn't work. Unfortunately Congress makes laws and the President enforces them using whatever force he considers necessary. As more computers get distributed to more people, the software problem will get larger and the voices asking for some action will get louder. If you don't want some Government functionary setting standards for how you will do your job, you personally will have to do something about the software problem. There are four basic choises: furnish the source code, internal documentation and test cases to all buyers so they can maintain the software themselves; furnish a written money back guarantee if any problems are found as long as any buyer is using the software; furnish the results of technical audits of your company's development practices; and wait for the Government to mandate some solution developed by [name your least favorite political group]. Personally I favor the third solution because I think that it is salable to our various companies' management and it can be implemented easily. The independent verification and validation (IV&V) contractors used by the DoD could be used for technical auditors. The major hasle is what to audit. The documentation mill specified by the DoD in Mil-Std-1679 and 2167 is both inflexible and inadequate. It does not recognize the difference between large and small software systems, or the need for different approaches to new problems. I would propose that the audits look at four areas: management, standards and practices, infrastructure, and personnel. As demonstrated in Fredrick Brooks' Mythical Manmonth the most vital ingrediant for good software is first and second level management that understand the software development problem. Twenty years later, nothing has changed: we still need management that understand software development and we still don't have it. Hugh amounts of money are being committed to the purchase or construction of software by people who are incapable of exercising due diligence. Many companies that develop software systems either don't have software development standards and practices (S&P's) documentes or those documents are ignored. Many of the S&P's were last updated when COBOL was developed. For these documents to do anyone any good the developers need to believe the procedures in the S&P's will help them develop good software. The auditors would check that the S&P's reflect current state of the art practices and the developers use those practices and procedures. Perhaps the most neglected part of the software development world is the infrastructure for testing, QA and configuration management. For too many companies the testing is ad hoc with no assurance of complete coverage or unsurprising behavior. Many companies can't tell the difference between testing and QA. I know of one company whose configuration management is a pile of boxes of floppy disks in the corner. The auditors would check for testing practices, QA checks, version control, change control, corrective action plans, and consistant application of the above. The auditors would also check the backgrounds, knowledge level, and continuing education of the developers. The knowledge level of the developers would probably be checked using the ACM self evaluation tests. Continuing education would be checked by participation in technical societies (ACM or IEEE Computer Society) or college courses. The background check would look for work on delivered products or college degrees. While this form of audit would not be easy the first few times, it would assure that the companies have the ability to develop software products. This auditing process is a future event so I am trying to get my company to look at the credentials of software suppliers. I am using a report from the Software Engineering Institute at Carnagie Mellon University entitled "A Method for Assessing the Software Engineering Capability of Contractors". The method rates the technical and managerial (including infrastructure) capabilities of companies. I feel that I can't recommend the purchase of software products unless I feel confortable with the software engineering capabilities of the producer. I recommend that you think about this and only recommend the purchase of software from companies you feel you can trust. Jerome Vollborn (Jerome_V_Vollborn@cup.portal.com or uunet!lci386!jerome) management.
diamond@csl.sony.co.jp (Norman Diamond) (10/23/89)
In article <23226@cup.portal.com> Jerome_V_Vollborn@cup.portal.com writes: >The auditors would also check the backgrounds, knowledge level, and >continuing education of the developers. Fine. >The knowledge level of the developers would probably be checked using >the ACM self evaluation tests. Not so fine. Read the copyright notices on those things. I hope ACM would have the integrity to sue abusers. >Continuing education would be checked by participation in >technical societies (ACM or IEEE Computer Society) or college courses. Excellent. -- Norman Diamond, Sony Corp. (diamond%ws.sony.junet@uunet.uu.net seems to work) Should the preceding opinions be caught or | James Bond asked his killed, the sender will disavow all knowledge | ATT rep for a source of their activities or whereabouts. | licence to "kill".