srctran@world.std.com (Gregory Aharonian) (05/22/91)
I am giving a talk during the first week of June on cost/benefit analysis and economics as applied to software reuse and engineering. One example I plan to use is the Software Engineering Institute (SEI) Distributed Ada Real-Time Kernel (DARK). My contention is that the costs in terms of tax dollars far outweighed the benefits the country received in terms of any new original information from the project' results. I am interested in what others think about the cost and benefits of this project. The DARK project was started a few years ago at SEI('88 or '89), funded by the Air Force's Electronic Systems Division. I do not know the exact amount of money allocated, but assuming that the effort required at least four man-years, had a small room of equipment to test the kernel, and average overhead rates, I figure that it cost at least 325,000 dollars for this project (four man years @ 50000, equipment at 50000 and overhead at 75000). "The purpose of DARK is to be a mechanism for supporting the execution of distributed real-time Ada applications in embedded computer systems. It provides a solution to scheduling and distributing tasks without modifying the Ada language or vendor-supplied runtime systems. The goal of the DARK project is not to develop any radically new technological breakthroughs, but to demonstrate that well-understood, familiar and reliable methods of building real-time, distributed systems could be done in Ada as well as any programming language". (This paragraph comes from some SEI report abstracts). After developing and experimenting with the code, the SEI tried licensing the kernel to industry in 1990. I assume that this effort failed, because in 1991, the Department of Commerce's National Technical Information Service (NTIS) announced it was selling the DARK system (the cost is a few hundred dollars for a magnetic tape and documentation). To the best of my knowledge, this is the current official status of DARK. I have not examined the code, but I assume that like other Ada code developed at SEI, the quality of the Ada source code is very high and reliable. As a background note, before, during and after this project, there were (and still are) at least a half dozen private ventures concerning real time Ada kernels, some from compiler companies like Alsys, some from real time kernel vendors such as Ready Systems, a few university efforts and at least one European Community funded project. My question is, how did (or how could) the country benefit by this investment? If the goal (as they stated in their documents) was to prove that it is possible to build decent real-time, distributed Ada systemss, then the DoD and the country could have learned this by sitting back and watching the activities of industry developments with Ada kernels, and actual embedded projects, for free. A low cost effort to summarize the results of existing activities (like the CECOM project) would have been sufficient. For this goal, the net cost/benefit of DARK is negative. An alternate goal might have been to have some SEI staff gain experience in this area. However the DoD could have sent them to one of these companies for training, at a much lower cost than what the DoD funded the SEI. Also, there was a rather nice ARMY CECOM effort (finished in 1989) that studied the problems of developing real time Ada code, and also offered Ada source code examples, as well as over 1000 pages of documentation, at no cost (contact Mary Bender at 201-544-2105). Again, for this goal, the net cost/benefit of DARK is negative. If a goal was to provide the country with an Ada real time kernel to insure that people use Ada (which must have been an unstated goal since they did try to license DARK to industry), then the venture failed because vendors developed their own real time kernels, and recently most agreed to offer or integrate in Ready Systems' real time kernel. Again, the cost/benefit of DARK is negative. Besides, the government shouldn't be in the position of developing products. If a goal was to provide a freely available, reusable piece of Ada real-time source code, then someone screwed up by having DARK available thru the NTIS, instead of being placed on the public archives on Internet. (It may be that DARK is available from SIMTEL, but to the best of my knowledge, DARK is not available by anonymous ftp from any Internet site). Again, the net cost/benefit of DARK is negative. If a goal was to provide an example of a self-contained project whose costs and benefits could be analyzed as an example of cost-benefit analysis, a project of immeasurable benefit to the Ada community, well, I haven't seen a single report from SEI (or an independent third party) performing such an analysis. In fact, in March of 1989 the General Accounting Office, in a report (GAO/IMTEC-89-9) on the general progress of Ada inside the DoD, complained (and the DoD concurred) that no one is collecting the data to justify the claims made on the benefits of reuse and Ada. The DARK project might have been a good opportunity to do so. My contention is that while the country benefitted from this project, the costs to the country were much greater, and that this analysis could have been performed before the project had ever started, by considering the social and economic aspects of software development and reuse on a national scale. All of the benefits could have been obtained at a much lower cost. There may be other benefits to this project, or that the costs were lower, but they escape me at the moment. Now 300,000 dollars is not much, but if similar waste is going on elsewhere, it soon adds up to a lot of tax dollars. I can believe great things about Ada, but I would like to see some numbers, especially when I am paying for it. (Of interest is that Ready Systems got out of the real-time Ada kernel business because there was no profit in it as a product. Finding out why would be a beneficial SEI study, and also why there are so few Ada library companies and fewer that survive.) I'd appreciate any comments of my analysis, either posted, or emailed to me at srctran@world.std.com (if you want privacy). I'll post the results. Attached are comments from one reviewer. Thanks. Gregory Aharonian Source Translation & Optimization "The best tools to use to defend are the ones being defended" ================================= ****************************************************************************** Comment: This is interesting to me because of my position at the *************, which is a state supported academic/industry/government consortium with the avowed goal of helping improve the state of the art and the state of the practice in software engineering, especially as it might benefit software-related industry. A smidgin more background about the center: The state provides `matching' funds against industrial contribution, with the goal (among other aspects of our mission) of making us a `cheap date' for industrial members to support or collaborate on ventures they might never do all by themselves. So while we aren't set up at all like SEI, there are rough similarities. I bring this up because DARK is along the same model of projects we might consider taking on: DARK probably LOOKED like * something worth doing (if nothing else, as a proof of concept that such things could be done in Ada.) * a somewhat risky project (people probably weren't very confident that it could be done.) * something that industrial teams might not do on their own (because of the risk). All of these apply just about uniformly to most things we do take on. But the main thing we do differently is that we NEVER take on a project without industrial involvement. Sometimes we feel that this is a straightjacket, since it's hard to convince places to support some things that we think need to be done, but don't look relevant enough over the usual 1-3yr horizons that they operate under. I think this is a difficult issue: DoD/DARPA, NSF, et al probably should support software technology research for which researchers can make a good case benefits computer science and engineering long term, but not if industry will develop it anyway. How can you know this in advance? Conclusions: 1. DARK was probably a waste; compounded by poor midcourse decisions like not generally releasing the software. 2. Had people at SEI been a little smarter they might have realized this before it happened. But that this was probably close enough to the borderline of the kinds of things that SEI should take on that I have a hard time assigning much blame. I might have made the same mistake.
spray@convex.com (Rob Spray) (05/22/91)
One concern I have always had about DARK is how it got funded targetted to a processor that is not available in a radiation-hardened configuration. Several embedded systems I am familiar with are required to use full MIL-SPEC processors. My experience with several distributed systems is that despite all good intentions, they are very processor dependent. I suspect that DARK would have been much harder on a processor like the Intel 80286 or 1750A that were the rad-hard choices that had an Ada compiler when DARK was started. Now, in that time-frame, the state-of-the-art for compilers and runtime systems for the 68K was far ahead of the other machines. Also, workstations like the SUN and Apollo were available for hosts and targets. So the costs and risks to DARK's preceived success were lower with the 68K. However, the result is a system that, IMHO, has minimal chance of being ported to a rad-hard environment. Tell me I'm wrong. --Rob Spray --spray@convex.com --214/497-4110 Disclaimer: This is my own cynical conjecture, and may not reflect the opinions of employers past or present. Your opinions may vary. Ted Holden's opinion will doubtless be different. But then I would bet I've worked on more rad-hard distributed Ada systems than he has.
vestal@SRC.Honeywell.COM (Steve Vestal) (05/22/91)
In article <spray.674858011@convex.convex.com> spray@convex.com (Rob Spray) writes:
Rob> One concern I have always had about DARK is how it got funded
Rob> targetted to a processor that is not available in a
Rob> radiation-hardened configuration. Several embedded systems
Rob> I am familiar with are required to use full MIL-SPEC processors.
Does mil-spec mean that a processor must be radiation hardened, or just that
it be class B, certain temperature range, certain packaging, etc.? "Radiation
hardened" isn't a yes/no thing; there are several parameters and several
degrees to radiation hardening. For example, of the two processors cited, I
am aware that there is a very rad-hard version of the 1750A. As far as I
know, there are versions of the i80x86 family that are perhaps less sensitive
to radiation in some ways than commercial parts (e.g. maybe tolerate somewhat
greater total dose) but may or may not have significantly better numbers in
all respects (e.g. sensitivity to single event upset). My speculation would
be that radiation hardening requirements for communication sattelites in high
orbit, shuttle or space station in low orbit, and aircraft, all vary. Also,
this is a systems issue, since one can trade shielding against the radiation
insensitivity of the circuitry used.
There are Ada development environments and executives for rad-hard computer
systems, but I don't think a high degree of radiation insensitivity is a
universal requirement on all military systems.
Steve Vestal
Mail: Honeywell S&RC MN65-2100, 3660 Technology Drive, Minneapolis MN 55418
Phone: (612) 782-7049 Internet: vestal@src.honeywell.com
spray@convex.com (Rob Spray) (05/22/91)
In <1991May22.143229.24667@src.honeywell.com> vestal@SRC.Honeywell.COM (Steve Vestal) writes: >In article <spray.674858011@convex.convex.com> spray@convex.com (Rob Spray) writes: >Rob> One concern I have always had about DARK is how it got funded >Rob> targetted to a processor that is not available in a >Rob> radiation-hardened configuration. Several embedded systems >Rob> I am familiar with are required to use full MIL-SPEC processors. >Does mil-spec mean that a processor must be radiation hardened, or just that >it be class B, certain temperature range, certain packaging, etc.? "Radiation >hardened" isn't a yes/no thing; there are several parameters and several >degrees to radiation hardening. For example, of the two processors cited, I There's a MIL STD (5400 Class 1? I can't remember) that specifies hardness, altitude, temperature range etc, etc. It's my understanding that most air/spaceborne MCCR systems must use processors that meet that Standard (e.g. Loral(Rolm) Hawks, AN/UYK-xx, ap101, 1750A etc) >orbit, shuttle or space station in low orbit, and aircraft, all vary. Also, >this is a systems issue, since one can trade shielding against the radiation >insensitivity of the circuitry used. Not always true. Most RFPs specify the Standards to be met. Shielding non-qualified parts is not always an option. >There are Ada development environments and executives for rad-hard computer >systems, but I don't think a high degree of radiation insensitivity is a >universal requirement on all military systems. Agreed, but a significant number of systems have that requirement, so the choice of the 68K for DARK, effectively ignored many issues faced by builders of such systems. The 68K has a reasonably orthogonal architecture and good memory management scheme for distributed processing. From a compiler writer's view, lessons learned on a VAX transfer to the 68K. So the 68K had the best Ada environment to implement DARK. (I suspect that proposing a VAX target would have been even more unacceptable.) However, there are no rad-hard 68Ks, so technology transfer of DARK to a significant percentage of the intended systems is hindered. --Rob Spray --spray@convex.com --Previous disclaimer is still in full effect.
hinton@gca.UUCP (Edward Hinton) (05/25/91)
In article <SRCTRAN.91May21140221@world.std.com> srctran@world.std.com (Gregory Aharonian) writes: > > I am giving a talk during the first week of June on cost/benefit >analysis and economics as applied to software reuse and engineering. One >example I plan to use is the Software Engineering Institute (SEI) Distributed >Ada Real-Time Kernel (DARK). My contention is that the costs in terms of tax >dollars far outweighed the benefits the country received in terms of any new >original information from the project' results. I am interested in what others >think about the cost and benefits of this project. First question: What is the scope of your talk? If it is limited to software and reuse and engineering in low-risk projects, then DARK is at best borderline as an example. If it includes high risk, how do you propose that cost/benefit analysis be applied? Are you assuming predictability and high success rates for high-risk projects? This would be contradictory. > After developing and experimenting with the code, the SEI tried >licensing the kernel to industry in 1990. I assume that this effort failed, >because in 1991, the Department of Commerce's National Technical Information >Service (NTIS) announced it was selling the DARK system (the cost is a few >hundred dollars for a magnetic tape and documentation). To the best of my >knowledge, this is the current official status of DARK. I have not examined >the code, but I assume that like other Ada code developed at SEI, the quality >of the Ada source code is very high and reliable. Second Question: Is SEI applying what it learned to other SEI projects? If so, failure may not be an appropriate word. > An alternate goal might have been to have some SEI staff gain >experience in this area. However the DoD could have sent them to one of these >companies for training, at a much lower cost than what the DoD funded the SEI. Third question: What kind of experience is the best teacher, experience doing, or experience watching and listening? >I bring this up because DARK is along the same model of projects we might >consider taking on: DARK probably LOOKED like > * something worth doing (if nothing else, as a proof of concept that > such things could be done in Ada.) > * a somewhat risky project (people probably weren't very confident that > it could be done.) > * something that industrial teams might not do on their own (because > of the risk). > Last question: Would inductry have teamed up on this? My point is, looking at cost/benefit analysis before the fact on high risk technological projects must take into account the POTENTIAL advancements, be they new technology, or wider acceptance of existing technology. The goal of proving something could be done seems reasonable compared to the cost (in my opinion). In fact, any software project of sufficient technical complexity to warrant government funds at all probably would cost at least the estimated $300,000 cited. Industry can and should do the easy cases if they are worth doing. In fact, I would fault SEI for not doing more on this project if it was worth doing in the first place. Their sights were not set high enough. As an analogy, getting to the moon probably provided little benefit to the country for the cost, but such high sights resulted in tremendous technological efforts which were well worth the cost in return for the other uses the technology was applied to. I don't know much about DARK itself, but I think SEI research apart from industry is too lightly being criticized. (Aside: I'm not generally in favor of rampant government spending, but I do think we can learn from the Japanese about the benefits to the country as a whole of government spending in high-risk technological areas.)
griest@ajpo.sei.cmu.edu (Tom Griest) (05/30/91)
In Article 5010 Gregory Aharonian writes: > I am giving a talk during the first week of June on cost/benefit > analysis and economics as applied to software reuse and engineering. One > example I plan to use is the Software Engineering Institute (SEI) Distributed > Ada Real-Time Kernel (DARK). My contention is that the costs in terms of tax > dollars far outweighed the benefits the country received in terms of any new > original information from the project' results. I am interested in what others > think about the cost and benefits of this project. First, I would not dream of giving a talk on something that you appear to have so many fuzzy facts about. If anyone in the audience knows about DARK you are likely to get ripped apart. Even if you get lucky and the audience accepts your guesses, because they are just guesses you are just as likely to give the wrong impression as the correct one. > The DARK project was started a few years ago at SEI('88 or '89), funded > by the Air Force's Electronic Systems Division. I do not know the exact > amount of money allocated, but assuming that the effort required at least four > man-years, had a small room of equipment to test the kernel, and average > overhead rates, I figure that it cost at least 325,000 dollars for this > project (four man years @ 50000, equipment at 50000 and overhead at 75000). If you want to do a true cost analysis, I suggest you find out exactly how many person-hours were spent on it and how much it cost (and who paid the bill). You must also study all of the spinoffs that resulted from this work if you want to be accurate. > [STUFF ABOUT DARK's PURPOSE DELETED] > After developing and experimenting with the code, the SEI tried > licensing the kernel to industry in 1990. I assume that this effort failed, > because in 1991, the Department of Commerce's National Technical Information > Service (NTIS) announced it was selling the DARK system (the cost is a few > hundred dollars for a magnetic tape and documentation). To the best of my > knowledge, this is the current official status of DARK. I have not examined > the code, but I assume that like other Ada code developed at SEI, the quality > of the Ada source code is very high and reliable. Licensing software source is an extremely tricky business. Since the mission of the SEI is to make information available for wide dissemination, I am not surprised that they decided to make data available via NTIS. I hope your not trying to imply that the project as a whole was a failure because they are providing the results through NTIS! Virtually all government funded projects provide their results through NTIS (or DTIC for classified data). > > As a background note, before, during and after this project, there > were (and still are) at least a half dozen private ventures concerning real > time Ada kernels, some from compiler companies like Alsys, some from real time > kernel vendors such as Ready Systems, a few university efforts and at least > one European Community funded project. Don't forget that DARK is specifically for DISTRIBUTED systems, and NOT simply a message passing mechanism on top of a network. However, many other researchers are also working on distributed operating systems. > > My question is, how did (or how could) the country benefit by this > investment? As with all research, the results can be very instructive to those WHO GET THE RESULTS AND STUDY THEM. Commercial vendors doing research, as you mention above, are very reluctant to publish the whole truth of their work (there is a financial disincentive unless it has been patented). Universities do provide useful information and sometimes at a lower cost because they occasionally "make do" with limited resources and often depend on graduate students to provide slave labor. However sometimes the results of the work reflects the inexperience of these graduate students. The SEI pays enough to get experienced people who are more likely to see subtle details that effect the results of studies than less experienced students. [ University folks please flame to /dev/null ... I think the vast majority of Univ. research is fine, but some of it could benefit from more review by industry. There is a need for both university and industrial researchers. Note also that SEI is run by CMU.] Since Ada is going through a revision process, the results of the SEI work could influence the Ada9X process. Alternatively is could provide guidance for others wanting to do similar projects with Ada. Even when research "fails" to meet it's objective, careful review of the work often provides ideas to other researches which benefit from the results of the prior projects. It's rather silly to suggest that because the results of a research project weren't watched like CNN during the first days of the Air War that the work was a waste. It is hard to predict the results of research and often the full results are not known for many years after the project has completed. > If the goal (as they stated in their documents) was to prove that it > is possible to build decent real-time, distributed Ada systems, then the DoD > and the country could have learned this by sitting back and watching the > activities of industry developments with Ada kernels, and actual embedded > projects, for free. A low cost effort to summarize the results of existing > activities (like the CECOM project) would have been sufficient. For this goal, > the net cost/benefit of DARK is negative. I would really like to see the equation you used to compute a negative number here. Other research which provides results "at a lower cost" does not indicate that the net benefit of DARK will not be positive. One point that must be made when observing work being done on "real" projects (that is, not research work) is that usually these projects do not have a time budget that allows finding out WHY everything operates they way it does. If a problem arises during a development, usually the quickest way to eliminate the problem is chosen. Often code is replaced with something "we know will work" rather than taking the time to find out why some "elegant solution" did not work. The emphasis is on coming up with any suitable solution, not an optimal one. If research was limited to the study of what developers did, we would probably use only the bubble sort for all sorting. Summaries of ongoing work and research are both needed to provide a balanced picture of the issues. > > An alternate goal might have been to have some SEI staff gain > experience in this area. However the DoD could have sent them to one of these > companies for training, at a much lower cost than what the DoD funded the SEI. > Also, there was a rather nice ARMY CECOM effort (finished in 1989) that > studied the problems of developing real time Ada code, and also offered Ada > source code examples, as well as over 1000 pages of documentation, at no cost > (contact Mary Bender at 201-544-2105). Again, for this goal, the net > cost/benefit of DARK is negative. Nothing is free. I can only guess that your "no cost" refers to the cost of getting the documentation. I know CECOM pays the people who do their research. The ARMY will sometimes pay for the duplication of reports, especially during periods when reports are being transitioned to NTIS. I suspect that these reports are now available from NTIS for a charge. I have also received free documentation from SEI. > > If a goal was to provide the country with an Ada real time kernel to > insure that people use Ada (which must have been an unstated goal since they > did try to license DARK to industry), then the venture failed because vendors > developed their own real time kernels, and recently most agreed to offer or > integrate in Ready Systems' real time kernel. Again, the cost/benefit of DARK > is negative. Besides, the government shouldn't be in the position of > developing products. In light of the fact (that you point out below) that Ready Systems is out of the Ada business, I fail to see the clear cost benefit in having a proprietary runtime which is no longer supported by the developer. > > If a goal was to provide a freely available, reusable piece of Ada > real-time source code, then someone screwed up by having DARK available thru > the NTIS, instead of being placed on the public archives on Internet. (It may > be that DARK is available from SIMTEL, but to the best of my knowledge, DARK > is not available by anonymous ftp from any Internet site). Again, the net > cost/benefit of DARK is negative. Not everyone has access to the net, but... Try anonymous ftp from: fg.sei.cmu.edu looks like you're the one who is in error. > > If a goal was to provide an example of a self-contained project whose > costs and benefits could be analyzed as an example of cost-benefit analysis, > a project of immeasurable benefit to the Ada community, well, I haven't seen > a single report from SEI (or an independent third party) performing such an > analysis. In fact, in March of 1989 the General Accounting Office, in a > report (GAO/IMTEC-89-9) on the general progress of Ada inside the DoD, > complained (and the DoD concurred) that no one is collecting the data to > justify the claims made on the benefits of reuse and Ada. The DARK project > might have been a good opportunity to do so. > There have been dozens of reports and public presentations on DARK. You may wish to get "Experiences Porting the Distributed Ada Real-Time Kernel", Brian Smith, Boeing Military Airplanes, June 1990 (Available through DTIC and NTIS: AD-A226 693, I think I paid $5 from DTIC.) I'm pretty sure that cost-benefit analysis of software reuse or government sponsored research were NOT the stated objectives of DARK. This is YOUR area of interest (and GAO's) and it seems you are not adequately prepared for it. You had better start with getting a way to measure software productivity. (When you figure that one out, let me know. I have a long list of people interested in THE answer!) > My contention is that while the country benefited from this project, > the costs to the country were much greater, and that this analysis could have > been performed before the project had ever started, by considering the social > and economic aspects of software development and reuse on a national scale. > All of the benefits could have been obtained at a much lower cost. There may > be other benefits to this project, or that the costs were lower, but they > escape me at the moment. Now 300,000 dollars is not much, but if similar waste > is going on elsewhere, it soon adds up to a lot of tax dollars. I can believe > great things about Ada, but I would like to see some numbers, especially when > I am paying for it. ".... by considering the social and economic aspects of software development and reuse on a national scale" ??? That certainly seems easy to do! -:) The computer industry is so volatile I don't see how a realistic cost analysis could be done. If it could be done, the cost analysis would certainly be as big a job as the research and probably cost more and have less reliable results. Do you know how many tons of money have been spent on actual development projects only to have the industry change direction before the product could get to market? Look at OS/2. I saw estimates by market forecasters in 1988 that it would be the OS on 70%-90% of the PC platforms by 1991. Now analysts say it is all but dead, being replace by MS Windows and Unix. Who knows? Will it be another PC Jr. or will it be what VMS used to be? > > (Of interest is that Ready Systems got out of the real-time Ada kernel > business because there was no profit in it as a product. Finding out why would > be a beneficial SEI study, and also why there are so few Ada library companies > and fewer that survive.) > How much would you pay for this study? Where is your cost-benefit analysis that this study would benefit American Tax Payers? I can probably guess the results of your study and save the tax payers $325,000: 1). The Ada compiler/support tool business is not exactly the "goose that laid the golden egg". 2). There is no standard interface between Ada compilers and the underlying runtime. This makes it costly to support a wide variety of compilers. (By the way, this issue may be addressed by Ada9X.) 3). Runtime vendors are somewhat dependent on the compiler vendors to obtain a customer base. As soon as the runtime vendor's business begins to show a large profit, the compiler vendor may invest the money it takes to provide nearly the same capability from their "new high-performance" runtime. It is extremely hard to compete against the compiler vendor which has complete control over the interface and front-line access to the customer base. 4). In the case of Ready Systems, there may have been "electro-political" forces in effect that caused a falling out between them and one of their compiler team-mates. This type of condition can cause runtime suppliers to get a "grass is greener" viewpoint and switch to a concentration on technologies that are less dependent on other software houses. 5). The future of runtime vendors will be brighter if a standard runtime interface is proposed in Ada9X, however I suspect that only companies that have a long commitment to Ada and good technology that is difficult to duplicate (or patented) will be profitable at supplying independent runtimes. There! You can make the check payable to: -))) Tom Griest LabTek Corporation IMHO, I think you should either pick a different project that you know more about, or spend the time to find out the details about DARK. If you can foretell all of the benefits of research before it is conducted as you suggest, you may want to make yourself available to SDIO, the Human Genome Project, the Cold Fusion Institute, High Temperature Superconductor Project, and others. Your time is much too valuable to waste on giving talks. Final note: I am much more interested in discussing the philosophy behind the model of distributed execution for DARK than trying to guess if this one project will "pay for itself". There are serious questions about what Ada9X is doing in the area of distribution and how DARK may or may not relate to this. ---------------- Disclaimer: I am not employed by either the AJPO, SEI, or CMU and have never received any compensation from the SEI. LabTek Corp. is an affiliate of the SEI and has found benefit from several of their projects. The opinions expressed above are my own and do not necessarily reflect those of LabTek Corporation or any of its customers.
jls@netcom.COM (Jim Showalter) (05/31/91)
I'd like to open up a related discussion on this thread. Based on the descriptions I've read here of DARK, it sounds like it is a real-time distributed-processing real-time runtime environment. This is very interesting to me, because I have been involved with a number of Ada projects over the past several years that needed just such a beast, and in each case the project invested considerable time and money in building its own homebrew version (none of which, to hazard a guess, was as well engineered as DARK). Given that we all agree that reuse is a fine and wonderful thing, a question naturally comes to mind: why did none of these projects opt to make use of DARK? Was it not well-publicized? Were there contractual obstacles to doing so? Were there infrastructural issues that got in the way? Are there technical problems with DARK that preclude its use on real-world projects? Are there projects that HAVE made use of DARK, and, if so, what differentiates them from the one's I'm familiar with (no flames that the differentiator is my participation!)? I think it would be quite useful for the SEI to do a formal study to find answers to these sorts of questions, since we might as an industry be able to do better next time. Thoughts? -- **************** JIM SHOWALTER, jls@netcom.com, (408) 243-0630 **************** *Proven solutions to software problems. Consulting and training on all aspects* *of software development. Management/process/methodology. Architecture/design/* *reuse. Quality/productivity. Risk reduction. EFFECTIVE OO usage. Ada/C++. *