sgf0z@watt.acc.Virginia.EDU (Steven Glenn Fox) (04/11/91)
I have been fighting with some SE terms recently and
am looking for some help in defining them and how
they relate to each other. The terms I am referring to are:
Software Quality (& Software Quality Assurance)
Software Risk
Software Testing
Verification and Validation
Software Reliability
To get the ball rolling, here are some of my observations:
1. Software Reliability is one component of Software Quality.
2..V&V is essentially synonymous to Software Testing.
3..Software Risk is the risk of a software project not
being completed, e.g. not fulfilling its specifications?
Does this make it a component of Software Reliability?
4. Software Quality is measuring quantitatively the quality
of a piece of software on a number of attributes as
defined by the specific project. These attributes often
are similar across many projects.
Thanks.
------------------
Steven G. Fox sgf0z@virginia.edu
University of Virginia uunet!virginia!sgf0z
Department of Systems Engineering
Charlottesville, Va 22903 (804) 982-2073song@berault.ics.uci.edu (Xiping Song) (04/12/91)
In article <1991Apr11.144028.20787@murdoch.acc.Virginia.EDU> sgf0z@watt.acc.Virginia.EDU (Steven Glenn Fox) writes: >I have been fighting with some SE terms recently and >am looking for some help in defining them and how >they relate to each other. The terms I am referring to are: > >Software Quality (& Software Quality Assurance) >Software Risk >Software Testing >Verification and Validation >Software Reliability > >To get the ball rolling, here are some of my observations: >1. Software Reliability is one component of Software Quality. >2..V&V is essentially synonymous to Software Testing. It is said that testing is a vehicle to validate software.
shimeall@taurus.cs.nps.navy.mil (timothy shimeall) (04/12/91)
In article <1991Apr11.144028.20787@murdoch.acc.Virginia.EDU> sgf0z@watt.acc.Virginia.EDU (Steven Glenn Fox) writes: >I have been fighting with some SE terms recently and >am looking for some help in defining them and how >they relate to each other. The terms I am referring to are: > >Software Quality (& Software Quality Assurance) >Software Risk >Software Testing >Verification and Validation >Software Reliability May I suggest that a good starting place for defining these terms is NOT coming up with new defintions, but rather examining existing defintions. A good place to start is the ANSI/IEEE Standard Glossary of Software Engineering Terminology (Std. 729-1983 or a revision), published by IEEE Computer Society Press. One dictionary can save a LOT of confusion. All of the terms you list above are included, except software risk. For that, may I suggest Barry Boehm's tutorial? (By the way, the standard technical definition of risk is something like "the probability of a given cost over a given period of time", you may wish to adapt that for software...) Tim -- Tim Shimeall ((408) 646-2509)
jcardow@afit.af.mil (James E. Cardow) (04/12/91)
sgf0z@watt.acc.Virginia.EDU (Steven Glenn Fox) writes: >I have been fighting with some SE terms recently and >am looking for some help in defining them and how >they relate to each other. The terms I am referring to are: >Software Quality (& Software Quality Assurance) >Software Risk >Software Testing >Verification and Validation >Software Reliability >To get the ball rolling, here are some of my observations: >1. Software Reliability is one component of Software Quality. Simplistic, but okay, I'll agree. If a definition is what your after, try Musa's "the probability of failure-free operation of a computer program for a specified period of time." Whatever definition you choose, it needs to be tied to performing as specified (note, that says nothing about the reliability of the spec.) >2..V&V is essentially synonymous to Software Testing. Sorry, can't agree with this one. Software Testing is a critical element of V&V, but then so is the review process. V&V definition in IEEE Std are pretty good. >3..Software Risk is the risk of a software project not > being completed, e.g. not fulfilling its specifications? > Does this make it a component of Software Reliability? Okay, this takes a little more thinking. Risk, as I see it, is the risk of not meeting the customer's value expectations for the software (note, you very well may be the customer) So risk is a composite of cost, schedule, performance (the machine and the people), technology innovation, etc, etc. One think I try to get across to my students is that risk is not all bad. Risk taking is what makes heroes, provided you can control and manage the level of risk. >4. Software Quality is measuring quantitatively the quality > of a piece of software on a number of attributes as > defined by the specific project. These attributes often > are similar across many projects. Well, I think the attributes are pretty well defined. The problem is, which are important for which project. For instance, portability may be a critical attribute in one project and totally trivial in another. Jim Cardow Air Force Institute of Technology Wright Patterson AFB, OH jcardow@blackbird.afit.af.mil
windy@pensoft.uucp (Windflower Gilbert) (04/12/91)
I will explain the terms in question as I understand them . I do QA for a living. The quoted definitions are from the IEEE Standard Glossary of Software Engineering Terms. "Software Quality: The totality oms to refer to robustness, lack of bugs, and other general quality statements about software. Software Risk probably refers to criticality. The Criticality of software is a function of whether or not the failure of the software could cause loss of life or significant financial loss. If software is "Critical" then it must be tested much more thoroughly. Beyond the point of diminishing returns for most software. Software Testing is exercising software in some predefined way with a specific focus to locate defects. "Verification : The process of determining whether or not the products of a given phase of the software development cycle fulfill the requirements established during the previous phase" "Validation: The process of evaluating software at the end of the software development process to ensure compliance with software requirements" V&V covers the whole development cycle. It may include things like prototype evaluations, and code inspections in addition to testing. "Software Reliability: The probability that software will not cause the failure of a system for a specified time under specified conditions." Generally, Software Reliability refers to extended periods of time. Reliability tested is conducted for increasing amounts of time, and often includes stress intended to resemble a "real implementation". I think a brief summary of the formal approach will serve to clarify. If I am responsible for all aspects of software quality for a particular product, I will write the following documents in the order given. A Software Quality Assurance Plan that will describe some aspects of the development process such as source code control, leadership responsibility and what design related documents are to be written . It will also identify what V&V methods will be implemented, whether or not the software is critical and how the development cycle relates to the QA. A V&V plan that will describe in a fair amount of detail each of the methods to be used. This may include code inspection approach, usability evaluation of a prototype, who will do unit testing, and types of software testing to be implemented (such as reliability testing). It will also identify what test plans will be written. A Test Plan may cover all or part of a product and details the testing of that part. For complex projects, there may be another layer here. Windflower Gilbert Pencom Software pensoft!windy@cs.utexas.edu
cole@farmhand.rtp.dg.com (Bill Cole) (04/13/91)
|> I have been fighting with some SE terms recently and |> am looking for some help in defining them and how |> they relate to each other. The terms I am referring to are: |> |> Software Quality (& Software Quality Assurance) |> Software Risk |> Software Testing |> Verification and Validation |> Software Reliability |> |> To get the ball rolling, here are some of my observations: |> 1. Software Reliability is one component of Software Quality. |> 2..V&V is essentially synonymous to Software Testing. |> 3..Software Risk is the risk of a software project not |> being completed, e.g. not fulfilling its specifications? |> Does this make it a component of Software Reliability? |> 4. Software Quality is measuring quantitatively the quality |> of a piece of software on a number of attributes as |> defined by the specific project. These attributes often |> are similar across many projects. |> From the bottom up: On what basis do you measure released software? You say 'attributes' but that's very non-specific. Weinberg states that the first measure of quality is whether of not the software does what we say it does. In very large software projects (my area of inquiry is database software), what attributes should be measured? Let's assume that I've tested it thoroughly and gone through the V&V process. Now, I've got several thousand users out there running this software and occasionally finding problems. Do I count the number of reported problems? or database corruptions? Or do I count some specific class of reported problems as bugs? I use the reported problems as a basis to know 'software quality', but I get lots of arguments from the academicians and pedagogues. Risk can have the component of what-happens-if-we-don't-do-the-project which can be measured against the rate of return for having done the project. The risk aspect can also include (as part of the quality issue) a statement of how-good-is-good-enough? If you test a product for three months and find 100 bugs of a non-serious nature, does it make sense to spend another three months looking for bugs? What's the risk associated with shipping software in its current state. V&V is NOT synonymous with software testing. These are purely positive tests; testing includes negative tests as well. In negative testing, you try to break the software, go beyond limits or look for limits. Reliability is, of course, a component of Quality. But who measures it? If you have 1000 users and 900 have no problem but 100 have varying problems, you have 10% of your user base saying the software is useless because of its reliability while 90% are happy/satisfied/not disgruntled. The views are my own, /Bill
cole@farmhand.rtp.dg.com (Bill Cole) (04/13/91)
|> May I suggest that a good starting place for defining these terms is |> NOT coming up with new defintions, but rather examining existing |> defintions. A good place to start is the ANSI/IEEE Standard |> Glossary of Software Engineering Terminology (Std. 729-1983 or a |> revision), published by IEEE Computer Society Press. One dictionary |> can save a LOT of confusion. |> |> All of the terms you list above are included, except software risk. |> For that, may I suggest Barry Boehm's tutorial? (By the way, the |> standard technical definition of risk is something like "the probability |> of a given cost over a given period of time", you may wish to adapt |> that for software...) |> The problem with definitions is that they lie there on a piece of paper and don't mean anything until they're applied. I would agree that a definition is a fine start, but the definition IS ONLY A START. How a definition is implemented in some specific task is the real definition. My own are the views, /Bill