novick@uo-vax1.UUCP (novick) (10/12/84)
[here's the response that was *supposed* to have gone out--DGN] >Just as damning as using an incompetent [software] advisor is >failing to use a competent one. If a doctor's error makes you a >cripple for life, and if he had available (and perhaps even used) >an expert system counceling a better course of treatment, is he >not guilty of malpractice? Does the doctor incur a different >liability than if he had used/not used a human consultant? Under the traditional tort law of medical negligence, the appropriate standard would be whether the unfortunate MD provided the level of care generally provided by other physicians under similar circumstances. Based on my limited knowledge of medicine, I would guess that this test would now still be resolved in favor of the erring doctor, since use of expert systems remains limited. On the other hand, an insightful attorney might be able to show if the expert system embodied the generally-available or exercised expertise in the field, and that the doctor did not have such expertise, then the doctor would be considered negligent--not because he failed to use the system but because his own level of knowledge was insufficient. However, the question of negligence is almost always decided by the jury as a question of fact, and a plaintiff's verdict would not surprise me. Additionally, there are some areas of social conduct involving public safety in which an entire industry's standards might held too low. The most famous example of this was a case in which the owner of a tugboat was held liable for failure to have a radio on board at a time when virtually no tugboats had radios. A similar analysis might then apply to doctors and expert systems. >The human consultant would normally bear part of the liability. >Since you can't sue an expert system, do you sue the company >that sold it? The programmer? The theoretician who developed >the algorithm? I'm sure there are abundant legal precedents for >all of the above. If one were to sue the sellers of an expert system, a number of problems might arise. First, the status of software as a "good" under the Uniform Commercial Code remains unclear, as far as I know. Second, either the injured plaintiff or the physician (on a third-party complaint) might sue the providers of the software on a strict products liability theory; however, the plaintiffs would probably have to show as a matter of law that the patient was a "user or consumer" of the software. There may be analogous law regarding, say, faulty X-ray equipment. In any event, the logical defendants would be the consultant and the company, on the pragmatic grounds that they would be most able to pay. The programmer or the theoretician might also be liable, either in strict liablity or in negligence, but only if their conduct either caused the system to be unreasonably dangerous or fell short of the conduct expected of an ordinary programmer or theoretician. > Systems for monitoring and interpreting >electrocardiograms are commonly adjusted at the "factory" to >match the diagnostic style of the purchasing physician. Suppose >that the doctor requests that this be done, or even does it >himself. Suppose further that he is incompetent at this type >of diagnosis (after all, he's buying a system to do it for him), >and that customization to match his preferences can be shown to >degrade the performance of the software. Is he liable for operating >the system at less than full capability? I assume so. Is the >manufacturer liable for making the adjustment, or for providing >him the means of doing it himself? I would assume that also. >What are the relative liabilities for all parties? Generally, doctors are considered "learned intermediaries" who are expected to know the field in which they practice. This has the effect of largely insulating suppliers of prescription drugs from liability to patients, where the physician was aware of the dangers of the product. A similar analysis might apply in this case. On the other hand, if the tuning feature were found to be a design defect which made the system unreasonably dangerous, or (more likely) if the seller failed to warn the physician of the dangers of tuning the system and this failure to warn made the system unreasonably dangerous, then the seller would be liable, assuming that the injured plaintiff proved that his or her injuries resulted from the design defect or the failure to warn the doctor. The relative liablities are too complex to discuss here, because they hinge largely on state-specific laws relating to comparative fault and strict products liability. In general though, one would expect certainly the seller and probably the doctor to be liable. As between the two, the doctor might be able to get contribution or indemnity from the seller if the system were found defective. --David Novick University of Oregon School of Law University of Oregon Dept. of Computer and Information Science uoregon!uo-vax1!novick
novick@uo-vax1.UUCP (novick) (10/12/84)
/***** uo-vax1:net.ai / sri-arpa!Laws@SRI-AI.ARPA / 2:01 pm Oct 6, 1984*/
From: Ken Laws <Laws@SRI-AI.ARPA>
Just as damning as using an incompetent [software] advisor is
failing to use a competent one. If a doctor's error makes you a
cripple for life, and if he had available (and perhaps even used)
an expert system counceling a better course of treatment, is he
not guilty of malpractice? Does the doctor incur a different
liability than if he had used/not used a human consultant?
The human consultant would normally bear part of the liability.
Since you can't sue an expert system, do you sue the company
that sold it? The programmer? The theoretician who developed
the algorithm? I'm sure there are abundant legal precedents for
all of the above.
For anyone with the answers to the above, here's an even more
difficult problem. Systems for monitoring and interpreting
electrocardiograms are commonly adjusted at the "factory" to
match the diagnostic style of the purchasing physician. Suppose
that the doctor requests that this be done, or even does it
himself. Suppose further that he is incompetent at this type
of diagnosis (after all, he's buying a system to do it for him),
and that customization to match his preferences can be shown to
degrade the performance of the software. Is he liable for operating
the system at less than full capability? I assume so. Is the
manufacturer liable for making the adjustment, or for providing
him the means of doing it himself? I would assume that also.
What are the relative liabilities for all parties?
-- Ken Laws
/* ---------- */sasaki@harvard.ARPA (Marty Sasaki) (10/13/84)
The world of Data Processing has already solved the kinds of problems
outlined in the various articles about the liability of expert systems.
Contracts for things like DBMSs always have a clause that specifically
says that the company is not liable for any damage to the client even
if the problem is obviously the fault of the software. The most that a
client can claim is the cost of the software package.
This has been tested in court several times and the decision is connected
directly with how the contract was written. If the contract was written
in a way which truly removed the liability from the software company, then
they won, otherwise the client won.
The legal issue is one entirely of legalese. This doesn't solve the moral
or ethical side though, which is what this group is really discussing.
--
Marty Sasaki
Havard University Science Center
sasaki@harvard.{arpa,uucp}robison@eosp1.UUCP (Tobias D. Robison) (10/17/84)
[I'm not a lawyer but I think that:] The contract terms of sale for an expert system should also be a factor in determining the systems liability. I imagine that the vendor of an expert system would include a strong statement that recommendations of the system must be interpreted by the customer in every case to determine reasonability, with the final determination being the sole responsibility of the customer. The vendor would probably restrict carefully the area of expertise that in which the system could be trusted, and might void even those recommendations as soon as a customer "customized" his machine. Finally, the vendor would also state that he was not liable for any legal problems arising out of recommendations by the system. It is possible that in a competitive market, some vendors would not be able to sell a system with such restrictions, but this is routinely done for most types of software. In some cases, the courts will void such [lack of] warranties, and hold the vendor responsible; in other cases, the vendor's restrictions will stand. When the vendor is able to limit his liability, he might still be responsbile for problems arising due to vendor negligence, or the failure of the product to perform in a way that any sensible person would think was a reasonable minimum of capability. - Toby Robison (not Robinson!) allegra!eosp1!robison or: decvax!ittvax!eosp1!robison or (emergency): princeton!eosp1!robison