[comp.sys.handhelds] The mythical bug-free code

Jake-S@cup.portal.com (Jake G Schwartz) (02/05/91)

Xeno writes:

> As a side-line, HP should have tested their ROMs more thoroughly...  If they
> didn't even try to invert a matrix larger than 8x8, they deserve the penalty
> of replacing every flawed calculator.  It is not OUR fault that the entire
> calculator must be replaced.  HP, in their infinite wisdom, could have
> worked out a scheme where the ROMs were replacable.  Software is covered
> by the same warranties that hardware is - it is to be free from manufacture
> defects... Revision Ds are flawed, and so are A, B, and Cs...  If I bought
> a car and the 2nd gear position on my automatic transmission control didn't
> work, I'd sure have it fixed even if I didn't use it!

Give me a break. There is no such thing as bug-free code, anywhere, 
especially when the code is 262 thousand bytes long. All there may be is
code where the bugs haven't been found yet. 

Let's suppose that HP's ROMs *did* invert 8 by 8 matrices correctly, but
they didnt invert 111 by 111 matrices correctly. How would you propose to
test THIS one ? How about if it inverted ALL matrices correctly except for
ones whose element in row 47 column 16 was a -1? How would you then pro-
pose to test THAT one out? If you have an answer, you are better than
most of us. My guess is that you just haven't thought our your statements
before typing them to this newsgroup.

Software is never free from defects..it's a fact of life. There are 
many hardware problems with socketing the ROM and allowing the user to
go in and change it. Static electricity is just one of these. Breaking 
pins is another. Opening and closing the machine easily without making it
too susceptable to dust and moisture is still another. It would simply
cost a signicant amount more if the ROM was socketed.

Testing four or five gears of a car is one thing. But testing each and
every permutation of 256K of code is another. There is no comparison. I
think they've done a pretty splendid job with what we've got. It simply
surprises me that HP is upgrading machines as much as they are. Lucky
for them that only a small minority of their HP48 users won't accept a 
really simple workaround for inverting matrices that is virtually as easy 
as the original method.

Jake Schwartz

akcs.dnickel@hpcvbbs.UUCP (Derek S. Nickel) (02/06/91)

Actually, I think HP *should* have tried inverting all possable matries
(with in the memory limits of the '48).  Of course it would never get to
market...

        dsn

akcs.falco@hpcvbbs.UUCP (Andrey Dolgachev) (02/07/91)

Derek brings up a good point, HP could have inverted all the possible
matrices, and it would have taken a hell of a while longer.  Along this
line, can you imagine if the testers tried every single HP command on
every possible combination of object type, and for every situation of
system flags, and then what happens if you press the on key right after
another key, and if you transfer 0 byte programs, etc., etc.  Sure, I
guess that it would be possbile, but it would take a hell of a long time
and I sure as hell wouldn't want the A version of the HP coming out about
now, even if it did have all the bug -fixes of the E.  Remember, the HP
has been out for a while now, with all the buyers testing it, and all the
bugs have still not been worked out.  Guys, remember one thing, I don't
think that if you compoiled all the bugs of the previosu version, put
them on one calculator, you would be in any dire straits.  Look at
Apple's System software, now that's buggy! Or Word, or Xtree, or many
commercial programs.  And anyways, the final point point is don't
complain, HP is replacing A-C and even D for FREE, so stop bitchin' and
start doing some constructive posts!
          ---Falco

bson@rice-chex.ai.mit.edu (Jan Brittenson) (02/07/91)

In a posting of [6 Feb 91 21:40:10 GMT]
   akcs.falco@hpcvbbs.UUCP (Andrey Dolgachev) writes:

 > Derek brings up a good point, HP could have inverted all the possible
 > matrices, and it would have taken a hell of a while longer.  Along
 > this line, can you imagine if the testers tried every single HP
 > command on every possible combination of object type, and for every
 > situation of system flags,

   This is almost trivial. It's called a test suite. A limited test
suite, i.e. a test suite that only exercises a couple of standard
parameters spread out over a conceivable range, as well as some
extreme cases to test the robustness of the code, would for sure have
caught the matrix inversion bug. Probably the zero-size kermit
transfer one, too. Bill Wickes mentioned the former was partly a
result of human error (if I remember correctly). I take this to mean
that something wasn't put into the test suite that should have been
there.


   You don't have to test every function for every possible system
flag setting - only for those that actually affect the behaviour.

   The reasons for this kind of test suites are fairly obvious: when
you fix something broken, you will know when something else breaks in
the process.


 > what happens if you press the on key right after another key, and if
 > you transfer 0 byte programs, etc.,

   We can only guess how this is tested... My guess is that the
development system allows a work station to exercise the calculator by
doing serial I/O, pressing keys, etc. This is difficult, though, and
much testing always has to be done manually to verify any interactive
system.

   I'm sure HP compared the costs of replacing units which people had
found defective to what longer testing would cost in terms of lost
market shares, lost revenues and increased development costs. I'm sure
HP, with plenty of "prestige margin," considered the rev A units ready
for shipping. This doesn't mean I think it's wrong to ask HP to fix
your calculator if it doesn't work properly. In fact, the
archive/clock bug pisses the hell out of me, and I will ship my rev D
calculator to HP as soon as I can live without it for a week or two.

						-- Jan Brittenson
						   bson@ai.mit.edu

edp@jareth.enet.dec.com (Eric Postpischil (Always mount a scratch monkey.)) (02/07/91)

In article <27b07042:1922.3comp.sys.handhelds;1@hpcvbbs.UUCP>,
akcs.falco@hpcvbbs.UUCP (Andrey Dolgachev) writes:

> . . .Along this
>line, can you imagine if the testers tried every single HP command on
>every possible combination of object type, and for every situation of
>system flags, and then what happens if you press the on key right after
>another key, and if you transfer 0 byte programs, etc., etc.

Actually, we can calculate how long this would have taken.  I'll suppose the
32Kb RAM is all there is.  I haven't followed the hardware details, so add to
that for whatever other machine states there are.  I'm assuming no plug-in
devices.  Let's see, that gives us 262,144 bits, so there are 2^262,144 machine
states.  At each instant, we can have a variety of possible inputs to the
machine:  clock tick, 49 keys, infared photocell high/low, receive wire high/low
-- 53 bits.  If
HP tested a googol of states/inputs per second, it would only take, let's see,
10 to the power of tens of thousands of universe life-times to completely
validate the machine.  No problem.


				-- edp (Eric Postpischil)
				"Always mount a scratch monkey."
				edp@jareth.enet.dec.com

akcs.falco@hpcvbbs.UUCP (Andrey Dolgachev) (02/09/91)

Jan,
You missed the point of my reply, which is this:  Don't worry about the
stupid bugs, when HP is going to be replacing you rcalculator for feee,
don't spend time on arguing how HP could have found out the bug, and
giving me a long-drawn out explanation of how HP could have followed a
simple procedure and found the bug, etc., etc.  This thread has gone out
of control, with far too many postings, arguments, flameings, , etc. on
something wich just started out as a comment or two.  INstead of posting
a reply to more arguments, why don't you and everyone else who is posting
on this thread (including me, at least my 2 replyies counting this one)
drop the this stupid thread and post something constructive.  You're
getting you calculator fixed for free, and in the meantime, try doing
something with it.
    ---Falco

And I am not affiliated with HP, etc., I just think that they are doing,
have done, and will do, awesome work with calculators, even if you can't
receive 0 byte programs on 'em.

bson@rice-chex.ai.mit.edu (Jan Brittenson) (02/09/91)

In a posting of [8 Feb 91 20:40:20 GMT]
   akcs.falco@hpcvbbs.UUCP (Andrey Dolgachev) writes:

 > when HP is going to be replacing you rcalculator for feee, don't spend
 > time on arguing how HP could have found out the bug, and giving me a
 > long-drawn out explanation of how HP could have followed a simple
 > procedure and found the bug, etc., etc.

   Having worked with software quality control (in fact, that's how
many people begin their careers) I find it an interesting issue. If
you don't want to discuss it, well then, don't. I was only trying to
spawn off a fruitful discussion on how HP-48 software can be tested in
general. If you can't stand the thought that someone adds something to
your articles, then you should stop posting, or openly declare this to
be the case, so you can be avoided.


 > You're getting you calculator fixed for free, and in the meantime, try
 > doing something with it.

   Perhaps if you had pondered the issue for another second or two
before starting the great text generator, you might have realized that
once something is done, it needs testing. Or do you push untested
code?

   Not that it bothers me, it's your name after all, not mine. But
please show some respect for those of us who _do_ care.

				* * *

   Now, back to the issue... Can someone explain how HP actually tests
the HP-48?

						-- Jan Brittenson
						   bson@ai.mit.edu

akcs.wiser1@hpcvbbs.UUCP (steven lee wiser) (03/28/91)

I appreciate how HP has given there time and money to develope the hp48sx
for what it is, the very best calculator one can built.  I hope that we
have grown so that the next machine that HP comes out with (Jaguar)
doesn't have to live up to the hertiage of the HP48sx with Lotus built in
and as it is built in with some very insignificant bugs that are
documented but are not fixed. So how will every body deal with that. I
hope with maturity developed from the 48sx experience.And not with the ax
that has been taken to HP for the 48...
Steve Wiser
Keep up the good work HP....