[comp.sys.atari.st] Testing that programs conform to TOS rules

knop@duteca (P. Knoppers) (02/05/90)

I am getting bored of all those arguments against improving TOS,
because it involves changing non-documented features/bugs of TOS.

What we need is a means to test that an application does not use facts
about TOS that it should not rely on. With such a beast it is possible
that:
- programmers verify that they have programmed according to the rules.
- project managers verify that their programmers have programmed
  according to the rules.
- buyers/users/owners of programs verify that programs are likely to 
  survive TOS upgrades.
- eventually, Atari creates a better TOS, because it is ensured that
  the majority of application programs for their computers will
  continue to work.
I believe that it is possible to create such a test suite.

What I am thinking of is a terminate-and-stay-resident program that 
intercepts most (if not all) trap vectors. The test suite will make 
sure that, in addition to really executing the requested trap:
- Successive mallocs NEVER return adjacent memory areas.
- The area just outside a malloc-ed area is not used by an application
  program. (This can be done by regularly checking that the contents of
  such memory areas have not been altered.)
- All registers that are not guaranteed to be preserved during a trap
  are scrambled before return from that trap.
- All undocumented information in low memory (which is private to TOS)
  is not there during normal program execution, but it is temporarily 
  restored DURING an intercepted trap.

In some of these cases this TSR program will be able to detect a
misbehaving application program as soon as it misbehaves, in other
cases the application program probably crashes the system shortly after
misbehaving.

There are probably many other undocumented properties of TOS that
might not be present in some future version. In general, the test suite
should make sure that undocumented properties will not work. The list
is only a start, there must be many more things that Atari can break
BECAUSE they are not documented.

Of course, the test suite will slow down the operation of the computer
during testing, that is the price that must be paid during development
to ensure better portability across TOS versions.

I am hoping for a vivid discussion on this topic.
- Do you think such a test suite is feasable ?
- Would YOU like to write such a test suite ?
- Which undocumented features/bugs should be added to the list ?
-- 
  _____       __  __  __  __   ____   _____   _____   ______  _____    _____ 
 |  _  \     |  |/  ||  \|  | /    \ |  _  \ |  _  \ |   ___||  _  \  /  ___|
 |   __/ _   |     < |      ||  ||  ||   __/ |   __/ |   >__ |     <  \__  \
 |__|   |_|  |__|\__||__|\__| \____/ |__|    |__|    |______||__|\__||_____/
P. Knoppers, Delft Univ. of Technology, The Netherlands, knop@duteca.tudelft.nl

david@bdt.UUCP (David Beckemeyer) (02/08/90)

I think a TOS validation test suite TSR is a really good idea.
The part about not having the TOS private variables there
while the program is running is tricky because of the TOS
interrupt handlers.  Also only Atari knows for sure where
these variables are, so only Atari could "really" do this part
correctly.

I have a system call profiler which I use to see what system
calls a program uses, which could be the start of such a
program, if anyone is interested in running with this idea.

I like it.
-- 
David Beckemeyer (david@bdt.UUCP)	| "I'll forgive you Dad...  If you have
Beckemeyer Development Tools		| a breath mint."
P.O. Box 21575, Oakland, CA 94620	|    Bart - "The Simpsons"
UUCP: {uunet,ucbvax}!unisoft!bdt!david	|