atreides@caladan.UUCP (System Administrator) (12/23/89)
There is always a lot of discussion about a variety of development
environments (and rightly so), but my interests lie more with testing
tools, and I have never seen a discussion on this topic.
So far I have found the Mac woefully lacking in this area.. but like
any Mac zealot the quest continues :-)
My question is, does anyone know of the existance of commercially
available tools for the Mac. Or if you have developed tools, would
you be willing to share them with the rest of the testing community on
the net?
I'm especially interested in tools for:
- Automation
- Code Coverage
- Code Complexity Measurement
etc...
I look forward to any information that comes to light.
--
Stuart Burden
root@caladan.uucpjspear@gryphon.COM (Jon Spear) (12/25/89)
In article <2bbd2b567@caladan.UUCP> atreides@caladan.UUCP (root/postmaster/etc) writes: >There is always a lot of discussion about a variety of development >environments (and rightly so), but my interests lie more with testing >tools, and I have never seen a discussion on this topic. > [...] >I'm especially interested in tools for: > > - Automation > - Code Coverage > - Code Complexity Measurement [note: I'd intended to mail this, but it bounced...] It is indeed hard to accept the Mac as a platform for developing highly reliable software when there seems to be a dearth of testing and code analysis tools available (not to mention the lack of a protected-mode operating system or a verified kernel or the popularity of languages like C that almost encourage unsafe coding [donning flame-retardant undies]). An automated test driver for general Mac applications is an interesting problem. The closest I know of is the monkey DA that posts random mouse and keyboard events, ... what more could you want? You can get a little mileage out of scripting/macro facilities like Tempo II, but that isn't quite a complete facility for maintaining a database of regression tests, running some subset, collecting and analyzing results, and displaying a summary report. Is this a problem with the Mac environment(s), or is it a general problem for systems that use a graphical user interface? The only test automation packages like this that I'm aware of are for Unix or VMS programs that are either batch-oriented or use simple character-oriented display user interfaces. Test coverage seems like an easier problem, but again, I've seen nothing for the Mac. The closest things I can think of are the performance analyzers that come with a couple environments. You know - the thingies that instrument your code to call a time-stamper subroutine that collects timing data on a per-module basis. [I haven't actually used these on the Mac -- do they instrument the code or sample the PC at some interrupt interval, or both?] It seems like it wouldn't be too hard to read that bucket file and identify which modules were called and which were not -- very crude coverage indicator, but better than nothing... Complexity metrics are also straightforward to port to the Mac, assuming you are using a language with standard syntax. But metrics are of less obvious value, other than curiosity, or maybe I've missed something. Do any complexity metrics exist that are reasonably simple to automate that have been shown to correlate well with the true maintainability or understandability of a piece of code? That are not easily fooled? That provide specific feedback on how to improve the code? I know Halstead's, McCabe's, and other metrics I've seen don't seem very useful in this respect. Please let us know what you find. -Jon -- ----- [The following address is soon to evaporate. New address sought.] Jon L Spear: jspear@gryphon.COM <routing site>!gryphon!jspear gryphon!jspear@elroy.jpl.nasa.gov "With computers we can make billions of mistakes every second!"
jess@apple.com (Jay Jessen) (01/22/90)
In article <2bbd2b567@caladan.UUCP> atreides@caladan.UUCP (System Administrator) writes: > In article <2bbd2b567@caladan.UUCP> atreides@caladan.UUCP (System Administrator) writes: > There is always a lot of discussion about a variety of development > environments (and rightly so), but my interests lie more with testing > tools, and I have never seen a discussion on this topic. > > So far I have found the Mac woefully lacking in this area.. but like > any Mac zealot the quest continues :-) > > My question is, does anyone know of the existance of commercially > available tools for the Mac. Or if you have developed tools, would > you be willing to share them with the rest of the testing community on > the net? > > I'm especially interested in tools for: > > - Automation > - Code Coverage > - Code Complexity Measurement > > etc... > > I look forward to any information that comes to light. Here at Apple, we have an environment to automate the testing of Mac applications through the user interface. User/Application interaction is what's being automated. The testing is driven by a scripting language which specifies Rwould-beS interaction between a user and the application under test. The test specification is given abstractly rather than literally as with many macro facilities. By abstract, I mean that the language primitives include commands to say things like Rselect the button titled OKS, Rdrag the frontmost windowS, Rselect the first menuItem under the windows menuS, etc... The system, is therefore adaptive to things such as varying monitor configurations, Apps that center dialogs, minor changes from one version of an App to the next, etc... In addition, higher level programming constructs are supported so that testing methodologies can be applied. One can specify code to facilitate things like Rif a dialog is present, log the message, dismiss the dialog, and continueS. Mechanisms are also supported to allow for modular development of scripts. In addition, this tool was designed from the very beginning to be a testing environment. We try to offload as much of the work away from the Mac under test and try to minimize intrusion of the Mac Application under test. Therefore, testing is done over Appletalk. ItUs possible to drive many MacUs from one Mac host. In fact, itUs possible to drive many MacUs from one host having each being driven by a different script. Each of the machines under test will have a small piece of code running which acts as an agent to pass raw information back to the host. This code works at a level low enough as to fool the application into thinking it is being manipulated by an actual user. It installs itself as an init. Presently, the tool exists as an MPW tool and scripts are written textually. There is also a sister application which can do literal mouse and keystroke recording/playback. The two tools can work cooperatively in that recordings can be played back at points within script execution. The environment is used extensively within Apple. Tom Chavez, the product manager for the tool is trying to make it available through APDA. You may contact Tom for information pertaining to the tool's availability. He is reachable as follows: Tom Chavez, Product Manager Apple Product Marketing Applelink: TOM.CHAVEZ tomc@apple.com I am also hoping to present this tool at the Worldwide Developer's conference this Spring. I am trying to arrange such a presentation, but cannot confirm it at this time. Thanks for your interest and I hope we can meet your testing needs with this tool. Jay Jessen Project Lead ------------------------ While I have your attention, let me also state that we have an immediate job opening on the project for an experienced and knowledgeable Mac programmer. The person should have at least a B.S. in Computer Science or Electrical Engineering. Strong interpersonal skills are a must as the person must be able to work within a project team and interact regularly with engineers of all sorts across Apple. Skills that would be an added bonus include Appletalk programming, C++ programming, knowledge of and/or application of testing methodologies. This position is in Cupertino, CA (Silicon Valley). I will accept e-mailed letters informing me of interest in the position, but please send all resumes and cover letters through U.S. mail to the address listed at the end of this posting. Apple Computer, Inc. has a corporate commitment to the principle of diversity. In that spirit, we welcome applications from all individuals including women, minorities, veterans, and disabled individuals. (C) 1990 Apple Computer, Inc. Apple, the Apple logo, Macintosh, AppleTalk, and MPW are all registered trademarks of Apple Computer, Inc. Jay Jessen, Software Engineer Apple Computer, Inc. MS 60BB 20525 Mariani Ave Cupertino, CA 95014 AppleLink: Jessen1
jess@apple.com (Jay Jessen) (01/22/90)
I see part of my first reply got screwed up. I apologize for this and repost... > In article <2bbd2b567@caladan.UUCP> atreides@caladan.UUCP (System Administrator) writes: > There is always a lot of discussion about a variety of development > environments (and rightly so), but my interests lie more with testing > tools, and I have never seen a discussion on this topic. > > So far I have found the Mac woefully lacking in this area.. but like > any Mac zealot the quest continues :-) > > My question is, does anyone know of the existance of commercially > available tools for the Mac. Or if you have developed tools, would > you be willing to share them with the rest of the testing community on > the net? > > I'm especially interested in tools for: > > - Automation > - Code Coverage > - Code Complexity Measurement > > etc... > > I look forward to any information that comes to light. Here at Apple, we have an environment to automate the testing of Mac applications through the user interface. User/Application interaction is what's being automated. The testing is driven by a scripting language which specifies "would-be" interaction between a user and the application under test. The test specification is given abstractly rather than literally as with many macro facilities. By abstract, I mean that the language primitives include commands to say things like "select the button titled OK", "drag the frontmost window", "select the first menuItem under the windows menu", etc... The system, is therefore adaptive to things such as varying monitor configurations, Apps that center dialogs, minor changes from one version of an App to the next, etc... In addition, higher level programming constructs are supported so that testing methodologies can be applied. One can specify code to facilitate things like "if a dialog is present, log the message, dismiss the dialog, and continue". Mechanisms are also supported to allow for modular development of scripts. In addition, this tool was designed from the very beginning to be a testing environment. We try to offload as much of the work away from the Mac under test and try to minimize intrusion of the Mac Application under test. Therefore, testing is done over Appletalk. ItUs possible to drive many Mac's from one Mac host. In fact, it's possible to drive many Mac's from one host having each being driven by a different script. Each of the machines under test will have a small piece of code running which acts as an agent to pass raw information back to the host. This code works at a level low enough as to fool the application into thinking it is being manipulated by an actual user. It installs itself as an init. Presently, the tool exists as an MPW tool and scripts are written textually. There is also a sister application which can do literal mouse and keystroke recording/playback. The two tools can work cooperatively in that recordings can be played back at points within script execution. The environment is used extensively within Apple. Tom Chavez, the product manager for the project is trying to make it available through APDA. You may contact Tom for information pertaining to the tool's availability. He is reachable as follows: Tom Chavez, Product Manager Apple Product Marketing Applelink: TOM.CHAVEZ tomc@apple.com I am also hoping to present this tool at the Worldwide Developer's conference this Spring. I am trying to arrange such a presentation, but cannot confirm it at this time. Thanks for your interest and I hope we can meet your testing needs with this tool. Jay Jessen Project Lead ------------------------ While I have your attention, let me also state that we have an immediate job opening on the project for an experienced and knowledgeable Mac programmer. The person should also have at least a B.S. in Computer Science or Electrical Engineering. Strong interpersonal skills are a must as the person must be able to work within a project team and interact regularly with engineers of all sorts across Apple. Skills that would be an added bonus include Appletalk programming, C++ programming, knowledge of and/or application of testing methodologies. This position is in Cupertino, CA (Silicon Valley). I will accept e-mailed letters informing me of interest in the position, but please send all resumes and cover letters through U.S. mail to the address given at the end of this posting. Apple Computer, Inc. has a corporate commitment to the principle of diversity. In that spirit, we welcome applications from all individuals including women, minorities, veterans, and disabled individuals. (C) 1990 Apple Computer, Inc. Apple, the Apple logo, Macintosh, AppleTalk, and MPW are all registered trademarks of Apple Computer, Inc. Jay Jessen, Software Engineer Apple Computer, Inc. MS 60BB 20525 Mariani Ave Cupertino, CA 95014 AppleLink: Jessen1
silverio@brahms.berkeley.edu (C J Silverio) (01/23/90)
In article <2bbd2b567@caladan.UUCP> atreides@caladan.UUCP (System Administrator) writes: My question is, does anyone know of the existance of commercially available tools for the Mac. Or if you have developed tools, would you be willing to share them with the rest of the testing community on the net? Well, there are some. Polytron makes a suite of these tools that runs under MPW and many common IBM-PC development environments. I haven't used any of their stuff. Their # is 1-800-547-4000 or (503)645-1150. They'd love to send you a free catalog. They advertise heavily in Computer Language magazine, a haven for people interested in the "engineering" aspects of Computer Science. I also spotted a pretty complete-looking suite from MMC AD Systems (408)263-0781 in Computer Language. Additionally, MPW 3.0 now contains Projector, a code-management module with a pretty spiff user interface. I only know about it from the article in Mac Tech Quarterly, Winter 1990 edition. An alternative is simply to call the maker of your favorite package, and ask them when they will port it to MPW. The Commando interface makes this task damn near trivial. You could even do it yourself. Recently, someone converted the C beautifier from GNU to an MPW Commando and posted it. have fun.