[comp.software-eng] CASE tool "stress tests"?

howell@COMMUNITY-CHEST.MITRE.ORG (12/08/89)

Any opinions on (or better yet, experiences with :-) the subject of
"scale up factors" for CASE tools?  For example, a tool that works just
fine for 3 people developing a "toy" system may break or suffer dramatic
performance loss or just not provide needed support (e.g. name space
partitioning) when you move to a big system and/or lots of users.  Any
specific stress points, any experiences good or bad with any CASE tools
(especially Teamwork), and any pointers to other work in this area would
all be appreciated a LOT.
Thanks,

+------------------------------------------------------------------------+
| Chuck Howell, M/S Z645     INTERNET: howell@community-chest.mitre.org  |
| The MITRE Corporation           OR   howell@mwunix.mitre.org           |
| 7525 Colshire Drive                                                    |
| McLean VA 22102-3481            FAX: (703) 883-5519                    |
+------------------------------------------------------------------------+

cberg@leadsv.UUCP (Charles R Berg) (12/12/89)

In article <8912081454.AA02621@boardwalk.mitre.org>
  howell@COMMUNITY-CHEST.MITRE.ORG writes:
>Any opinions on (or better yet, experiences with :-) the subject of
>"scale up factors" for CASE tools?  For example, a tool that works just
>fine for 3 people developing a "toy" system may break or suffer dramatic
>performance loss

I am the technical lead for a group of 8 people using IDE's Software-
through-Pictures.  We have six licenses, running on workstations dedicated
to our use.  The workstations are diskless Sun 3/140's, with 12 MBytes
of memory.  The network supports 175 workstations with 14 fileservers, and
a total of 27 GBytes of disk.  I would rate the performance level of StP
as poor.  Since the execution time of many StP database requests is a
function of database size, note that our databases range in size from
1.3Mbytes to 5.4Mbytes in size.  Our current estimates are that each of
these databases is around 10-20% of anticipated final size.  (PS, we have
21 separate databases - see comments below.) (PPS, we are using a beta
release of 4.2, which has the recent fix to improve  database performance.)

* Opening the Main Menu window from an icon can take up to 15 seconds.
* Generating the dictionary from a typical dataflow diagram can take 10
  minutes.
* Generating the dictionary from a typical data structure diagram can take
  up to 25 minutes.
* We have document scripts that can take from between 4 and 24 hours to run.
* I have received phone calls from other IDE users that take up to 4 days to
  rebuild there entire dictionary from scratch.

>or just not provide needed support (e.g. name space partitioning)

Name space partitioning?  IDE has no such concept.  We are capturing the
design of 750,000 lines of c code, running on 17 different processors.  In
order to 'partition' our names, we had to build 17 different project
databases.  Then, we built additional sub-system databases to capture the
interfaces between the 17 lower-level units.  Then, we built one last
database to capture the sub-system interfaces.  All databases are
symbolically linked to all files in parent databases.  Maintenance of those
links, and ensuring that changes in higher level databases are generated
into lower level databases is quite a chore.

>Any specific stress points, any experiences good or bad with any CASE tools

So far, our experiences with IDE have not been very good.  We are starting
to get visible results, and we are pleased with them, but the level of
effort it has taken to get where we are is significantly greater than we
expected (ie: I have done nothing but work on templates, scripts, bug
fixes, work-arounds, etc full-time, since April.)  In addition, the level
of support from IDE has been quite minimal.  In the 18 months we have been
using this tool, I have reported dozens of bugs and problems.  I have not
yet seen a single one of these bugs or problems corrected.  Now, to be fair,
I have not yet received the production release of 4.2, and many of the bugs
I found were in the beta release.  Hopefully, some of them got fixed.

>any pointers to other work in this area would all be appreciated a LOT.

Mike Beaver (beaver@sun.com) is the chairman of the Northwest chapter of
the IDE User Group.  He recently sent a letter to IDE reporting on the
most recent User Group meeting.  In that report, he chastised IDE for spending
too much effort on developing new products, new interfaces, and new bugs, and
not enough effort on fixing whats currently in users hands.

It is not my intent to air dirty laundry in public.  However, I caution
anyone looking at CASE technology to evaluate the company, as well as its
products, and determine the level to which they can and will support you in
doing your work.

>Thanks,

You're welcome.

Chuck
leadsv!cberg