john@tirnan.UUCP (John Richartz) (06/10/89)
Thanks to Mark Runyan (runyan@hpirs.HP.COM) and Mark Valentine
(mark@spider.co.uk) for responding to my suggestion for discussion
of these issues.
It strikes me that comp.software-eng might be an appropriate place
for discussion, since it seems to be fairly low volume now, and
they're currently pretty hung up on whether software engineering
really exists and whether you're better off training an Engineer
to develop software than to teach a Software guy engineering.
Mr. Valentine does, I believe, characterize the situation fairly
well. I've worked with several (mostly ad-hoc) variants of
development based on either SCCS or RCS and find that the structure
is generally not in place to provide a development model that
allows for realistic development of a Product and any variants and
descendents of it (admittedly this often results from lack of
organizational focus on the problems). I believe that this is due
to an inherent weakness in the model of a product as the state of a
particular set of source files, rather than as the collection of
source files and configuration specifics that define a particular
instance of the product.
"Source Control" is only a part of the problem and one can feel
fairly comfortable knowing that the developers have retained a file
revision history for each file, and one can certainly snapshot the
development at product release by treating the product as the list
of files in the source control system at some revision (the
software bill of materials kind of approach, useful with both SCCS
and RCS). But this gets really messy when you've got either
multiple similar products, or perhaps site specific versions. So we
immediately find ourselves considering a higher level abstraction of
the product - and ending up with "Configuration Control/Management"
rather than just source archiving.
So - comp.software-eng?
john richartz
!tektronix.tek.com!reed!tirnan!johnrender@m.cs.uiuc.edu (06/13/89)
Written 11:36 am Jun 10, 1989 by john@tirnan.UUCP: > Mr. Valentine does, I believe, characterize the situation fairly > well. I've worked with several (mostly ad-hoc) variants of > development based on either SCCS or RCS and find that the structure > is generally not in place to provide a development model that > allows for realistic development of a Product and any variants and > descendents of it (admittedly this often results from lack of > organizational focus on the problems). I believe that this is due > to an inherent weakness in the model of a product as the state of a > particular set of source files, rather than as the collection of > source files and configuration specifics that define a particular > instance of the product. I'm curious as to what you consider "configuration specifics." By this do you mean such machine-dependent characteristics as word-size and byte-ordering or do you have a higher level notion? I ask because I'm one of the people doing research into software configuration management and I'm interested in what other people see as shortcomings of the SCM and version control systems they have used. If you haven't seen it, you might wish to take a look at the Shape system being distributed to the public in comp.sources.unix. It is an example of the current trend in UNIX-based SCM systems and represents some considerable advances over SCCS and RCS. It seems to do things that you sound like you need. > "Source Control" is only a part of the problem and one can feel > fairly comfortable knowing that the developers have retained a file > revision history for each file, and one can certainly snapshot the > development at product release by treating the product as the list > of files in the source control system at some revision (the > software bill of materials kind of approach, useful with both SCCS > and RCS). But this gets really messy when you've got either > multiple similar products, or perhaps site specific versions. So we > immediately find ourselves considering a higher level abstraction of > the product - and ending up with "Configuration Control/Management" > rather than just source archiving. As you mention, SCCS and RCS fall short when you need to do SCM for larger projects. There are several recent systems which attempt to address this problem. You might wish to take a look at the proceedings from last year's IEEE workshop on software version control for examples. Also take a look at the last proceedings of the SIGSOFT/SIGPLAN Symposium on Practical Software Development Environments. They talk about 3 or 4 systems (Shape is among them) which do some of the things you seem to want. For more background, there was a bibliography on the subject in SIGSOFT Software Engineering Notes 11:3 (July 1986). It's pretty complete up to the time it was published and lists 76 articles. > So - comp.software-eng? A glib response would be "So - what?" I don't really know what you want to talk about. If you want suggestions for solutions to your problems, take a look at the proceedings I mentioned. I can also re-post a list of vendors of SCM systems that someone posted a month or two ago. If you want to discuss the issues involved in SCM, I'd be happy to, though I have to have someplace to start. Possible topics include database support for SCM, distributed SCM, formal models of SCM, language-based support of SCM, object-oriented SCM, improving the efficiency of SCM systems, SCM in programming-in-the-large, and integrating SCM into a development methodology. Anybody want to throw out the first ball? > john richartz > !tektronix.tek.com!reed!tirnan!john Hal Render render@cs.uiuc.edu
pgh@stl.stc.co.uk (Peter Hamer) (06/13/89)
In article <133@tirnan.UUCP> john@tirnan.UUCP (John Richartz) writes: > > I've worked with several (mostly ad-hoc) variants of > development based on either SCCS or RCS and find that the structure > is generally not in place to provide a development model that > allows for realistic development of a Product and any variants and > descendents of it ... > > "Source Control" is only a part of the problem ... I know the problem well! A configuration management system called Shape has just been distributed on comp.sources.unix. You will probably find the papers included in the distribution well worth the effort of unpacking all 33 parts; they are very interesting. Shape is in some ways yet another SCCS/RCS combined with yet another Make. However, it is built on an 'Attribute FileSystem' which maintains a distinction between objects and their attributes. This enables Make requests of the form 'select the newest version of all components I am working on and the newest published version of all other components' to be used. Similarly, some of the problems of variants seem to be addressed; both when the variants are realised as separate sources, and when they are realised as a common source with different compiler-time flags. I would be interested in other peoples comments on the approach adopted by the Shape team, and any feedback from people using it [or trying to use it]. Please read the Shape papers and don't rely on the accuracy of my summary.
joe@gistdev.UUCP (06/13/89)
> Written 11:36 am Jun 10, 1989 by tirnan.UUCP!john in comp.software-eng > [...] I've worked with several (mostly ad-hoc) variants of > development based on either SCCS or RCS and find that the structure > is generally not in place to provide a development model that > allows for realistic development of a Product and any variants and > descendents of it (admittedly this often results from lack of > organizational focus on the problems). I believe that this is due > to an inherent weakness in the model of a product as the state of a > particular set of source files, rather than as the collection of > source files and configuration specifics that define a particular > instance of the product. I think that this is true in several regards. When we set up our source code control system (we use SCCS, and yes, we looked at RCS), we found (like everyone else), that controlling source files is not enough. You need the ability to track the state of a "program" (I'll define that in just a moment) including not just source code, by documentation, make files, testing files, and any other associated text. Our model of what a "program" is varies depending on the nature of the software. A program is usually a traditional set of source code that compiles to make a binary. However, a program could be a set of text files, a simple UNIX shell script, a source code library, a binary library -- anything that can be tracked back to a set of controlled text files and can be given a collective version number. Our SCCS implementation includes a set of UNIX shell utilities which track versions of each "program", and the scripts do the house-keeping as to which versions of which files comprise the program. For our products, then, we maintain an even higher level set of scripts which track which programs (remember, a program could even be a simple set of text files) made up each version of the product. In this way, from a single product version number, we can easily back-track to each version of each file that comprised that product. We have found that this model has worked for us, even in situations where parallel copies are under development, or patches must be made to older releases. But I think the key is that just tracking source files is not enough, because that's just too low a level to reasonably track. It is better to control things at the program level, and let the computer deal with the details of individual files for you. Then you can concentrate on the organization issues mentioned above. ------------------------------------------------------------------------------- Joe Brownlee | The best diplomat I know is a fully activated phaser bank. GIST, Inc. | -- Montgomery Scott 1800 Woodfield Dr. | Pay attention to what I say, and you might start a trend. Savoy, IL 61874 | ARPANET: joe%gistdev@uxc.cso.uiuc.edu (217) 352-1165 | UUCP : {uunet,pur-ee,convex}!uiucuxc!gistdev!joe -------------------------------------------------------------------------------
johnm@uts.amdahl.com (John Murray) (06/14/89)
In article <133@tirnan.UUCP>, john@tirnan.UUCP (John Richartz) writes: > > [... Having used] either SCCS or RCS[, I] find that the structure > is generally not in place to provide a development model that > allows for realistic development of a Product and any variants and > descendents of it (admittedly this often results from lack of > organizational focus on the problems). . . . > [...Things get] really messy when you've got either > multiple similar products, or perhaps site specific versions. A development group frequently suffers from tunnel vision, to the extent that it cannot see how it fits into the entire organization. This seems to be a real problem where both hardware and software are being developed simultaneously, or more particularly, when existing software is being modified to run with new hardware. I like to think of this activity as software "maintenance" rather than "new design". However, organizations prefer to act as if they were designing from scratch, even though they subsequently find themselves porting software fixes forward from earlier versions, and so on. Here's an interesting logistics problem. The prototype hardware has a bug which will take several weeks to fix. A temporary fix is applied to software to get around the problem and allow alpha testing to continue. Software development is still continuing at the same time. When the hardware problem is resolved, how do you back off the temporary fix in an orderly fashion? (This is basically an instance of the site-specific problem, I suppose). Even establishing an appropriate mechanism for tracking the changes can be a paperwork nightmare. At Amdahl, one system we use tracks problem fixes and new s/w enhancements simultaneously, and no source changes are allowed without a corresponding entry in the tracking database. (See Software Engineering Notes, ACM, Oct 1988 for my paper on it.) Merging such software control with the corresponding hardware change control system seems difficult, however; I'd be very interested in hearing opinions on this subject. - John Murray, Amdahl Corp, Sunnyvale, Ca. (My own opinions, etc.)
hull@dinl.uucp (Jeff Hull) (06/15/89)
In article <39400023@m.cs.uiuc.edu> render@m.cs.uiuc.edu writes: >Possible topics include database support for SCM, distributed SCM, >formal models of SCM, language-based support of SCM, object-oriented SCM, >improving the efficiency of SCM systems, SCM in programming-in-the-large, >and integrating SCM into a development methodology. Anybody want to throw >out the first ball? While having nothing to contribute at this time, I would like to read about formal models of SCM (with English descriptions) and object-oriented SCM. -- Blessed Be, Jeff Hull ...!ncar!dinl!hull 1544 S. Vaughn Circle 303-750-3538 It was great when it all begaaaaan, Aurora, CO 80012 I was a regular <USENET> faaaan, ....
runyan@hpirs.HP.COM (Mark Runyan) (06/16/89)
>/ hpirs:comp.software-eng / render@m.cs.uiuc.edu / 6:00 pm Jun 12, 1989 / >As you mention, SCCS and RCS fall short when you need to do SCM for larger >projects. There are several recent systems which attempt to address this >problem. You might wish to take a look at the proceedings from last year's >IEEE workshop on software version control for examples. Also take a look >at the last proceedings of the SIGSOFT/SIGPLAN Symposium on Practical Software >Development Environments. They talk about 3 or 4 systems (Shape is among them) >which do some of the things you seem to want. For more background, there >was a bibliography on the subject in SIGSOFT Software Engineering Notes 11:3 >(July 1986). It's pretty complete up to the time it was published and lists >76 articles. While I haven't completed reading the references you gave, I have noted one major issue it what you are pointing out. There may exist several SCM systems out there in the market place, but getting them "in house" is not a simple process. I'm familiar with several systems that "almost" do what we need here in our lab, but they all have some shortcoming that makes it difficult to convince management to purchase the systems. As an example, what I've read so far on SHAPE indicates that it is a good replacement for make and RCS, which makes it sound like we could easily replace our ad hoc system with SHAPE. However, in my scanning of the information, I've yet to encounter any reference on how SHAPE works with networks. If my development is spread out on several machines, can SHAPE "transparently" access the source for me, or am I still going to need the ad hoc support procedures to manage network access? This doesn't seem like a major problem to most people, but it will be a stumbling block to selling management on spending the development time to replace our tools with a tool set that doesn't do "everything". As another example, I'll pick on an Un-named Product X (UPX). This system appeared to do everything, but it had a basic design assumption that would make it impossible for us to accept. Most SCM systems assume there is one major line of development with the occasional "branch" or divergent development. Performance usually suffers if you are on one of these divergent developments. However, in a research lab, there is almost always divergent development going on. The next two or three releases or new hardware. These "branches" sometimes fade into nothing when the project is proven to be unnecessary, but they must not suffer from poor performance. What I'm trying to say is, there are several tools in existance, but using them almost always requires that you adopt the work style required by the tools and this work style is most probably different than your current one. This requires changing the work habits of several engineers, some of which may still not understand the need for SCM. Mark Runyan
render@m.cs.uiuc.edu (06/19/89)
Sorry if it seems I'm talking too much, but no one else from the SCM
research people seems to be jumping in. Onward ...
Written 9:35 am Jun 16, 1989 by runyan@hpirs.HP.COM:
[Several good points about the problems of distributed SCM and getting
developers to accept SCM restriction.]
From what I've read, Shape does not specifically address the problems of
distributed SCM except to say that you can use it with NFS but will run
into some troubles. Just to make things clear, I am in no way connected
with the Shape people and have only just gotten their software. I haven't
even tried it out much, so my knowledge of it is limited to reading one
of their papers and looking at the release docs a bit.
To me it seems that the problems of distributed SCM are similar to the
problems of distributed databases, i.e. distributed data and concurrent
updates. I know of non-DB SCM systems which attempt to solve this, among
them Apollo's DSEE (which unfortunately requires Apollo hardware and OS
support to work) and DVSS, a distributed version server for CAD applications.
The groups that build their SCM systems around a database (representing
development modules as data items and putting version control into the DBMS)
can presumably use distributed DB techniques to solve the problem.
Surprisingly, this has not seemed to be a popular research topic, but it
may just be that I haven't seen the right papers.
Of the approaches to distribution that I've read about, the popular one is
the idea of putting all your controlled modules in one place (a server)
and accessing them only using controlled checkin/checkout commands. Depending
on the method/system you use, you can exclusively lock modules for update,
presumably preventing overlapping modifications. The problem is that this
write-locking hoses you if you want multiple mods done simultaneously by
different developers. It hoses you even further if the system prevents any
access to a module undergoing update. My opinion is that the system should
allow the users to specify the locking strategy to use, either one checkout
at a time or as a system-wide policy. RCS gives you some flexibility, the
prototype SCM system I built gives you a little more, and I don't doubt other
systems do as well.
Another problem with distribution is speed of access. This can be solved
by caching versions and by establishing "domains of control" that localize
the modules and versions needed by a particular group on a particular
machine. The identification of the modules then becomes a function of
the work breakdown for a project; when a project is laid out, the modules
a particular group will be responsible for can be located on their machine(s).
This would not only speed up access, it will encourage encapsulation of
the code they develop.
The problems you mention with branching versions is somewhat new to me.
I don't know of any current SCM systems that don't allow branching versions,
unless they have been developed by shops where such things are no-nos.
If there is a degradation in performance in working on branches, it may
be due to the storage of the branching versions. Most source code versions
are stored as deltas, lists of the textual differences between one version
and it's successor or predecessor. To checkout a version, the deltas must
be applied to some "base" version that is stored in its entirety. In
RCS, this version is the most recent "trunk" version, the one that is
on the main line of descent from the initial version of the module.
To checkout a branch version, RCS must first apply all the differences
between the base and the version at which the branch begins and then
apply the deltas along the branch until the desired one is recreated.
This can take time, and there hasn't really been an improved version of
this scheme devised. Off the top of my head, I guess you could store
heavily used versions in their entirety regardless of whether they
are on a branch or not. This would take up more space but would reduce
the time it takes for checkin/checkout. From a research point of view,
I think someone should put together a SCM system and test it in the
field using different storage schemes, mixing deltas, full copies and
compressed copies to see the time/space behavior and the user response.
You may even be able to do such a study based purely on update frequency
and the growth characteristics of module version trees in a particular
shop. This would show you things like how often revisions are generated,
how many branches are made, and how often old versions are accessed.
I know of a little work like this done years ago with SCCS, but nothing
recently.
Despite the fact that a good SCM could greatly benefit both developers and
managers, selling SCM and SCM systems to people seems to be a major chore.
Part of the problem is that it imposes constraints on developers. I don't
know if you can get around this anymore than you can get around the fact
that using a strongly-typed programming language will prevent you from
doing some things. Fortunately, if the system is well designed, the things
that it prevents you from doing are things that you shouldn't be doing
anyway. For example, one programmer should not be modifying the same version
as another without knowing what the other is doing. Unfortunately, there are
still a lot of SCM things we don't know how to support cleanly and painlessly.
Those of us doing SCM research are trying to remedy this, but it's slow going
and not outrageously popular.
Hal Render
render@cs.uiuc.educampbell@redsox.bsw.com (Larry Campbell) (06/20/89)
In article <9630001@hpirs.HP.COM> runyan@hpirs.HP.COM (Mark Runyan) writes:
(some stuff with which I agree, and then this statement that I couldn't
resist picking at:)
-This requires changing the work habits of several engineers, some of which
-may still not understand the need for SCM.
If they don't understand the need for SCM, then they're not engineers,
they're programmers, and pretty feeble ones at that.
--
Larry Campbell The Boston Software Works, Inc.
campbell@bsw.com 120 Fulton Street
wjh12!redsox!campbell Boston, MA 02146steve@umigw.MIAMI.EDU (steve emmerson) (06/20/89)
In article <39400028@m.cs.uiuc.edu> render@m.cs.uiuc.edu (Hal Render) writes: > >The problems you mention with branching versions is somewhat new to me. >I don't know of any current SCM systems that don't allow branching versions, >unless they have been developed by shops where such things are no-nos. Since shape has been mentioned (and I'm investigating it) I thought I'd mention that it is one SCM system that doesn't explicitly allow branching. Instead, I believe one can use its "variant" concept to achieve similar functionality. The authors argue that branching isn't *that* important in practice, and, in those cases where it is important, the end result is usually a variant (i.e. parallel) line of development. (These assertions have been true in my work). -- Steve Emmerson Inet: steve@umigw.miami.edu [128.116.10.1] SPAN: miami::emmerson (host 3074::) emmerson%miami.span@star.stanford.edu UUCP: ...!ncar!umigw!steve emmerson%miami.span@vlsi.jpl.nasa.gov "Computers are like God in the Old Testament: lots of rules and no mercy"
steinar@fdmetd.uucp (Steinar Overbeck Cook) (06/21/89)
In article <785@redsox.bsw.com>, campbell@redsox.bsw.com (Larry Campbell) writes: > In article <9630001@hpirs.HP.COM> runyan@hpirs.HP.COM (Mark Runyan) writes: > > > -This requires changing the work habits of several engineers, some of which > -may still not understand the need for SCM. > > If they don't understand the need for SCM, then they're not engineers, > they're programmers, and pretty feeble ones at that. I could not agree more with that last statement :-). I have been following this discussion only for a short time so please excuse me if I have missed something, but I have only seen the product shape mentioned (which I got from the net, but have not used yet). Are there any other SCM products available around ? I would be very interested in hearing about them. I'm currently developing a SCM system for in-house use. It is based upon a database which contains all the information related to the software development phase of a project, and SCCS which we use for storing the source code itself. I must hurry and add that we have a very heterogeneous environment with 3 different operating systems on 2 different hardware platforms, connected by a XNS LAN. Being far away from USA and the university atmosphere, I do feel a litle lonely developing a SCM without too many impulses from the outside. I am very pleased that someone started this discussion. -- Steinar Overbeck Cook, Fellesdata a.s, P.O. Box 248, 0212 OSLO 2, NORWAY Phone : +47 2 52 80 80 Fax : +47 2 52 85 10 E-mail : ...!mcvax!ndosl!fdmetd!steinar or steinar@fdmetd.uucp <The opinions expressed, if any, do not represent Fellesdata a.s>
maslen@polya.Stanford.EDU (Thomas Maslen) (06/21/89)
In article <39400028@m.cs.uiuc.edu> render@m.cs.uiuc.edu writes: >To me it seems that the problems of distributed SCM are similar to the >problems of distributed databases, i.e. distributed data and concurrent >updates. I know of non-DB SCM systems which attempt to solve this, among >them Apollo's DSEE (which unfortunately requires Apollo hardware and OS >support to work) and DVSS, a distributed version server for CAD applications. Hope this isn't too commercial for this newsgroup, but... are people also aware of Sun's NSE? It runs on Suns under (I believe) SunOS 3.5 (and 4.0 Real Soon Now, if not already). It lets one manipulate revisions of source trees (not just single source files) and has some notion of variants for derived files. Reflecting a difference between industry and academia, parts of it are implemented in the kernel so that your favourite binary (editor, compiler, whatever) can work happily with NSE (no relinking needed). >Of the approaches to distribution that I've read about, the popular one is >the idea of putting all your controlled modules in one place (a server) >and accessing them only using controlled checkin/checkout commands. Depending >on the method/system you use, you can exclusively lock modules for update, >presumably preventing overlapping modifications. The problem is that this >write-locking hoses you if you want multiple mods done simultaneously by >different developers. It hoses you even further if the system prevents any >access to a module undergoing update. My opinion is that the system should >allow the users to specify the locking strategy to use, either one checkout >at a time or as a system-wide policy. RCS gives you some flexibility, the >prototype SCM system I built gives you a little more, and I don't doubt other >systems do as well. One of the interesting decisions made by the NSE folks was to recognize that exclusive locking is often too restrictive. Their model lets you "check out" a revision (which gives you a consistent snapshot of the system) but can allow others to add revisions in the meantime. When you finally go to check yours back in, if changes have been made in the interim, you go through a (largely automated) reconciliation/merge process. Disclaimer: I spent a very interesting summer in a group at Sun doing something vaguely related to the NSE, so I'm hardly unbiased. Thomas Maslen maslen@polya.stanford.edu
dnk8842@ultb.UUCP (D.N. Kadoch) (06/22/89)
In article <8100004@gistdev> joe@gistdev.UUCP writes: >I think that this is true in several regards. When we set up our source code >control system (we use SCCS, and yes, we looked at RCS), we found (like >everyone else), that controlling source files is not enough. You need the >ability to track the state of a "program" >including not just source code, by documentation, make files, testing files, >and any other associated text. I use RCS and ran into the same problem. My quick solution was to use forced updates, meaning every time I wanted to check something in, EVERY source/doc was checked in even if some had not changed. That way, I can just check out a specific revision and get the entire working source quickly. This method is probably not too good when several people are working on a project, or at least when more than one person is editing a source file at the same time. I also had ALL the files checked out whenever working on the program, which is not very common practice. But for a single person working on a program, it is adequate. -- /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ > Dani Kadoch @ Computer Science House @ Rochester Institute of Technology < > USNail: 14 Loden Lane, Rochester, NY 14623 (716) 359-4647 < > UUCP:..!rochester!rit!ultb!dani MCIMail:dani BITNET:dnk8842@ritvax <
render@m.cs.uiuc.edu (06/22/89)
Written 2:02 am Jun 21, 1989 by maslen@polya.Stanford.EDU: >Hope this isn't too commercial for this newsgroup, but... are people >also aware of Sun's NSE? It runs on Suns under (I believe) SunOS 3.5 >(and 4.0 Real Soon Now, if not already). It lets one manipulate >revisions of source trees (not just single source files) and has some >notion of variants for derived files. Reflecting a difference between >industry and academia, parts of it are implemented in the kernel so >that your favourite binary (editor, compiler, whatever) can work >happily with NSE (no relinking needed). I've heard of NSE, but haven't seen any write-ups of it. (If anyone knows of a good one, I'd appreciate the reference.) From what you've described, it sounds a lot like Apollo's DSEE, which has been out for about 5 years now. One advantage SUN has, obviously, is a larger market for their product. I'd be interested in hearing of any conceptual/technical improvements NSE has over DSEE. Hal Render render@cs.uiuc.edu uiucdcs!render
campbell@redsox.bsw.com (Larry Campbell) (06/23/89)
There is a commercial product around called Aide-de-Camp, put out by a local company called something like Software Maintenance and Development Systems. I've not used it, but I have looked through the documentation, and it looks very promising. The main guy at SMDS, Richard Harter, is on the net somewhere (hi, Richard!); perhaps he'll see this and offer more information. -- Larry Campbell The Boston Software Works, Inc. campbell@bsw.com 120 Fulton Street wjh12!redsox!campbell Boston, MA 02146
psrc@pegasus.ATT.COM (Paul S. R. Chisholm) (06/23/89)
One of the questions raised in this discussion (in an article I lost) is about keeping track of versions of the *product*, not just versions of the source files. (Like other members of this discussion, I think that "source files" includes specifications, design documents, test plans and scripts, end user manuals, etc.) One useful method of doing this is to use "slists". An slist is a list of which versions of which source files you're using at any given time. (In the SCCS world, it's a "what" of the sources*, plus some header information with percent strings in it.) An slist is generated mechanically from the source files, and itself kept under source code control. (*Originally, an slist was a "what" of the product. This lead to an odd chicken-and-egg problem: the slist is a "what" of the product, but the product has to have the slist version ID in it! We had a kludgey two step manufacturing process for this. Checking the consistency of the version IDs in a product is still a good idea.) Slists are used to get a snapshot of the product from the source code database. If I want to look at the currently distributed version of a certain product, I don't have to look for dates; I "get" ma.sl at the appropriate version (2.103.1.3 or some such), and then "get" the appropriate source files at the versions listed in the slist. If I want to turn it over to system test for manufacturing, I just tell them the slist name and number; they get the slist, the makefile and other source files at the appropriate versions, and build. "Branch deltas" (supporting a tree of different changes) are a handy feature; read the get(1) and delta(1) manual pages for details. In practice, we find that they work very nicely. Another approach is to support a number of parallel "generics". For example, I might have a 2.0 generic (which includes bug fix releases such as 2.01 and 2.02), and a 2.1 generic (which will include a whole host of new features, when it eventually is released). Each change is applied to the appropriate generics. (Adding a new way of editing would only be added to the 2.1 generic, but every generic would get a bug fix that prevented a core dump.) By the way, for any of you building products on a UNIX(R) System V (release 3 only?) platform, I strongly recommend using the mcs command to get rid of redundant SCCS IDs. Your code might not have these, but the files in /usr/include have lots of long ones. I trimmed 100Kbytes of ugly fat off of one program with this, and shrunk another by forty percent! Paul S. R. Chisholm, AT&T Bell Laboratories att!pegasus!psrc, psrc@pegasus.att.com, AT&T Mail !psrchisholm I'm not speaking for the company, I'm just speaking my mind. UNIX(R) is a registered trademark of AT&T. So far as I know, outside of SCCS utilities that form a foundation for internal tools, we don't sell tools that support slists.
andy@coma.UUCP (Andreas Lampen) (06/23/89)
In article <39400028@m.cs.uiuc.edu>, render@m.cs.uiuc.edu writes: > ... Just to make things clear, I am in no way connected > with the Shape people and have only just gotten their software. I confirm that, I am one of the developers of SHAPE =:-) . > To me it seems that the problems of distributed SCM are similar to the > problems of distributed databases, i.e. distributed data and concurrent > updates. I know of non-DB SCM systems which attempt to solve this, among > them Apollo's DSEE (which unfortunately requires Apollo hardware and OS > support to work) and DVSS, a distributed version server for CAD applications. The retrieval of SCM specific data from remote machines in a network is only one part of the problem "distributed SCM". When focusing on the aspect of identifying and building configurations within SCM systems, we have to discuss at least two different aspects of distribution: - distributed identification of source components and - distributed system building. The first point is conceptually quite uncritical. I agree that this is mainly a technical problem that has basically been solved yet. BTW, SHAPE's approach to use NFS in order provide network support was a pragmatic decision (due to lack of time) and I think it works quite reasonable. Hal is right when he writes that there still are some problems but most of them will be solved in the near future. Nevertheless I would like to introduce an own network service but we have many other plans with higher priority. The second point (distributed system building) is much more complicated. It sounds simple to say "I have a lot of modules to be compiled and linked together, so I construct single compile jobs, spread them all over my network, gather the results, and link the whole stuff together". This is what eg. DSEE and some "make"oids do. But this works only in a homogeneous environment of only APOLLOs or only SUN3s or only VAXes. Even then you get in trouble when the software environment (eg. /usr/include) is different on different machines in your network. DSEE solves this problem by copying not only the source file that is to be compiled but also *all* other files needed for the compilation to the compiling machine. With a mechanism like this you are totally lost in a heterogeneous environment. For example our network at TU Berlin consists of a large number of SUN3s, several VAXes (running BSD 4.3, Ultrix, and VMS), an IBM mainframe, some HP machines running HP/UX and and and. In our discussion on the problem of distributed system building in a heterogeneous network we came up with the concept of 'compatibility classes'. A 'compatibility class' would typically comprise a set properties that define a notion of 'compatibility' for binaries. All machines in a network that are equivalent in terms of these properties should then be assigned one (or more ?) such class(es). Significant parameters include: machine type, operating system (-version), compiler version, object file format, file system environment... . But this is still no *general* solution for managing distributed system building. For example we will have to provide different levels of (binary-) compatibility: in our case, VAX/43BSD and VAX/Ultrix are compatible on object-file level but (sometimes!) incompatible on executable level (different run time systems). What influence will dynamic loading have ? How should cross compilers be managed ? I'll come up with an answer to these questions next week =:-). Andy ----- P.S.: For people interested in a technical discussion on SHAPE I recommend to subscribe to our brand new mailing list on SHAPE. Send you subscription request to "shape@coma.uucp" or "shape@db0tui62.bitnet". -- Andreas Lampen, TU Berlin UUCP: andy@coma (unido!coma!andy) BITNET: andy@db0tui62
joe@gistdev.UUCP (06/23/89)
> Written 11:26 pm Jun 22, 1989 by pegasus.ATT.COM!psrc in comp.software- > One useful method of doing this is to use "slists". An slist is a list > of which versions of which source files you're using at any given time. > (In the SCCS world, it's a "what" of the sources*, plus some header > information with percent strings in it.) An slist is generated > mechanically from the source files, and itself kept under source code > control. This is exactly how our implementation of SCCS product control works. We keep an "slist" for each module with all of the versions of the source files which were used to build it. This is great for versions of individual programs, but what about the product? We also keep a product "slist". We have standards to be sure that the appropriate labelling is compiled into each released binary (or even source or text file), and we then do a "what" on the entire product tree into a master "slist", which itself is controlled by SCCS! So, for example, a customer has a problem in the Foo Editor which is part of our WhizBang product. The customer has WhizBang version 4.2. I pull the list of binaries in WhizBang 4.2 and search (grep -- after all, this is under UNIX) for the label which says "Foo Editor" (it's SCCS Module Name, which is is the master slist), and find it was version 2.1. I can then pull the source to the 2.1 Foo Editor, which our tools produce by using "get" to get the appropriate version of each file based on the program slist, and I'm in business. As stated before, even if subsequent releases have been made since 2.1 of the Foo Editor, I can still make a branch version to fix this for the customer. This kind of control is invaluable. I have worked in environments where there was no control of any kind, and noone had the slightest idea what customers had. I can't even imagine going back to that kind of environment today! ------------------------------------------------------------------------------- Joe Brownlee | Captain, please -- not in front of the Klingons. GIST, Inc. | -- Mr. Spock, Star Trek V 1800 Woodfield Dr. | Pay attention to what I say, and you might start a trend. Savoy, IL 61874 | ARPANET: joe%gistdev@uxc.cso.uiuc.edu (217) 352-1165 | UUCP : {uunet,pur-ee,convex}!uiucuxc!gistdev!joe -------------------------------------------------------------------------------
render@m.cs.uiuc.edu (06/24/89)
Written 7:50 am Jun 22, 1989 by dnk8842@ultb.UUCP: > I use RCS and ran into the same problem. My quick solution >was to use forced updates, meaning every time I wanted to check >something in, EVERY source/doc was checked in even if some had not >changed. That way, I can just check out a specific revision and get >the entire working source quickly. This method is probably not too >good when several people are working on a project, or at least when >more than one person is editing a source file at the same time. I >also had ALL the files checked out whenever working on the program, >which is not very common practice. But for a single person working on >a program, it is adequate. There is a set of PD shell scripts called CVS which supports use of RCS by a team of programmers. It's available from the comp.sources.unix repository on uunet.uu.net using anonymous ftp. I don't know how good it is. Hal Render render@cs.uiuc.edu
maslen@polya.Stanford.EDU (Thomas Maslen) (06/24/89)
In article <39400029@m.cs.uiuc.edu> render@m.cs.uiuc.edu writes: > >I've heard of NSE, but haven't seen any write-ups of it. (If anyone knows >of a good one, I'd appreciate the reference.) From what you've described, I'm told that the best reference is "Object Management in a CASE Environment" by E. Adams, M. Honda, and T. Miller, Proceedings of the 11th International Conference on Software Engineering, Pittsburgh, May 1989. I haven't yet seen this myself. >it sounds a lot like Apollo's DSEE, which has been out for about 5 years >now. One advantage SUN has, obviously, is a larger market for their product. >I'd be interested in hearing of any conceptual/technical improvements NSE >has over DSEE. It is certainly intended to deal with some of the same problems as DSEE. I'm not sure I'm competent to do a real comparison with DSEE -- apart from anything else, it's too long since I last read the DSEE papers. However, I *believe* that the NSE approach to checkin/checkout, mentioned in my last posting, was deliberately chosen to be more generally useful than a rigid reserve/replace model. Thomas Maslen maslen@polya.stanford.edu
simmel@ginosko.samsung.com (Sergiu Simmel) (06/25/89)
Has anybody used (at least evaluated, if not used in a real project) any
SCM other than SCCS, RCS, or ad-hoc systems built around these two?
I am interested in SCMs that would run in a Unix environment.
Can anyubody recommend such a system? And if so, please elaborate on the
advantages (to you and your project) over SCCS and RCS.
Please mail to address below, and I will summarize.
Thank you.
================================+==========================================
Sergiu Saul Simmel | VOICE: 508-685-7200x133 FAX: 508-685-4940
SAMSUNG Software America, Inc. | INTERNET: simmel@ginosko.samsung.com
One Corporate Drive | UUCP: {decvax!{gsg,cg-atla},uunet,
Andover, MA 01810 | ulowell}!ginosko!simmel
================================+==========================================campbell@redsox.bsw.com (Larry Campbell) (06/25/89)
I'm not too familiar with SCCS, but with RCS it seems that you could use the "state" field to identify product versions. For instance, if we're going to make a snapshot/baselevel of the source library and call it V1.01.02, we could do: rcs -sV1.01.01 RCS/* Later, if we decide that a bug fix to foo.c needs to be considered part of (backed in to) that baselevel, we just: rcs -sV1.01.01 foo.c To build the baselevel: co -sV1.01.01 RCS/* make We've just started using RCS and haven't actually tried this yet, but it seems like it should be a serviceable approach. -- Larry Campbell The Boston Software Works, Inc. campbell@bsw.com 120 Fulton Street wjh12!redsox!campbell Boston, MA 02146
steve@umigw.MIAMI.EDU (steve emmerson) (06/25/89)
In article <791@redsox.bsw.com>, Larry Campbell <campbell@redsox.UUCP>
describes a possible methodology for configuration management using the
"state" attribute of individual RCS files. In the description, an
example is given in which a bug-fixed module is "back-dated" into an
already released version.
IMHO, this should be done with extreme caution --- if at all. If one
updates a module, but still associates it with a previous version of the
total package, then one has effectively by-passed the primary purpose
of a configuration management system: namely, to unambiguously bind a
release to a set of modules.
It would be better, IMHO, to increment the release level (i.e. "state")
of all other modules --- regardless of the difficulty of doing so.
(Please accept my aplogies if I've misunderstood the referenced article).
--
Steve Emmerson Inet: steve@umigw.miami.edu [128.116.10.1]
SPAN: miami::emmerson (host 3074::) emmerson%miami.span@star.stanford.edu
UUCP: ...!ncar!umigw!steve emmerson%miami.span@vlsi.jpl.nasa.gov
"Computers are like God in the Old Testament: lots of rules and no mercy"runyan@hpirs.HP.COM (Mark Runyan) (06/26/89)
>/ steinar@fdmetd.uucp (Steinar Overbeck Cook) / 11:33 pm Jun 20, 1989 / >(Larry Campbell) writes: >>(Mark Runyan) writes: >>> -This requires changing the work habits of several engineers, some of which >>> -may still not understand the need for SCM. >> If they don't understand the need for SCM, then they're not engineers, >> they're programmers, and pretty feeble ones at that. >I could not agree more with that last statement :-). Of course, I, too, would tend to agree, but then, I am the person responsible for SCM here. It appears that everyone in this discussion believes that SCM is a good idea, but remember that there are various levels. While I believe that being able to tie all the source, documentation, etc together for a given product is a great idea, I may tend to balk at the need for a configuration control board, especially on a "small" to "medium" sized project. Anyway, there *are* various degrees and levels of software configuration, and several of those who have posted appear to be interested in tools and/or procedures that have made life easier for others. If you can start fresh, perhaps DSEE from Apollo or NSE from Sun would be a good idea (assuming you can afford their equipment). However, some of us can't just start over, we have to use the existing equipment and systems and try to find a solution that works. Whatever that solution is, it *has* to be simple to use and train people in. Getting engineers to switch work habits is difficult. Mark Runyan
runyan@hpirs.HP.COM (Mark Runyan) (06/26/89)
>/ maslen@polya.Stanford.EDU (Thomas Maslen) / 12:02 am Jun 21, 1989 / >One of the interesting decisions made by the NSE folks was to >recognize that exclusive locking is often too restrictive. Their >model lets you "check out" a revision (which gives you a consistent >snapshot of the system) but can allow others to add revisions in the >meantime. When you finally go to check yours back in, if changes have >been made in the interim, you go through a (largely automated) >reconciliation/merge process. If there are any technical papers available to the general public on how this reconciliation/merge process is accomplished, I would be extremely interested in reading them! The "need" for exclusive locks exists because of the difficulty of merging changes back in (at least, in my experience). If NSE solves this problem to handle the merge automatically, I'd like to know how its done. (Of course, it may be a company secret for SUN, in which case, I'll not know. But if SUN wants to make it a paper at the next USENIX Workshop on Software Management, I'll be the attentive one in the front row... :-) Mark Runyan
runyan@hpirs.HP.COM (Mark Runyan) (06/26/89)
>/ psrc@pegasus.ATT.COM (Paul S. R. Chisholm) / 9:26 pm Jun 22, 1989 / >One of the questions raised in this discussion (in an article I lost) >is about keeping track of versions of the *product*, not just versions >of the source files. ... >One useful method of doing this is to use "slists". An slist is a list >of which versions of which source files you're using at any given time. While the term "slist" is new to me, the concept isn't, and I agree that this appears to be a good way to track what versions go into a product. (I've encountered this concept in "The DOMAIN Software Engineering Environment for Lage Scale Software Development Efforts" by David Leblang, Robert Chase, and Gordon McLean as printed in the May 1985 issue of IEEE. The term "thread" was used and is part of Apollo's DSEE product). Some care must be taken, though, by those who are working on complex projects. In one project I worked on at another company, the project kept the list of versions (which I'll call an slist for now). The project was fairly complex with several parts. The management team discovered that version A of their product worked (more or less), but version B had a bug, so they wanted to experiment with the slists to make a version A/B to combine the best of both worlds. They didn't quite have a feel for the dependancies of the files, and their modified slist checked out "garbage" that wouldn't compile. The lesson I learned from that was: An slist is a good history tool, but not an efficient development tool. Mark Runyan
runyan@hpirs.HP.COM (Mark Runyan) (06/26/89)
>/ campbell@redsox.bsw.com (Larry Campbell) / 8:27 pm Jun 24, 1989 / >I'm not too familiar with SCCS, but with RCS it seems that you could >use the "state" field to identify product versions. ... The problem of using the "state" field or even the symbolic name field to identify a product's versions is there is no recording of changes to the state field or symbolic name field of an RCS file. An external logging mechinism should probably be used to record that version A was in state X1 at Time T1, but was set to state X2 at Time T2. Another point to consider when using the state field is if your "file" is used in multiple products. If Product P1 was in State S1 and Product P2 was in S2, and both P1 and P2 used the same version, then what state should it be set to? The symbolic names of RCS allow a user to point multiple tags at the same revision number. (Sorry, my own SCM problem deals with source files that exist for multiple source products undergoing development by several different teams. If you aren't in that environment, then don't worry about this paragraph... :-) Mark Runyan
campbell@redsox.bsw.com (Larry Campbell) (06/26/89)
In article <357@umigw.MIAMI.EDU> steve@umigw.miami.edu (steve emmerson) writes:
-In article <791@redsox.bsw.com>, Larry Campbell <campbell@redsox.UUCP>
-describes a possible methodology for configuration management using the
-"state" attribute of individual RCS files. In the description, an
-example is given in which a bug-fixed module is "back-dated" into an
-already released version.
-
-IMHO, this should be done with extreme caution --- if at all. If one
-updates a module, but still associates it with a previous version of the
-total package, then one has effectively by-passed the primary purpose
-of a configuration management system: namely, to unambiguously bind a
-release to a set of modules.
Not at all -- there is no ambiguity, as long as we've not yet released
-- i.e., shipped -- the bits. You're right, though, if we had actually
shipped bits, we'd have to bite the bullet and create a new version.
For example -- let's say it's the final release of a product, but someone
forgot to take the "Warning -- BETA test version" message off the main menu.
We freeze the configuration (creating an appropriate state), do a build,
run the QA test suite, everything checks out OK, we build a tape and then
someone notices the botch. So now we just check out the offending module,
make the edit, back the module into the release, and re-build the release.
As long as we haven't shipped the tape, I see no harm here.
-It would be better, IMHO, to increment the release level (i.e. "state")
-of all other modules --- regardless of the difficulty of doing so.
I disagree. Chances are that other modules have already had edits that
you DON'T want going in to the release -- i.e., edits for the next major
development version.
There is one problem with the scheme I suggested, though, which I noticed
only after I posted the original article. Unfortunately, a revision can
have only one state.
This is a problem because we believe strongly in code reuse. We have
several modules that are used by more than one product. Ideally, we'd
like to be able to say something like:
This is rev 2.4 of foo.c, and it belongs to:
Product A version 1.05
Product B version 2.06
Product C version 1.34
I can't think of any good way (short of building an add-on wart) to do
this in either RCS or SCCS.
DEC's source code control system, CMS, allows you to assign revisions
(they call them generations) to one or more "classes". In the example
above, I'd have classes called, say, "Product_A_V1.05", "Product_B_V2.06",
and "Product_C_V1.34", and rev 2.4 of foo.c would belong to all three
classes.
--
Larry Campbell The Boston Software Works, Inc.
campbell@bsw.com 120 Fulton Street
wjh12!redsox!campbell Boston, MA 02146vic@zen.co.uk (Victor Gavin) (06/26/89)
In article <357@umigw.MIAMI.EDU> steve@umigw.miami.edu (steve emmerson) writes: >In article <791@redsox.bsw.com>, Larry Campbell <campbell@redsox.UUCP> >describes a possible methodology for configuration management using the >"state" attribute of individual RCS files. > >It would be better, IMHO, to increment the release level (i.e. "state") >of all other modules --- regardless of the difficulty of doing so. I've been waiting for someone else to mention this, but since no one has come forward... What's wrong with named versions? Just "rcs -nVersion_1 RCS/*" (or use rcsfreeze, which checks in your files for you as well), and everything has a consistent ``name'', you no longer have to keep each file at the same revision code or make unwieldy abuse of features designed for other uses. vic "This is one man's opinion" -- Sam Malone -- Victor Gavin Zengrange Limited vic@zen.co.uk Greenfield Road ..!mcvax!ukc!zen.co.uk!vic Leeds England +44 532 489048 LS9 8DB
warw@cgch.UUCP (Anthony R. Wuersch) (06/27/89)
In article <133@tirnan.UUCP>, john@tirnan.UUCP (John Richartz) writes: > > [... Having used] either SCCS or RCS[, I] find that the structure > is generally not in place to provide a development model that > allows for realistic development of a Product and any variants and > descendents of it (admittedly this often results from lack of > organizational focus on the problems). . . . ("I think," "I believe," or "I would suggest" applies to all of this. I'll leave these phrases out of the rest.) Let's try an organizational focus, known where I've been as "release engineering". It suggests a context, a problem, and a solution that non-developers understand. Based on that, some evaluation criteria. Grant that "development" is embedded in a larger organization. Grant that Products flow out of smaller suborganizations, one of which is "development". The organization claims that Products have stable forms and properties. The organization can reproduce Products and can distribute them. Products circulate in the organization and go out to users. [NOTE: a post-release bug fix is an example of a Product] Finally, grant that a lot of important talk in the organization refers to specific Products and sets of Products. People inside and outside the organization need to precisely distinguish between one Product and another, and between one set of Products and another set of Products. People also need to be able to refer to a single Product, or to a set of related Products. Developers may *not* need to precisely distinguish between one Product and another, however, if the difference is only a small configuration or a couple of keystrokes. Developers may delay specificity until the later stages of a design. Developers also tend to forget the fine points of Product reproduction, such as the need for a standard environment in which Products will be assembled. One rarely sees a machine in use by developers which keeps its original standard environment. Release engineering is a standard group chartered by the organization to guarantee the proper identification of Products, and to ensure that the organization can reproduce Products by assembling them from parts in standard environments which are available for the long term, using available techniques. An organization needs release engineering when distinguishing between and referring to Products is *more* important to most people in the organization than it is to developers, and when Product reproduction occurs *away* from normal arenas of development. A proper ID must be well-defined, i.e., *any* trained person (not just developers) should be able to answer a question "Does ID $x cover Product instance $i?" with the correct yes or no. Expressed differently, an ID is well-defined if those trained to know what it means can and do use it correctly. The release engineering (RE) group: + executes and improves the assembly process for a Product in a plain standard environment, until it knows the Product is reproducable. It works from instructions given by development, and a goal which development claims that following the instructions will achieve. Development and the group work together until the group is satisfied. + secures all parts for assembly so that they aren't lost or changed; guarantees that all initial conditions for assembly are fulfilled now and for the expected life of the Product. + approves IDs for Products. + accurately answers questions of what Products an ID covers. + ensures that Product IDs are visible (for instance, in software and in report forms) and used correctly throughout the organization. Configuration in development takes on a meaning in relation to release engineering. A configuration is a change to a product which leads to a new Product ID approved by release engineering. SCM systems with mail system interfaces would interest me a lot. For instance, SCM systems which set up IDs so that mail could be sent to a release ID as a bulletin board for anyone interested in the release, and so that mail folders could be aggregated for people interested in a release and following subreleases (i.e., the 1.1, 1.11, 1.12, 1.13, ... mail). I would say just one thing more: It is almost always easier to give missions [charters] to *people* and let them gain experience in carrying them out, then to design systems which will do them better than the people. A learning curve is better than none. For instance, no SCM system I know of satisfies the release engineering *visibility and correct use* requirement. Nor can any SCM system show that IDs are well-defined in the organization. RE needs good people. One RE group I worked with just wanted source code on a tape, instructions on a separate sheet of paper (handwritten was fine if they could read it --- they'd transcribe and edit it anyhow), and binary to "cmp" a processed result against. Simple instructions preferred. Complex scripts frowned upon. Instructions could refer to product IDs already approved by RE. Assembly took place on a standalone machine not connected to any network. All user-visible software had a standard call to read a file containing the product ID. The RE group wrote this file and ran the software to see that its product ID was visible. These are my opinions alone. Toni Wuersch (CIBA-GEIGY Scientific Computing Center) c/o CIBA_GEIGY AG, R-1045.3.32, P.O.Box, CH-4002 Basel, Switzerland UUCP: warw@cgch.UUCP ( ..!mcvax!cernvax!cgch!warw ) Internet: warw%cgch.uucp@uunet.uu.net Phone: (+41) 61 697 31 43 BITNET: warw%cgch.uucp@cernvax.bitnet Fax: (+41) 61 697 32 88
pgh@stl.stc.co.uk (Peter Hamer) (06/27/89)
In article <793@redsox.bsw.com> campbell@redsox.UUCP (Larry Campbell) writes: > Ideally, we'd like to be able to say something like: > > This is rev 2.4 of foo.c, and it belongs to: > Product A version 1.05 > Product B version 2.06 > Product C version 1.34 > >I can't think of any good way (short of building an add-on wart) to do >this in either RCS or SCCS. I haven't done it [yet] but surely you could do something based on the -n [or -N] option of RCS? Something like ... rcs -nProduct_A_1v05:2.4 foo.c,v rcs -nProduct_B_2v06:2.4 foo.c,v rcs -nProduct_C_1v34:2.4 foo.c,v A rlog foo.c will then include the information symbolic names: Product_C_1v34: 2.4; Product_B_2v06: 2.4; Product_A_1v05: 2.4; You can then check out the appropriate version with commands like co -rProduct_A_1v05 foo.c,v or even co -rProduct_A_1v05 *.c,v if you want to check out all source for this release. P.S. Thanks to those contributing to this discussion. It has certainly encouraged me to take another look at the facilities offered by SCCS/RCS. P.P.S. Comp.source.unix distributed a make-oid called cake a year or two back which might also be of interest. Its main attractions were: a MUCH more general way of specifying rules; allowing intermediate files to be deleted (eg if ~.a is used to generate ~.b, which is used to generate ~.c, then if ~.b has been deleted cake will notice that ~.c is older than ~.a and infer that it doesn't need to re-make either ~.b or ~.c); dependencies need to be hard-coded into the 'makefile', but determined at run time by use of a scanning utility (handy for $includes, etc).
andy@coma.UUCP (Andreas Lampen) (06/28/89)
In article <793@redsox.bsw.com>, campbell@redsox.bsw.com (Larry Campbell) writes: > There is one problem with the scheme I suggested, though, which I noticed > only after I posted the original article. Unfortunately, a revision can > have only one state. > > This is a problem because we believe strongly in code reuse. We have > several modules that are used by more than one product. Ideally, we'd > like to be able to say something like: > > This is rev 2.4 of foo.c, and it belongs to: > Product A version 1.05 > Product B version 2.06 > Product C version 1.34 > > I can't think of any good way (short of building an add-on wart) to do > this in either RCS or SCCS. Not true! You can use RCS's mechanism of naming each revision with any number of symbolic names. By this you can attach to Revision 2.4 of foo.c symbolic names like "Product_A-1.05", "Product_B-2.06", *and* "Product_C-1.34". The problem is, that this information is not easy to retrieve with genuine RCS commands (rlog is *very* verbose). Generally, the main precondition for a proper release management is the ability to store the "right information" at the "right time" in the "right place" and to be able to retrieve this information in an appropriate way. I think you can do the things you want with SHAPE. SHAPE's object base (AFS) provides a mechanism to store any number of (freely named) attributes, each of which can hold any number of values, with each version. So, although the SHAPE toolkit does not provide a proper release management (yet), you can introduce a mechanism to keep track of the history of your releases quite easily. As an example you may look at the Shapefile for AFS in the SHAPE distribution. There is a target - release: which is followed by a sequence of (sometimes cryptic) commands that - generate a new release number for AFS and a c-function returning this release number (to be bound into the system) - generate a unique symbolic name for the new release - publish all unpublished (== new) versions belonging to the new release - and marks *each* component version belonging to the release with the generated symbolic name. All these actions are done when typing the command "shape release". Later on, with the command "vl -n Product_A-1.05" you can get an overview of the names and version numbers of all component versions belonging to Release 1.05 of Product_A in the current directory. The question "Which Product versions does version 2.4 of foo.c belong to" is a bit more complicated to answer. The command "vl -U __SymbolicName__ -ux foo.c\[2.4\]" leads to the output - foo.c[1.0]: __SymbolicName__=Product_A-1.05 - Product_B-2.06 - Product_C-1.34 which might not be 100% the output you want. But with a small shell script, a bit sed, a bit awk... even this can be managed with small effort. Andy -- Andreas Lampen, TU Berlin UUCP: andy@coma (unido!coma!andy) BITNET: andy@db0tui62
sartin@hplabsz.HPL.HP.COM (Rob Sartin) (06/28/89)
In article <357@umigw.MIAMI.EDU> steve@umigw.miami.edu (steve emmerson)
refers to the problems of using the RCS "state" attribute to mark
releases.
The two problems of using "state" that way are 1) backdating of modules
to appear "as if" they were in the release and 2) only one "state" is
attached to each revision. I've dealt with that in the past by using
the rcs "symbolic name" feature to attach a name to the source used for
each release.
To do a release you need to traverse all of the directories containing
RCS files marking the latest revision with the release name using rcs -n
and some magic command to find the latest revision. The advantage of
this method is that you don't have to artificially update unchanged
modules between releases.
As one example I was involved in a product that had an emergency bug fix
release (release 4.0) that required modifications to about 3 modules out
of hundreds and hundreds. Using the "state" method I would have had to
either 1) fake the new release to be 3.0 without being able to document
the differences between the original 3.0 and the bug fix release or 2)
do forced checkins of hundreds and hundreds of unchanged modules to get
the state to "Rel4_0". Using the "symbolic names" feature all I had to
do was add symbolic names to the latest revision of the 3 changed files
and 'rcs -nRel4_0:Rel3_0' for everything else.
Somewhere we had a modified version of rcs that supported attaching
symbolic names to the latest revision, eliminating the need for the
"magic command" mentioned before.
Rob Sartin internet: sartin@hplabs.hp.com
Software Technology Lab uucp : hplabs!sartin
Hewlett-Packard voice : (415) 857-7592will@PRESTO.IG.COM (Will Nelson) (06/28/89)
In article <788@redsox.bsw.com>, campbell@redsox.bsw.com (Larry Campbell) writes: > There is a commercial product around called Aide-de-Camp, put out by a local > company called something like Software Maintenance and Development Systems. > I've not used it, but I have looked through the documentation, and it looks > very promising. The main guy at SMDS, Richard Harter, is on the net somewhere > (hi, Richard!); perhaps he'll see this and offer more information. > -- > Larry Campbell The Boston Software Works, Inc. > campbell@bsw.com 120 Fulton Street > wjh12!redsox!campbell Boston, MA 02146 Mr. Harter discusses Aide-de-Camp in his article: Version Management and Change Control. Unix World 6(6): 95-102, 1989. -- Will Nelson Internet: will@presto.ig.com Intelligenetics, Inc. Uucp: orc!ig.com!presto!will 700 East El Camino Real (415) 962-7363 Mountain View, CA 94040
cheung@unc.cs.unc.edu (Clement Cheung) (06/28/89)
In article <39400029@m.cs.uiuc.edu> render@m.cs.uiuc.edu writes: > > >I've heard of NSE, but haven't seen any write-ups of it. (If anyone knows >of a good one, I'd appreciate the reference.) From what you've described, >it sounds a lot like Apollo's DSEE, which has been out for about 5 years >now. One advantage SUN has, obviously, is a larger market for their product. >I'd be interested in hearing of any conceptual/technical improvements NSE >has over DSEE. > >Hal Render >render@cs.uiuc.edu uiucdcs!render There is an article on NSE in the Proceedings of the 11th International Conference of Software Engineering. That article only details how NSE can help parallel development without touching on other aspects of SCM. By the way, I have been reading over 20 technical papers on SCM systems in the past few weeks but I didn't find any which deal with management of object codes. Also lacking is the relationship between SCM and quality assurance. Almost all systems assume that object codes can be derived when necessary. But let suppose. Suppose your software runs on different Unix platforms. It is a huge system. Your development team has finished work for platforms 1 and 2 and have passed the torch to QA. Meanwhile development has begun on platforms 3 and 4. At this point 2 groups of people will be working on 4 sets of source codes and 4 sets of executables. Suppose the executables for each platform is 20 Mb. So we are talking about 80+ Mb of objects code floating around. This is a strain on disk space plus QA will be demanding bug fixes for the software for platform 1 and 2. I see that it is necessary to come up with a scheme to manage the object codes that are flowing around. (The 2 sets of object code for QA are there to stay because QA people will be playing with them constantly.) I would appreciate any reference for managing object code. Clement Cheung.
johnm@uts.amdahl.com (John Murray) (06/28/89)
In article <10217@polya.Stanford.EDU>, maslen@polya.Stanford.EDU (Thomas Maslen) writes: > [...suggests the folllowing as a reference for NSE] > > "Object Management in a CASE Environment" by E. Adams, M. Honda, > and T. Miller, Proceedings of the 11th International Conference > on Software Engineering, Pittsburgh, May 1989. I believe one/more of the authors also wrote a discussion of NSE in a recent issue of Dr. Dobb's Journal. Another source of NSE info is the technical report produced by Sun itself. which was researched and written by William Courington. Contact your local Sun salesbody for a copy, I would think. The problem of merging/reconciling revisions made by different people still involves human interaction in NSE. As far as I can see, the essential difference in NSE is the human interface with the person doing the merging job. It seems that, in most auto-merging systems, data is assumed to be "auto-merge"able unless the two revisions hit the same lines of code. When that happens, then the human controller must make a decision. NSE does appear to make this operation fairly user-friendly. Revision editing systems like IBM's VM/CMS UPDATE, which works with the standard editor XEDIT, can fairly easily be contrived to do a similar job of merging disparate revisions, while requesting the user to resolve individual line conflicts. The essential difficulty with any form of auto-merge is not handling the case of two developers revising the SAME LINE of source. It's really how to ensure that Jack's change to line 25 is FUNCTIONALLY COMPATIBLE with Jill's change to line 75. - John Murray, Amdahl Corp. (My own opinions only. No Amdahl endorsement intended.)
cheung@unc.cs.unc.edu (Clement Cheung) (06/28/89)
In article <9630003@hpirs.HP.COM> runyan@hpirs.HP.COM (Mark Runyan) writes: > >If there are any technical papers available to the general public on how >this reconciliation/merge process is accomplished, I would be extremely >interested in reading them! The "need" for exclusive locks exists because >of the difficulty of merging changes back in (at least, in my experience). >If NSE solves this problem to handle the merge automatically, I'd like >to know how its done. (Of course, it may be a company secret for SUN, >in which case, I'll not know. But if SUN wants to make it a paper at >the next USENIX Workshop on Software Management, I'll be the attentive >one in the front row... :-) > >Mark Runyan There is nothing that will automatically merge 2 lines of development into one yet. What NSE does is to put up some windows on your workstation showing the 2 versions and highlighting the differences. A person has to merge the differences manually using the diff utility. One of the assumption the authors made is that the chance that 2 teams want to work on the same section of code is rare as well as semantic conflicts. This is described in the paper on NSE in the Proceedings of the 11th Conference on Software Engineering. As an aside, there has been research conducted on merging versions of programs considering the semantics, not just blind merge. There is a paper on this research in the 8th Conference on the Principles of Programming Languages Record by Reps, Prins, and Horwitz. (Prins is one of the faculty here at UNC.) As far as I know, the algorithm they came up with can handles only simple programming constructs. It may be awhile before the algorithm can work on full blown languages like C. Clement Cheung
cheung@unc.cs.unc.edu (Clement Cheung) (06/28/89)
In article <1625@zen.co.uk> vic@zen.UUCP (Victor Gavin) writes: >What's wrong with named versions? > >Just "rcs -nVersion_1 RCS/*" (or use rcsfreeze, which checks in your files >for you as well), and everything has a consistent ``name'', you no longer >have to keep each file at the same revision code or make unwieldy abuse of >features designed for other uses. > > vic > >Victor Gavin Zengrange Limited If you say co -l2.0 sysrc.c , RCS will give sysrc.c of version 2.0 or less depending what is the latest version. But if you give a name for version 2.0 of sysrc.c then RCS will only return version 2.0 of sysrc.c if it has it. But identifying a version by name is not powerful enough because it does not convey any information about that particular version. A lot of the research SCM systems identify a version by attributes values. These attributes describe the properties of that version of the program. This concept is already in DSEE. A version that matches the specified attributes values is selected. In DSEE you can also have preferences for your selection. You lists the most desirable properties before the less desirables properties. Clement Cheung.
maslen@polya.Stanford.EDU (Thomas Maslen) (06/28/89)
In article <18MZ02qG393v01@amdahl.uts.amdahl.com> johnm@uts.amdahl.com writes: >The problem of merging/reconciling revisions made by different people >still involves human interaction in NSE. As far as I can see, the >essential difference in NSE is the human interface with the person >doing the merging job. It seems that, in most auto-merging systems, >data is assumed to be "auto-merge"able unless the two revisions >hit the same lines of code. When that happens, then the human >controller must make a decision. NSE does appear to make this >operation fairly user-friendly. Right. I shouldn't have used a nebulous phrase such as "(largely automated)". My apologies to anyone who was misled. >The essential difficulty with any form of auto-merge is not handling >the case of two developers revising the SAME LINE of source. It's >really how to ensure that Jack's change to line 25 is FUNCTIONALLY >COMPATIBLE with Jill's change to line 75. Wot he said. Remember the failure rate of previous cooperative efforts by Jack and Jill. Seriously, if anyone has any insights on this, I'd like to hear about them too. In the absence of anything smarter, it's good to have a system that insists on regression-testing things before blessing them as official revisions, but that's assuming rather a lot about the tests. Thomas Maslen maslen@polya.stanford.edu
steve@umigw.MIAMI.EDU (steve emmerson) (06/28/89)
In article <412@coma.UUCP> andy@coma.UUCP (Andreas Lampen) wrote about
SHAPE(1). This caused me to wonder how it could handle a real-world
software configuration problem example which was recently posted: namely,
correcting a bug in an already-released module which has already been
subsequently modified.
The situation is this: version 2.0 of "foo.c" has been released with version
1.0 of The Product. It has since been modified and is currently at version
3.1 of version 1.7 of The Product. A bug report on the 2.0 "foo.c" comes in.
What to do?
The originator of this example (sorry, I've forgotten the name) solved the
problem by creating a (soon to be a dead-end) branch off of the 2.0 version
of "foo.c" and shipping that off to the user. Shape(1), on the other hand,
disallows branches. I assume other configuration/revision management systems
also disallow branching.
How, then, can this situation be handled by such non-branching systems?
--
Steve Emmerson Inet: steve@umigw.miami.edu [128.116.10.1]
SPAN: miami::emmerson (host 3074::) emmerson%miami.span@star.stanford.edu
UUCP: ...!ncar!umigw!steve emmerson%miami.span@vlsi.jpl.nasa.gov
"Computers are like God in the Old Testament: lots of rules and no mercy"biocca@bevb.bev.lbl.gov (Alan Biocca) (06/28/89)
In article <9630003@hpirs.HP.COM> runyan@hpirs.HP.COM (Mark Runyan) writes: >>/ maslen@polya.Stanford.EDU (Thomas Maslen) / 12:02 am Jun 21, 1989 / >>One of the interesting decisions made by the NSE folks was to >>recognize that exclusive locking is often too restrictive. Their >>model lets you "check out" a revision (which gives you a consistent >>snapshot of the system) but can allow others to add revisions in the >>meantime. When you finally go to check yours back in, if changes have >>been made in the interim, you go through a (largely automated) >>reconciliation/merge process. >If there are any technical papers available to the general public on how >this reconciliation/merge process is accomplished, I would be extremely >interested in reading them! The "need" for exclusive locks exists because >of the difficulty of merging changes back in (at least, in my experience). >If NSE solves this problem to handle the merge automatically, I'd like >to know how its done. (Of course, it may be a company secret for SUN, >in which case, I'll not know. But if SUN wants to make it a paper at >the next USENIX Workshop on Software Management, I'll be the attentive >one in the front row... :-) I attended a presentation at the Sun Users Group West Regional Conference two days ago by Masahiro Honda, the NSE Engineering Manager for Sun. I was in the front row. The session wasn't very well attended, interestingly enough. I have no experience with this product and have heard little about it prior to this brief exposure, so some errors are likely in the following. The Sun Network Software Environment is intended to support Programming in the Many type problems -- many machines, many phases, many tools, many types of objects, many developers. It is designed to integrate well with CASE. NSE manages sets of objects with many versions and many releases. The unusual feature of NSE is 'optimistic concurrency control'. This is apparently the default and allows multiple developers to acquire the same module at the same time for parallel development. It is possible to configure NSE so this is not allowed, so the 'normal' locking type environment is a subset of NSE's capability. They (the developers) have found that allowing parallel development is more of a gain than not doing so, in spite of the greater problem of reconciliation of parallel paths. If management has properly partitioned the developer's tasks so they aren't working on the same problem they tend to touch different areas of the same files in ways that don't conflict. Obviously counter examples would be easy to construct, but the goal is to allow parallelism where possible. I suspect the larger a program gets the less likely it is that programmers are tripping over each other -- providing they are really working on unrelated changes. Controlling edits on a file basis is pretty arbitrary -- depending on how functions are grouped into files this can be fine (one function per file) or impossible (one file period). When a developer attempts to check-in a version that has been checked in since he checked it out NSE tells him he must 'resolve' first. He then does a 'resync' to get the changes followed by a 'resolve' to integrate them, and finally a 'reconcile' to put everything back. The merge/resolve operation can be handled a number of ways. NSE provides a tool (merge-tool?) that will attempt to automatically merge the files textually. It has no syntactical understanding of the files and will successfully complete automatically only if the changes to the two files are non-overlapping. Where they overlap a user interface consisting of three windows and a control panel allows the user to control the process directly (or select an editor and go completely manual). Marty's comments indicate that they have found this system to work quite well in practice. Toward release time they setup 'slots' for each developer to reconcile his sections to prevent a developer from needing to resync/merge more than once during the same reconcile. He also indicated that Apollo's DSEE was similar but was considerably older and didn't take advantage of some newer technology, didn't support the NSE optimistic concurrency control, and didn't support multiple parallel development (bug fixing on old releases while developing new version). On this last point (parallel development) there was some confusion, so I could have read that wrong. Marty also indicated that NSE was more extensible to user defined types than DSEE. NSE does provide for merging of changes to one subtree back to another. This supports the bringing of bug-fixes from old release support into the new version development. I asked him what projects inside Sun are using NSE and he indicated that most new projects are -- the difficulty of changing in older projects is of course a major impediment to any new system. He specifically mentioned the languages group and several others that are using it now. He also said the SunOS group is going to begin using it on their next major rewrite (coming soon). NSE uses VCS rather than RCS or SCCS to control versioning of source code. It has tools to read RCS, SCCS, and makefiles to automatically construct environments for existing projects to ease a mid-project transition. It relies heavily on NFS and uses SunPro make which automatically handles include dependencies without 'make depends', etc. NSE supports development for single-source multiple architecture (we know why they need that) called 'variants'. This produces multiple 'dependent' files from the same source. A very important feature of NSE is the programmer's environment. NSE sets up an environment for the programmer that apparently contains all the files required for the task at hand. The files are 'logical' -- not copies but effectively links -- until changes are made (at which time a copy is made magically). There is also a way to 'force' real copies in the first place in cases where this is desirable (performance, etc). Creation of these environments by hand is a tedious job -- automation here would be a big help. My only complaint about NSE at the moment is the cost. It's very expensive to get your feet wet on the first license. The system is really designed to be site- or group- licensed and the pricing is not bad there, but for a small-medium project it is tough to justify, especially when you cannot setup a working test case. Perhaps Sun should have a cheap single-user version for small project/demo use. Anyone really familiar with NSE please correct/add comments -- I've been eagerly awaiting some real in-depth info from NSE users and this appears to be a good forum for it. Alan K Biocca Lawrence Berkeley Laboratory AKBiocca@lbl.gov
bronson@mfci.UUCP (Tan Bronson) (06/29/89)
In article <1625@zen.co.uk> vic@zen.UUCP (Victor Gavin) writes: ]In article <357@umigw.MIAMI.EDU> steve@umigw.miami.edu (steve emmerson) writes: ]>In article <791@redsox.bsw.com>, Larry Campbell <campbell@redsox.UUCP> ]>describes a possible methodology for configuration management using the ]>"state" attribute of individual RCS files. ]> ]>It would be better, IMHO, to increment the release level (i.e. "state") ]>of all other modules --- regardless of the difficulty of doing so. ] ]I've been waiting for someone else to mention this, but since no one has come ]forward... ] ]What's wrong with named versions? ] ]Just "rcs -nVersion_1 RCS/*" (or use rcsfreeze, which checks in your files ]for you as well), and everything has a consistent ``name'', you no longer ]have to keep each file at the same revision code or make unwieldy abuse of ]features designed for other uses. ] The advantage of marking every file is it's trivial to branch. Usng only symbolic names means that when you need to create a branch you can potentially have a to: find all names which point to that branch point create the new point where that branch will work from (RCS does this) create new points for the other symbols. Here are a few examples which describe what I'm trying to say above: The first branch is easy (for Ckp1), note that only Ckp1 is affected. Before: Head:1.1; Ckp1:1.1 Ckp2:1.1 Ckp3:1.1 After: Head 1.1; Ckp1:1.1.1.1 Ckp2:1.1 Ckp3:1.1 The next branch is messier (for Ckp2), as we'd like to keep a separate branch for Ckp2 (see below) Before: Head 1.1; Ckp1:1.1.1.1 Ckp2:1.1 Ckp3:1.1 After: Head 1.3; Ckp1:1.1.1.1 Ckp2:1.2.1.1 Ckp3:1.3 Alternatively one could make Ckp2:1.1.1.1.1, but requires Ckp1 to be changed to Ckp1:1.1.1.2 to keep RCS `branching automatically' (see below) If one creates a new `RCS revision' for every version, the resulting RCS looks something like this: Head 1.4; Ckp1:1.1 Ckp2:1.2 Ckp3:1.3 This is revision wasteful, but you can now `co -lCkp1' and when you check in the file a branch will be created automatically by ci. My configuration management package uses the later mechanism now, but people object to revision number changing when nothing has changed, so I've been trying to figure out how one could do `delayed' branching. Has anyone solved the general case? Delayed branching is great for code which stays the same, but but once you have to do the delayed branch it's a mess. Maybe that's the time to something like: Before: Head:1.1; Ckp1:1.1 Ckp2:1.1 Ckp3:1.1 After: Head 1.4; Ckp1:1.1 Ckp2:1.2 Ckp3:1.3 and of course this can be further complicated by the existance of other branches and revisons. any ideas? (I want to live within the framework of RCS) Tan Bronson Multiflow Computer Inc UUCP(work): {yale,uunet}!mfci!bronson 30 Business Park UUCP(home): {yale,mfci}!bronson!tan Branford, Ct 06405 Phone(work):(203)-488-6090
runyan@hpirs.HP.COM (Mark Runyan) (06/29/89)
>/ cheung@unc.cs.unc.edu (Clement Cheung) / 5:15 pm Jun 27, 1989 / >But identifying a version by name is not powerful enough because it does >not convey any information about that particular version. A lot of the >research SCM systems identify a version by attributes values. These >attributes describe the properties of that version of the program. >This concept is already in DSEE. A version that matches the specified >attributes values is selected. In DSEE you can also have preferences >for your selection. You lists the most desirable properties before >the less desirables properties. Obviously being able to define the properties of the revisions you are interested in is a good (perhaps even great) idea. DSEE would probably solve most of the technical problems I have, except one... I don't have an Apollo system. Note that the use of RCS symbolic names allows a person to maintain an "slist" (mentioned in previous article) with the RCS file. Unfortunately, there isn't anyway to restrict when a symbolic name can be changed or who can do, and the past status of a symbolic name in an RCS file is not kept under any sort of history control. My experience to date has been that RCS and SCCS are not sufficient for complete software configuration management, especially on projects with more than 10 engineers and more than 10,000 lines of code. Usually a front-end must be built that uses RCS or SCCS to do the version control. There *are* systems such as NSE and DSEE, but I'm not sure how well they do in an heterogeneous environment. Personally, I favor DSEE (but then I *do* work for HP :-). Programs such as Aide-De-Camp or CMT (from Expertware) are "portable" (they run on my systems anyway), but they seem to be such juggernauts that I can't find the time to get a demo set up and see if they solve our technical issues. (Not only that, but as products that must be bought, the "need" for these must be justified.) Maybe these tools are the answer and I just need to settle down and work with them... Mark Runyan
campbell@redsox.bsw.com (Larry Campbell) (06/29/89)
Yep, it looks like symbolic names (rcs -n) will do what I want. Thanks to all who pointed this out. (I did RTFM, but TFM did not make it entirely clear how one could check out a module by symbolic name.) -- Larry Campbell The Boston Software Works, Inc. campbell@bsw.com 120 Fulton Street wjh12!redsox!campbell Boston, MA 02146
render@m.cs.uiuc.edu (06/29/89)
Hoo hah! Things have really taken off! Written 6:35 pm Jun 27, 1989 by cheung@unc.cs.unc.edu: >By the way, I have been reading over 20 technical papers on SCM systems >in the past few weeks but I didn't find any which deal with management >of object codes. Also lacking is the relationship between SCM and quality >assurance. Almost all systems assume that object codes can be derived >when necessary. But let suppose. > [Problem statement of controlling 2 sets of source and binary for 4 > different platforms.] I'm not sure what the problem is that you are trying to state. If the problem is "we don't have enough disk to keep all this stuff around at the same time," then you're hosed unless you can schedule work so that you don't need everything around at once and can reproduce stuff on demand. You can ensure this by recording the manufacture steps used to create the executables and making sure that all the inputs to the manufacture steps are either on disk or are reproducible. If the problem is "we have to keep clear which object code corresponds to which source and make sure that this is kept straight," then take a look at the DSEE description of the derived object pools. Everything that is made as the result of some DSEE manufacture step is placed in the DOP and uniquely identified. There is no way for Joe Johnson to accidentally overwrite Bill Benson's object file, and each person's objects are distinguishable from the other's. I dunno, I may be missing your point, in which case I welcome enlightenment. Hal Render render@cs.uiuc.edu p.s. for a nice theoretical look at this, take a look at @inproceedings{borison, author="Ellen Borison", title="{``A Model of Software Manufacture''}", booktitle="Proceedings of the IFIP International Workshop on Advanced Programming Environments", address="Trondheim, Norway", month=jun, year=1987, pages="197--220" }
paul@athertn.Atherton.COM (Paul Sander) (06/30/89)
There is a product on the market that addresses many of the issues that have
been discussed recently here in regards to source code management. It's
called Software BackPlane. Some of its features are:
- Maintains sequential versions of text or binary files
- Permits files to be collected into a project
- Permits files to be shared among projects
- Permits development branches to be taken and later merged (this is a merge
at the file level, not a content merge like the one featured in NSE)
- Has a checkin/checkout mechanism that permits a file to be modified by only
one person at a time
- It runs on SunOS, VMS, and Ultrix
The product addresses more problems as well, but these are some of the more
prominent points discussed recently.
More information can be had by writing to:
Atherton Technology
1333 Bordeaux Drive
Sunnyvale, California 94089 USA
I'd like to discuss this product further, but as I am an employee of Atherton,
this posting is really a commercial which many on the net might consider to be
in bad taste. But as I've followed this discussion for some weeks and have
seen no mention of our product, I felt I had to bring it up.
--
Paul Sander (408) 734-9822 | If a machine is powerful enough
paul@Atherton.COM | to have a DWIM button, why bother
{decwrl,sun,pyramid}!athertn!paul | with the button? -- Eric Blackpcg@aber-cs.UUCP (Piercarlo Grandi) (06/30/89)
In article <9630005@hpirs.HP.COM> runyan@hpirs.HP.COM (Mark Runyan) writes: >/ campbell@redsox.bsw.com (Larry Campbell) / 8:27 pm Jun 24, 1989 / >I'm not too familiar with SCCS, but with RCS it seems that you could >use the "state" field to identify product versions. ... The problem of using the "state" field or even the symbolic name field to identify a product's versions is there is no recording of changes to the state field or symbolic name field of an RCS file. An external logging mechinism should probably be used to record that version A was in state X1 at Time T1, but was set to state X2 at Time T2. This is ok of course. But IMNHO this whole discussion is obviously vitiated by the usual unfamiliarity of sw eng types with other fields of compsci. In particular sw eng requires a lot of database technology, and database research also embodies a lot of sw eng (e.g. data dictionaries, etc...), but I have seen very few sw eng people realize that they ought to know about database research. Case in point: temporal databases. I always had this feeling that versioning has been studied a lot by temporal database researchers, and it does not apply just to databases of sw components. It's depressing to see sw eng research groping in the dark because of a dilettante approach to databases :-(. -- Piercarlo "Peter" Grandi | ARPA: pcg%cs.aber.ac.uk@nsfnet-relay.ac.uk Dept of CS, UCW Aberystwyth | UUCP: ...!mcvax!ukc!aber-cs!pcg Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@cs.aber.ac.uk
david@cullsj.UUCP (David Taylor) (07/04/89)
In article <1043@aber-cs.UUCP>, pcg@aber-cs.UUCP (Piercarlo Grandi) writes: > It's depressing to see sw eng research groping in the dark because of a > dilettante approach to databases :-(. > -- A brief bibliography, please? Thanks, David L. Taylor
cheung@unc.cs.unc.edu (Clement Cheung) (07/04/89)
In article <39400035@m.cs.uiuc.edu> render@m.cs.uiuc.edu writes: > > >Written 6:35 pm Jun 27, 1989 by cheung@unc.cs.unc.edu: >>By the way, I have been reading over 20 technical papers on SCM systems >>in the past few weeks but I didn't find any which deal with management >>of object codes. Also lacking is the relationship between SCM and quality >>assurance. Almost all systems assume that object codes can be derived >>when necessary. But let suppose. >> [Problem statement of controlling 2 sets of source and binary for 4 >> different platforms.] > >I'm not sure what the problem is that you are trying to state. If the >problem is "we don't have enough disk to keep all this stuff around at >the same time," then you're hosed unless you can schedule work so that you >don't need everything around at once and can reproduce stuff on demand. >You can ensure this by recording the manufacture steps used to create >the executables and making sure that all the inputs to the manufacture >steps are either on disk or are reproducible. > >If the problem is "we have to keep clear which object code corresponds to >which source and make sure that this is kept straight," then take a >look at the DSEE description of the derived object pools. Everything >that is made as the result of some DSEE manufacture step is placed in >the DOP and uniquely identified. There is no way for Joe Johnson to >accidentally overwrite Bill Benson's object file, and each person's >objects are distinguishable from the other's. > >I dunno, I may be missing your point, in which case I welcome enlightenment. > >render@cs.uiuc.edu It sounds attractive to keep all parameters of the software build steps so that hopefully you can reproduce an exact copy of the executables at a later time. However what is not said is that you must put your compilers, linker, loader under version control as well. Moreover, if we are talking about manufacturing large scale software then you are not going to regenerate your executables when needed. You simply won't have time to do that. Suppose you are asked to produce a fix for this large scale software version 3.5 within 24 hours, you might just be spending all those time rebuilding the executables. That leaves you no time to test your fix. On the other hand, if the executables is available, the fix can be applied by patching that executables. So this is one reason why software house tends to keep their executables around. I don't think I explained this clearly the first time. Realistically, can it be guaranteed that the regenerated executables is 100% identical to the one you built the first time everytime ? [Archiving happens when you are in leisure; regeneration happens when you are time crunched.] Bearing the aforesaid in mind, suppose your software runs on different Unix platforms. Since they are not binary compatible, you want to keep a copy of each of the versions for each of the platform. This is a lot of copies of the executables for the same large scale software. There simply is a need to device some scheme to manage the executables. The assumption that all SCM system made concerning regenerating executables when needed breaks down in this scenario (IMHO). The production aspect of configuration management (or is there a better term) is simply neglected. I guess this is closer to what you called release engineering. If there is a byte oriented delta technique, could that be used to find the deltas of object codes ? Clement Cheung cheung@unc.cs.edu
geoff@tom.harvard.edu (Geoff Clemm) (07/05/89)
In article <1043@aber-cs.UUCP> pcg@aber-cs.UUCP (Piercarlo Grandi) writes:
But IMNHO this whole discussion is obviously vitiated by the usual
unfamiliarity of sw eng types with other fields of compsci. In particular sw
eng requires a lot of database technology, and database research also
embodies a lot of sw eng (e.g. data dictionaries, etc...), but I have seen
very few sw eng people realize that they ought to know about database
research.
This comment is obviously vitiated by the usual unfamiliarity of Mr Grandi
with the group he is criticizing. The recent version control activity in
this newsgroup was clearly focussed on how widely available tools can be used
to solve some specific VC problems, thus the emphasis on tools such as RCS
and SCCS, rather than the database supported systems. When the database
systems are as stable, inexpensive, and generally available as RCS and SCCS,
the discussions will focus on them instead. Very few sw eng people do _not_
realize that database research is important for improving version control
support, but unlike Mr. Grandi, they realize that providing database support
for version control is a difficult task, and unless they are in the business
of building version control systems, this fact does not help solve their
problems.
Case in point: temporal databases. I always had this feeling that versioning
has been studied a lot by temporal database researchers, and it does not
apply just to databases of sw components.
The fact that there is not a single version control product on the market
based on temporal databases might explain the lack of interest by software
engineers. Again, Mr Grandi confuse software engineers with the builders of
version control systems.
It's depressing to see sw eng research groping in the dark because of a
dilettante approach to databases :-(.
It's depressing to read so much drivel. :-(.
Geoffrey Clemmcrm@romeo.cs.duke.edu (Charlie Martin) (07/05/89)
In article <10295@polya.Stanford.EDU> maslen@polya.Stanford.EDU (Thomas Maslen) writes: >In article <18MZ02qG393v01@amdahl.uts.amdahl.com> johnm@uts.amdahl.com writes: >>The problem of merging/reconciling revisions made by different people >>still involves human interaction in NSE.... > >>The essential difficulty with any form of auto-merge is not handling >>the case of two developers revising the SAME LINE of source. It's >>really how to ensure that Jack's change to line 25 is FUNCTIONALLY >>COMPATIBLE with Jill's change to line 75. > >Wot he said. Remember the failure rate of previous cooperative >efforts by Jack and Jill. Seriously, if anyone has any insights on >this, I'd like to hear about them too. Assuming that "functionally compatible" means "doesn't break anything that wasn't already broken" this could be pretty troublesome, since it's equivalent to deciding equivalence. I suspect that absent a program-proof facility and maintaining the proofs over changes, the *best* that can be done is run regression tests automagically. i hate mailers that dont have any room to allow one to include sufficient context in the body of the article. never trust a program that thinks its smarter than you are. Charlie Martin (crm@cs.duke.edu,mcnc!duke!crm)
stank@anvil.WV.TEK.COM (Stan Kalinowski) (07/05/89)
In article <791@redsox.bsw.com> campbell@redsox.UUCP (Larry Campbell) writes: >I'm not too familiar with SCCS, but with RCS it seems that you could >use the "state" field to identify product versions. For instance, >if we're going to make a snapshot/baselevel of the source library and >call it V1.01.02, we could do: > > rcs -sV1.01.01 RCS/* We are currently using a tool that is a layer on top of RCS that does a similar thing. It uses the symbolic name feature of RCS instead of the state field. Using the symbolic name allows multiple product versions to share the same version number of an individual file, thus eliminating redundant "forced" check-ins on files that didn't change between releases. This makes it easy to identify which files changed between releases or builds of different "flavors". The symbolic name "Release_V1_01_01" would be attached to a given version (12.34) with the following command: rcs -nV1_01_01:12.34 <filename> The syntax of RCS does not allow decimal points (".") in the symbol string so we generally substitute the underbar ("_") character. An added benifit of all this is that a simple "rlog -h <filename>" quickly reveals all activity on the given file. We also maintain a version list for each build that contains the version information for every file used in that build. I think someone called this an "slist". Basically, the file is a C source file with an RCS "$HEADER" as a literal C string definition that can be reported by the software on demand. (Actually we parse it and report in our own format.) The header information for all files used to build the system appears as a C comment in the same file. The file is, of course, maintained with RCS and symbolic names are attached significant versions. With this file, it is possible to reconstruct every build of the system that was ever released and even the development builds that occurred in between. Mind you, it may take awhile for RCS to checkout really old versions because all the deltas must be applied, each in turn. I should also note that in our approach, we have abandoned modification dates as a means of knowing when files need to be checked out, several parallel projects share the common RCS file for a given module and we don't want a change on one version branch to cause recompilation for a project on a different branch. At checkout time, the version number associated with a symbolic name that marks the "latest-and-greatest" is compared with the version in the local directory and a new one is checked out as needed. Of course the engineer has the option of building against the "good old stuff" if he's afraid of bugs introduced by his cohorts. What I'm curious about is, how are people managing their binaries? Right now we are using a scheme that allows engineers to use binaries kept out on our fileserver for local development builds but we aren't very smart about maintaining dependancy information. (They can only grab the latest released binaries of whole subsystems, not individual module binaries. Sometimes, when a global header file changes, they must do much local recompiling of their own local expermental subsystem, duplicating effort that was done on other machines.) I have thought of using an RDBMS to track the dependancies but they all seem to be horribly slow. (Another group here is using that approach.) stank US Mail: Stan Kalinowski, Tektronix, Inc., Interactive Technologies Division PO Box 1000, MS 61-028, Wilsonville OR 97070 Phone:(503)-685-2458 e-mail: {ucbvax,decvax,allegra,uw-beaver}!tektronix!orca!stank or stank@orca.WV.TEK.COM
stank@anvil.WV.TEK.COM (Stan Kalinowski) (07/05/89)
In article <9630003@hpirs.HP.COM> runyan@hpirs.HP.COM (Mark Runyan) writes: >If NSE solves this problem to handle the merge automatically, I'd like >to know how its done. (Of course, it may be a company secret for SUN, I don't know how NSE solves it, but, we had Apollo present DSEE here and it looked like their system used regular 'ol "merge" or "rcsmerge". Do a man on these if you have RCS on your system. Of course, they may be using their own propriatary implementations, but the functionality seemed to be the same as "merge". (i.e. it asked for help when the changes overlapped. There was no language dependant checking for things like variable reuse etc.) stank US Mail: Stan Kalinowski, Tektronix, Inc., Interactive Technologies Division PO Box 1000, MS 61-028, Wilsonville OR 97070 Phone:(503)-685-2458 e-mail: {ucbvax,decvax,allegra,uw-beaver}!tektronix!orca!stank or stank@orca.WV.TEK.COM
pcg@aber-cs.UUCP (Piercarlo Grandi) (07/06/89)
In article <GEOFF.89Jul4152416@tom.harvard.edu> geoff@tom.harvard.edu (Geoff Clemm) writes:
When the database systems are as stable, inexpensive, and generally
available as RCS and SCCS, the discussions will focus on them instead.
You have a somewhat optimistic idea of what a DBMS is... RCS and SCCS are
"database systems"? Since when? (I know that an argument can be made that
the unix filesystem itself is a database system for files, but that is
stretching definitions more than a bit...).
And what about University Ingres, as "stable, inexpensive, and generally
available"?
Very few sw eng people do _not_ realize that database research is
important for improving version control support,
And not just version control support.
Still, except for very few laudable cases, most research (not to speak of
commercial developments) on IPSEs and other sw engineering tools is based
(at best) on extensions to file system concepts. In 1981-2 I had observed at
the Honeywell corporate research center people doing interesting work using
Multics' RDBMS engine for sw engineering. I had hoped that things had
progressed from that, but many sw eng environments are still pretty much
behind that.
Even interesting things like SHAPE are based on an AFS, which is an ad hoc
extension to a file system. It is powerful, admittedly, but has all the
obvious disadvantages of an ad hoc solution (and the idea behind it,
described in a Stanford doctoral dissertation, was clearly intended to be
just a clever improvement on filesystem system technology).
Conversely, at least in principle, it would be much more interesting to use
a true DBMS, e.g. Ingres, as a substitute for a file system, rather than
viceversa (e.g. to store all the mass of unruly info currently contained in
UNIX ad hoc "databases" like /etc/passwd, /usr/lib/sendmail.cf, etc...), or
even for pathname resolution.
The fact that there is not a single version control product on the market
based on temporal databases might explain the lack of interest by
software engineers.
Or maybe you should have written "might be explained by the lack of
interest".
On the other hand, unfortunately I would be hard pressed to name a single
widely used commercial temporal database management system. But at least
research has been done on them, and there is understanding of the issues
(and even some prototype implementation).
As to version control, my reckoning is that the ad hoc approach still
dominates (nothwithstanding some interesting work, like W. Tichy's a few
years ago on and/or trees).
For example, how many sw eng environents/tools out there do support as
easily and obviously such DBMS features as:
* transactions
* recovery
* auditing
* schemas/data dictionary as part of the database itself
* interactive ad hoc queries
* 4GL interfaces
* ....
that are assumed as given when using a proper DBMS? My impression is that
most sw engineering tools and environments around are still in the
equivalent of Cobol+ISAM age... (or in the case of SCCS/RCS, the Librarian
one).
It's depressing to read so much drivel. :-(.
Well, you could cheer up me and everybody else by being more specific and
trying to provide some good example of the widespread influence of database
research on IPSEs and sw engineering tools. I can easily think of isolated
meritorious works (e.g. ASPECT), but I would really appreciate it if you
could dispel my gloomy perception of the status of the mainstream.
--
Piercarlo "Peter" Grandi | ARPA: pcg%cs.aber.ac.uk@nsfnet-relay.ac.uk
Dept of CS, UCW Aberystwyth | UUCP: ...!mcvax!ukc!aber-cs!pcg
Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@cs.aber.ac.ukolender@cs.colostate.edu (Kurt Olender) (07/06/89)
In article <3777@orca.WV.TEK.COM> stank@anvil.WV.TEK.COM (Stan Kalinowski) writes: In article <9630003@hpirs.HP.COM> runyan@hpirs.HP.COM (Mark Runyan) writes: >If NSE solves this problem to handle the merge automatically, I'd like >to know how its done. (Of course, it may be a company secret for SUN, Sun's NSE does not handle the merge problem automatically when there is a conflict between two versions (i. e. they modify the same section of code). It does provide a window-based tool to help a user reconcile the conflicts. BTW, the source of my information is a reasonably detailed technical report published by Sun that describes the features of and tools included in NSE. I'm not sure where it came from, since it was routed to me from others, but it is likely that it would be available through a Sun sales rep, since it is intended for potential buyers of NSE. The citation is "The Network Software Environment", William Courington, Sun Microsystems, Inc., April 1989. From the back page, I suspect the part number is FE197-0/20K, but I wouldn't bet my next paycheck on it. -- ============================================================================ || Kurt Olender | Internet: olender@cs.colostate.edu || || Phone: (303) 491-7015 | UUCP: hao!handel!olender || ============================================================================
davis@mtgzz.att.com (p.a.davis) (07/07/89)
In article <1052@aber-cs.UUCP> pcg@cs.aber.ac.uk (Piercarlo Grandi) writes: >Conversely, at least in principle, it would be much more interesting to use >a true DBMS, e.g. Ingres, as a substitute for a file system, rather than >viceversa (e.g. to store all the mass of unruly info currently contained in >UNIX ad hoc "databases" like /etc/passwd, /usr/lib/sendmail.cf, etc...), or >even for pathname resolution. Oh it would, would it ? And to forever have such information bound up in as part of some highly complex, obscure, binary-oriented system, in which there is very little access to the information unless you go in through the specified channels ? Give us a break - the proliferation of ridiculous systems like Ingres, Dbase and Oracle is one of the most worrying on the computing horizon. Extending their reach into SW engineering and system admin could turn into the worst nightmare ever. If we have to have RDMS on Unix, lets stick to those that adhere to the philosophy of the whole system (such as /rdb) and forget cryto-DB's. (By which I mean ASCII files in simple filterable format). > * 4GL interfaces From "Relational Databases under Unix" (or some such book by the author of /rdb)... * the Unix shell (s) - the best 4GL yet invented Remember: build a system a fool could use, and only a fool will want to. -- Internet: davis@mtgzz.att.com 1930 Green St., PA 19130 (215-981-0614) uucp: ...!att!mtgzz!davis @ AT&T Integrated Systems Division, NJ "All the above opinions are my own, which may or may not increase their value."
Luis@postgres.Berkeley.EDU (Luis Miguel) (07/11/89)
> For example, how many sw eng environents/tools out there do support as > easily and obviously such DBMS features as: > > * transactions > * recovery > * auditing > * schemas/data dictionary as part of the database itself > * interactive ad hoc queries > * 4GL interfaces > * .... > > that are assumed as given when using a proper DBMS? My impression is that > most sw engineering tools and environments around are still in the > equivalent of Cobol+ISAM age... (or in the case of SCCS/RCS, the Librarian > one). > I can say in 2 words why traditional RDBMS's and the features cited above are not part of SW eng. tools: DISASTROUS PERFORMANCE !!!!!!! Runing simple queries on commercial RDBMS's takes on the order of 100 msecs on a Sun 3/260. With this performance, one cannot hope to do much work in a reasonable amount of time (I can cite papers on people who have tried but have been defeated by the performance figures). As useful as these services are, until they can be provided with reasonable performance, they will be largely confined to those areas where they are essential. /Luis Luis Miguel. Computer Science Division, UC Berkeley. arpanet: luis@postgres.Berkeley.EDU uucp: {ihnp4,decvax}!ucbvax!postgres!luis at&t: (415) 642-9585
pgh@stl.stc.co.uk (Peter Hamer) (07/13/89)
My first attempt seemed to get lost, sorry if this duplicates. In article <15328@pasteur.Berkeley.EDU> Luis@postgres.Berkeley.EDU (Luis Miguel) writes: > traditional RDBMS's ... are not part of SW eng. tools [because of] > DISASTROUS PERFORMANCE !!!!!!! ... (I can cite papers on people who have > tried but have been defeated by the performance figures). I am sure you are right, and remember being told that the Teamwork environment was moved from a RDBMS onto a B-tree manager for just this reason (no figures thought). Could you please reference the papers, as I would like to follow them up? Peter
luis@postgres.Berkeley.EDU (Luis Miguel) (07/18/89)
> > traditional RDBMS's ... are not part of SW eng. tools [because of] > > DISASTROUS PERFORMANCE !!!!!!! ... (I can cite papers on people who have > > tried but have been defeated by the performance figures). > > I am sure you are right, and remember being told that the Teamwork environment > was moved from a RDBMS onto a B-tree manager for just this reason (no figures > thought). > > Could you please reference the papers, as I would like to follow them up? > > Peter Here is a list, in bib format, of some of the articles. Enjoy......... /Luis %A Larry Rowe %T Report on the 1989 Software CAD Databases Workshop %I Computer Science Division-EECS, U.C. Berkeley %D March 1989 %J To be Published %K scad report %A Bienvenido Velez %T Data Base Representations For Programs %D November 1988 %C University of California, Berkeley %R MS Report %K C program database %A Philip A. Bernstein %T Database System Support for Software Engineering -- An Extende d Abstract -- %D 1987 %A James R. Larus %T An Interactive Program Analysis System For Franz Lisp %D October 1982 %C University of California, Berkeley %R MS Report %K lisp database program analysis %A Margaret H. Butler %T Persistent LISP: Storing Interobject References in a Database %D November 1987 %C University of California, Berkeley %R Report No. UCB CSD 88-401, PhD Thesis %K persistent lisp database %A Scott E. Hudson %A Roger King %T Object-Oriented Database Support for Software Development Envi ronments %J Proceedings of ACM SIGMOD 1987 Annual Conference %D May, 1987 %C San Francisco, California %A Mark Bigger %T A Database Model for Software Development Environments %D 1986 %K UCLA Eng-Math, LD791.8 C6 B482 %A Mark Linton %T Queries and Views of Programs Using a Relational Database Syst em %R PhD Thesis %D December 1983 %C University of California, Berkeley %K omega programs relations %A Michael Powell %T Database Support For Programming Environments %J Proceedings Eng. Design Applications of ACM-IEEE Data Base Wee k %C San Jose, Ca. %D May 1983. %A Mark Linton %T Implementing Relational Views of Programs %J Proc of the ACM SIGSOFT-SIGPLAN software engineering symposium on practical software development environments. %C Pittsburgh, Penn %D April 1984 %K relational views programs %A R. Snodgrass %T Monitoring a Software Development Environment: a Relational Ap proach %J Proc. of the ACM SIGSOFT-SIGPLAN Software Engineering Symposiu m on Practical Software Development Environments. %C Pittsburgh, Penn %D April 1984 %K monitoring environment relational %A S. Horwitz %A T. Teitelbaum %T Relations and Attributes: A Symbiotic Basis for Editing Enviro nments %J Proc. of the ACM SIGSOFT-SIGPLAN Software Engineering Symposiu m On Practical Software Development Environments. %C Pittsburgh, Penn %D April 1984 %K relations attributed grammars environments Luis Miguel. Computer Science Division, UC Berkeley. arpanet: luis@postgres.Berkeley.EDU uucp: {ihnp4,decvax}!ucbvax!postgres!luis at&t: (415) 642-9585
stewartw@warpdrive.UUCP (Stewart Winter) (07/21/89)
In article <1052@aber-cs.UUCP> pcg@cs.aber.ac.uk (Piercarlo Grandi) writes: >For example, how many sw eng environents/tools out there do support as >easily and obviously such DBMS features as: > > * transactions > * recovery > * auditing > * schemas/data dictionary as part of the database itself > * interactive ad hoc queries > * 4GL interfaces > * .... > >that are assumed as given when using a proper DBMS? My impression is that You raise an excellent point, however, using a conventional relational DBMS is not going to be the way to go. What you are talking about is exactly what object databases are meant for. Unfortunately, we are all waiting for someone to write a usable ODB. Given the amount of effort by companies like IBM and HP, they can't be too far away. ODB research has more or less come to the conclusion that relational DBMS's do not provide the required performance levels to serve as the basis for an ODB; otherwise I suspect that ODBs would already be around. Stewart -- Stewart Winter Cognos Incorporated S-mail: P.O. Box 9707 VOICE: (613) 738-1338 x3830 FAX: (613) 738-0002 3755 Riverside Drive UUCP: uunet!cognos!stewartw Ottawa, Ontario "The bird for the day is .... crimson rosella." CANADA K1G 3Z4
render@m.cs.uiuc.edu (07/23/89)
Written 5:52 pm Jun 14, 1989 by hull@dinl.uucp: >While having nothing to contribute at this time, I would like to read >about formal models of SCM (with English descriptions) and >object-oriented SCM. Well, the thing about formal models of SCM and OO SCM is that there isn't much published on either. There are several papers describing version control and SCM systems that give data models for them. There are few papers that specifically give models; Walter Tichy, Karen Huff, Ellen Borison and Dennis Heimbigner have written such papers. I'll list the references at the bottom of this posting. As for OO SCM, a few people are working on an OO data model for a SCM, including myself. One group is the Arcadia project, with whom Dennis Heimbigner is related (I believe). Unfortunately I haven't seen any published models as such, though I'm working on mine. Many of the object-oriented database people address version control in their systems, both from the point of maintaining multiple versions of data to dealing with dynamic schema evolution. There have been a couple of good papers on models of this, and I'll also append those references. The only actual OO programming system that addresses SCM that I know of is called Orwell. It's a Smalltalk-based system that was designed to allow team programming, and deals with concurrent updates, change control and the like. It's not real sophisticated, but it addresses the issues. Oh, for those interested I know of one paper on the rigorous development of a version control program using VDM. I like it because it's the one of the few attempts to apply formal development methods to SCM. Anyway, I don't know how to make a short introduction into this stuff, though I may try if I get inspired over the weekend. If anyone has any specifics to talk about, I'll try to contribute. Also, I'm always interested in hearing about any other related work or papers people know of. Hal Render render@cs.uiuc.edu ----- %Here are the papers concerning SCM models (all references in bibtex format): @inproceedings{borison, keywords=":scm:data models:process models:", author="Ellen Borison", title="{``A Model of Software Manufacture''}", booktitle="Proceedings of the IFIP International Workshop on Advanced Programming Environments", address="Trondheim, Norway", month=jun, year=1987, pages="197--220" } @article{Ditt88, keywords=":vc:dbms:data models:", title="{``Version Support for Engineering Database Systems''}", author="Klaus R. Dittrich and Raymond A. Lorie", journal="IEEE Transactions on Software Engineering", volume="14", number="4", month= apr, year=1988, pages="429--437" } @inproceedings{heimbigner2, keywords=":arcadia:sde:scm:data models:", title="{``A Graph Transform Model for Configuration Management Environments''}", author="Dennis Heimbigner and Steven Krane", booktitle="Proceedings of the ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments", address="Boston, Massachusetts", month=nov, year=1988, pages="216--225" } @techreport{heimbigner3, keywords=":arcadia:scm:sde:data models:relations:", title="{Active Relations for Specifying \& Implementing Software Object Management}", author="Dennis Heimbigner and Leon Osterweil and Sutton, Jr., Stanley", school="Department of Computer Science, University of Colorado", number="CS-CS-406-88", month= jul, year="1988"} @inproceedings{huff81, keywords=":sde:dbms:scm:data models:", author="Karen E. Huff", title="{``A Database Model for Effective Configuration Management in the Programming Environment''}", booktitle="Proceedings of the 5th International Conference on Software Engineering", address="San Diego, CA", month= "March", year=1981, pages="54--61" } @inproceedings{lint84, keywords=":dbms:data models:scm:views:", author="Mark A. Linton", title="{``Implementing Relational Views of Programs''}", booktitle="Proceedings of the ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments", address="Pittsburgh, PA", month= "April", year=1984, pages="132--140" } @incollection{lock83, keywords=":scm:vc:sde:data models:er:", title="{``Analysis of Version and Configuration Control in a Software Engineering Environment''}", author="Peter C. Lockemann", booktitle="Entity-Relationship Approach to Software Engineering", Editor="C.G. Davis and S. Jajodia and P.A. Ng and R.T. Yeh", publisher="Elsevier Science Publishers", address="North-Holland", year=1983, pages="701--713" } @inproceedings{minsky85, keywords=":darwin:sde:scm:data models:", title="{``Controlling the Evolution of Large Scale Software Systems''}", author="Naftaly H. Minsky", booktitle="Workshop on Software Engineering Environments for Programming-in-the-Large", address="Harwichport, MA", month= "June", year=1985, pages="1--16" } @inproceedings{grids, keywords=":scm:data models:", title="{``Grids: A New Program Structuring Mechanism Based on Layered Graphs''}", author="Harold L. Ossher", booktitle="Eleventh Annual ACM Symposium on Principles of Programming Languages", month= jan, year="1984", pages="11--22"} @techreport{clemma2, keywords=":clemma:scm:data models:", author="Hal S. Render and Roy H. Campbell", title="{The Design of the CLEMMA Configuration Librarian System}", institution="Dept. of Computer Science, University of Illinois at Urbana-Champaign", number="UIUCDCS-R-88-1458", month=Oct, year=1988 } @techreport{staudt, keywords=":sde:scm:data models:", keywords=":transformgen:software maintenance:sde:", title="{TransformGen: Automating the Maintenance of Structure-Oriented Environments}", author="Barbara Staudt and Charles Krueger and David Garlan", school="Department of Computer Science, Carnegie Mellon University", number="CMS-CS-88-186", month= nov, year="1988"} @inproceedings{tichy81, keywords=":vc:scm:data models:", title="{``A Data Model for Programming Support Environments''}", booktitle="Proceedings of the IFIP WG 8.1 Working Conference on Automated Tools for Information System Design and Development", author="Walter F. Tichy", month= oct, year="1981"} @inproceedings{klahold, keywords=":vc:data models:dbms:", title="{``A General Model for Version Management in Databases''}", booktitle="Proceedings of the 12th International Conference on Very Large Databases", author="Peter Klahold and Gunter Schlageter and Wolfgang Wilkes", address="Kyoto, Japan", month= aug, year="1986", pages="319--327"} @incollection{lock83, keywords=":scm:vc:sde:data models:er:", title="{``Analysis of Version and Configuration Control in a Software Engineering Environment''}", author="Peter C. Lockemann", booktitle="Entity-Relationship Approach to Software Engineering", Editor="C.G. Davis and S. Jajodia and P.A. Ng and R.T. Yeh", publisher="Elsevier Science Publishers", address="North-Holland", year=1983, pages="701--713" } @inproceedings{perr87b, keywords=":sde:vc:mi:data models:", author="Dewayne E. Perry", title="{``Software Interconnection Models''}", booktitle="Proceedings of the 9th International Conference on Software Engineering", address="Monterey, CA", month= "March", year=1987, pages="61--69" } %Here are the OO papers: @article{orwell, keywords=":scm:oo:pl:", title="{``Orwell: A Configuration Management System for Team Programming''}", author="Dave Thomas and Kent Johnson", journal="ACM SIGPLAN Notices", volume="23", number="11", month= Nov, year="1988", pages="135--141" } @inproceedings{beech, keywords=":oo:dbms:vc:", title="{``Generalized Version Control in an Object-Oriented Database''}", author="David Beech and Brom Mahbod", booktitle="Proceedings of the 1988 IEEE Fourth International Conference on Data Engineering", address="Los Angeles, California", month= feb, year="1988", pages="14--22" } @inproceedings{katz87, keywords=":oo:vc:dbms:data models:", title="{``Managing Change in a Computer-Aided Design Database''}", author="R. H. Katz and E. Chang", booktitle="Proceedings of the 13th International Conference on Very Large Data Bases", address="Brighton, England", month= sep, year=1987, pages="455--462" } %Here's the rigorous development paper: @article{cott84, keywords=":vc:sde:svce:", title="{``The Rigorous Development of a System Version Control Program''}", author="Ian D. Cottam", journal="IEEE Transactions on Software Engineering", volume="SE-10", number="2", month= mar, year=1984, pages="143--154" }
render@m.cs.uiuc.edu (07/24/89)
I wasn't going to get into this discussion, but I saw a lack of support for some good things Mr. Grandi had to say, in combination with less than reasonable attacks on his position. Written 8:30 am Jul 7, 1989 by davis@mtgzz.att.com: >In article <1052@aber-cs.UUCP> pcg@cs.aber.ac.uk (Piercarlo Grandi) writes: >>Conversely, at least in principle, it would be much more interesting to use >>a true DBMS, e.g. Ingres, as a substitute for a file system, rather than >>viceversa (e.g. to store all the mass of unruly info currently contained in >>UNIX ad hoc "databases" like /etc/passwd, /usr/lib/sendmail.cf, etc...), or >>even for pathname resolution. > >Oh it would, would it ? And to forever have such information bound up >in as part of some highly complex, obscure, binary-oriented system, >in which there is very little access to the information unless you go >in through the specified channels ? (Based on Mr. Davis' address, I'll assume that UNIX is his OS of choice.) What do you think UNIX is if not a binary-oriented system with very limited access channels? There's no support for record-oriented I/O (something that operating systems had twenty-five years ago) and a primitive notion of different types of file. The fact that the only thing which distinguishes a C source file from an object file is the suffix attached to its name indicates its level of sophistication. With all the advancement in programming language technology you'd expect some of it to filter down to command languages and programming environments, but you see almost none of it in UNIX. A lot of time and effort have gone into making DBMS that support a higher-level view of the data and operations. This work applied to file systems could provide developers with a lot nicer working environment than UNIX and [tkc]sh. >Give us a break - the proliferation of ridiculous systems like Ingres, >Dbase and Oracle is one of the most worrying on the computing horizon. >Extending their reach into SW engineering and system admin could turn >into the worst nightmare ever. This statement demonstrates little knowledge of the current trends in software engineering. Most development environments that I'm familiar have some sort of DBMS built into them, primarily because most file systems don't provide the facilities you need (transaction mgmt., data structuring, UI development tools, etc.). I really don't see why you consider the DBMS systems you cite ridiculous, unless it is because you never felt comfortable using them. You have to admit they allow you to do several things you can't in UNIX without a lot of effort and hacking around. I'm not saying you can't emulate a DBMS using various UNIX tools, but you are almost assuredly going to lack several facilities and suffer a severe performance loss over a well-constructed DBMS. >If we have to have RDMS on Unix, lets stick to those that adhere to >the philosophy of the whole system (such as /rdb) and forget >cryto-DB's. (By which I mean ASCII files in simple filterable format). "Crypto-DB"? That's a novel term. If by this you mean a DB which stores it's data in a form other than one which is human-readable, then you'd better stay away from any application which has more than a few thousand data items to manage. Storing things in human-readable form is fine when you've got a few thousand pieces of data to look through, but when you've got a hundred thousand or more than you had better use a storage format (such as B-trees) that isn't going to be a time-hog for look-ups. ASCII files are fine for storing your list of personal phone numbers, but not for a cross-reference of all the defined functions and global variables in a 3 million line software system. For things like that you need a DBMS. Of course you'll need special access routines, but what do you think function libraries are for? If read and write routines were sufficient, we wouldn't have bothered with scanf and printf. The same is true for other types of data store. >> * 4GL interfaces > >From "Relational Databases under Unix" (or some such book by the >author of /rdb)... > > * the Unix shell (s) - the best 4GL yet invented Right. And some people still say, "COBOL -- the best programming language yet invented." You'd better learn to take such statements with a grain of salt if you hope to ever progress beyond the current state of the art. >Remember: build a system a fool could use, and only a fool will want to. Good quote, but if I read you right, you'd still love for us to be programming in machine code and on cards. Hey, punch cards were fun when I was back in college, but I'd rather not be forced to still use them, if it's okay with you. Hal Render render@cs.uiuc.edu
render@m.cs.uiuc.edu (07/24/89)
Written 12:19 pm Jul 10, 1989 by Luis@postgres.Berkeley.EDU: > I can say in 2 words why traditional RDBMS's and the features cited > above are not part of SW eng. tools: > > DISASTROUS PERFORMANCE !!!!!!! > > Runing simple queries on commercial RDBMS's >takes on the order of 100 msecs on a Sun 3/260. >With this performance, one cannot hope to do much work >in a reasonable amount of time >(I can cite papers on people who have tried but have been defeated by the > performance figures). From my reading in the area, I've come to think that part of the problem is implementation-dependent. There are DBMS machines that can handle queries very quickly because they've optimized their software and hardware to support such things. However, if you take a DBMS like INGRES and put it on a generic UNIX box, you're going to get performance degradation because of the incompatibilities between the differing models of interaction. I think a solution would be to design an integrated file system/OBMS that supported both file and DBMS functions and was optimized for both. I've heard of various people trying to build such things, but I can't say how successful they've been so far. The results will be slower than a dumb file system, because you just can't add functionality to a system without suffering some performance slow-down. There ain't no such thing as a free lunch. However, the number of new, higher-level operations provided should outweigh the slowdown in older, lower-level operations, and this will be what makes the system worthwhile. >As useful as these services are, until they can be >provided with reasonable performance, they will be >largely confined to those areas where they are essential. Agreed, but the number of areas in which such things are essential is growing. I don't think you can sell a modern IPSE without DBMS capabilities, and to write them off as being too time-expensive is to ignore their usefulness. Hal Render render@cs.uiuc.edu
render@m.cs.uiuc.edu (07/24/89)
Written 1:15 pm Jul 20, 1989 by stewartw@warpdrive.UUCP: > You raise an excellent point, however, using a conventional relational >DBMS is not going to be the way to go. What you are talking about is >exactly what object databases are meant for. Unfortunately, we are all >waiting for someone to write a usable ODB. Given the amount of effort >by companies like IBM and HP, they can't be too far away. I'm sure IBM and HP appreciate your support, but I think it quite possible that the big breakthrough in OODBMS work will come elsewhere. There are several indie companies working on them, and at least one (Ontologic) is into it's second generation of product. You must not assume, however, that the new OODBMS will be have better performance than the RDBMS for the kinds of applications software engineers plan. An OODBMS has at least as many difficult things to support as an RDBMS. I do think that ODBMS are more applicable to SE applications, so hopefully they will be optimized in a manner that makes them faster than an RDBMS for what we need. > ODB research has more or less come to the conclusion that relational >DBMS's do not provide the required performance levels to serve as the >basis for an ODB; otherwise I suspect that ODBs would already be around. This is a very good point you raise: if you want performance, you can't layer incompatible systems. For this reason, a UNIX box running a RDBMS application will be slow, just as a OODBMS built on top of an RDBMS will be slow. What needs to be done is a ground-up implementation that can be optimized at every level. Once an OODBMS has been built in this way, we can then determine how usable they will be for software engineering. Hal Render render@cs.uiuc.edu
simpson@trwarcadia.uucp (Scott Simpson) (07/26/89)
In article <39400049@m.cs.uiuc.edu> render@m.cs.uiuc.edu writes: >that the big breakthrough in OODBMS work will come elsewhere. There are >several indie companies working on them, and at least one (Ontologic) is >into it's second generation of product. You must not assume, however, >that the new OODBMS will be have better performance than the RDBMS for >the kinds of applications software engineers plan. An OODBMS has at We have been using the first version of Ontologic's OODB called VBase. It uses a new language invented by Ontologic called COP/TDL. TDL is sort of a combined data definition/procedure interface specification language. You define your schema with TDL. COP is the programming language used to write methods. It is a superset of C including exceptions, seamless persistent objects and triggers. (By seamless, I mean persistence is a part of the language. It is not embedded (like ESQL) or procedure calls). VBase also supports meta-data. Types are first-class objects too. Unfortunately, Ontologic discovered that the market is too resistant to the introduction of a new language so they have dropped it and introduced a C++ version called OB2 that uses procedure calls for persistence. The new version had to chuck seamlessness out the window because they didn't want to modify the C++ language to support persistent objects. They also didn't write their own compiler this time. They decided that this was too resource intensive an activity so now they support AT&T's cfront and Glockenspiel's compiler. Ontologic's claims that the big win with OODBs is the narrowing of the semantic gap between your problem domain and your programming language. We don't live in a tuple world. We live in an object world. Rather than mapping the whole world to tuples doesn't it make more sense to map the world to objects? We have found OODBs to be a big win. We mapped an ER model directly into objects in VBase. The semantic gap between our problem domain and our programming language was small. VBase had some problems though. We found it somewhat slow for our needs and since it is a first version, it is somewhat buggy. Ontologic provided a debugger (somewhat like dbx) though and and SQL interface to the database. Also, the concurrency model was quite flawed. VBase and OB2 provide a component library. The new OB2 product is supposed to be much faster than VBase and orders of magnitude faster than RDBMSs. One of the reasons RDBMSs are slow is that one object may be represented by a bunch of tuples. You have to retrieve and map all these tuples to get the information on your object. In OB2, you simply go and retrieve the object. Ontologic again provides an SQL interface but they have extended it with OODB features. Some of the extensions provide a great speed improvement over a standard SQL syntax. For example, you can directly refer to objects following a path down the type tree. SELECT (Company.subsidiary.budget) ... where Company.subsidiary.budget may require a relational join in an RDBMS to retrieve that field. In a large database, this may result in retrieval of a lot of tuples that simply get thrown away. The new OB2 version also provides an improved concurrency model. Ontologic claims the back end is not specific to the C++ type system and they will be providing interfaces for other languages such as Ada and Fortran provided there is sufficient interest. (You can't transfer types between different language type models though. They haven't solved that problem.) A fellow coworker commented that VBase had many of the features of object management systems such as Atherton and CAIS. I don't how to comment on that but it will be interesting to compare the features of OODBs now and what people envision as object managers later. Scott Simpson TRW Space and Defense Sector usc!trwarcadia!simpson (UUCP) trwarcadia!simpson@usc.edu (Internet)
render@m.cs.uiuc.edu (07/31/89)
Just to clarify an earlier message, I like UNIX. I use it all the time. My statement about it lacking record I/O was intended to show that it is not the be-all and end-all of operating systems and programming environments. It was a poorly chosen example because, as people have pointed out, record I/O is not that big a deal. Still, there are various high-level features which I think should be implemented in a layer below that of the user (mostly for the sake of performance). UNIX is admirable for the number of extraneous things it doesn't put in the kernel, but there are things that I wished it had there. hal.
stewartw@warpdrive.UUCP (Stewart Winter) (08/02/89)
In article <39400049@m.cs.uiuc.edu> render@m.cs.uiuc.edu writes: > >Written 1:15 pm Jul 20, 1989 by stewartw@warpdrive.UUCP: >>exactly what object databases are meant for. Unfortunately, we are all >>waiting for someone to write a usable ODB. Given the amount of effort >>by companies like IBM and HP, they can't be too far away. >I'm sure IBM and HP appreciate your support, but I think it quite possible >that the big breakthrough in OODBMS work will come elsewhere. There are Well, the breakthrough could come from anywhere, but the standards will certainly come from someone bigger. Whether I like it or not, the only standards worth supporting in a business product are those endorsed by an IBM or HP-SUN-etc group. Bucking the trend set by such a group doesn't work and hoping that some small companies system will become a standard without the support of the large hardware vendors doesn't happen. And we do need a standard for ODBs so that all the third party systems out there can access and manage the objects in these databases. I can't see coding an access layer for each and every different ODB in the marketplace. So you may not believe that IBM et al will lead the way, but they certainly make the path viable. Stewart -- Stewart Winter Cognos Incorporated S-mail: P.O. Box 9707 VOICE: (613) 738-1338 x3830 FAX: (613) 738-0002 3755 Riverside Drive UUCP: uunet!cognos!stewartw Ottawa, Ontario "The bird for the day is .... canary winged parakeet." CANADA K1G 3Z4