draves@harvard.ARPA (Richard Draves) (02/26/85)
I recently ran across an undocumented flag for cpp that makes the
following entry for makefiles convenient. It automagically
updates file dependencies. If your cpp doesn't have this
feature, or you don't like depending on undocumented features,
you can write you own shell script to accomplish the same thing.
# This has only been tested on 4.2BSD
CPP = /lib/cpp
ED = /bin/ed
CFILES = main.c
depend :
{ echo '/^# DO NOT DELETE THIS LINE/+1,$$c'; \
for i in $(CFILES); do $(CPP) -M $$i; done; \
echo '.'; \
echo 'w'; \
} | $(ED) - Makefile
# File Relationships
# DO NOT DELETE THIS LINE -- make depends on it.
# This line will be replaced by your file dependencies.
Rich
--
"a picture in the head is a gory murder in an art gallery"
-- Stephen Kosslynguy@rlgvax.UUCP (Guy Harris) (02/28/85)
> I recently ran across an undocumented flag for cpp that makes the > following entry for makefiles convenient. It automagically > updates file dependencies. If your cpp doesn't have this > feature, or you don't like depending on undocumented features, > you can write you own shell script to accomplish the same thing. > > # This has only been tested on 4.2BSD > > CPP = /lib/cpp > depend : > { echo '/^# DO NOT DELETE THIS LINE/+1,$$c'; \ > for i in $(CFILES); do $(CPP) -M $$i; done; \ Well, I just tested it on 4.2BSD and it says 0: unknown flag -M Somebody at your site must have added it to "cpp". You can do a "cpp -E" and suck out all the "# <linenumber> <include-file>" lines and make a list of included files out of that. "-M" probably just suppresses all output except lines indicating that an include file is being read. Unfortunately, if "cpp" isn't a separate program, or if it produces different kinds of output for included files, or if it doesn't support "cpp -E", you're out of luck. Guy Harris {seismo,ihnp4,allegra}!rlgvax!guy
moss@Brl-Vld.ARPA (Gary S. Moss (AMXBR-VLD-V)) (02/28/85)
Don't know about vanilla 4.2BSD, but we have the BRL-enhanced 4.2 with Doug Gwyn's SVR2 emulation running on a 780, and I implemented Richard Drave's make depend with success (thanks Richard). The only problem I had, was since I had to use /lib/cpp to get the -M flag (/usr/5lib/cpp which emulates S5 doesn't have it) I had to add -I/usr/5include to my other -I's to tell /lib/cpp where string.h was (BSD calls it strings.h (don't know if they are related, but if so, it would be nice if they were the same name)). So, anyone know if this -M flag has been added at BRL or why Guy missed out on it? &- Moss -&
kupfer@ucbvax.ARPA (Mike Kupfer) (03/02/85)
The -M option apparently got added to Berkeley cpp some time after the
4.2 tapes started out the door. (No, I don't know if it's at Harvard
because they got it from Berkeley, or whether they did it themselves.)
By the way, "cc" was changed at the same time to understand -M, so that
"make depend" lines can read
cc -M ${INCPATH} ${SRCS} | \
...
thus eliminating the need for shell scripts calling cpp.
--
Mike Kupfer
kupfer@Berkeley
...!ucbvax!kupfer
"weil, do haess Ahnung vun dae Technik, vun der ich nix verstonn."guy@rlgvax.UUCP (Guy Harris) (03/02/85)
> So, anyone know if this -M flag has been added at BRL or why Guy missed > out on it? I got some mail from Thomas Breuel at Harvard saying Old 4.2 distribution tapes don't have it, and I'm not sure whether it's officially out... so I guess we missed out on it by getting too early a 4.2 tape. (Somebody told me once that there was only going to be one 4.2BSD tape, so people wouldn't have to worry about "well, gee, I have an old tape"; I guess that changed.) I checked a local 4.3 (more-or-less) system and it has the flag. I guess it's coming in 4.3. It's a very small change. They redirect the output of "cpp" to "/dev/null" (literally), and the only output that goes to the normal place is the equivalent of "# <linenum> <file>" lines, except that only one gets produced per included file, and the syntax is that of a make dependency line. The same goal can be achieved with a filter; such a filter was posted to "net.sources" ages ago. Either the filter, or the -M flag, should be picked up and stuck into every UNIX in existence (picked up by vendors, for the benefit of their licensees), because the absence of (semi-)automated dependency generators is a nuisance. People don't keep their dependency lists up to date, things don't get remade when a header changes, and all sorts of bugs crop up. Or, worse, they only appear when ".o" files are removed and the modules are recompiled, so the bug appears rather distant in time from the actual change to the header file. -- Guy Harris {seismo,ihnp4,allegra}!rlgvax!guy
arnold@gatech.UUCP (Arnold Robbins) (03/08/85)
In Article <544@rlgvax.UUCP> Guy Harris (rlgvax!guy) writes > Either the filter, or the -M flag, should be picked up and stuck into every > UNIX in existence (picked up by vendors, for the benefit of their licensees), > because the absence of (semi-)automated dependency generators is a nuisance. > People don't keep their dependency lists up to date, things don't get remade > when a header changes, and all sorts of bugs crop up. > -- > Guy Harris > {seismo,ihnp4,allegra}!rlgvax!guy I just wanted to mention that ATT's Unix Toolchest, which allows you to sign up for software which is delivered electronically, provies a tool call "makefile", which will read through your software in the current directory, and generate a makefile for you. If there is a makefile there, it will keep any comments you had when it produces the new one. It is available (source!) for a reasonable price, only $95. I didn't note what the binary sublicensing fee is, but I doubt it is a lot if the original price is only $95. The number for the toolchest (in case any missed it) is 201-522-6900. It is a 1200 Baud dial-in, which puts you into a cute little menu system, with pop-up help menues, and everything. -- Arnold Robbins CSNET: arnold@gatech ARPA: arnold%gatech.csnet@csnet-relay.arpa UUCP: { akgua, allegra, hplabs, ihnp4, seismo, ut-sally }!gatech!arnold Help advance the state of Computer Science: Nuke a PR1ME today!
guy@rlgvax.UUCP (Guy Harris) (03/09/85)
> I just wanted to mention that ATT's Unix Toolchest, which allows you to > sign up for software which is delivered electronically, provies a tool > call "makefile", which will read through your software in the current > directory, and generate a makefile for you. If there is a makefile there, > it will keep any comments you had when it produces the new one. It is > available (source!) for a reasonable price, only $95. For what it's worth, the following, courtesy of the University of California at Berkeley (or whoever did it first) will do much the same, if you have "/lib/cpp -M" (or its equivalent; somebody posted a script that filtered the output of a regular "cpp" to produce dependencies): depend: for i in ${SOURCES}; do \ /lib/cpp -M $$i >>makedep; \ done echo '/^# DO NOT DELETE THIS LINE/+2,$$d' >eddep echo '.kb' >>eddep echo '$$r makedep' >>eddep echo 'w' >>eddep cp Makefile Makefile.bak ed - Makefile < eddep rm eddep makedep echo '# DEPENDENCIES MUST END AT END OF FILE' >> Makefile echo '# IF YOU PUT STUFF HERE IT WILL GO AWAY' >> Makefile echo '# see make depend above' >> Makefile if you stick # DO NOT DELETE THIS LINE -- make depend uses it and two blank lines after it at the end of your Makefile, and if you have ${SOURCES} as a macro which expands to the names of all your source files. -- Guy Harris {seismo,ihnp4,allegra}!rlgvax!guy
kim@enea.UUCP (Kim Walden) (03/15/85)
> From article 3517 in net.lang.c: > > ... [a make dependency generating tool] should be picked up > and stuck into every UNIX in existence (picked up by vendors, > for the benefit of their licensees), because the absence of > (semi-)automated dependency generators is a nuisance. > People don't keep their dependency lists up to date, things > don't get remade when a header changes, and all sorts of bugs > crop up. Or, worse, they only appear when ".o" files are removed > and the modules are recompiled, so the bug appears rather distant > in time from the actual change to the header file. > -- > Guy Harris > {seismo,ihnp4,allegra}!rlgvax!guy I agree entirely with Guy concerning the importance of automated dependeny generators. There seems to be an abundance of such generators spreading around, ranging from simplistic sed scripts to more ambitious work, but the generators I have seen, including the ones from Berkeley, Stanford and AT & T, are all wrong. The reason for this is quite fundamental. Scanning source files recursively to find out exactly what files will be included in what other files, which lately seems to have been added as an extra option to the C compiler, simply does not work. In many cases, files include other files, which are not source files, e.g. the typical #include "scanner.c" in a yacc program. Such files may not be present at the time of dependency generation, or if they are, the may be obsolete, and thus leave out dependencies or introduce faulty ones. One cannot force them to be up to date either, since this would require running make, but make cannot run properly before its dependencies have all been generated. Thus we have an inevitable hen-and-egg situation. The solution is to use ONLY source files as a basis for dependency generation. I have described this at some length in the article "Automatic Generation of Make Dependencies", Software Practice & Experience, vol. 14(6), pp. 575-585, June 1984, and I will talk about it on the EUUG Usenix Conference in Paris, April 1-3. When standard make suffix conventions are used (renaming files like y.tab.c and lex.yy.c to get the base file name of the respective input files etc.), simple transformation rules can be used to deduce from the source file names and include statements extracted from them just what include statements will be present in the generated files, without actually creating these. This also has the advantage of being easily parameterized to handle include mechanisms in other languages than C. I have a program that takes a complete set of source files, extracts all include statements from them, and generates the correct set of dependencies using a set of default suffix transformation rules, without requiring any generated files to be present. If Berkeley is interested, I would be willing to include it as user contributed software to bsd4.3. -- Kim Walden ENEA DATA Sweden UUCP: {seismo,decvax,philabs}!{mcvax,ukc,unido}!enea!kim ARPA: decvax!mcvax!enea!kim@berkeley.arpa mcvax!enea!kim@seismo.arpa
jimbi@azure.UUCP (Jim Bigelow) (03/19/85)
[] There is a command, mkmf (make makefile), in the 4.2BSD distribution, 3rd party contibuted software, spms ( software project management system ) that creates a new makefile from a template, if one doesn't already exist. It automatically makes the dependancy lists. This version is not fool proof, but it DOES work for simple and non-fancy dependancies, it's fast and simple to use. To update a makefile for new files and dependancies just type "mkmf". Jim Bigelow Tektronix
throopw@rtp47.UUCP (Wayne Throop) (03/25/85)
> > ... > > People don't keep their dependency lists up to date, things > > ... > > Guy Harris > > I agree entirely with Guy concerning the importance of automated > dependeny generators. > ... > The solution is to use ONLY source files as a basis for > dependency generation. > ... > Kim Walden Kim's solution seems quite good given the way make works. If you are willing to change make's model of the world, another solution becomes available. In particular, if a make-like tool allowed dependencies to be specified dynamically, the problem of intermediate files being needed before make is invoked (to allow dependencies to be discovered) becomes a non-problem. There are many ways for make to be modified to allow for dynamic inputs, but suppose a syntax something like foo.o: foo.c (foo.c; find_includes foo.c) cc -c foo.c foo.c: foo.x make_c_from_x foo.x The parenthesized text specifies a list of inputs to the "command" at the end of the list. The output of the command will be a list of include files, and the effect is as though that list of files had been supplied instead of the parenthesized text. What make would do for this makefile fragment would be to invoke make_c_from_x, then invoke find_includes (discovering any include file dependencies of foo.c), and then cc foo.c, producing foo.o. This is essentially an "incremental make-file generator". It has problems, such as what to do about include files that include other files, and so on, but these problems can be overcome. Note also that make would then need a database of already-run dynamic input lists, for efficency (otherwise it would need to re-run find_includes every time make runs, not just when foo.c changes).
kim@enea.UUCP (Kim Walden) (03/26/85)
In article <157@azure.UUCP> jimbi@azure.UUCP (Jim Bigelow) writes: >[] >There is a command, mkmf (make makefile), in the 4.2BSD distribution, 3rd party >contibuted software, spms ( software project management system ) that creates >a new makefile from a template, if one doesn't already exist. It automatically >makes the dependancy lists. This version is not fool proof, but it DOES work >for simple and non-fancy dependancies, it's fast and simple to use. >To update a makefile for new files and dependancies just type "mkmf". > >Jim Bigelow >Tektronix Unfortunately, mkmf generates wrong dependencies, in most cases, when generated files are involved. I would hardly call standard use of yacc or lex as 'fancy' by any meaning of the word. Almost correct is not good enough when hundreds or thousands of dependencies are generated. If you have to make manual corrections after each dependency generation, the whole point is missed. -- Kim Walden ENEA DATA Sweden UUCP: {seismo,decvax,philabs}!{mcvax,ukc,unido}!enea!kim ARPA: decvax!mcvax!enea!kim@berkeley.arpa mcvax!enea!kim@seismo.arpa
kim@enea.UUCP (Kim Walden) (04/10/85)
In article <9201@rtp47.UUCP> throop@rtp47.UUCP (Wayne Throop) writes: >> ... >> The solution is to use ONLY source files as a basis for >> dependency generation. >> ... >> Kim Walden > Kim's solution seems quite good given the way make works. If you are > willing to change make's model of the world, another solution becomes > available. In particular, if a make-like tool allowed dependencies to > be specified dynamically, the problem of intermediate files being needed > before make is invoked (to allow dependencies to be discovered) becomes > a non-problem. > > There are many ways for make to be modified to allow for dynamic inputs, > but suppose a syntax something like > > foo.o: foo.c (foo.c; find_includes foo.c) > cc -c foo.c > foo.c: foo.x > make_c_from_x foo.x > > The parenthesized text specifies a list of inputs to the "command" > at the end of the list. The output of the command will be a list of > include files, and the effect is as though that list of files had been > supplied instead of the parenthesized text. What make would do for > this makefile fragment would be to invoke make_c_from_x, then invoke > find_includes (discovering any include file dependencies of foo.c), > and then cc foo.c, producing foo.o. > > ... I do not agree with your proposal, because: 1. Very few people really understand the implications of make's basic model as it is, so the least thing we would want to do is to complicate it further. 2. It would not solve the problem anyway. In the example, make itself forces the generated file foo.c up-to-date before invoking a command to extract include lines from it. But an extraction command will have to deal with nested includes, and when an INCLUDED file is a generated file, the command cannot force it up-to-date, and hence cannot proceed to search the file for more include lines. The hen-and-egg syndrome is still there, and cannot be circumvented. -- Kim Walden ENEA DATA Sweden UUCP: {seismo,decvax,philabs}!{mcvax,ukc,unido}!enea!kim ARPA: decvax!mcvax!enea!kim@berkeley.arpa mcvax!enea!kim@seismo.arpa
throopw@rtp47.UUCP (Wayne Throop) (04/13/85)
In <853@enea.UUCP>, Kim Walden objects to the notion of dynamic dependency derivation, saying: >I do not agree with your proposal, because: > > 1. Very few people really understand the implications > of make's basic model as it is, so the least thing > we would want to do is to complicate it further. > > 2. It would not solve the problem anyway. > > In the example, make itself forces the generated file foo.c > up-to-date before invoking a command to extract include lines > from it. > But an extraction command will have to deal with nested includes, > and when an INCLUDED file is a generated file, the command cannot > force it up-to-date, and hence cannot proceed to search the > file for more include lines. > > The hen-and-egg syndrome is still there, and cannot be > circumvented. I find myself agreeing with point 1. Adding yet another wart to make is not the answer. However, point 2 turns out not to be the case, since I use an existing make-like tool that dynamically derives dependencies. Kim's objections are quite valid, but my original posting (in retrospect) did not adequately present my position. Let me try to clarify. First, my example was illustrative only. I did not mean to imply that I thought that warping the existing make was the proper way to proceed. I chose a make-like syntax, since most readers are faminiar with it. The actual syntax of the tool I use is very un-make-like, as is it's basic model of the world. Second, when I said in my original article that the "include file includes another file" problem could be solved, I had reason to be pretty sure I was right. Because it has been solved. Granted, it has problems with conditionally included files, but then, so does Kim's tool. I made no claim to dynamic derivation's superiority, just that it was a viable alternative. (On the other hand, since user-specified dependency rules can easily be added, it probably IS better in cases where non-standard derivations are used. Back on the original hand, it is NOT better in cases where all the transforms you want to apply are known to a make-file generator.) So: how does it deal with the include file problem? Well, the basic model is that for each buildable item there is a derivation action, and a construction action. The derivation action tells the dependency manager what items need to be built before the construction action can take place. If these are include files, they have derivation actions that specify that the recursively included files must be available before the level-one include file can be considered available. Given the sources foo.c #include "a.h" foo(){} a.h #include "b.h" b.h /* no more includes */ Our automated build tool produced this trace: % 01 VISITING foo.c.cc % 02 DERIVING foo.c.cc % 03 VISITING foo.c.c_source % 03 Changed: File sources:foo.c % 03 Invoking build macro for foo.c.c_source % 03 END VISITING foo.c.c_source {Ok} % 02 Changed: File foo.c % 02 Invoking derive macro for foo.c.cc % 02 END DERIVING foo.c.cc {Ok} % 02 VISITING a.h.c_source % 03 DERIVING a.h.c_source % 03 Changed: File sources:a.h % 03 Invoking derive macro for a.h.c_source % 03 END DERIVING a.h.c_source {Ok} % 03 VISITING b.h.c_source % 04 DERIVING b.h.c_source % 04 Changed: File sources:b.h % 04 Invoking derive macro for b.h.c_source % 04 END DERIVING b.h.c_source {Ok} % 03 Changed: File sources:b.h % 03 Invoking build macro for b.h.c_source % 03 END VISITING b.h.c_source {Ok} % 02 Changed: File sources:a.h % 02 Changed: File b.h % 02 Invoking build macro for a.h.c_source % 02 END VISITING a.h.c_source {Ok} % 01 Changed: File foo.c % 01 Changed: File a.h % 01 Invoking build macro for foo.c.cc % 01 END VISITING foo.c.cc {Ok} A lot of huffing and puffing to go through for a fairly simple compile, but note that in these cases foo.c, a.h and b.h are all GENERATED FILES! That is, they didn't exist in the file system at the start of the "make", but were instead in a source archive. The "Invoking build macro for <mumble>.c_source" are the "extract from archive" actions. Thus there is nothing to prevent the derive action for a.h to force b.h to be created on the fly, if b.h is produced by something else. In fact, this sort of thing is done in many of our automated builds. The crucial ability here is that derive actions can communicate with the dependency manager. The chicken-and-egg problem does not arise here. In fact, if there are no circular dependencies, I don't see how it CAN arise. And, if there are circular dependencies, I don't see how any automated method can do much better. The crucial points I am trying to make: - Dynamic derivation is conceptually simple. In a dynamic derivation, a derive rule needs only to know locally what is going on. EG, I have a C file as input and I want to know what include files there are. I don't need to worry about the fact that the C file was produced YAPG (yet another program generator), nor do I care what the original source was, or even if there WAS an original source in any traditional sense. In a from-source derivation, the deriver needs to know globally what transforms are going to be made. - Dynamic derivation is flexible. It is easy to add new derivation rules, and these new rules don't need to know about how the entire world fits together. - Dynamic derivation is practical. It exists in a working system, and further development is proceeding on these tools. It turns out that I am not at liberty to distribute these tools (and they don't run under unix anyhow), but I can do the next best thing and distribute the idea. As Kim pointed out, enhancing make is not the best way to implement dynamic dependancy generation. Creating a simpler but more flexible tool "from scratch" seems a better idea. (A reasonable first-cut version of the tool itself can be implemented in a couple of man-weeks.) -- Wayne Throop at Data General, RTP, NC <the-known-world>!mcnc!rti-sel!rtp47!throopw