erikb@cs.vu.nl (Erik Baalbergen) (10/16/87)
Here is a (long) summary of the responses to my article "Thoughts about make" which I posted a while ago to this newsgroup. Furthermore, I included fragments of recent news articles which are related to the topic. I'll repeat my original article, intermixed with comments. At first I would like to thank Bart Schaefer, Jim Leinweber, Craig Norborg, Dan Hoey, Dave Bullis, Isaac Balbin, Larry Yaffe, Snoopy (snoopy@doghouse), Stan Shebs, Tan Bronson, Kral Braun, Eduardo Krell, Marty Brilliant, George Tomasevich, John Chambers, Gary Adams, Steve Losen, Peter da Silva, Glenn Fowler, Henry Spencer and Pat Place for their replies and suggestions. The replies contain references to the following makes and make-like systems: nmake, or 'newmake' - AT&T Toolchest mk - Hume's mk (Summer '87 Usenix conf at Phoenix) Sequent's DYNIX make CCSLAND - (Tan Bronson) DEFSYSTEM - on Lisp machines several "hacked" makes (cake, ...) Please mail any further comments and suggestions (and possibly any references to make-like utilities and papers about make) to me. Happy reading, Erik Baalbergen <erikb@cs.vu.nl> ******************************************************************************** The following is a request for thoughts and comments on the Unix "make" utility. "Make" has been in use by Unix programmers for almost 10 years, and many programs and programming systems nowadays come with at least one [Mm]akefile. Many programmers have become familiar with specifying a set of rules, in order to keep their programs up to date, but as their software and the software development utilities became more complex, they had to twist and turn to let "make" know how to proceed after some change. Enhancements to `make' have been introduced to deal with the growing supply of software development and maintenance tools (e.g. sccs). Even parallel versions of "make" were developed, to make use of the large computing power of connected processors. All these augmented "make"s are based on the same declarative language introduced by S.I. Feldman's "make" in 1978. ******************************************************************************** Some general remarks: --- | From: Eduardo Krell (AT&T Bell Laboratories, Murray Hill) | You should take a look at nmake (aka newmake, aka 4th generation make), | which I think is superior to mk. I already sent a note to the original | poster describing nmake and how it solves almost all the complaints | he had over old make. | | nmake is available to the outside world thru AT&T Toolchest. | nmake has been described in past Usenix conferences. Some key features are: | | no need to specify dependencies on #include'd files. nmake does it | automatically. Moreover, it will generate -I flags for cpp to look up | the directories where the header files were found. | You can use a | | .SOURCE.h : incl headers /usr/local/include | | and it will look for all the #include'd files in those directories | (and then /usr/include, of course). | | nmake remembers the command line flags used to compile every file, so, | for instance, if you change a -D flag or a -I flag to look for header | files in a different directory, nmake will recompile. It keeps a state | file with all the info on how every file was generated and the time | stamp. "touch"ing a file won't fool nmake. | | The makefiles are compiled so that future runs of nmake won't require | the Makefiles to be read and parsed again. Of course, if you've changed | the Makefiles, they will be recompiled automatically. | | nmake has really a knowledge base on Unix (but most of it is not built | into nmake, but rather put into a "Makerules" file which has all the | "metarules"). For instance, If I have a library libfoo.a that contains | a couple of .o's generated from their corresponding .c's and a yacc | output file, all I need to specify in nmake is | | libfoo.a :: a.c b.c x.c gram.y (BSD) | | What that one line will do is compile a.c, b.c and x.c into the .o's, | with a -DBSD flag depending on what BSD has been set previously | in the makefile. It will run yacc on gram.y, move y.tab.c to gram.o, | run ar to create the archive or update the archive and remove the .o's. | | There are many more features, but this gives you a feeling for what you | can do. --- | I am an inexperienced user of "make". I have used it in the usual way | to make programs out of a specified set of .c and .h files, and it's | fairly easy to use that way. I also tried to use it in an unusual way | to maintain and use a database, and couldn't make it work well. | | It doesn't bother me that "make" is system-dependent, because I use | plain UNIX(tm). It does bother me that the manual entry is hard to | read. I infer that "make" was originally a low-level facility, and | gradually acquired built-in rules, overloaded syntax, many special | features, and limited expert knowledge about specific command systems. | | I think it's time to rewrite the manual entry so it first describes the | commands "make" knows about, and then explains how to tell "make" about | commands it doesn't already know. "Make" should be described as an | expert system that can be taught more, not as a low-level system with | special rules and features built in. Then "make" should be rewritten | to match the new manual entry. | | By the way, it seems appropriate for an expert system to have, like | "make", a non-procedural language. But after you get past the special | language of "make", you are writing procedural shell commands, with | (unfortunately) a slightly different syntax from the shell. | | As an alternative, the "newer-than" and "older-than" test arguments in | ksh can be used to get the effect of "make" in a completely procedural | language. The shell command file is about three times the size of the | equivalent makefile, but the dependencies and operations are explicit, | easy to understand if you are familiar with shell programming, and more | flexible if you use the full power of shell parameters. That's usable, | but "make" remade as an expert system would be better. --- | You might want to look at Andrew Hume's "mk" paper in the latest Usenix | proceedings. There is a public-domain implementation of it underway. --- | About a year ago I got to the point where I had to hack up my own private | version of make. When we added C++ to our development environment a number | of alternate and mixed environments were needed. We use the Masscomp RTU | operating system. They are BSD and System V compatible at the system call | interface and have provided a "universe" command which determines which | set of headers and libraries to be used via "conditional symbolic links" | in the file system. In addition to the Unix "universe" it is also | necessary to distinguish in our environment the C++ header files from | standard /usr/include headers. The C++ command "CC" on our machine is | actual a script that encompasses much of the C++ specific redirection from | standard headers. The make rules were easy to augment at first my adding | ".C" and ".~C" rules for C++ sources. The problem gets a bit sticky when | you start to add C++ compatible yacc and lex files. | | A number of the stated shortcomings of make are actual deemed as features | in our "masterpack" environment. Since make relies of the shell environment | we can easily modify it's behavior globally by setting up the environment | before a large make run. (We relink all of our software nightly(4 hours) | and rebuild all libraries weekly(14 hours), because of the large volume of | software being changed and the number of bugs that are uncovered by building | from scratch.) --- | By the way, Make is not the only program of its kind. Lisp machines | have a "DEFSYSTEM" construct, and I believe a group at Carnegie-Mellon | University was developing a similar program for Ada or Bliss or Mach or | something. Sorry I can't pass along references, but you may be able to | find a survey article if you look hard. --- | First, I must applaud S. Feldman for a useful program; many non-unix | systems still lack something with equivalent functionality. Second, | I agree with all of your criticisms. I'm currently using 4.3 BSD | make, which has a broken implementation of VPATH. Recently I've been | working on projects with RCS source one place, object files another, | and about 4 ultimate destinations for the dozens of things (C | programs, preprocessed shell scripts, pattern files, ...) being made. --- | A fellow here [...] Somogyi rewrote make to add many things, | he is looking for beta test sites I believe. [...] | The name of the program is cake. --- | Mk has many other nifty features that I won't go into. --- | At the Summer '87 Usenix conference in Phoenix I attended a talk by | David Hume entitled "Mk: a successor to make" Hume's new "mk" (pronounced | "muck") looks a lot like make and addresses quite a few of make's shortcomings. | We do not have mk (I wish we did) so I am taking my info from Hume's | paper in the Proceedings. As I understand it, mk will be (is ?) an | AT&T toolkit product. Anyone out there know how to get it? --- | All these comments refer to "nmake" or "new make", also known as 4th | generation make, heavily used at Bell Labs. --- | One view of make is that it is a logic programming language, and that | makefiles are logic programs. The idea of achieving a goal is obvious, | and a logic formulation also give pattern-matching for the more general | rules and the possibility for backtracking to achieve a goal in some other | way (say, copying a .o over the network instead of recompiling). --- | I know I have not answered all of your questions, but that is | because you left many questions of mine unanswered. For example, you | state that make is strongly shell and environment dependant. I really | don't see it being this dependant, and more so, I really don't see it | as being a detrement to make. I would really enjoy it if you could | expand it on most of your points since I tend to disagree with most of | them on the surface level, although I don't know your reasoning. --- I'm interested to hear from both experienced and non-experienced "make" users how they think about the following statements. --- | I'm experienced; your complaints are mostly well-taken, though I've a | few comments. --- ******************************************************************************** 1) `Make' has some shortcomings, among which the following - there is no way to specify that a single command produces more than one file --- | nmake understand about intermediate and generated files. The default rules | understand about y.tab.c and y.tab.h, for instance. --- | possible with nmake -- also possible with mk (9th edition, 1987 USENIX) --- | Wrong; you just put multiple names before the ';'. I've done this, | for instance with yacc (which produces two output files), and it works. --- | Hmm, I've never run into problems with this, but there must be a way around it. | How do the default rules deal with yacc (which produces y.tab.[ch])? --- | I am not sure what you mean by that. You can have more than one target | in a rule. --- | If you have multiple targets in the same dependency rule and the rule is | followed by a recipe, mk assumes that the recipe creates all the targets. --- ******************************************************************************** - due to the specification of explicit shell commands, Makefiles are strongly shell and environment dependent --- | [nmake] | on UNIX the shell is the way you get things done, and the Bourne | shell seems to be the common shell for non-interactive scripts | | environment dependence is a big problem, and nmake addresses this | by always giving the environment the lowest precedence --- | A problem is that many things in make *look* similar to sh syntax, but | are different enough to bite you. --- | use the line SHELL=/bin/sh and it's pretty safe --- | Well, the commands have to be in SOME language, and the shell language is | better than most alternatives. I suspect that, unless you could come up | with one a LOT better, that most users would just stick to the shell. --- | True. The make here ignores $SHELL and always uses the Bourne shell (as far as | I can tell. In some cases one wants an environment-dependent Makefile, but for | things like USENET distributions it would be better to be able to specify what | is in the make environment. --- | make is a Unix tool. nmake has a lot more knowledge of Unix than plain make | does. I don't see that as a liability, it's a feature. You can use all the | functionality and power of the shell and the Unix model. --- | This is also true of mk, but recipes are not one-line shell scripts executed | via "sh -c". The whole recipe is executed as one shell script. --- ******************************************************************************** - side effects in shell commands are not taken care of; there is no check whether a successful command results in the target to exist --- | nmake understands about intermediate files and targets. In any event, if | the target exists, it is compared against the info in the state file. --- | True, you only find out if the command fails. --- | nmake handles this, although some heuristics are necessary for virtual | targets (such as clean and clobber) --- | nmake keeps a state file where all this information is kept. --- | This is a feature. For instance: | all: foo bar ; # No commands here. | clean: ; rm *.o a.out # No dependencies or output. | These are very useful makefile entries; requiring a "valid" entry that | makes something out of something else would break them. --- | A worse problem is commands that succeed but exit with non-zero status. | However, make ought to at least re-check the date of the target to make sure | that it really WAS made (i.e., that an old version wasn't left unchanged by a | failing command). --- | I don't know if mk addresses this. --- ******************************************************************************** - transformation (implicit) rules work under the assumption that each kind of file has its own suffix; therefore, "make" has little or no knowledge of the type of a file, but relies on the file naming convention (How could you make "make" tell the difference between a Lex source and Lisp file, which both names end in ".l") --- | this is a deficiency that is being looked at but not currently solved | mk handles this by providing generalized pattern matches --- | Good point. Unfortunately, in a system like Unix with typeless files, | it is hard to see how this could be done cleanly. In essence, the "suffix" | is used in Unix to indicate type, and in your example, the "system" is | being abused by using the same suffix for two types. If you follow the | custom that ".foo" means a file whose contents are of type "foo", then | make works pretty well. --- | Give them different names (like... .lisp! After all, you don't HAVE to have | 3-or-less-character file extensions on UNIX. --- | The only other way I can think of to identify the types of files would be to | put a comment on the first line; but then either make has to understand all | sorts of comment forms or else a particular "make-comment" form has to be | understood by all the other programs that might deal with the files. Yyuucckk, | in either case. --- | You can't because Unix files are not types. This is really a complaint about | Unix and not make. --- | Mk doesn't restrict you to single character extensions. You can construct | rules such as | %.o:%.lisp | ln $stem.lisp $stem.l | lisp -o $stem.o $stem.l | rm $stem.l $stem.lisp | | Note that mk puts the variable "stem" in the environment of the recipe | script. Stem contains whatever the "%" matched. You can easily keep source | in one directory and objects in another by having rules like | %.o:../src/%.c | $CC -c $CFLAGS ../src/$stem.c | | Mk also allows arbitrary regular expressions in metarules if the % facility | isn't general enough. --- ******************************************************************************** - it is, e.g., hard to incorporate a "make dependencies" within a single run of "make"; make is not able to dynamically add or change rules --- | nmake dynamically determines source file include dependencies --- | This is often considered a feature. It might be described by saying that | make implements a non-procedural (declarative) language. This isn't quite | true, since commands are run sequentially, and the order is well-defined | by the postorder scan of the dependency tree. But it's a bit hard to think | of a way around it that doesn't complicate make's language significantly | Remember that one of the reasons for make's popularity is its simplicity. --- | I've seen Makefiles that have commands to append to the Makefile and then call | make again, but this is AWFUL. I try to set up Makefiles with enough variables | that "recursive" makes can supply new variables in the command line. I may | have 3 or 4 ways to build the same program, with different target names for | each compilation. --- | The static view of dependencies that make has is a continual nuisance. | The fact that make offers no help in generating dependencies is a problem. | I use a locally written program (called 'Am' - for "Automake") to generate | makefiles. This program takes what you might regard as a skeleton | Makefile and recursively tracks down dependencies (by actually reading | the source files and understanding C's #include statements). Obviously | language specific, but invaluable nevertheless; particularly in the | development stage of a software projects involving many source files. --- | in nmake, you can have nmake rules as the action part of a rule, in effect, | adding or changing rules at run-time. | >Does this mean that 'nmake' is called recursively? | not, but you can change the rules (by having a rule has the action block of | another rule). --- | mk can include the standard output of a recipe as part of the "mkfile" --- ******************************************************************************** - time is the only file attribute that is taken into account when comparing files; one can think of having a more general file comparison mechanism, looking at more file attributes --- | [nmake] | build (AT&T Technical Journal) has generalized this by allowing a | shell script to determine if a prerequisite has changed -- this can be | expensive though -- the trick is to generalize without losing efficiency | | the "out of date" test must be a small percentage of the total time | to build the target --- | One example I've run across is that I've wanted to have entries that | say something like "If foo doesn't have the right permissions, then | set them." It'd also be useful to be able to ensure that a directory | exists, is in fact a directory, and has certain permissions; you rarely | care about the update time in directories. --- | Like what? --- | Such as what attributes? If make is to be fairly general, it can't look inside | the files for information. Perhaps checking that files had appropriate | permissions would be feasible .... --- | Why would you want check attributes other than the times? Of what | use would this be? --- | we've looked at this and we might add a checksum or file size field when | comparing files. --- | Mk identical to make on this point. --- | The fact that make only looks a file modification times is a serious | limitation. A common occurance is having a top-level .h file which is | included in many .c files, and deciding you want to change a comment in | the .h file. Suddendly, you have to suffer through an hour long | recompilation. Improving this is clearly tough - I have no practical | suggestions. --- ******************************************************************************** - it is easy to loose control when altering the Makefile, unless you add "Makefile" to each target's dependency list; a famous example is adding or deleting a -Dxxx flag in CFLAGS --- | Or, equivalently, passing such parameters on the command line. My favorite | is something like 'make foo "CFLAGS=-DEBUG"' when the file has '#ifdef EBUG'. --- | nmake has state variables, and newer versions automatically determine | state variable dependencies, using these to generate -D options | for C compilations -- nmake also keeps track of prerequsite list | order changes and action block command changes --- | On the other hand, you may not want to recompile the whole world when | adding -DDEBUG to one .c file. I would like to see much better handling | for turning debug flags on/off. (hint: comand line options?) --- | This is indeed a hairy problem. In fact, you'd really like to be able | to say "If foo.o was compiled without -Dxxx, then rebuild it." --- | Again, I avoid this with multiple target names that are just dummies for the | same "real" program, and make the commands for the dummy targets include | "touches" and the like to cover the altered definitions or whatever. --- | I don't think mk addresses this. --- | Such dependancies should not be put into a makefile. They should be | in an include file, of which the dependancies would take care of that. The | only thing that should be on the CFLAGS line should be dependancies that | affect most of the source, in which case, doing a 'make clean' is always | advised. --- | nmake keeps track of the command line options it used to compile each file, | and it they are different next time, it will recompile. The reason why nmake | can do that is that it keeps a state file with all the information about the | last run. plain make doesn't have that and thus can't "remember" this. --- ******************************************************************************** 2) The "command" parts in the Makefile should be Unix-flavour (and shell) independent and environment independent. --- | nmake helps here by partitioning its info input: default rules, | global rules and user makefiles (all separate from the nmake engine) --- | Largely true, but there should be some way of extending the behavior of | Make. No matter what extensions you provide, they will probably not be | enough. At least, it should be possible to generate a program in one | rule that is run in another rule, and the syntax for running that | program might differ from shell to shell. | | I agree, though, that much of the shell coding is a clumsy work-around | for Make's inadequacies. --- | Well, the only way to do this is to invent a new command language, and try | to make people adopt it. Again, this would only be accepted if it were very | much better than the existing language. Even then, people would say "Why | should I learn yet another obscure programming language, when the old make | will let me use a language I know?" Granted, the makefiles from Unix systems | aren't of much use to, say, OS/MVS users. If you figure out a simpler way | to do the commands than with shell commands, let us all know! --- | Why? The commands themselves aren't. That's what default rules are for. --- | This takes away an awful lot of the power of make. Perhaps some sort of | "conditional-dependency" along the lines of conditional compilation in C could | be built in to make, with system-dependent keywords (like "vax" in cpp) | supplied "automatically." --- | Mk runs all recipes through sh(1). I guess that's as shell dependent as | you can get. --- | The shell is the interface between the user and the system. Making use of | shell features (like I/O redirection and environment variables) is extremely | useful. Why do you want to take that away? --- ******************************************************************************** 3) The declarative "make" language should incorporate subroutines to facilitate the description of almost identical "make" code, which otherwise occurs more than once within a single Makefile. The user has to be able to specify conditional and repetitive execution of commands, using only the declarative language. (One often sees an "if" statement programmed in a shell command.) --- | I use shell scripts or recursive calls of the makefile to get around that. | They are both slow, though. --- | nmake has constructs and macro expansion operators that help solve the | info duplication problems --- | - In cases where multiple objects need to be installed in a | destination directory with extra processing, you end up with several | make lines per object, first to make it, and then to install it. | It is distressing to write 47 lines of the form: | | B = ../../bin | $B/foo : ./foo ; install -s -m 4555 -o grumble -g fooey $? $@ | | when you ought to be able to merely say something like | | foo bar baz ... -> $B | install ... --- | Good idea. Can you come up with a syntax that is easier to use than the | usual repetitions? Remember that the current method of specifying default | rules is sufficiently complicated that many users don't bother with it. | It's easier to replicate the commands via an editor than it is to master | the somewhat obscure notation in a make ".foo.bar:" entry. The result | is also easier for readers to understand. I've often found myself going | through makefiles and inserting explicit commands, both to get around the | common problems of make generating incorrect commands, and also to rewrite | the makefile so other humans could understand them better. --- | This would be helpful in some cases. One example I fought with recently | involved a Makefile in one directory referencing .c files in other directories; | specifying the dependencies was something awful. --- | Since whole recipes are executed as one shell script, you are free to | write very long and complicated recipes using all the power of sh(1). | Since mk passes all variables to recipes through the environment, the text | of a recipe is copied untouched from the mkfile to the shell. --- | nmake as "metarules". For instance, to specify that a program "foo" is built | by compiling "foo1.c", "foo2.c" and by a yacc file "gram.y", you just say | | foo :: foo1.c foo2.c gram.y -lPW | | nmakme will compile foo1.c and foo.c with the -c option, will run yacc on | gram.y, move y.tab.c to gram.c, compile gram.c and load all the .o's into | foo with the libPW.a library. All you need is the one line above. | | Another metarule example: libraries | | libx.a :: x1.c x2.c x3.c | | again, that one line does the following: | | x1.c, x2.c and x3.c are compiled. The .o's are archived into libx.a | and then removed. --- | This can be done by using the shell interaction with make. For example, | a common occurance in makefiles here is the use of a ${LOOP} variable which | allows us to do operations on all subdirectories. Below is our LOOP variable | and an example make rule for this. | | ]SUBDIR= ansi ansitar arc | ] | ]LOOP= for i in ${SUBDIR}; do\ | ] echo $$i:;\ | ] cd $$i;\ | ] [ -f Makefile ] || co Makefile;\ | ] make ${MFLAGS} DESTDIR=${DESTDIR} $@;\ | ] cd ..;\ | ] done | ] | ]clean: FRC | ] ${LOOP} | ] --- ******************************************************************************** 4) Parallelism, if supplied, should be transparent to the Makefile. --- | in nmake it is transparent up to the point of mutual exclusion | (e.g., only one yacc at a time because of y.tab.c) which requires | explicit assertions -- a command line option limits the maximum | number of concurrent actions (default 1) --- | This implies that it is done automatically. This idea makes me a bit | nervous. It's true that I sometimes get annoyed at a lot of serial | commands that could be done in parallel more quickly. But I accept | it, because then when something fails, I don't sit there being mystified | by what went wrong. Many makefiles have lots of subtle interdependencies | on the things being made. It's often hard enough to get them right with | a deterministic algorithm; I'd probably give up on a parallel make after | the first time it got the order wrong. --- | Within limits. Parallelism, if supplied, should not necessarily be automatic. | I like the way Sequent's DYNIX make handles it -- | # Compile all dependencies sequentially | target : dependecy-list | # Compile all dependencies in parallel | target : & dependecy-list | # Compile depend1 and depend2 in parallel, then do depend3 | target : depend1 & depend2 depend3 | Variable expansion is done before parsing for parallel constructs, so -- | # "Portable" parallel make -- use "make P='&'" for parallel | target : $(P) dependency-list --- | Mk does this, but you need to specify the number of targets that | can be built simultaneously (default=1). --- | the -jn option will cause nmake to run up to "n" jobs in parallel. | >Is there a facility to define a mutex, to specify that some commands, e.g. | >two 'lex'es, may not run in parallel? | No, but I think it can be done by changing the default rules. --- ******************************************************************************** 5) Each user or group of users should have its own "make environment", which defines system-dependent macros and actions (e.g. ".makerc", which defines CC and specifies how to create "file.foo" out of "file.bar"); --- | This has to be managed with extreme care. How shall User A give | software to User B if their .makerc files are incompatible? --- | You probably already have this. Most extant versions of make routinely | use anything in your shell environment. I use this constantly. For | instance, right now I have "DEBUG=2" in my environment, which is used | by my personal C debugging package to initialize its debug package. My | makefiles tend to say things like "-DDEBUG=$(DEBUG)", which passes the | environment value to cc. And so on. --- | Probably, yes. --- | Doesn't this conflict with statement (2)? --- | I don't think mk offers this. --- | nmake supports private rule files, exactly what you want. --- ******************************************************************************** 6) Make should be integrated with a source-code revision-control system. (sccs, rcs) --- | This is a real problem. We have a system that has what I call a 'fake' | makefile, because of the lack of integration. --- | [nmake] | this is not a clear choice -- software generation specifications are | distinct from software version specifications -- rather than putting | version specs into make, a better solution might be a separate version | control systems that cooperates with make --- | And something needs to be done about RCS's defacto reversing of the meaning | of the write-permission bits. (I want to be able to turn off write | permission and not have make/RCS silently update the file on me!) --- | I have worked on (hacked) make here for a bit. We have ported the 5.3 make | to our convergent tech mightyframes. We use RCS. | The changes I made: | 1) change searchdir() to search all the directories in VPATH, so we can | set VPATH=$(RCSDIR) and make will find rcs files as dependents. | 2) added rules for rcs which are enabled by a flag. | 3) for rcs files (*,v) spawn a subprogram to return rcs's idea of the | timestamp respective to a revision. | | And some bugs with macro expansion in $(NAME:x=y), VPATH and setting up | environment variables before execing commands. | In the queue is fixing $(NAME1$(NAME2)). --- | I would like see some rules which combine with the .PREFIXES hack which | was made to make for RCS, which would mean if this rule every fires then | make sure we delete any files it created when we're all done. | | .SUFFIXES: | .SUFFIXES: .c,v .c .o | .PREFIXES: . /usr/foo/srcdirs/RCS | | .c,v.c: | co -q $(RCSRELEASE) $< | $(CC) | | # when a file foo.o needs to be built a foo.c (or foo.c,v) are looked for | #using .PREFIXES as a list of directories where the source files might be found. | | There is also a VPATH in the newest version fo make which was mentioned | in a Usenix talk a while ago. I believe this is identical --- | But not locked into just one! With the right extension language, you | should be able to write a guide for interfacing to brand-X source | archive. --- | Many of them have been. Unfortunately, the result is too complicated for | most users to master, so it doesn't get used by more than a few hackers. | I have lots of bad experiences with such features; I tend to give up and | just use simple, explicit entries instead. Then it does what I want. --- | Rephrase as: Either the shource code control system should use suffixes, or | MAKE should have some better way of describing default rules. More anon. --- | It might be possible to handle this now if the (e.g.) RCS files are in the same | directory. Does a dependency like | .c,v.c : ; co $*.c | work, or does the comma in the RCS suffix mess it up? --- | [mk] | You would have to do this yourself with metarules. --- | No! We had this done at this site for a few months. It disoriented | many users to a great degree and generally made things confusing. It is | easy enough to create rules for your own source code revision system that | binding these rules into make is foolish! --- | nmake understands sccs, and it would be trivial to add rules for rcs. |> How does nmake solve the 'prefix' problem? | nmake understands both prefixes and suffixes. --- ******************************************************************************** 7) Which version of "make" are you using? What are the shortcomings you observed? --- | Make works nicely for simple things, and become a big headache | for anything out of the ordinary. --- | I use nmake all of the time. nmake has been available in the | AT&T Toolchest since 8/85 (although this version has not been | updated since then). The original nmake paper appears in the | Summer 1985 Portland USENIX proceedings. I have acces to the | latest version and the comments will be on this version. --- | Probably 4.2 BSD. I think we have nmake floating around but I haven't | had time to try it. --- | I have a hacked version of the PD version of make posted to the net a | while ago. | I've added in .PREFIXES (directories where source can be found), some | locking which assures only person runs make at a time in the same directory, | and a built in command 'docmd' which allows: | docmd -S. -S./RCS -C"lint foo.c" foo.c | which says that one should find foo.c and then run the command specified | with the -C argument. If we had to check out a copy of foo.c we delete it after | the command had completed. | | [in a later message:] | [...] but there are some 'easy' enhancements: | .INIT - a rule which is executed before anything is done | .DONE - the last thing executed before the task exits | +FOO=VAR - allow this as a command rule (using + as a prefix) --- | 4.2BSD --- | Several. This is a rather general Sys5.2 machine. I'm also using a version | by Microsoft that runs on MS/DOS; I don't like it much. Instead of the usual | non-procedural language of Unix make, the Microsoft make processes the entries | (all of them) in the order they are read. This is a major loss of both power | and readability. --- | System V make, Aztec 'C' make on the Amiga. A make of my own devising on the | IBM-PC and the VAX under VMS. --- | We have the 4.3 BSD distribution make (sccs id version numbers range from 4.3 | to 4.12). The biggest shortcoming I've found is the problem with source files | in other directories. --- | 1) I use Sys 5.2 derived version of make on a Ridge 3200. --- | nmake, or "newmake" or 4th generation make. --- ******************************************************************************** 8) Are there any other "things" you would like to express in "make", but which are hardly (or not) possible? Did you ever met a situation in which you were forced to use non-make constructs? (Examples are welcome!) --- | I want to remove the *.o files after they are stored away in a *.a library. | The 'make' generates .o files for all .c files or their dependencies (usually .h | files) that are newer than the library. The 'make' finds out whether any .o | files got created by finding out whether a string like '*.o' expands into | something else. I could not find a way to do it in 'make', so I added a shell | script. It goes something like this: The script is invoked as | something.sh *.o | The script does | if [ "$*" != '*.o' ] | then | ar rv archive.a $* | rm -f $* | fi | The apostrophes prevent metacharacter expansion inside the if [ ], | so if unquoted *.o expands, the script knows that there are *.o files. | I think that our 'make' (UNIX 5.2) can deal with libraries, but this program | has evolved from the Mashey shell to Bourne shell to Korn shell, and I have | not always kept up with changes. --- | if-then-else | | e.g. | MACH=`uname -m` | if (MACH = VAX) | foo | else if (MACH = 32k) | bar | else if (MACH = 68k) | barfola | fi | | Perhaps make needs to be completely re-thought out from scratch. | A shell script would often be easier and faster than using make. --- | - You can't specify how to use multiple dependents, or even that all | of them have to be used in cases where only one is out of date. This | makes it impossible to use default rules for things like: | | foo : foo.KSH fix.sed ; sed -f fix.sed <foo.KSH >$@ | | A $1 style notation would cure that. The situation gets worse if more | than one sed or awk pass is required (I wrote a "sub" for "substitute" | shell script ...) | --- | Often. I usually just have make execute a shell script. Sometimes the | script invokes another make, perhaps after a cd or changing the search | path or modifying the environment or .... --- | The version of make I have does not handle libraries very gracefully. | I typically want to place all (or most) of my .o files in a library, | and then delete the .o files. I also like to do the same thing | with lint's .ln files. And finally, when multiple files are out of date, | I'd like to be able to do all the recompilations first, and then put | everything in the library just once. With my version of make, doing | all of this in a fault tolerant way is next to impossible. (Typical | problem - one of the source files fails to compile. You want the | .o files of the successful compilations placed in the library and | then deleted, but do not want to disturb the library version of the | failed compilation.) --- | Hmm ... I'm not really a "make-niac" so the worst I've had to do is call some | shell scripts from make to satisfy odd combinations of conditions. I know of | people who pratically live in "make," using it a sort of a meta-shell, so | those types probably have all sorts of things they'd like to do but can't. --- ******************************************************************************** 9) Do you have any suggestions for further enhancements, apart from the ones stated above? --- | Some things that would be nice to have: | 1) conditionals and loops in the declarative language | 2) knowing the name of the current make file | 3) a warning when the command exceeds make's buffer size, (if you are lucky | sh will complain when this happens). this would be less of problem with 1) --- | I have been developing a front end to make, which uses a high level | description file, [...] | | It is part of my 'CCSLAND' product which supports doing configuration | control for large unix based software products. All makefile parameters | are included as additional description file. --- | Well, I have occasionally wished that I could tell make to loop until it | fails to find anything that isn't up to date. I've also occasionally wanted | to say "If foo depends on bar and is out of date, update it; but if foo | is missing, don't bother building it." I've also wanted to say "If foo | exists, delete it" without getting a "no such file" error message when | it doesn't exist. Do you know a way to do this? --- | The syntax for make default rules are abominable. They have had to be kludged | beyond belief to handle sccs. When I implemented my own make I threw them | all out of the window and used the following (Lattice on MS-DOS example, from | memory): | | # Running out of floppies. C: is vdisk.sys | # Make lattice put temp files in RAM. | C:*.q: *.c | lc1 $? -i$(INCLUDES) -oC:$?.q | | *.obj: C:*.q | lc2 C:$?.q -o$?.obj | | *.com: C:*.com | copy C:$?.com $?.com | del C:$?.com | | C:*.com: *.obj | link $(LIB)/cc.obj+$?.obj,NUL,$?.com,$(LIB)lcc.lib | | #to make "make foo" work: | | *: *.com | | It should be obvious what's going on here. To make it work I force a space | after the : seperating wildcard targets from dependencies. In UNIX, sccs | would be resolved with: | | *: s.* | get s.$? | | The code for handling the wildcards is a bit ugly in my implementation, but | we can't all be perfect. There is no .SUFFIXES pseudo target... it's not needed. | Normally you would put your wildcards in a file named "default". On MS-DOS this | file is searched for first in the current directory, then in $LIB. In VMS it's | searched for in the current directory then in your login directory. I guess | you could call it .makerc. --- | Handling of directory structures would be nice. --- | I include, below, a message I wrote a couple of years back about the | problem of multi-step make rules. Buried in there is another | problem--it is impossible to invoke Make recursively, passing all the | options (e.g. the -f switch). | | Included message: | | >Date: 20 Mar 1985 16:47:20 EST (Wed) | >From: Dan Hoey <hoey@nrl-aic.ARPA> | >Subject: Multi-step implicit rules for make(1) | >To: unix-wizards@brl-tgr.ARPA | > | >In <177@osiris.UUCP> eric@osiris.UUCP (Eric Bergan) has a problem with | >make's use of implicit rules. An example of the problem can be seen | >with a makefile like | > | > .SUFFIXES: .D .C .B .A | > | > CONVERT_A_TO_B=cp | > CONVERT_B_TO_C=cp | > CONVERT_C_TO_D=cp | > | > .A.B: | > $(CONVERT_A_TO_B) $< $@ | > .B.C: | > $(CONVERT_B_TO_C) $< $@ | > .C.D: | > $(CONVERT_C_TO_D) $< $@ | > | > all: foo.D | > | >Executed in a directory with only makefile and foo.A, this make will | >complain that it doesn't know how to make foo.D. | > | >As both howard@cyb-eng.UUCP (Howard Johnson) and greg@ncr-tp.UUCP (Greg | >Noel) have pointed out, the problem is that for an implicit make rule | >".A.B" to be invoked foo.A must either exist or be mentioned as a | >target or dependent in some rule. Thus the problem can be circumvented | >by the addition of the dependency | > foo.D: foo.B foo.C | >or even | > foo.C: foo.B | >in the makefile. One problem with adding lines like this is that they | >must be added for bar.D, baz.D, ad infinitum. Another problem is that | >the methods fool make's error handling. If we instead add the line | > foo.D: foo.B | >make will create only foo.B, with no complaints. The problem is that | >it treats "foo.D" as a target like "all" in this case, not as a file | >that must be created. | > | >We now turn to methods for getting make to use implicit rules to create | >foo.D. Both Howard and Greg propose adding a rule like | > .A.D: | > $(CONVERT_A_TO_B) $< $*.B | > $(CONVERT_B_TO_C) $*.B $*.C | > $(CONVERT_C_TO_D) $*.C $@ | >(their solutions are identical except for Greg's cleanup line | > rm -f $*.B $*.C | >) to the makefile. In this case, with four suffixes involved, we might | >want to also add rules for .A.C: and .B.D:. I find these methods | >unsatisfactory because of the proliferation of the information about | >the commands used for converting one object into another. For | >instance, if the procedure for $(CONVERT_B_TO_C) took the arguments in | >the opposite order, we would have to change the information in two or | >three rules. | > | >I would suggest that the rules to be added should tell make to | >use the previously defined implicit rules, for example | > .A.D .B.D: | > @make $(MFLAGS) $*.C $@ | > rm -f $*.C | > .A.C: | > @make $(MFLAGS) $*.B $@ | > rm -f $*.B | >There are a few things to watch out for here. For one thing, the | >inclusion of $(MFLAGS) does not pass all of make's invocation to the | >subsidiary make. If you expect to invoke this script as, for instance | >``make CONVERT_B_TO_C=mv'', you must set up the recursive invocations to | >say ``@make CONVERT_B_TO_C=$(CONVERT_B_TO_C) $(MFLAGS) ...''. It is | >clear this could get out of hand. And if you like to try things out | >with ``make -f fakefile'' you had better forget this approach entirely. | > | >Another thing to watch out for is the ordering of ".SUFFIXES:". The | >dependents of .SUFFIXES must be in reverse temporal order for this to | >work. Since lines like ".SUFFIXES: .A" append to the list of suffixes, | >this is sufficient if you want to add preprocessors to your source | >files. If you want to add postprocessors to targets already known to | >make, be sure to start with a ``.SUFFIXES:'' line with no dependents. | > | >It would of course be nicer if we could get make to know how to | >recursively apply its own implicit rules as it does for explicit rules, | >but I suppose there are problems with that. | > | >Dan Hoey | >Navy Center for Applied Research in Artificial Intelligence --- ******************************************************************************** 10) Do you have any experience with concurrent versions of make? --- | No. But I have used the shell '&' operator inside makefiles. I have also | told make to invoke shell scripts that ran commands in parallel. What could | a parallel make do in addition that would be worthwhile? --- | I don't know what a "concurrent verion of make" is. If you mean a version | of make that can support several actions run in parallel, nmake already | has that. --- | I've used DYNIX make (example above). Sequent says they can rebuild the whole | DYNIX kernal in 3 hours with their parallel make, as opposed to 24+ hours with | sequential make. --- ******************************************************************************** PS This article is not meant to be an offence against "make". I still feel that it is a great and useful tool! It's just that I'm proposing enhancements which could be incorporated in any future 'make's. --- | Agreed; it is one of the more brilliant innovations to come from Bell Labs. | The fact that it isn't quite perfect shouldn't detract from that. --- | anyway I'd like to be kept up to date on your effort towards a new | version of make! --- | I encourage you to try to design a superior version fo make. | Please post more specific proposals to the net. Good luck. --- -- Erik H. Baalbergen <erikb@cs.vu.nl> Dept. of Mathematics & Computer Science Vrije Universiteit (tel +31 20 5488080) P.O. Box 7161 Amsterdam The Netherlands
andrew@alice.UUCP (10/22/87)
i was unaware of this discussion so i am responding rather tardily. Most of the repsonses made on mk's behalf were accurate (thanks henry?). by the way, toolchest and TOADS has mk now and educational people are being sent tapes (mhuxd!macor for details). 1) rules producing more than one file: explicitly supported. -) sh specific: not too much. mk doesn't care about commands because it doesn't look at them. the only bug is mk thinks it knows how the shell expands names, metacharacters etc. -) side-effects in rule bodies: mk checks the times after the recipe executes so that a rule firing does not mean the target is updated. -) suffixes are not enough: agreed. mk gives more pattern stuff than suffices but does not suport types for non-aggregates. It does try to figure out what type of thing an aaggregate is (archive etc). -) dynamic rules: mk supports (in a clumsy way). -) general 'depends-on' mechaism: i thought about this real hard and could not think of an efficent way to do this. forking a process is a loser but is necessary in general. -) altering Makefile: mk manipulates files that depend on files. for subfile resolution (at least C-wise) use nmake. 2) recipes should be unix and shell independent: hahahaha. should they be C, or fortran, perhaps? 3) subroutines: i am not too sure what is meant here. mk strongly supports shell in recipes which gives you most of what you want. the other help it gives is rules for similar targets: (cmda|cmdb):R: \\1.o # R->regular expression cmda or cmdb $CC $CFLAGS -o $target $prereq 4) parallelism: mk runs $NPROC streams at once. the only problem is with braindead programs (like yacc but not lex) which produce output in a constant name. Even yacc can work in parallel by a (admittedly) more complicated rule which goes to another directory to do its work. what makes this work for mk is $nproc which is unique per recipe %.o: %.y mkdir /tmp/$nproc; cp $stem.y /tmp/$nproc (cd /tmp/$nproc; yacc $stem.y; mv y.tab.c $stem.c) $CC $CFLAGS -c /tmp/$nproc/$stem.c && rm -fr /tmp/$nproc (i am not real proud of this but it works and doesn't need special mechanism inside mk). 5) customising: i am against this in principle but as mk gets its BUILTINS from the environment, anything is possible. 6) support for sccs/rcs: just another metarule (which is just as it should be). 7) which make do you use? mk, of course. I am scared by nmake. although V8 make is usable, every other make i have used is so braindead i can't use it anymore. Sun's make is particularly bad; however mk runs real fast on a sun. 8) wish-list/hard to do: mk is general enough that i personally can do all i want. however, chris fraser supports a compiler that involves five levels of directories and where programs at lower-levels are made, then run to make upper source files (all with one mkfile). processing is different on a per directory basis, too. his problems would be best solved by binding variables to upper level targets while generating the dependency graph for targets below. E.g. comp: pdpcc vaxcc vaxcc: vax/compiler :bind 'O=vo' for lower targets pdpcc: pdp/compiler :bind 'O=po' for lower targets %/compiler: %/*.$O # get the right .[pv]o's $stem.cc -o $target $prereq it would also be nice to have macros in mk. 9) enhancements: extending the general depends-on relation. If you don't have mk, you would need to add transitive closure, real recipes (not one-liners) and at least, %-style metarules. 10) concurrent make: DYNIX make is an extraordinary hack; good enough but restricting concurrency to one line rule bodies?? come on! PS) forgive us, stu: make is and was a fine accomplishment. however, it was implemented for a specific environment and purpose. time to go on.