DEAN@USC-ECL (02/28/83)
From: Jeff Dean <DEAN@USC-ECL>
Date: 10 Jan 1983 1126-PST
Comments?
---------------
Return-path: GJC@MIT-MC
Received: from MIT-MC by USC-ECL; Friday, 7 Jan 83 11:24:48-PST
Date: 7 January 1983 14:23-EST
From: George J. Carrette <GJC @ MIT-MC>
Subject: Release of VAX-"New Implementation of LISP"
To: DEAN @ USC-ECL
cc: BUG-NIL @ MIT-MC
In-reply-to: The message of 6 Jan 1983 2105-PST from Jeff Dean <DEAN at USC-ECL>
Date: 6 Jan 1983 2105-PST
From: Jeff Dean <DEAN at USC-ECL>
To: GJC
Re: Release of VAX-"New Implementation of LISP"
Do you know anything about NIL for Unix?
[Kashtan suggested that if we brought it first up under EUNICE that
we could make a unix'able "A.OUT" binary file with minimal hassle.]
Our mission has always been to get the highest performance, and most
generally useful, LISP-SYSTEM running on the VAX. (Or indeed, on a
particular piece of hardware). The Unix implementation on the VAX is
an inamicable environment in which to be pursuing such a goal.
Furthermore, our users view the LISP-SYSTEM as their
system-programming environment, and it is, with dynamic linking,
datastructures and primitives well suited to the VAX instruction set,
and the integration of compile-time-intent with runtime information, a
far superior environment to one conceivable within Unix philosophy,
and beyond the scope of Unix system programmers to implement at reasonable
cost.
The above tirade aside for a moment, we do have the machine resources
to construct an inferior performance NIL to run under Unix for those
people who cannot afford to run VMS/EUNICE. These could be made available
to interested parties. In fact, since NIL is public domain there isn't
anything to stop anybody from bringing it up under Unix.
-GJC
-------guy (03/01/83)
Mr. Carrette did not specify why a superior environment is "(in)conceivable within Unix philosophy". The statement about "datastructures and primitives well suited to the VAX instruction set" indicates his objection to UNIX may be that it is oriented towards portable software. True, but there is nothing in UNIX that prohibits one from directly using the machine language (other than the lack of a macro assembler, perhaps). I'm not quite sure what the "UNIX philosophy" refers to, either. UNIX can support a wide range of applications, constructed in a wide range of styles, with varying degrees of success. If there are some things that the operating system (I refer here to the kernel; if a library routine gets in your way, any application programmer can substitute their own) does poorly, constructive criticism of UNIX would be valuable. (I would like to see UNIX able to handle as wide a range of applications as other general-purpose operating systems - such as VMS - with the same degree of success.) (Also note that either the IPC mechanisms in 4.2BSD or the IPC mechanisms in USG UNIX 4.1 and later may have significantly higher performance than VMS mailboxes; can anyone with experience with these - preferably in the latest releases of those operating systems - comment on this?) I don't know why he claims that a proper implementation of NIL is "beyond the scope of Unix system programmers to implement at reasonable cost"; I presume he means that it would be too much work to change UNIX to support this implementation of NIL. Yes, VMS does have some features that UNIX (as of, say, Seventh Edition or 4.1BSD or USG 3.0) doesn't; however, 4.2BSD may have what he needs. Again, some constructive criticism of UNIX would be helpful; I see no evidence that UNIX, by the very nature of its internal design, is incapable of doing more than a few things well. Guy Harris RLG Corporation ...!decvax!mcnc!rlgvax!guy ...!seismo!rlgvax!guy
zrm (03/03/83)
What has kept Unix responsive to real, important needs is the fact that is was never responsive to niggling details. Throughout a long and unsupported developement only things that were interesting and worthwhile were done to Unix. Hence no "real time" support in an operating system meant to run on non-distributed minicomputers, whereas DEC's operating systems have always been subject to pressure from their user community to be all things to all people. So VMS is not "Software for the 80's", but "Software for the 60's" when all the grotty little real time systems that VMS has to be compatible with were designed. "You mean I can't hook up my U200 RJE station to my UNIX?" Sorry. The do-it-all philosophy sells machines, up to a point, and it provides much ammunition for the creeping featurists. But now that hardware is cheap enough so that any task requiring guarenteed response can have a machine dedicated to it, there is no need to have your 11/780 be able to ignore the 25 people using it to turn a valve off RIGHT NOW. Similarly, if you want to run a very full blown LISP, buy a LISP Machine and don't waste your time with a machine that can't support the most fundamental LISP constructs anywhere near as efficiently. UNIX is a system for a niche, and that niche happens to be quite broad. It is not all encompassing however and I think you will see an explosion of real time systems written in C to let a UNIX centered environment deal with real time needs and you will also see, e.g., LISP Machines and other specialized workstations harmonizing batter with UNIX. Cheers, Zig
tjt (03/12/83)
There are several specific deficiencies of UNIX with respect to NIL (New Implementation of LISP) that I am aware of: 1) The assembler/linker environment is extremely simple: the assembler has no macros. The UNIX assemblers were intended primarily as the final pass of a compiler and therefore macros were considered unnecessary. Preprocessors such as m4 are not equivalent to e.g. DEC MACRO-11 and MACRO-32 which implement conditionals based on internal assembler state such as whether a symbol is defined. However, I do not know offhand whether m4 is in fact sufficient for the types of macros used in NIL, and I am unfamiliar with the NIL source code. The bigger problem that NIL runs into is the restriction to only 3 program sections -- .text, .data and .bss. The VAX/UNIX assembler is a little better but still highly restrictive. NIL takes advantage of multiple program sections to initialize several symbol tables throughout the source code by having each module append initializations to separate program sections. Early versions of C had only fairly simple mechanisms for initializing static variables. Even now there are few high-level languages that allow the type of sophisticated static initializations possible by appending or merging initializations in separate source modules (perhaps ADA, but I would have to check a manual to find out for sure). If you don't think this would be useful, take a look at the source of James Gosling's EMACS if you have it: the symbol table is initialized by executable code in each module. This makes it slightly easier to add new primitives although it is necessary to both define the code for the primitive and to include initialization code to add the primitive to the internal symbol table. A richer language for distributed data initializations and suitable macros would allow these to be better integrated. i.e. the code for the primitive could be immediately preceded by the initialization of the symbol table information. Obviously, there are alternatives to the approach used by NIL, but the biggest problem is either the time required to convert the source code and/or to build appropriate tools to solve the problem differently. 2) NIL expects to manage its address space as a collection of pages, each with independent access rights. In particular, when NIL dynamically loads a new function it would like to make the new "text" segment read only. The address space of a NIL process often becomes alternating "text" and "data" pages. This memory management scheme is incompatible with the three-segments used in standard unix (text, data and stack). This is not an insurmountable (or even a serious) barrier to implementation of dynamic linking as Franz Lisp clearly demonstrates. However, it is clearly useful to make all code read-only if possible when debugging. This type of memory access control is not possible under 4.1BSD, although may be possible under 4.2 (again, I would have to check a manual). In any case, this is an example of system service provided by most "paged" operating systems not currently provided by UNIX. The lack of independently settable access modes for individual pages does cause insurmountable problems with NIL's stack management. I don't remember the details, but the VMS implementation of NIL maintained two "stacks" separated by a hole in virtual memory: when the upper stack ran into the hole, the lower stack was moved down and more memory allocated to the upper stack. I think the lower "stack" was in fact some other type of expandable data structure. In any case, the point is not whether this is a good scheme for memory allocation but rather that it is impossible to implement in 4.1BSD. In summary, NIL and UNIX are largely incompatible as they currently exist. In some ways, this is not surprising since NIL was apparently not written with portability in mind. Much of the early source code was written in macro-32 and bliss-32. This is at least partially due to the initial lack of a NIL compiler. However, I don't think much thought was devoted to which parts of NIL were operating system dependent and/or machine dependent. At the same time, UNIX lacks some features that make porting the current version of NIL impossible, and may very well have made any implementation of NIL awkward. I think it is clear that UNIX operating systems for larger machines (i.e. with paged address spaces) will need to evolve, and I think that 4.2BSD is a step in the right direction. However, it is unfortunate that any such evolution of UNIX will unavoidably conflict with UNIX on non-paged machines, and must be born as the price of progress.