[comp.text.tex] TeX - The Language

pauld@stowe.cs.washington.edu (Paul Barton-Davis) (05/14/91)

In article <1991May11.013248.16286@nntp-server.caltech.edu> marcel@cs.caltech.edu (Marcel van der Goot) writes:
>We've recently had a number of postings about ``the TeX language,'' its
>perceived shortcomings, and comparisons with other typesetting systems.
>
>From pauld@stowe.cs.washington.edu (Paul Barton-Davis):
>> The hacking problem is primarily caused by the fact that the TeX
>>
>> [ .. stuff from me & Edward Wang about TeX's language deleted .. ]

>Also, the history of programming shows you wrong: ``hacking'' (used
>in the negative sense) occurs in all programming languages. The hacking
>problem is caused by programmers who lack knowledge about programming
>techniques.

OK, I used "hacking" in the wrong sense. What I meant by this was the
requirement to write "odd" solutions even for quite easy problems. I
don't consider myself lacking in knowledge of programming techniques,
but I can't bring many of them to bear on a TeX problem.

>
>From edward@priam.Berkeley.EDU (Edward Wang):
>> I agree.  I can think of three things wrong with the Tex language:
>> textual substitution macros (rather than functions or even
>> Lisp-like macros),
>
>TeX is indeed based on macro substitution, rather than on the constructs
>found in conventional programming languages. But then, TeX is not a
>general-purpose programming language, it is specifically designed for
>typesetting text. It is much more of an application program than a
>programming language. An important difference with a ``normal'' programming
>language is that the typical user wants to present TeX with the text
>to be typeset. 
		
ABOLSUTELY NOT ! The typical user, rather regrettably, wants to present
TeX with instructions on *how* their text is to be typeset. This is why
using TeX *is* a "programming" activity, and as such, should really
have proper support for it. If the details get hidden by a layer like
LaTEX, some much the better, but TeX is a mark up language first and
foremost, and because of this, needs proper programming support.


>As far as I can tell, the only way in which you can have the freedom of
>presenting your input as almost plain text, with very little knowledge
>about the details of the underlying programming language, is with a
>language based on macro substitution. A conventional programming language
>with strict syntax means that your input has to obey very strict rules.

Like balanced delimiters ? Like \begin{} ... \end{} pairs ? Like
backslashes to identify keywords ? Like only 9 arguments to macros ?
Like double-#'s for delayed expansion ? These are all just as
troublesome as the "strict rules" C or Lisp presents me with. Given a
reasonable editor, its not at all difficult to conform to these rules.

>It is true, TeX's macros take some getting used to, because of the complexity
>of the system. What you get back for that is an unparallelled freedom in
>the form your (normal text) input can take. You can make TeX look like
>LaTeX (which is only somewhat different), but you could for instance also
>make TeX look like troff. With a normal programming language that
>is impossible: A C program always looks like a C program, a pascal program
>always looks like a pascal program, and a lisp program always looks like

Not true. I've seen C code that looks like SmallTalk. The difference
with TeX is that you can change the syntactic sugar (what character
does what), but you can't change the fundamental language structures.
With C, its not easy, but is possible, to given yourself a whole new
style of programming. Trying to use TeX's macro language to enable
users to have greater ease-of-use when inputting TeX is like trying to
use Fortran to implement linked lists or other pointer-based data
structures: you can do it, but it costs and its cumbersome.

>a lisp program. You cannot decide, ``let's give an end-of-line the meaning
>of a semi-colon'' so that you don't have to write semi-colons in C; You
>cannot decide to use square brackets instead of parentheses in lisp. And
>these are only very minor variations, many TeX macros do much more.

I would disagree. TeX macros do much *less*. They are gloss, not
substance, all too often. Useful to a specific paper or article, but
not much use for documentstyles.

>Is it important to have so much control over the form of your input? Yes,
>it is. Not so much for you programmer types. Programmers are used to
>obeying strict syntax rules. But it is important for people who do the
>actual writing of the books, papers, letters, etc. For those people the
>ease of *usage* of the macros is important; TeX can make that very easy.

LaTeX, a good example of TeX made easy, is, at the admission of its
author, full of kludges and "hacks" that try to deal with the fact
that TeX does *not* make it easy.

>Of course, that means that the *construction* of macros becomes more
>complicated.
>
>In short: TeX's macro language makes input of text much easier than
>any conventional type programming language could; the price is the
>increased burden for the macro designer.
>

Precisely. Hence Tim & my decision to separate the macro language from
the extension language. We were trying to use lisp for the latter, and
then layer TeX on top, for use as an option.

>Paul Barton-Davis adds to that:
>> Add to that - too many primitives. Even Common Lisp, perhaps the most
>> burdened common language when it comes to primitives, doesn't come
>> close to TeX (this is a top-of-the-head assertion that I should really
>> check). Things like vbox and hbox should, in my opinion, be some way
>> above the primitive level. [...]
>
>As I said, TeX is more an application program than a programming language.

I think's TeX's chief problem is trying to be both.

>The main purpose is to typeset text, not to program macros. Therefore,
>the comparison with Lisp doesn't make too much sense, I think. Why should
>vbox and hbox be at a different level? (I assume you mean, expressed in
>terms of other primitives.)
>
>and:
>> only traditional typesetting
>> systems. TeX's language is a pretty good model of that process, but
>> doesn't reflect the much more powerful models that PostScript and
>> other PDL's have given us.
>
>Well, TeX is a typesetting system, whereas (as you say) PostScript is
>a page-description language. If you take a standard book on PostScript
>(say the reference manual or so) then it will almost certainly tell you
>the difference. Very few people want to write their text directly in
>PostScript. Instead, PostScript is usually produced as output from some
>other typesetting system (for instance, TeX). TeX's PDL is the dvi format,
>which is indeed less general (but more efficient for the printer) than
>PostScript format.
>
>Also, aren't you shooting yourself a bit in the foot here: I think that
>PostScript has more primitives than TeX.
>

Hmmm. 
	1) PostScript is orders of magnitude more flexible than TeX,
		so I don't consider its proliferation of primitives
		to be reflecting a basic problem. Its hard to find
		a PostScript command which could obviously be built
		from a lower level - those that can are few in number,
		and are often implemented this way (internally).

	2) There's about as much difference between a PDL and a
		typesetting language as there is between a 
		compiled and interpreted - some, but not that
		much. If PostScript could do what TeX does
		with math and text, and wasn't RPN, I would use
		it 100% of the time. But it doesn't, so I use
		TeX to shield myself from the horrors of the stack,
		and to gain access to TeX's power. The TinT project
		was about trying to get the best of both worlds.
		
>(Paul Barton-Davis)
>> TeX's model is based on
>> ignoring the contents of each box, which works for general text, where
>> each set character can be accurately described by a box along with a
>> few extra details to handle areas where it is not contained by the
>> box. However, this doesn't work for images, and is very difficult to
>> work with when curves are heavily in use.
>
>Well, who claims that TeX is good for all kinds of typography? TeX is

Not me. I'd just *like* to be able to use it for all kinds ... why
should I have to make a choice when I want to do a page with complex
math and wierd shape pictures too ?

>intended for typesetting things like books, papers, and letters. In
>such environments even pictures are typically indeed just boxes. How
>often do you see a picture in the newspaper that has the text flowing
>around George Bush's head (say)? Never: everything is cut at right angles.
>Such things, as well as typesetting along fancy curves, individual spacing
>between characters, etc., *do* occur in, for instance, advertisements
>and on posters. Sure, TeX is not very suitable for such things.
>

Yes, I know they're not common, but the Mac models have made them
*easy*. Why shouldn't us TeX-types have access to the same kind of
flexibility ?

>Finally: No perfect program has ever been written. TeX isn't perfect,
>and PostScript isn't perfect either.
>

Agreed. That's why we started TinT, and why one day, it might be finished.
It won't be perfect, but at least it doesn't have to be renamed (the
ultimate recursive acronym: TinT Is Not TinT ?) :-)))

Tim, if you're still reading this group, do you want to comment on
-- 
Paul Barton-Davis <pauld@cs.washington.edu> UW Computer Science Lab	 

"People cannot cooperate towards common goals if they are forced to
 compete with each other in order to guarantee their own survival."

tom@ssd.csd.harris.com (Tom Horsley) (05/15/91)

My favorite example of the ability of TeX guru's to fool themselves into
believing TeX is easy to use comes from the intro to "The Joy of TeX" (I am
almost certain that is the book, but I know it is one of the well known TeX
based books).

In it, the author describes how the book started as a project to attend a
two week seminar, than then spend a few weeks putting the notes in order and
coming out with a small booklet on TeX. Five *years* later, the book was
finally finished, then it goes on to explain how easy it is to use TeX.
I love it! It took the author 5 years, but now he thinks its easy :-).

P.S. In case you can't tell, I also wish TeX was based on a much more
general purpose programming language instead of the almost-language it
implements now. Writing (or worse yet, reading) TeX macros is by far the
most challenging programming task I have ever faced. I am not sure that I
will ever overcome all the obstacles...
--
======================================================================
domain: tahorsley@csd.harris.com       USMail: Tom Horsley
  uucp: ...!uunet!hcx1!tahorsley               511 Kingbird Circle
                                               Delray Beach, FL  33444
+==== Censorship is the only form of Obscenity ======================+
|     (Wait, I forgot government tobacco subsidies...)               |
+====================================================================+

geyer@galton.uchicago.edu (05/15/91)

TeX is sui generis.  If it had been written like other programming languages,
it would be a lot easier to program in.  But it wouldn't be anywhere near as
good at what it does.

Charles Geyer
Department of Statistics
University of Chicago
geyer@galton.uchicago.edu

tim@cstr.ed.ac.uk (Tim Bradshaw) (05/16/91)

[ This is a followup to several threads that seem to have become
  split, so is probably confusing -- I wish news could work as a DAG
  rather than a tree.  I also missed the beginning of the thread, but
  it seems to have my name in it at various points (hey, fame!), so I
  feel justified in slightly random comments! 

  It's also very long for which I apologise. ]
  
As far as I can see, people are arguing about TeX's macro language,
and comparing it to various other systems.  

I should make my position clear: I find TeX the most arcane and
unpleasant system to use I have ever seen, assembler included.  I've
written fairly considerable (though small cf some other readers of
this group I'm sure) things in TeX, and even when I did it practically
full time for months I still ran into behaviour that I could spend a
day or more trying to understand.  It says something that latex.tex
has the algorithms as comments: not comments in the code explaining
what is going on, but the whole thing written out more comprehensibly.

* So: comments on what people have said:

marcel@cs.caltech.edu (Marcel van der Goot) writes:

> As far as I can tell, the only way in which you can have the freedom of
> presenting your input as almost plain text, with very little knowledge
> about the details of the underlying programming language, is with a
> language based on macro substitution. A conventional programming language
> with strict syntax means that your input has to obey very strict rules.

This is perhaps true *if* you insist, as TeX does, on inlining the
programming language with the input to the resulting program.  While
this is the way that typesetting systems have worked historically, I
think it is very unfortunate.  Once you stop doing this then you can
have a programming language that looks however you want it to, and an
input stream likewise: I don't have to type Emacs lisp at emacs to use
it as an editor.

> It is true, TeX's macros take some getting used to, because of the complexity
> of the system. What you get back for that is an unparallelled freedom in
> the form your (normal text) input can take. 

I wish you did!  Other people have mentioned some of the limitations.

> You can make TeX look like
> LaTeX (which is only somewhat different), but you could for instance also
> make TeX look like troff. With a normal programming language that
> is impossible: A C program always looks like a C program, a pascal program
> always looks like a pascal program, and a lisp program always looks like
> a lisp program. You cannot decide, ``let's give an end-of-line the meaning
> of a semi-colon'' so that you don't have to write semi-colons in C; You
> cannot decide to use square brackets instead of parentheses in lisp. And
> these are only very minor variations, many TeX macros do much more.

Don't know too much about what you can do with C.  You can make Lisp
(by which I mean Common Lisp) look like whatever you like modulo some
stupidities in the design of the reader which are now fixed.  I use
the Lisp reader to read stuff that looks like: 
`*o-bar | {*r *rr} *# -> oo.'.  Prolog people seem mostly to work by
redefining the syntax to be some suitable grammar formalism.

> For those people the
> ease of *usage* of the macros is important; TeX can make that very easy.

Really?  The users who complain to me about how difficult LaTeX is
disagree.  Most of the problems could probably be fixed by redoing
LaTeX, but some problems are not fixable.  The worst problem is the
absence of any kind of error system.  *However* carefully and cleverly
you define the input syntax, people are going to get it wrong: when
they do you should be able to catch the error, work out what caused
it, and at the very least print some kind of useful message suggesting
a fix. In particular you should be able to trap errors produced by
the system itself and print something more sensible than `overfull
hbox'.  Preferably you should be able to define ways of continuing
from classes of errors.

> Of course, that means that the *construction* of macros becomes more
> complicated.

Well it's certainly complicated.

> When TeX is compared with other programs/systems, one should keep in
> mind what TeX is intended for. Since TeX is intended for typesetting
> books and the like, it should not be compared as if the primary use
> of TeX is as a programming language. 

While this is off the point I am trying to make I think it's anm
extremely important thing to bear in mind when fighting TeX.  Knuth
wrote it (I suspect) to typeset one particular series of books in one
particular way.  It's amazing that it *is* so generally useful when that
is borne in mind.

pauld@stowe.cs.washington.edu (Paul Barton-Davis) writes:

> Trying to use TeX's macro language to enable
> users to have greater ease-of-use when inputting TeX is like trying to
> use Fortran to implement linked lists or other pointer-based data
> structures: you can do it, but it costs and its cumbersome.

Ha! wonderful!  Actually you can't do it anyway, not without dynamic
memory allocation.


* And ideas on how it might be:

Although TeX is the only system I know of that's really any good for
serious typography, I think it has problems that are not fixable,
especially now it is frozen.  These are in two places:

	The language is horrid. Probably for several reasons but at
	least a factor is this business of inlining code and data.

	TeX's model of the world isn't powerful enough to do more than
	it was designed to do -- why should it be?  Unfortunately people
	now *want* to do more than it was designed to do.

So I think in due course some successor for TeX is needed and will
appear.  I am interested in what such a system might look like.  Here
are some characteristics I think are important:

	Text and programming should be separate (see above).

	It needs to be extensible: this means a real programming
	language.  In particular I'd like good error handling 
	including user-defined error classes &c.  I'd also like at
	least some kind of object-oriented system, so I can specialise
	things that already exist rather than reinventing them.

	It needs to be *dynamically* extensible:  I want to be able
	to load code into a running system (like TeX can).

	It needs to have no limits: I don't want to have to recompile
	the system when I run out of nesting depth.  Even PCs can now
	run big programs and do dynamic memroy management I hope.

	It needs to produce output as good looking as TeX can (but I
	want to be able to change the line-breaking algorithm if need be!).

	It needs a more flexible model of the world than TeX has, so I
	don't have to use scissors and glue so often.

Paul mentioned TinT (disagree about the capitalisation btw Paul!):
this was the result of some early ideas along these lines.  I have
millions of other ideas of how things should be done but this article
is too long already.  I think however that it is important that people
discuss these things: that way perhaps whatever replaces TeX will be
some good.

--tim

Tim Bradshaw.  Internet: tim%ed.cstr@nsfnet-relay.ac.uk
UUCP: ...!uunet!mcvax!ukc!cstr!tim  JANET: tim@uk.ac.ed.cstr
"...wizzards & inchanters..."