brad@looking.ON.CA (Brad Templeton) (12/20/89)
Posting-number: Volume 9, Issue 80
Submitted-by: brad@looking.ON.CA (Brad Templeton)
Archive-name: newsclip/part11
#! /bin/sh
# This is a shell archive. Remove anything before this line, then unpack
# it by saving it into a file and typing "sh file". To overwrite existing
# files, type "sh file -c". You can also feed this as standard input via
# unshar, or by typing "sh <file", e.g.. If this archive is complete, you
# will see the following message at the end:
# "End of archive 11 (of 15)."
# Contents: doc/append.mm.1
# Wrapped by allbery@uunet on Tue Dec 19 20:10:03 1989
PATH=/bin:/usr/bin:/usr/ucb ; export PATH
if test -f 'doc/append.mm.1' -a "${1}" != "-c" ; then
echo shar: Will not clobber existing file \"'doc/append.mm.1'\"
else
echo shar: Extracting \"'doc/append.mm.1'\" \(44635 characters\)
sed "s/^X//" >'doc/append.mm.1' <<'END_OF_FILE'
X.nr Ej 1
X.nr Hb 4
X.nr Hs 3
X.ds HP +4 +2 +1 +0 +0 +0 +0
X.de xd
X.sp 1V
X.br
X\\$1 \s+2\fB\\$2\fP\s-2
X.sp 0.7V
X..
X.de Bb
X.sp 0.5V
X.in 0.4in
X.nf
X..
X.de Be
X.sp 0.5V
X.in -0.4in
X.fi
X..
X.de St
X.sp 0.8V
X.in 0.4in
X\\$1
X.in -0.4in
X.sp 0.8V
X..
X.HM A
X
X
X
X.H 1 "Statement Reference"
X.P
XA NewsClip program consists of a list of declarations, functions
Xand procedures.
X
X.H 2 "Lexical Structure"
X.P
XLexical tokens include keywords, symbols, quoted
Xstrings, integer constants, character constants, newsgroup names and
Xidentifiers.
X.P
XIdentifiers start with a letter and consist of alphanumerics and/or the
Xunderscore
Xcharacter. There is no limit on the length, although system resident
XC compilers and linkers may impose one. Identifiers are case-insensitive.
X.P
XComments that begin with \fB/*\fP and end with \fB*/\fP may be placed
Xbetween any two tokens.
X.P
XC Preprocessor directives may be included at any line. These begin with
Xa \fB#\fP in column 0 and are documented in the C preprocessor manual.
X.P
XThe following are reserved keywords. They may be written in either
Xupper, lower or mixed case:
X
X.in 0.3in
X.ll -0.3in.P
X\fBaccept, adjust, array, break, case, continue, database, datetime,
Xdefault, else, extern, for, forward, goto,
Xhas, header, if, in, int, is,
Xnewsgroup, parse, procedure, reject, return, string,
Xswitch, userid, while\fP
X.in -0.3in
X.ll +0.3in.P
XNumbers are integers. They consist of digits and may be preceded by
Xa minus sign. Character constants consist of a single character in
Xsingle quotes. Escape sequences, using a backslash, may be used. A
Xsingle quote may be escaped to generate that constant.
X.P
XString constants are bound in double quotes. Escape sequences start
Xwith a backslash. These are: \fB\\n\fP - newline, \fB\\r\fP - carriage
Xreturn, \fB\\t\fP - tab, \fB\\f\fP - form feed, \fB\\"\fP - double quote,
X\fB\\\\ \fP - backslash, \fB\\ \fP \fIoctal num\fP -- arbitrary ascii
Xcharacter.
X.P
XNewsgroup constants consist of a \fB#\fP followed by a string of
Xalphanumerics, underscores, plus signs, dashes or dots. \fBIs\fP tokens
Xconsist of the string ``is'' followed by a space and a newsgroup constant
Xwithout the \fB#\fP prefix character.
X.P
XOther escape sequences generate a backslash and the non-special ``escaped''
Xcharacter.
X
X.H 2 "Declarations"
X.P
XDeclarations may appear at the global level or at the local level, ie. inside
Xa subroutine. Global declarations last for the entire program. Local
Xdeclarations may hide global declarations, but are only valid within the
Xfunction or procedure in which they are found.
X.P
XAll declarations involve types:
X
X.H 3 types
X.P
XThe following are valid types:
X.sp 0.5V
X.in 0.4in
X
X.sp 0.7V
X.ti -0.4in
Xint
X.P
XAn integer. Size is machine dependent. Usually 16 or 32 bits.
XThe integer type is also used for boolean (conditional) expression values,
Xwith 1 or non-zero representing true, and 0 representing false.
X.sp 0.7V
X.ti -0.4in
Xdatetime
X.P
XAn integer large enough to hold a date/time number. Such numbers are
Xthe number of seconds since midnight, GMT, Jan 1, 1970.
X.sp 0.7V
X.ti -0.4in
Xstring
X.P
XA character string.
X.sp 0.7V
X.ti -0.4in
Xuserid
X.P
XA USENET style \fBFrom:\fP line userid. Normally this is the character
Xstring of the mail address. The ``real name'' comment can also be
Xextracted.
X.sp 0.7V
X.ti -0.4in
Xnewsgroup
X.P
XA number which represents a newsgroup. May be used as a string
Xin any expression, wherein the string with the name of the newsgroup is
Xprovided. Numbers which represent newsgroups named in the NewsClip
Xprogram will be unique. Others may be re-used on a per-article basis.
X.sp 0.7V
X.ti -0.4in
X\fIsimple-type\fP array
X.P
XAn array of any of the above types. (The above types are known as
Xsimple types.) Such variables may be indexed by
Xintegers using
Xsquare brackets, and their size may be queried. Array variables and values
Xare really pointers to arrays.
X.sp 0.7V
X.ti -0.4in
Xdatabase
X.P
XA special type that can be indexed with square brackets using string value
Xindices. The elements are integers.
X.in -0.4in0
X
X.H 3 "Compatible Types"
X.P
XThe types \fBnewsgroup\fP, \fBuserid\fP and \fB\fP string are known
Xas string-like types. Expressions of these three types can be used in an
Xexpression wherever a string is required.
X.P
XThe \fBint\fP and \fBdatetime\fP types can be mixed in expressions, the
Xresult is a \fBdatetime\fP.
X.P
XTypes are assignment compatible if they are the same type, or if a string-like
Xtype is being assigned to a string, or if an \fBint\fP or \fBdatetime\fP
Xtype is being assigned to an \fBint\fP or \fBdatetime\fP.
X.P
XA \fBnewsgroup\fP expression can be assigned to an \fBint\fP. The \fBint\fP
Xvariable or argument gets the newsgroup number. The reverse assignment is not
Xallowed.
X.P
XArrays of assignment compatible types are not necessarily assignment
Xcompatible. Arrays of string-like types, however, can be used with \fBin\fP
Xand \fBhas\fP operators where an array of strings is required.
X
X.H 3 "Variable Declarations"
X
X.Bb
X\fItype\fP \fIvariable-identifier\fP;
Xextern \fItype\fP \fIvariable-identifier\fP;
X.Be
XThe first form declares a variable of the specified type. The second form
Ximports a variable from outside the program. You may not make an external
Xdeclaration for one of your own variables. Predefined header variables
Xmay only be imported at the global level, not within a procedure or function.
X.H 3 "Header Variables"
X
X.Bb
Xheader \fIsimple-type\fP \fIvariable-identifier\fP : \fIfieldname\fP;
Xheader \fIsimple-type\fP array \fIvariable-identifier\fP : \fIfieldname\fP, \fIdelims\fP;
X.Be
XThese forms declare variables that are initialized for each article from
Xthe appropriate header line in the article. The header line is selected
Xfrom the \fIfieldname\fP string.
X.P
XIn the second form, an array is declared. The array will be parsed using
Xthe delimiter string provided. Any span of characters from the delimiter
Xset counts asnd one delimiter. Thus with ``;:,'' \fB\:\:;:\fP counts as
Xone delimiter.
X.P
XIf the field name begins with a lower case letter, the header field will
Xbe lower cased before being parsed into the header variable.
X.P
XIf the delimiter list starts with a capital \fBS,\fP then the S will not
Xbe a delimiter, but rather indicates that white space should be stripped
Xfrom the front and back of all array elements. (Naturally, this is only
Xuseful when space itself is not a delimiter.) Spaces in the middle of
Xelements will not be removed. The \fBKeywords\fP line is delimited with
X``S,''
X.P
XHeader declarations may only appear at the global level, outside procedures
Xor functions.
X
X.H 3 "External & Forward Subroutines"
X
X.Bb
Xextern \fItype\fP \fIfuncname\fP ( \fItypelist\fP );
Xforward \fItype\fP \fIfuncname\fP ( \fItypelist\fP );
Xextern procedure \fIfuncname\fP ( \fItypelist\fP );
Xforward procedure \fIfuncname\fP ( \fItypelist\fP );
X.Be
X.P
XThese two forms declare a procedure or function. The \fBextern\fP forms
Ximport the routine from outside the program. The \fBforward\fP form
Xdeclares a routine that will be defined later in the program.
X.P
XThe \fItypelist\fP is a comma delimited list of argument types. It may
Xbe null.
X.P
X\fBForward\fP declarations may only appear at the global level, not inside
Xsubroutines.
X
X.H 3 "Externals"
X.P
XThere are two types of externals. NewsClip defined externals are
Xpart of the language. They must be imported with exactly the right
Xtype and form. It is an error otherwise. It is also possible to import
Xarbitrary symbols, which must be present in modules linked with the program.
X.P
XThere are two kinds of NewsClip externals, header variables and
Xothers. It is an error to include an \fBextern\fP for a header variable
Xwithin a procedure or function.
X
X.H 2 "Subroutines"
X.P
XProcedures:
X.Bb
Xprocedure
X\fIprocname\fP( \fIarglist\fP )
X{
X \fIlocal-declaration-list\fP
X \fIstatement-list\fP
X}
X.Be
X.P
XFunctions:
X.Bb
X\fItype\fP
X\fIfuncname\fP( \fIarglist\fP )
X{
X \fIlocal-declaration-list\fP
X \fIstatement-list\fP
X}
X.Be
X.P
XProcedures and functions may only be declared at the global level.
XA \fIprocname\fP or \fIfuncname\fP is a unique global identifier.
XAn \fIarglist\fP is a list of argument declarations, which take the
Xform \fItype\fP \fIarg-identifier\fP.
X.P
XArguments act as local variables. The argument list and type of a
Xprocedure or function must match any \fBforward\fP declaration for it.
X.P
XThe \fIlocal-declaration-list\fP is a list of semicolon terminated local
Xdeclarations. The \fIstatement-list\fP is a series of semicolon terminated
Xstatements to execute when the routine is called.
X.P
XThe \fBaccept\fP and \fBreject\fP statments may only be used in procedures.
XIn procedures, \fBreturn\fP must have no arguments, in functions, an argument
Xmatching the \fItype\fP must be given on a \fBreturn\fP statement.
X
X.H 2 "Statements"
X.P
XAny statement may be labeled. To do so, proceed the statement with
Xan identifier (label) and a colon.
X
X.H 3 "Compound Statement"
X.P
XA compound statement may be used whenever a single statement is required.
XIt consists of an open brace, a list of zero or more semicolon-terminated
Xstatements, and closing brace. Declarations may not be included within the
Xcompound statement.
X
X.H 3 "Procedure Call"
X
X.St "\fIprocname\fP( \fIarglist\fP )"
X.P
XThe named procedure is called. The \fIarglist\fP is a comma delimited list
Xof expressions, or nul. The count of arguments must match the declaration
Xfor the procedure, and each argument must be assignment compatible with the
Xformal parameter declared for the procedure.
X.P
XIf, after the procedure returns, the magnitude of the score is greater than
Xor equal to 30,000, the calling procedure terminates. (Note that outside
Xof this context scores that go too high can become negative.)
X
X.H 2 "Assignment Statements"
X
X.H 3 "Assignment"
X
X.St "\fIvariable\fP = \fIexpr\fP"
X.P
XThe value of the expression is assigned to the variable. The types must
Xbe assignment compatible.
X
X.St "\fIvariable\fP++"
X
X.St "++\fIvariable\fP"
X.P
XThe integer variable is incremented. Remember, this is a statement, not
Xan operator.
X
X.St "\fIvariable\fP-\-"
X
X.St "-\-\fIvariable\fP"
X.P
XThe integer variable is decremented.
X
X.H 3 "Adjust"
X
X.St "adjust \fIexpr\fP"
X.P
XThe integer expression is added to the article \fBscore\fP.
X
X.H 3 "Parsed Assignment"
X
X.St "parse \fIvariable\fP = \fIstring-expr\fP"
X.P
XThe simple type variable is assigned the value parsed from the string
Xargument and converted to the proper type.
X
X.St "parse \fIarvariable\fP = \fIstring-expr\fP, \fIdelims\fP"
X.P
XThe variable becomes a newly created array parsed from the string, with
Xelements delimited by spans of characters from the delimiter string.
XIf the delimiter string starts with \fBS\fP, then S is not used as a
Xdelimiter, but white space is stripped from front and end of all elements.
X
X.H 3 "Array Create"
X
X.St "\fIarrayvar\fP = array \fIexpr\fP"
X.P
XThe array variable becomes a fresh, empty array of size dictated by the
Xinteger expression. The indices will range from 0 to the size-1.
X
X.H 2 "Conditional Statements"
X
X.H 3 "If"
X
X.Bb
Xif( \fIexpr\fP )
X \fIstatement\fP
X.Be
XIf the value of the integer expression is non-zero, the \fIstatement\fP is
Xexecuted. Otherwise, nothing is done.
X.Bb
Xif( \fIexpr\fP )
X \fItrue-statement\fP;
X else
X \fIfalse-statement\fP
X.Be
XIf the value of the integer expression is non-zero, the \fItrue-statement\fP is
Xexecuted. Otherwise, the \fIfalse-statement\fP is executed.
X.P
XAn \fBelse\fP clause binds to the most recent \fBIf\fP statement.
X
X.H 3 "switch"
X
X.Bb
Xswitch( \fIexpr\fP )
X \fIcompound-statement\fP
X.Be
X.P
XThe compound statement will include case labels, and optionally a
X\fBdefault\fP label. A case label is placed on a statement by prefacing
Xit with:
X.St "case \fIconst\fP :"
X.P
Xwhere the constant should match the type of the \fBswitch\fP expression.
XEach constant should be unique within the \fBswitch\fP. One \fBdefault\fP
Xlabel may be present as well.
X.P
XUpon execution, the expression is evaluted. If a case label exists for
Xthe resulting value, control transfers there. If not, and a default label
Xexists, control transfers to the default statement. If no default exists,
Xcontrol transfers to after the \fBswitch\fP statement.
X
X.H 2 "Loop Statements"
X
X.H 3 "for"
X
X.Bb
Xfor( \fIinit-assignment\fP; \fIcondition-expr\fP; \fIassignment\fP )
X \fIstatement\fP
X.Be
X.P
XThe \fIinit-assignment\fP is executed. Then the integer \fIcondition-expr\fP is
Xevaluated. If it is non-zero, the \fIstatement\fP and second \fIassignment\fP
Xare executed. Control then transfers back to the above evaluation of
Xthe \fIcondition-expr\fP.
X.P
XThe assignment statements may be null. The condition may also be null
Xin which case it always evaluates as true.
X
X.Bb
Xfor( \fIscalarvar\fP in \fIarray/database\fP )
X \fIstatement\fP
X.Be
XA loop is executed once for every element in the array or database.
XThe scalar variable must be of the constituent type of the array in the
Xarray case, or of string type in the database case.
X
X.H 3 "while"
X
X.Bb
Xwhile( \fIexpr\fP )
X \fIstatement\fP
X.Be
X.P
XIf the integer \fIexpr\fP is non-zero, the \fIstatement\fP is executed.
XThis is repeated as long as the \fIexpr\fP is non-zero.
X
X.H 2 "Transfer Statements"
X
X.H 3 "goto"
X
X.St "goto label"
X.P
XControl transfers to the statement within the current subroutine that
Xhas the named label.
X
X.H 3 "continue"
X
X.St "continue"
X.P
XControl transfers to the evaluation of the test condition of the
Xcurrent \fBwhile\fP or \fBfor\fP statement, or to the next entry in the
X\fBfor .. in\fP format.
X
X.H 3 "break"
X
X.St "break"
X.P
XControl transfers out of the enclosing \fBswitch\fP, \fBfor\fP or
X\fBwhile\fP statement, terminating the latter two.
X
X.H 3 "return"
X
X.St "return"
X.P
XThe current procedure is terminated.
X
X.St "return \fIexpr\fP"
X.P
XThe current function is terminated and returns the value of the \fIexpr\fP.
XThe expression must be assignment compatible with the declared function
Xreturn type.
X
X.H 3 "accept"
X
X.St "accept"
X.P
XThe \fBscore\fP is set to a high positive number and the procedure terminates.
X
X.H 3 "reject"
X
X.St "reject"
X.P
XThe \fBscore\fP is set to a low negative number and the procedure terminates.
X
X.H 3 "accept if"
X
X.St "accept if \fIexpr\fP"
X.P
XIf the integer expression is non-zero, an \fBaccept\fP is done.
X
X.H 3 "reject if"
X
X.St "reject if \fIexpr\fP"
X.P
XIf the integer expression is non-zero, a \fBreject\fP is done.
X
X.H 2 "Variable"
X.P
XA variable is either a declared variable identifier, or a declared
Xarray or database variable identifier followed by an open square bracket,
Xan index expression and a closing square bracket.
X.H 3 "Array Index"
X.P
XAn element (index on) an array variable has the same type as the component
Xtype of the array. The index must be an integer. Indices range from
X0 to the size of the array, less one.
X.P
XIndexing on a database variable results in a variable of integer type.
XThe index must
Xbe a string expression. When used on the left side of an assignment,
Xthe specified index will be created with a value of 0 if it does not
Xalready exist in the array. When such an index is used elsewhere,
Xthe index will not be defined within the database and the value 0 will
Xbe returned.
X.P
XUse of a defined database array index results in the update of the access
Xtime for that index.
X
X.H 2 "Function Call"
X
X.St "\fIfuncname\fP( \fIexpr-list\fP )"
X.P
XThe named function is called. The \fIarglist\fP is a comma delimited list
Xof expressions, or nul. The count of arguments must match the declaration
Xfor the function, and each argument must be assignment compatible with the
Xformal parameter declared for the function. The value returned is the
Xvalue given as an argument to a \fBreturn\fP statement in the function.
X
X.H 2 "Expression Units"
X.P
XThe basic units of expressions are variables, function calls, integers,
Xstring constants, newsgroup constants, \fBis\fP primitives and expressions
Xin parentheses. Integer expression units may be negated with a unary minus.
XThey may also be negated as boolean values with the not (\fB!\fP) operator,
Xwhich turns 0 to 1, and non-zero values to 0.
X
X.H 2 "Dyadic Operators"
X.P
XTwo-argument operators for integers include multiplication (\fB*\fP),
Xdivision (\fB/\fP),
Xmodulus (\fB%\fP),
Xaddition (\fB+\fP)
Xand subtraction (\fB-\fP)
X.P
XComparison operators for integers include
Xless than (\fB<\fP),
Xless than or equal to (\fB<=\fP),
Xgreater than (\fB>\fP) and
Xgreater than or equal to (\fB>=\fP).
X.P
XString and integer values can be compared for equality (\fB==\fP) and
Xinequality (\fB!=\fP). String comparison is exact.
X
X.H 3 "IN"
X
X.St "\fIelement\fP in \fIarray/database\fP"
X.P
XThe in operator tests for the presence of a scalar, or any element from
Xan array in an array of a compatible type or a database. Only
Xstring compatible types can be searched for in a database.
X.P
XThe operator returns 1 if the element is present, 0 if not. The
X\fB!in\fP operator returns the reverse.
X.P
XIf a search is made in a database, the matched index gets its access date
Xupdated. If a search is made for an array in a database,
Xonly one database index will get an updated access date, even if there
Xare several matches.
X
X.H 3 "HAS"
X
X.St "\fItext/database/str-array/string\fP has \fIdatabase/str-array/string\fP"
X.P
XThis operator searches for the presence of a regular expression or series of
Xregular expressions in a string, array of strings, database or text region.
X.P
XThe language of regular expressions is that taken by \fBegrep\fP, and is
Xdescribed elsewhere. If any of the regular expressions (strings are used
Xas regular expressions) from the pattern side matches any of the strings
Xon the string side or within the text region, 1 is returned. 0 is returned
Xotherwise. The \fB!has\fP operator returns the reverse.
X.P
XWhen databases are involved, the indices are either searched or used as
Xpatterns. The integer values are not involved. If a database of patterns
Xis used, the ``first'' (in hash-order, which is meaningless to the
Xuser) pattern to match gets its access date updated.
X
X.H 3 "Logical Operators"
X.P
XThe \fB&\fP operator \fBand\fPs two integer operands together as bits. The
X\fB|\fP operator \fBor\fPs them. Both operands are always calculated before
Xthey are combined. The \fB^\fP operator returns the integer that is
Xthe result of exclusive \fBor\fPing the bits of the two operands.
X.P
XThe \fB&&\fP operator returns 1 if both operands are true, 0 otherwise.
XThe \fB||\fP operator returns 0 if either of its operands is true, 0 if
Xboth are false (zero). In the case of \fB&&\fP, if the first operator
Xis false, the 2nd is not even evaluated. With \fB||\fP, if the
Xfirst operand is true, the 2nd is not evaluated.
X
X.H 3 "Query"
X
X.St "\fIconditional-expr\fP ? \fItrue-expr\fP : \fIfalse-expr\fP"
X.P
XThe integer conditional expression is evaluated. If true, the \fItrue-expr\fP
Xis evaluated and returned. If false, the \fIfalse-expr\fP is evaluated and
Xreturned.
X
X.H 2 "Operator Precedence"
X.P
XThe operator precedence of NewsClip is the same as that of C, with
X\fBhas\fP and \fBin\fP at the level of the equality operators. Operators
Xat the same level of precedence bind left to right unless otherwise stated.
XHere is the precedence table, from strongest to weakest:
X.AL
X
X.LI
X\fB() []\fP \fIfunc\fP\fB()\fP
X.LI
X\fB- !\fP (unary, right to left)
X.LI
X\fB* / %\fP
X.LI
X\fB+ -\fP
X.LI
X\fB< > <= >=\fP
X.LI
X\fB== != has !has in !in\fP
X.LI
X\fB&\fP
X.LI
X\fB^\fP
X.LI
X\fB|\fP
X.LI
X\fB&&\fP
X.LI
X\fB||\fP
X.LI
X\fB? :\fP (right to left)
X.LE
X.H 2 "Entry Points"
X.P
XEntry points are procedures that are called by the NewsClip library
Xwhen a program is linked with it. They are essential to the language, but
Xare primarily part of the library. The library supports several modes
Xof operation. The \fInewsrc\fP mode is described here.
X.P
XEntry point procedures are all optional. If the user does not provide
Xsuch procedures, a null routine is inserted.
X.P
XThe program begins by calling \fBInit\fP. Then for each newsgroup,
X\fBstartgroup\fP is called, and a loop is done over the unread, unprocessed
Xarticles, after which \fBendgroup\fP is called. Finally, \fBterminate\fP
Xis called.
X.P
XIn the loop, for each unread article the header is processed and the
Xvarious variables are initialized. The \fBscore\fP is set to 1 (default
Xaccept) and \fBarticle\fP is called. Action is taken based on the
Xscore at the end of \fBarticle\fP.
X.P
XThe \fBcommand\fP entry point is called, with a string argument, whenever
Xa command is passed down in \fIpipe\fP mode. The procedure is expected to
Xact on the command and issue a \fBaccept\fP, or issue a \fBreject\fP.
X
X
X
X.H 1 "Contrast with C"
X.P
XThe NewsClip language has been designed to be very similar to the
XC language. The general syntax, operators and flavour are the same.
XIt has, however, been stripped down quite a lot, and at the same time
Xa few news related extensions have been added.
X.P
XC programmers should fit right in, once they get past a few major
Xand minor differences. These are:
X.BL
X
X.LI
XAssignment is quite different. There is only one assignment operator,
Xnamely \fB=\fP, although there are also increment \fB++\fP and
Xdecrement \fB-\-\fP operators. Assignment operators, including
Xincrement/decrement, can \fBnot\fP be used in the middle of expressions.
XThey can only be used in statements on their own.
X.LI
XThere are only a few basic types, and you can't define your own types.
XA couple of the basic types are structured, but structures are not
Xreferenced in the C manner.
X.LI
XThere are no pointers in the conventional sense. Pointer operators
Xlike monadic \fB*\fP, \fB->\fP and \fB&\fP are not present.
X.LI
XBit shifting operators are not present.
X.LI
XCase is ignored in symbol names such as variables. This means
X\fBHelloThere\fP is the same variable as \fBhelloTHERE\fP.
X.LI
XStrings are a basic type, and chars are not. You can't really manipulate
Xthe individual characters in strings the way you can in C. On the other
Xhand, operators such as \fB==\fP work on strings.
X.LI
XThere is a difference between procedures and functions. Functions always
Xreturn results, procedures never do, and they are declared as procedures.
X.LI
XThe comma operator for multiple expressions is not present.
X.LI
XNon integer values are not to be used in boolean expressions. Instead,
Xyou should compare string, array, database, userid and newsgroup values
Xwith special defined ``nil'' constants.
X.LI
XThere is no casting, and there is no \fBsizeof\fP operator.
X.LI
XYou can't make declarations inside any compound statement \fB{ . . . }\fP,
Xonly at the top of a routine.
X.LI
XDeclarations of parameter types for routines are done in a Pascal style,
Xwith \fBint f( int a, string b )\fP and all external procedures and functions
Xmust have their arguments declared, ANSI C style.
X.LI
XYou can't reference an undefined procedure or function, you must forward
Xdeclare it in the global declarations section.
X.LI
XThere are new operators -- \fBhas\fP and \fBin\fP, along with new
Xstatements \fBaccept\fP, \fBreject\fP, \fBadjust\fP and a new variation
Xof the \fBfor\fP loop.
X.LI
XArrays are dynamic. There are some special statements for dynamic arrays.
X.LI
XThere are no initializers for declared variables. If you wish to
Xinitialize, do it with code. The standard default of 0 from C will be
Xthe case for all user defined global variables.
X.LI
XMemory for most user strings and arrays is allocated from a temporary pool
Xthat is cleared with each new article.
X.LE
X.P
XThis may not seem a lot like C to you, but in fact you will find that for
Xthe purpose of examining news articles, you will still be able to write the
Xcode you want. We have removed some of the problem-causing constructs
Xof C without removing the power needed to write good news filtering
Xprograms.
X.P
XYou may ask why we removed so much, when there was little work required
Xto leave it in. In fact, if we wanted to, NewsClip could have been
Ximplemented with a lexical preprocessor for C and lots of clever macros.
X.P
XInstead we have a compiler for a reduced language. The reasons for this
Xare:
X.AL
X
X.LI
XIt allowed a nicer, easier-to-read syntax for language extensions, like
Xthe \fBhas\fP operator.
X.LI
XWe can fully error check the user program so that the C compilation stage
Xdoes not generate errors.
X.LI
XLater, we can implement an interpreter for the language so that it can
Xbe run by people who do not have C compilers. Allowing the full power
Xof C would have made this bulky with little gain.
X.LE
X.P
XFinally, we have allowed the \fBextern\fP declaration to make external
Xreferences to C variables and functions found in the general C libraries or
Xwritten by the user and linked in with the newsclip library. This allows
Xaccess to the full power of C (or other languages) to those with the skill.
X.P
XThis power will not be available to users of the planned interpreter, but
Xwe did not want to make such access impossible to customers who have the
Xtools. Source code customers \fIcan\fP change the NewsClip compiler
Xsource code and extend the language, but we would rather they simply
X\fBextern\fP in their own routines, as needed.
X
X
X
X
X.H 1 "Predefined Variables & Routines"
X
X.H 2 "Predefined Header Variables"
X.P
XThe following variables are special globals that refer to fields from the
Xheader of a USENET article. They must be declared with the types shown.
XIf you wish to parse header lines using other types, you can declare explicit
Xheader lines as described in the section on header declarations.
X.P
XThese globals are special. Any reference to them causes the appropriate
Xheader line to be searched for and parsed. The more such variables you
Xrefer to, the more involved header processing will be. As such, do not
Xmake external reference to any of these variables that you do not intend to
Xactually use.
X.P
XIf a header line is not present, the associated variable will contain the
Xappropriate \fInil\fP value for its type. Some variables refer to required
Xheader items and will never be nil. These are \fBfrom\fP, \fBmessage-id\fP
Xand \fBSubject\fP. If a \fBNewsgroups:\fP line is missing, the article is
Xdeclared to be invalid and is rejected.
X.P
XA special set of variables beginning with the letter ``r'' exist. These
Xare variables for which default values exist, based on required header lines.
XWith these variables, if the optional header field they refer to is not
Xpresent in the article, you will get the default required field.
X.P
XAssigning to predefined header variables is not advised, although it
Xis possible.
X.P
XAll header variables are mapped to lower case, with the exception of
X\fBxref\fP, \fBmessage\_id\fP and \fBreferences\fP. This makes searching
Xeasier. This mapping can be turned of with explict header declarations
Xor by setting the \fBpreserve\_case\fP variable to TRUE.
X
X.xd "userid" "approved"
X.P
XThe article's \fBApproved:\fP header line, or \fBnilstring\fP if it was not
Xpresent.
X.xd "string" "control"
X.P
XThe article's \fBControl:\fP header line, or \fBnilstring\fP if it was not present.
X.xd "datetime" "date"
X.P
XThe posting date of the article, from the header \fBDate:\fP field.
X.xd "newsgroup array" "distribution"
X.P
XThe array of distributions from the \fBDistribution:\fP field, or \fBnilarray\fP
Xif that is not present. Note that on USENET, distributions are specified with
Xnewsgroup names, even if the values given, like \fBcomp\fP are newsgroup
Xhierarchies, rather than newsgroups. Note as well that NewsClip provides some
Xfacilities to compare and examine distributions in a quantitative way.
X.xd "newsgroup array" "rdistribution"
X.P
XThis will be the \fBDistribution\fP array if that header field is present.
XOtherwise, it will be the \fBNewsgroups\fP array, which is always present
Xin a Usenet article. This is the way distribution works on a Usenet
Xarticle -- if no explicit distribution is given, the distribution is defined
Xby the \fBNewsgroups:\fP line.
X.xd "datetime" "expires"
X.P
XThe date of expiry for the article, or 0 if no explicit expire date is given.
X.xd "newsgroup array" "followup_to"
X.P
XThe array of newsgroups to which followups to a given article should be posted,
Xor nilarray if no explicit list is given.
X.xd "newsgroup array" "rfollowup_to"
X.P
XThis array is the \fBfollowup_to\fP array if the \fBFollowup-to:\fP header line
Xis present, and is the \fBNewsgroups\fP array, otherwise. As the latter is
Xalways
Xpresent, this variable will never be nil. On Usenet, if a user follows up
Xan article with no explicit \fBFollowup-to:\fP line, the followups go to the
Xgroups on the \fBNewsgroups:\fP line.
X.xd "userid" "from"
X.P
XThe author of the article, from the \fBFrom:\fP line. Under Usenet rules, this
Xfield must always be defined. If it is missing, a dummy userid will be
Xinserted. This field will never be equal to \fBniluserid\fP.
X.xd "string array" "keywords"
X.P
XAny keywords or phrases, delimited by commas, from a \fBKeywords:\fP header
Xline, are placed in this array. If there are none, this is \fBnilarray\fP.
XSpaces and tabs are removed from the front and end of any keywords or
Xphrases.
X.xd "int" "lines"
X.P
XThe number of lines in the body of the article, as given by the \fBLines:\fP
Xheader item. Please note that this value is often inaccurate. If you wish
Xan exact count, you may use \fBlinecount(body)\fP which actually reads the
Xarticle body and counts the lines. This number will be 0 if the field is
Xmissing.
X.xd "string" "message_id"
X.P
XThe article's message-id. Under USENET rules, this is a unique string that
Xno other message will use. This field is required in all articles. If it
Xis somehow missing, a dummy (non-unique) string will be provided, so that
Xyou never have to worry about getting \fBnilstring\fP.
X.P
XThe \fBmessage-id\fP string is not mapped to lower case. All comparisons on
Xmessage-ids should be exact.
X.xd "newsgroup array" "newsgroups"
X.P
XThis gives the newsgroups from the article's \fBNewsgroups:\fP header line.
XAll articles must have such a line or they are rejected out of hand.
X.P
XThis header variable is unlike all the others. First of all, it need not
Xbe declared as an external -- it is always present. Furthermore, you
Xmay not define your own header to parse the \fBNewsgroups:\fP line in
Xa different way. It is always parsed as an array of newsgroups and
Xplaced in this variable.
X.xd "string" "organization"
X.P
XA free-format string describing the poster's location. This is taken from
Xthe \fBOrganization:\fP header line and is \fBnilstring\fP if that is not
Xpresent.
X.xd "string array" "path"
X.P
XThis array comes from the \fBPath:\fP header line, which should always be
Xpresent on a usenet article. The array will be the list of sites that the
Xarticle passed through to get to you. The last element in the array,
X\fBpath[count(path)-1]\fP will not be a site name, but may instead by the
Xlocal userid of the poster on the posting site. The array is parsed using
Xall the major network characters as delimiters, namely ``\fB!:@%\fP'' and comma.
X.P
XIf the program is used in \fIbatch\fP mode to feed another site, you will
Xwant to reject all articles that have the feed site's name in the
X\fBpath\fP array.
X.xd "string" "posting_version"
X.P
XThis old header line is not used much any more. If defined, it will be
Xa string describing the news posting program used to post the article.
X.xd "string array" "references"
X.P
XThis is an array of the message-ids of the articles to which this article
Xis a followup. This is useful if you wish to kill off an entire discussion
Xrooted at a given message. If you know the message-id of the original
Xmessage, you can search for all messages that refer to that in their
X\fBreferences\fP array.
X.P
XThe \fBreferences\fP array is not mapped to lower case. All comparisons on
Xmessage-ids should be exact. The boolean (an integer that is 0 or 1)
Xvariable \fBfollowup\fP tells
Xwhether \fBreferences\fP is defined or not.
X.xd "userid" "reply_to"
X.P
XThis is the requested reply address on the message. Sometimes posters wish
Xreplies to go to a different address from the one given on the \fBFrom:\fP
Xline. If present, this userid variable gives that address.
X.xd "userid" "rreply_to"
X.P
XThis is the \fBreply_to\fP variable, or if that is not present, it is the
X\fBfrom\fP variable. The \fBFrom:\fP field is required on every usenet posting,
Xso this variable should never be \fBniluserid\fP. This is the way replies
Xwork on usenet -- if there is no \fBReply-to:\fP field, replies are to go to
Xthe userid in the \fBFrom:\fP field.
X.xd "userid" "sender"
X.P
XThis is the userid from the \fBSender:\fP field, if present. If a message
Xis actually posted by a person different from the one specified in the
X\fBFrom:\fP field, this field will be present to specify who the real poster
Xwas.
X.xd "userid" "rsender"
X.P
XThis is the \fBsender\fP variable, or if that is nil, the \fBfrom\fP variable.
XThis should always be defined in a usenet message, and defines the true
Xsender of the message.
X.xd "string" "subject"
X.P
XThis is the free format subject string. All messages are required to have
Xa subject string, so this will never be nilstring. The \fBsubject\fP string
Xwill still have any ``Re:'' prefixes on it. You should apply the
X\fBdrop_re\fP function on it and store the result in your own variable if
Xyou plan on working on the pure subject.
X.xd "string" "summary"
X.P
XAn optional free format string, from the \fBSummary:\fP line. This is an
Xexpanded subject.
X.xd "string array" "xref"
X.P
XThis is a special array generated by the local news processing program. The
Xfirst element should be the name of your site. Subsequent elements will consist
Xof newsgroup names, a colon, and local article numbers. These indicate the
Xvarious places a cross-posted article is linked on the local machine.
X.P
XThis field is not mapped to lower case. If this field is imported,
Xit will be used by the various \fInewsrc\fP processing modes. You don't
Xhave to use it yourself. The presence of a proper \fBXref:\fP line on
Xa rejected crossposted article will cause all the cross references to
Xalso be rejected without being processed.
X.P
XIf you do not wish this
Xto happen, then define your own special header definition for the \fBXref:\fP
Xline, and do not import this predefined one.
X.H 2 "Distribution"
X.P
XThe NewsClip language includes a facility to examine article distribution
Xon a quantitative level.
X.P
XNormally article distribution is expressed in terms of special distribution
Xkeywords, which are actually strings from the newsgroup naming space. To
Xthe usenet software, for example, ``comp'' is treated like a newsgroup.
XTo the distributing software, it matches all newsgroups that begin with
X``comp'' and a dot.
X.P
XSome distribution fields are newsgroup hierarchies, and others are names
Xwhich don't actually refer to a newsgroup and are used only for distribution.
XThese include things like local, regional, statewide and national
Xdistributions like ``usa.'' (Many local regions keep their own newsgroup
Xhierarchy for strictly local groups, and thus use these strings both as
Xdistributions and the root of a hierarchy.)
X.P
XNewsgroup distributions are roughly ordered, in that some are larger than
Xothers. Using NewsClip one can compare the size of distributions. For
Xeach newsgroup or distribution (they are really the same thing) you can
Xextract a distribution level with the \fBdlevel\fP function.
X.P
XThe number you extract is arbitrary and defined by the person who installed
Xthe newsclip system on your computer. In general, it should correspond
Xroughly to the number of computers that are expected to get an article with
Xthe specified distribution. For example, for local articles, the level is 1.
XFor articles with the ``world'' distribution, it is currently around 12,000.
X.P
XIn general, you should avoid using hard decimal constants, but instead
Xrefer to defined distribution levels for predefined newsgroup constants
Xusing the \fBdlevel\fP function.
X
X.xd "int" "dlevel( newsgroup n )"
X.P
XGiven a value of type newsgroup, this returns the distribution level
Xfor that newsgroup. This is the approximate number of sites the article
Xis expected to reach.
X.P
XThis is a predeclared function. You do not have to import it.
XBe warned that \fBdlevel\fP does not work on unknown newsgroups (groups
Xthat are not named in the program or the \fB.newsrc\fP or \fB.newsrclas\fP
Xfiles) unless \fBdistribution\_level\fP is also imported. This is for
Xefficiency reasons.
X.xd "int" "distribution_level"
X.P
XThis variable gives the expected distribution level for the current
Xarticle. This is calculated by first getting the maximum distribution
Xlevel from fields in the \fBDistribution:\fP header line as well as the
Xmaximum distribution level from newsgroups in the \fBNewsgroups:\fP line.
XThe minimum of these two is then taken and assigned to the variable.
X.P
XOn usenet, a \fBDistribution:\fP line only restricts the distribution of
Xan article. It does not expand it. Thus, the minimum of the two values
Xis taken.
X.P
XThe following are predefined newsgroup name constants.
X(Newsgroup name constants are all prefaced by a # character in
XNewsClip source code.)
XThey exist for the
Xsole purpose of having the \fBdlevel\fP function used on them. This
Xreturns the approximate number of sites in the given geographic region.
XThis allows expressions like \fBdistribution_level <= dlevel(#state)\fP
Xto test for articles of statewide or lower distribution.
X.P
XYour own local distributions should also have been defined by the
Xperson who installed the NewsClip system. If you are
Xin New Jersey, you can also thus check statewide distribution with
X\fBdistribution_level <= dlevel(#nj)\fP.
X.xd "" "#world"
X.P
XA predefined newsgroup constant with a \fBdlevel\fP equal to the estimated
Xtotal number of sites in the entire world.
X.xd "" "#usenet"
X.P
XThe typical number of sites for usenet articles. Some usenet groups go
Xoff of usenet into other networks, so this will be slightly less than
Xthe dlevel for #world on some sites.
X.xd "" "#continent"
X.P
XThe size of the continental distribution applicable to the local site.
XRegions like North America, Europe or Australia.
X.xd "" "#country"
X.P
XThe size of the local country distribution.
X.xd "" "#state"
X.P
XThe size of the local statewide distribution. Should probably be the same
Xas #province, which is a synonym.
X.xd "" "#province"
X.P
XThe size of the local provincial distribution. Should probably be the same
Xas #state, which is a synonym.
X.xd "" "#region"
X.P
XThe size for the local regional (county, multi-city area etc.) distribution.
X.xd "" "#city"
X.P
XThe size for a distribution within a city.
X.xd "" "#organization"
X.P
XThe distribution within your entire organization. In some cases, like
X``att'', this could actually be larger than a region or even a state.
X.xd "" "#department"
X.P
XA departmental distribution, if you have one, otherwise the same as your
Xsubnet or local computer.
X.xd "" "#subnet"
X.P
XAny smaller distribution, otherwise the same as #department, #organization
Xor #local
X.xd "" "#local"
X.P
XA predefined ``newsgroup'' with a \fBdlevel\fP of one.
X.H 2 "Useful Variables"
X.P
XThe following variables which you can import can all be useful.
X.xd "newsgroup" "main_newsgroup"
X.P
XThe ``current'' newsgroup. This normally only has meaning when working
Xin \fIpipe\fP or \fInewsrc\fP type modes. In \fIfilter\fP mode it will be set
Xto the first newsgroup in the \fBnewsgroups\fP array.
X.xd "newsgroup" "dir_newsgroup"
X.P
XThis special variable can be set so that the \fB~d\fP code in database
Xfilenames will expand to the name of the chosen newsgroup, with dots
Xreplaced with slashes. See ``database routines'' for more details.
X.xd "int" "followup"
X.P
XThis is actually a boolean variable. It is true if the current article
Xis a followup, which is to say if it contains a \fBReferences:\fP line.
XIt is false on independently generated articles.
X.xd "int" "score"
X.P
XThis is the running score that you control with \fBadjust\fP statements.
XIf you declare it on your own, you can reset it, adjust it yourself or
Xexamine it. In particular, you may wish to examine it in the post-article
Xroutine, should you define one.
X.xd "string" "article_filename"
X.P
XThis is the name of the file containing the current article. Be warned
Xthat in some modes of operation, this will not be defined. For example,
Xa filename will not be available when talking via PIPE to a newsreader
Xthat reads articles using the NNTP protocol.
X.P
XThe filename may also be the name of a temporary file, and thus not the
Xname of the permanent article spool at all.
X.xd "int" "article_number"
X.P
XThis is the article number of the article in the ``main newsgroup.''
XThis is only defined in processing modes that work on newsgroups, such
Xas the \fInewsrc\fP mode and the \fIpipe\fP mode. This will not be defined
Xin the \fIfilter\fP mode of operation.
X.xd "int" "article_bytes"
X.P
XThis is the number of bytes in the entire article. Under normal
Xcircumstances, this is obtained from the operating system. On Unix, it
Xis found in the \fIinode\fP.
X.P
XWhen in a processing mode where articles are not read from independent
Xfiles, reference to this function causes the entire article to be read
Xin so that the size may be calculated.
X.P
XNote that if the size of the article is larger than the largest integer,
Xthe largest integer will be returned.
X.xd "int" "num_links"
X.P
XOn a Unix system, this gives the number of links to the file containing the article.
XIf there is no article file, this will be 1.
X.xd "datetime" "write_time"
X.P
XThis contains the date/time that the article file was last written to.
XIt is extracted from the operating system. If this is not available, the
Xcurrent time will be used.
X
X.H 2 "Article Body Sections"
X.P
XUsing the \fBhas\fP operator, one can search for text patterns in sections
Xof the article body. The various sections of the body have predefined names.
XYou do not have to declare them. Note that searching in these text regions
Xnaturally requires reading in some or all of the article body,
Xwhich can add considerably to processing time.
X.xd "" "body"
X.P
XThe entire body of the article. (\fBPre-declared\fP)
X.xd "" "text"
X.P
XThe entire text of the article, but not the signature. (\fBPre-declared\fP)
X.xd "" "signature"
X.P
XThe signature section only. The signature is delimited from the text by
Xthe presence of a line that matches a certain pattern.
X(See \fBset_signature_start(str)\fP) (\fBPre-declared\fP)
X.xd "" "newtext"
X.P
XOnly that portion of the text that is deemed to be ``new,'' as opposed to
Xincluded from a previous article. May people write followup articles
Xwhich include the text of the parent article prefixed by some character,
Xusually a greater than sign. With this variable, you can search only the
Xnew contributions for key patterns. (\fBPre-declared\fP)
X(See \fBset_include_prefix(str)\fP)
X.xd "" "included"
X.P
XOnly the sections of the text (and not the signature) that were included
Xfrom a previous article. Included sections are defined as those that
Xstart with a specified pattern. (\fBPre-declared\fP)
X(See \fBset_include_prefix(str)\fP)
X.H 2 "Body Size Functions"
X.P
XThe following functions return statistics about the size of various
Xregions within the article body. Note that calling these functions
Xrequires that the news filtering program read in the body of the article,
Xwhich can add considerably to processing time.
X.xd "int" "byte_count( text_section )"
X.P
XReturns the number of bytes in the given text section. If this number is
Xlarger than the largest positive integer, that largest positive integer
Xis returned. (\fBPre-declared\fP)
X.xd "int" "line_count( text_section )"
X.P
XReturn the number of lines in the given text section. (\fBPre-declared\fP)
X.xd "procedure" "set_include_prefix( string )"
X.P
XSets the regular expression used to match lines which are to be considered
Xincluded from a previous article. There is no need to place a ``\fB^\fP''
Xat the front of the pattern, as it will only be matched at the start of
Xthe line. Whitespace before the pattern is ignored.
XThe default is ``\fB[:>#%]\fP'' -- a set of possible prefix characters.
X.xd "procedure" "set_signature_start( string )"
X.P
XSets the regular expression used to match the line which is considered
Xto be the delimiter between the text of an article and the signature.
XThere is no official delimiter, but a common practice is a line with
Xjust two dashes. Many other methods exist, and no pattern will be
Xperfect. The default is ``\fB^-\- *$\fP,'' which allows two dashes
Xfollowed by any number (including zero) of spaces.
XThe \fB^\fP at the front is important in this case.
X.H 2 "Control Variables"
X.P
XThese variables control parameters of pattern searches in the article body
END_OF_FILE
if test 44635 -ne `wc -c <'doc/append.mm.1'`; then
echo shar: \"'doc/append.mm.1'\" unpacked with wrong size!
fi
# end of 'doc/append.mm.1'
fi
echo shar: End of archive 11 \(of 15\).
cp /dev/null ark11isdone
MISSING=""
for I in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ; do
if test ! -f ark${I}isdone ; then
MISSING="${MISSING} ${I}"
fi
done
if test "${MISSING}" = "" ; then
echo You have unpacked all 15 archives.
rm -f ark[1-9]isdone ark[1-9][0-9]isdone
else
echo You still need to unpack the following archives:
echo " " ${MISSING}
fi
## End of shell archive.
exit 0