rsalz@bbn.com (Rich Salz) (06/18/91)
InterNetNews, or INN, is a news transport system. The core part of the package is a single long-running daemon that handles all incoming NNTP connections. It files the articles and arranges for them to be forwarded to downstream sites. Because it is long-running, it can be directed to spawn other long-running processes, telling them exactly when an article should be sent to a feed. This can replace the "watch the logfile" mode of nntplink, for example, with a much cleaner mechanism: read the batchfile on standard input. InterNetNews assumes that memory is cheap and fast while disks are slow. No temporary files are used while incoming articles are being received, and once processed the entire article is written out using a single writev(2) call (this includes updating the Path and Xref headers). The active file is kept in memory (a compile-time option can be set to use mmap(2)), and the newsfeeds file is parsed once to build a complete matrix of which sites receive which newsgroups. InterNetNews uses many features of standard BSD sockets including non-blocking I/O and Unix-domain stream and datagram sockets. It is highly doubtful that the official version will ever provide support for TLI, DECNET, or other facilities. INN is fast. Not many hard numbers are available (that is one requirement of being a beta-site), but some preliminary tests show it to be at least twice as fast as the current standard NNTP/C News combination. For example, Jim Thompson at Sun has had 20 nntpxmits feeding into a 4/490, and was getting over 14 articles per second, with the CPU 11% utilized. I was getting 10 articles/second feeding into a DECstations 3100, with the program (running profiled!) 50% idle and the load average under .7. (It is a scary thing to see several articles filed with the same timestamp.) The sys file format is somewhat different, and has been renamed. The arcane "foo.all" syntax is gone, replaced with a set of order-dependant shell patterns. For example, instead of "comp,comp.sys.sun,!comp.sys" you would write "comp.*,!comp.sys.*,comp.sys.sun"; to not get any groups related to binaries or pictures, you write "!*pictures*,!*binaries*". There are other incompatibilities as well. For example, ihave/sendme control messages are not supported. Also the philosophy is that that invalid articles are dropped, rather than filed into "junk." (A log message is written with the reason, and also sent back to the upstream feed as part of the NNTP reject reply.) The active file is taken to be the definitive list of groups that an article wants to recieve, and if none of an article's newsgroups are mentioned in the active file, then the article is invalid, logged, and dropped. The history and log files are intended to be compatible with those created by C News. I want to thank Henry and Geoff for their kind permission to use DBZ and SUBST. You will need to be running C News expire or a B2.11 expire that has been modified to use DBZ. The InterNetNews daemon does not implement all NNTP commands. If sites within your campus are going to post or read news via NNTP, you will need the standard NNTP distribution. The daemon will spawn the standard nntpd if any site not mentioned in its "hosts.nntp" file connects to the TCP port. InterNetNews includes a replacement for the "mini-inews" that comes with the standard NNTP distribution. This can be used on any machine that posts news and connects to an NNTP server somewhere; its use is not limited to INN. At some point I hope to have a replacement nntpd optimized for newsreaders, and an NNTP transmission program. These will remove the need for any external software beyond the C News expire program. If you would like to beta-test this version, please FTP the file pub/usenet/INN.BETA from cronus.bbn.com for directions. It will be a fairly tightly-screened beta: DO NOT ASK ME FOR COPIES! Once the system is stable, it will be freely redistributable. I hope to have the official release by August 7, so that schools can bring the system up before the semester starts. /rich $alz -- Please send comp.sources.unix-related mail to rsalz@uunet.uu.net. Use a domain-based address or give alternate paths, or you may lose out.