donnalyn@uunet.UU.NET (Donnalyn Frey) (05/02/91)
UUNET Communications Services Funds C News Development UUNET Communications Services, a non-profit corporation, is funding development of the C News software used by thousands of USENET network readers and administrators. The improved software will enhance the reliability of USENET message transmission thoughout the world-wide network. Software Tool & Die of Brookline, Massachusetts has received funding for the development. The enhanced C News will be available to all at no charge. The C News performance enhancements fall into five categories: Performance, NNTP, System Management, Documentation, and Installation and Support. Major enhancements include improvements to: % UUCP batching for large sites % NNTP transmission to and from large numbers of sites % processing of complex sys files, which control the flow of news to other sites % performance bottlenecks by converting shell scripts to C programs % documenting the news system for the transition from B News to C News. The many enhancements will assist both large and small USENET sites, smooth the transition from the older B News to C News, and improve portability between many hardware platforms. The improved C News will be available in the spring of 1992. UUNET Communications Services info@uunet.uu.net
rickert@mp.cs.niu.edu (Neil Rickert) (05/02/91)
In article <131119@uunet.UU.NET> donnalyn@uunet.UU.NET (Donnalyn Frey) writes: >UUNET Communications Services, a non-profit corporation, is funding >development of the C News software used by thousands of USENET network >The C News performance enhancements fall into five categories: > >% processing of complex sys files, which control the flow of news to >other sites Talking of complex sys files, why not a sys file compiler which generates the transition tables for a finite state automaton. Each incoming article (or at least the PATH, DISTRIBUTION and NEWSGROUPS lines) would be fed through the FSA to efficiently discover the list of sites to which this should be forwarded. -- =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*= Neil W. Rickert, Computer Science <rickert@cs.niu.edu> Northern Illinois Univ. DeKalb, IL 60115 +1-815-753-6940
kucharsk@solbourne.com (William Kucharski) (05/02/91)
In article <131119@uunet.UU.NET> donnalyn@uunet.UU.NET (Donnalyn Frey) writes: >% performance bottlenecks by converting shell scripts to C programs I thought the reason for the shell scripts in the first place was to avoid placing lots of news policy in C code as does B News... -- | William Kucharski, Solbourne Computer, Inc. | Opinions expressed above | Internet: kucharsk@Solbourne.COM | are MINE alone, not those | uucp: ...!{boulder,sun,uunet}!stan!kucharsk | of Solbourne... | Snail Mail: 1900 Pike Road, Longmont, CO 80501 | "It's Night 9 With D2 Dave!"
henry@zoo.toronto.edu (Henry Spencer) (05/02/91)
In article <1991May1.193937.29962@mp.cs.niu.edu> rickert@mp.cs.niu.edu (Neil Rickert) writes: > Talking of complex sys files, why not a sys file compiler which generates >the transition tables for a finite state automaton. Each incoming article >(or at least the PATH, DISTRIBUTION and NEWSGROUPS lines) would be fed >through the FSA to efficiently discover the list of sites to which this >should be forwarded. Geoff and I have talked about things like this in the past. Profiling has consistently said that it's not worth the trouble: newsgroup matching is not that large a share of the processing time. Of course, we were running timings on sites with modest numbers of outgoing feeds, and a place like uunet might well get different profiling results... -- And the bean-counter replied, | Henry Spencer @ U of Toronto Zoology "beans are more important". | henry@zoo.toronto.edu utzoo!henry
brendan@cs.widener.edu (Brendan Kehoe) (05/02/91)
In <1991May1.205754.7384@solbourne.com>, kucharsk@solbourne.com writes: >In article <131119@uunet.UU.NET> donnalyn@uunet.UU.NET (Donnalyn Frey) writes: > >% performance bottlenecks by converting shell scripts to C programs > >I thought the reason for the shell scripts in the first place was to avoid >placing lots of news policy in C code as does B News... Because a lot of the shell scripts are *never* modified. I rewrote a few of them (anne.jones, tear, and spacefor) in C, and the speed increase and reduction in load on the system's markedly clear. [Plug, plug: you can ftp them on ftp.cs.widener.edu in pub/cnews.set.shar.Z]. I can't wait to see if inews gets turned into C. :) Brendan -- Brendan Kehoe - Widener Sun Network Manager - brendan@cs.widener.edu Widener University in Chester, PA A Bloody Sun-Dec War Zone "Does this person look relaxed to you? Well, it's actually an experiment of Contour's new 565-E chair!"
henry@zoo.toronto.edu (Henry Spencer) (05/02/91)
In article <6MA+RMH@cs.widener.edu> brendan@cs.widener.edu (Brendan Kehoe) writes: > Because a lot of the shell scripts are *never* modified. I rewrote a >few of them (anne.jones, tear, and spacefor) in C, and the speed >increase and reduction in load on the system's markedly clear... The load on the system from anne.jones and tear shouldn't be significant unless you're using inews for gatewaying (generally not a great idea); they get invoked only for local postings. They *are* slow enough to be annoying if you're desperately anxious to get your words of wisdom out onto the net this very minute, but patience is a virtue... Spacefor, on the other hand, is a difficult case. Basically it deserves to be in C for performance reasons, but there *isn't* a C interface for determining free disk space on a lot of old systems. Unfortunately, not only is a df/awk approach slow, but df output format turns out to be far more variable than we thought. Current plans, partly implemented, are to provide a C version of the guts of it (keeping the parameter settings in a shell file makes sense, so you don't want to convert *all* of it) for those with suitably modern systems, while retaining the df-based one for old systems. > I can't wait to see if inews gets turned into C. :) Again, it will not be rewritten entirely in C, because much of it is not performance-critical and does contain imbedded policy decisions. However, a re-partitioning of functions and a *partial* C rewrite is in order, and has been on the "to do" list for quite some time. -- And the bean-counter replied, | Henry Spencer @ U of Toronto Zoology "beans are more important". | henry@zoo.toronto.edu utzoo!henry
rls@svcs1.UUCP (Bob Strait) (05/02/91)
donnalyn@uunet.UU.NET (Donnalyn Frey) writes: |>UUNET Communications Services, a non-profit corporation, is funding |>development of the C News software used by thousands of USENET network |>readers and administrators. The improved software will enhance the |>reliability of USENET message transmission thoughout the world-wide |>network. Software Tool & Die of Brookline, Massachusetts has received |>funding for the development. The enhanced C News will be available to |>all at no charge. |> . |> . |> . |>The many enhancements will assist both large and small USENET sites, |>smooth the transition from the older B News to C News, and improve |>portability between many hardware platforms. The improved C News will |>be available in the spring of 1992. Hear! Hear! Yeah ... UUNET is a real money-grubbing, fascist outfit ....(sarcasm). ST&D, anything we can do to help, give a yell! I'm all for you...go for it! -- BUBBA (aka Bob Strait) ...!mips!svcs1!rls Silicon Valley Computer Society Sunnyvale, CA --
henry@zoo.toronto.edu (Henry Spencer) (05/02/91)
I wrote:
>[comments on C News plans]
I guess I ought to be clear about one thing, lest someone misunderstand:
I work for neither UUNET Communications Services nor Software Tool & Die,
and don't speak for either in any way. (Geoff does work for ST&D now, and
if you think that's a coincidence, I have a bridge to sell you...) My
comments reflect, usually, approximate consensus between Geoff and me on
future plans, and we're still the ones doing most of the work. The folks
paying Geoff's salary do have some say in the matter :-), though, so if
they have strong opinions about something, plans could get revised. Treat
my comments on plans as a first approximation only. (Mind you, that's all
they've ever been, since we've revised plans often...)
I should also note, in case anyone is wondering, that Geoff and I are both
very happy about these developments. Among other considerations, having
one of us actually *paid* to work on this stuff should speed up a lot of
long-overdue improvements.
--
And the bean-counter replied, | Henry Spencer @ U of Toronto Zoology
"beans are more important". | henry@zoo.toronto.edu utzoo!henrypeter@taronga.hackercorp.com (Peter da Silva) (05/04/91)
brendan@cs.widener.edu (Brendan Kehoe) writes: > Because a lot of the shell scripts are *never* modified. I rewrote a > few of them (anne.jones, tear, and spacefor) in C, and the speed > increase and reduction in load on the system's markedly clear. Of course you had already modified anne.jones: > Message-Id: <6MA+RMH@cs.widener.edu> I have made several mods to anne.jones, and I'm surprised that there is a detectable increase in performance overall. After all, anne.jones only gets called for local articles. -- (peter@taronga.hackercorp.com) `-_-' 'U`
sjk@aura.nbn.com (Scott J. Kramer) (05/21/91)
In article <131119@uunet.UU.NET> donnalyn@uunet.UU.NET (Donnalyn Frey) writes:
From: donnalyn@uunet.UU.NET (Donnalyn Frey)
Newsgroups: news.software.b
Date: 1 May 91 19:22:21 GMT
...
Major enhancements include improvements to:
...
% processing of complex sys files, which control the flow of news to
other sites
One suggestion: allow the 2nd field of the "sys" file, which can often
be rather unwieldy to maintain, to specify the pathname to a file
containing the newsgroup subscription/distribution info.
--
Scott J. Kramer UUCP: {sun,ucbvax}!pixar!aura!sjk
P.O. Box 3392 Internet: sjk@aura.nbn.com
San Rafael, CA 94912henry@zoo.toronto.edu (Henry Spencer) (05/27/91)
In article <SJK.91May20143218@aura.nbn.com> sjk@aura.nbn.com (Scott J. Kramer) writes: >One suggestion: allow the 2nd field of the "sys" file, which can often >be rather unwieldy to maintain, to specify the pathname to a file >containing the newsgroup subscription/distribution info. An easier solution than trying to load all this support functionality into relaynews is to use the existing Unix tools to build a complex sys file from a simpler description. -- "We're thinking about upgrading from | Henry Spencer @ U of Toronto Zoology SunOS 4.1.1 to SunOS 3.5." | henry@zoo.toronto.edu utzoo!henry
cudep@warwick.ac.uk (Ian Dickinson) (05/28/91)
In article <1991May27.163350.6608@zoo.toronto.edu> henry@zoo.toronto.edu (Henry Spencer) writes: >An easier solution than trying to load all this support functionality into >relaynews is to use the existing Unix tools to build a complex sys file >from a simpler description. Makes sense. On a related issue (ie. too lazy to look it up in the source :-) ): I have one site which has a very complex sys line. At the moment, it is split over about a dozen seperate lines. Only about 3 need to exist, but it's split further to make it nicer to edit. Would relaynews run faster if I concatenated some lines? -- \/ato /'\ /`\ Ian Dickinson TED KALDIS FOR PRESIDENT! /^^^\/^^^\ vato@warwick.ac.uk /TWIN/TEATS\ @c=GB@o=University of Warwick@ou=Computing Services@cn=Ian Dickinson / \
henry@zoo.toronto.edu (Henry Spencer) (05/29/91)
In article <JQY_QZ_@warwick.ac.uk> cudep@warwick.ac.uk (Ian Dickinson) writes: >I have one site which has a very complex sys line. >At the moment, it is split over about a dozen seperate lines. >Only about 3 need to exist, but it's split further to make it nicer to edit. >Would relaynews run faster if I concatenated some lines? Probably not significantly. It's not clear whether you've split it using backslashes or whether you have multiple complete sys lines for the site. In the former case there will definitely be no difference; in the latter I would be surprised if there were. -- "We're thinking about upgrading from | Henry Spencer @ U of Toronto Zoology SunOS 4.1.1 to SunOS 3.5." | henry@zoo.toronto.edu utzoo!henry
brad@looking.on.ca (Brad Templeton) (05/29/91)
Never missing the opportunity to do a plug, let me point out that if your sys line is really complex, perhaps you should not use the SYS file and feed with dynafeed instead. With dynafeed you give the sites a .newsrc file, just like a reader on your system, and the feeding program reads the .newsrc and feeds them the "unread" news. They can change the .newsrc without your intervention, and in fact can have it updated on a daily basis form their arbitron reports. It is on uunet, in the ~ftp/clarinet directory. It's free in most cases. -- Brad Templeton, ClariNet Communications Corp. -- Waterloo, Ontario 519/884-7473
darcy@druid.uucp (D'Arcy J.M. Cain) (05/29/91)
In article <1991May29.013013.23820@looking.on.ca> Brad Templeton writes: >Never missing the opportunity to do a plug, let me point out that if your >sys line is really complex, perhaps you should not use the SYS file and >feed with dynafeed instead. Or use gensys, a package I am just finishing up the final testing on. Should be available by the weekend. >With dynafeed you give the sites a .newsrc file, just like a reader on >your system, and the feeding program reads the .newsrc and feeds them >the "unread" news. They can change the .newsrc without your intervention, >and in fact can have it updated on a daily basis form their arbitron reports. Same idea I guess except that the list are automatically generated by what your users and your downstream users subscribe to. I default to checking the list once an hour instead of daily though. If you have a few levels of feed using the system and you also have bad timing it could take a few full periods to start the flow of a new group. >It is on uunet, in the ~ftp/clarinet directory. It's free in most cases. Don't know where it will be archived yet but it will be free in all cases. -- D'Arcy J.M. Cain (darcy@druid) | D'Arcy Cain Consulting | There's no government Toronto, Ontario, Canada | like no government! +1 416 424 2871 |
brad@looking.on.ca (Brad Templeton) (05/30/91)
In article <1991May29.124844.9496@druid.uucp> darcy@druid.uucp (D'Arcy J.M. Cain) writes: >In article <1991May29.013013.23820@looking.on.ca> Brad Templeton writes: >>the "unread" news. They can change the .newsrc without your intervention, >>and in fact can have it updated on a daily basis form their arbitron reports. >Same idea I guess except that the list are automatically generated by what >your users and your downstream users subscribe to. I default to checking >the list once an hour instead of daily though. If you have a few levels >of feed using the system and you also have bad timing it could take a few >full periods to start the flow of a new group. NOt same idea "except," sounds like same idea exactly. The "daily" is of course just an option about how often you run it in the cron. It sounds like you've re-implemented exactly the same system. Why? Or are there some extra features you did not mention? -- Brad Templeton, ClariNet Communications Corp. -- Waterloo, Ontario 519/884-7473
emv@msen.com (Ed Vielmetti) (05/30/91)
> you've re-implemented exactly the same system. why?
if i had to guess, it would be the absence of restrictions on
distribution of the code for nefarious (read: commercial) purposes.
--Ed
(a nefarious commercial purpose now and again)cudep@warwick.ac.uk (Ian Dickinson) (05/30/91)
In article <1991May29.013013.23820@looking.on.ca> brad@looking.on.ca (Brad Templeton) writes: >With dynafeed you give the sites a .newsrc file, just like a reader on >your system, and the feeding program reads the .newsrc and feeds them >the "unread" news. They can change the .newsrc without your intervention, >and in fact can have it updated on a daily basis form their arbitron reports. I'm also newsadmin on the other site - so communication is not a problem. Disk space is, however, so tight that the arbitron results are almost worthless :-( But I would recommend the dynafeed arbitron replacement to anyone - it's fast. Cheers, -- \/ato /'\ /`\ Ian Dickinson TED KALDIS FOR PRESIDENT! /^^^\/^^^\ vato@warwick.ac.uk /TWIN/TEATS\ @c=GB@o=University of Warwick@ou=Computing Services@cn=Ian Dickinson / \
darcy@druid.uucp (D'Arcy J.M. Cain) (05/30/91)
In article <1991May29.215556.18947@looking.on.ca> Brad Templeton writes: > [discussion of his dynafeed and my gensys deleted] >It sounds like you've re-implemented exactly the same system. Why? Or >are there some extra features you did not mention? Gee Brad, why send mail and then thrash it out in news as well? Anyway as I said in email I haven't seen your dynafeed and in fact only heard about it after I had my package almost finished. As for extra features, from the description from others I suspect it has fewer features which is fine by me. I tried to make it as simple as possible. Once I have posted my software perhaps I will pick up dynafeed and see if there is anything I want to steal but I have purposely not looked at it while I was writing my package. Anyway why bother discussing it. I expect to post gensys within a few days and anyone, including you, can look at it and decide if it has any value. No point in cluttering up the net until then. In the meantime I'll mail you the man page for your own information. -- D'Arcy J.M. Cain (darcy@druid) | D'Arcy Cain Consulting | There's no government Toronto, Ontario, Canada | like no government! +1 416 424 2871 |
rbj@uunet.uu.net (Root Boy Jim) (06/01/91)
henry@zoo.toronto.edu (Henry Spencer) writes: ?sjk@aura.nbn.com (Scott J. Kramer) writes: ?>One suggestion: allow the 2nd field of the "sys" file, which can often ?>be rather unwieldy to maintain, to specify the pathname to a file ?>containing the newsgroup subscription/distribution info. ? ?An easier solution than trying to load all this support functionality into ?relaynews is to use the existing Unix tools to build a complex sys file ?from a simpler description. Indeed. The obvious thing that comes to mind is keeping each site's entry in it's own file. A simple makefile entry might look like: sys: FRC cat sys.* > sys With a bit more work, any recently changed entrys might be sanity checked beforehand. And I suppose sys will have to be locked. Besides, relaynews already has enuf files open, without having to open another one for each of your 700 newsfeeds :-) ?-- ?"We're thinking about upgrading from | Henry Spencer @ U of Toronto Zoology ?SunOS 4.1.1 to SunOS 3.5." | henry@zoo.toronto.edu utzoo!henry Ah yes. I understood 3.5. I hardly recognize 4.1.1. -- [rbj@uunet 1] stty sane unknown mode: sane