todd@ivucsb.sba.ca.us (Todd Day) (07/05/89)
Just recompile Cnews using dbz, and boy does it scream now! Whereas before on my UNIXPC, it would take about 10-15 secs per article, it now takes less than a second per article. Combined with the major speed win of expire, Cnews is a definite improvement over Bnews. -- Todd Day | todd@ivucsb.sba.ca.us | ivucsb!todd@anise.acc.com "Santa Barbara - so many babes... so few who will talk to me."
grady@fxgrp.fx.com (Steven Grady) (07/06/89)
In article <1989Jul4.183539.19106@ivucsb.sba.ca.us> todd@ivucsb.sba.ca.us (Todd Day) writes: >Just recompile Cnews using dbz, and boy does it scream now! Whereas >before on my UNIXPC, it would take about 10-15 secs per article, it >now takes less than a second per article. OK, so I've heard many people say dbz is a win. My problem is that we use NNTP to read news from client machines (although we receive news via UUCP). NNTP wants to read the history DBM files. We have suns (SunOS 4.0), and I've compiled nntp with NDBM. Are the dbm files compatible? Is the dbm interface compatible with dbz? It appears as though I could switch to dbz by defining DBM in nntp/common/conf.h, and adding the dbz library to the server LIBS. Is this correct? Should I do it? Steven ...!ucbvax!grady grady@postgres.berkeley.edu "Guards, beat this man brutally for daring to try to confuse me!"
amanda@intercon.uu.net (Amanda Walker) (07/06/89)
In article <1989Jul6.003048.25616@fxgrp.fx.com>, grady@fxgrp.fx.com (Steven Grady) writes: > It appears as though I could switch to dbz by defining DBM in > nntp/common/conf.h, and adding the dbz library to the server LIBS. > Is this correct? Should I do it? You may also have to change all of the ifdefs in misc.c from '#ifdef USG' to '#ifdef USGHIST', but aside from that, it works great. I'm running nntpd with dbz on System V here, and it's nice and fast. Tastes great, and less filling :-)... -- Amanda Walker <amanda@intercon.uu.net> InterCon Systems Corporation
henry@utzoo.uucp (Henry Spencer) (07/06/89)
In article <1989Jul6.003048.25616@fxgrp.fx.com> grady@postgres.berkeley.edu (Steven Grady) writes: >... NNTP wants to read the history DBM files. We have suns >(SunOS 4.0), and I've compiled nntp with NDBM. Are the dbm >files compatible? Is the dbm interface compatible with dbz? The interface is compatible (incomplete, but it should be complete enough). The files are not compatible. -- $10 million equals 18 PM | Henry Spencer at U of Toronto Zoology (Pentagon-Minutes). -Tom Neff | uunet!attcan!utzoo!henry henry@zoo.toronto.edu
zeeff@b-tech.ann-arbor.mi.us (Jon Zeeff) (07/06/89)
In article <1989Jul4.183539.19106@ivucsb.sba.ca.us> todd@ivucsb.sba.ca.us (Todd Day) writes: > >Combined with the major speed win of expire, Cnews is a definite >improvement over Bnews. > Have you done any tests on how much time Cnews and Bnews take to completely process one or more batches of news all the way from rnews to individual articles in the spool directory? My tests indicate that cnews is slower (due to all the front end scripts and the copy to in.coming) and I'm interested in other's tests. I may write a fast front end for relaynews. I agree about cnews expire being nice - I use it with B news (thanks to some patches that were posted quite awhile back) to make the fastest system I've tried (I haven't been able to try TMNN). -- Are you making the world | zeeff@b-tech.ann-arbor.mi.us a better place? | Ann Arbor, MI
henry@utzoo.uucp (Henry Spencer) (07/08/89)
In article <9568@b-tech.ann-arbor.mi.us> zeeff@b-tech.ann-arbor.mi.us (Jon Zeeff) writes: >Have you done any tests on how much time Cnews and Bnews take to >completely process one or more batches of news all the way from rnews >to individual articles in the spool directory? My tests indicate that >cnews is slower (due to all the front end scripts and the copy to >in.coming)... I trust you're testing with newsrun as of the first patch, not the original release, and with a substantial number of batches? If so, I have my doubts that you can streamline it much without losing something (clean error handling, disk-overflow prevention, locking, the ability to postpone processing on a busy system, etc.). If you never get bad batches, never overflow disks, always have plenty of resources available for processing, and never get simultaneous startup attempts, then yes, simplifications are possible. If. -- $10 million equals 18 PM | Henry Spencer at U of Toronto Zoology (Pentagon-Minutes). -Tom Neff | uunet!attcan!utzoo!henry henry@zoo.toronto.edu
karl@ddsw1.MCS.COM (Karl Denninger) (07/09/89)
In article <9568@b-tech.ann-arbor.mi.us> zeeff@b-tech.ann-arbor.mi.us (Jon Zeeff) writes: >In article <1989Jul4.183539.19106@ivucsb.sba.ca.us> todd@ivucsb.sba.ca.us (Todd Day) writes: >> >>Combined with the major speed win of expire, Cnews is a definite >>improvement over Bnews. > >Have you done any tests on how much time Cnews and Bnews take to >completely process one or more batches of news all the way from rnews >to individual articles in the spool directory? My tests indicate that >cnews is slower (due to all the front end scripts and the copy to >in.coming) and I'm interested in other's tests. I may write a fast >front end for relaynews. No way. Then again, we ran "B" news with "spoolnews" on, so the system would copy the files ANYWAYS when first invoked. Thus, no saving or loss there. Now, with that in mind, we have found that 1MB of news, for example, which used to take some 30-40 minutes to process to completion with "B" news now takes less than 10 minutes to do! That's an effective speed increase of some 300-400%. Expire is no contest - speed increases there are somewhere between 300-1000%! I am impressed. I am less impressed with: 1) "C" news articles don't have a "Lines: xxx" header. This stinks, and broke one of our local interface programs in a mysterious way until I finally tracked it down today. Articles which originate from "B" news sites have the header, those which originate on a "C" news site do not. This particular problem is at least slightly serious. 2) "C" news has little or no documentation. For example, "expire" flags had to be discerned by reading the code. I don't mind reading the code (after all, I had to compile and tweak it), but c'mon -- not even a description of the formats in the files themselves, although there is a comment character (#) supported?! That wasn't nice! 3) Some obscure things that you used to be able to do with the "sys" file don't work anymore. In particular, enclosing a set of shell statements in "(" and ")", which used to work in "B" news, fails under "C" news. I had to relocate our "funny items" to a shell script and call that from the sys file. No biggie, but a puzzler for a few minutes. 4) All my wonderful statistics don't work anymore. Oh well. Time to do some code hacking to get some semblance of information back in the log files, and some tweaking on the statistics code I used to use. I am quite impressed, however, with: 1) Speed. "C" news is gawd-awful fast. And I _LIKE_ that. We actually have our machine back during things like expire and unbatch. Nice. 2) The ability to junk a newsgroup without storing it. I don't know if "B" could do this, but if it could I never saw it in the docs. It works great with "C" news.... and has made me more than a little happy. 3) Policy decisions are made in shell scripts -- which means I can change them without having to recompile code. This is a major plus. 4) The batcher looks like it could be set up to handle our AKCSNet conferences as a batched feed -- which would make some of our neighbor sites which prefer to receive newsgroups as AKCS conferences very happy (previously we had to create and import them here, as "B" news couldn't handle creating batches while passing them through some other program easily). That appears to be solved with "C" news. The specifics have to be worked out here first.... and the doc problems mean that people who aren't familiar with Usenet and software configuration probably won't even ATTEMPT to run "C" news. A pity, as "C" news looks like a winner overall. >I agree about cnews expire being nice - I use it with B news (thanks to some >patches that were posted quite awhile back) to make the fastest system I've >tried (I haven't been able to try TMNN). TMNN is supposed to be killer-fast, but unfortunately it breaks EVERYTHING. I tried it out and quickly discarded it -- it had problems when we checked it out in the feed process, and utterly scrambled "rn"... We're staying with "C", although I did keep the "B" release around until I was darn sure that "C" was going to work out. One word of warning -- I had "B" news set up such that it would deal with VERY tight disk space. "C" doesn't seem to be able to get that fine-grained control that I liked with the B2.11.17 release I was using here (admittedly with some heavy hacks in the batching code!) Then again, disk space isn't as tight as it used to be here, so that doesn't look like it will be a problem. This will only be of concern to those sites that regularly get below 10kblocks in their spool areas..... -- Karl Denninger (karl@ddsw1.MCS.COM, <well-connected>!ddsw1!karl) Public Access Data Line: [+1 312 566-8911], Voice: [+1 312 566-8910] Macro Computer Solutions, Inc. "Quality Solutions at a Fair Price"
henry@utzoo.uucp (Henry Spencer) (07/10/89)
In article <1989Jul9.043700.11769@ddsw1.MCS.COM> karl@ddsw1.MCS.COM (Karl Denninger) writes: >1) "C" news articles don't have a "Lines: xxx" header. This stinks, and > broke one of our local interface programs... I regret to point out that your program was already broken: "Lines:" has always been optional according to the specs (RFC1036 is the current spec). >2) "C" news has little or no documentation. For example, "expire" flags had > to be discerned by reading the code... Uh, last time I looked, expire's flags were fully documented in man/expire.8. Did you not get that file for some reason? The documentation is admittedly a bit sketchy for total novices, but for experienced news administrators the contents of the "doc", "man", and "notebook" directories should be reasonably sufficient. -- $10 million equals 18 PM | Henry Spencer at U of Toronto Zoology (Pentagon-Minutes). -Tom Neff | uunet!attcan!utzoo!henry henry@zoo.toronto.edu
karl@ddsw1.MCS.COM (Karl Denninger) (07/10/89)
In article <1989Jul9.043700.11769@ddsw1.MCS.COM> karl@ddsw1.MCS.COM (Karl Denninger) writes: I retract my comment quoted below; it appears that the copy I was building from was missing the "man/*" subdirectory! I have unpacked the Usenet posted copy, and now have the manual pages. Oops. The remainder of the distribution I was building from is/was identical to the Usenet distribution, so the rest of my comments still apply. >2) "C" news has little or no documentation. For example, "expire" flags had > to be discerned by reading the code. I don't mind reading the code > (after all, I had to compile and tweak it), but c'mon -- not even a > description of the formats in the files themselves, although there is a > comment character (#) supported?! That wasn't nice! -- Karl Denninger (karl@ddsw1.MCS.COM, <well-connected>!ddsw1!karl) Public Access Data Line: [+1 312 566-8911], Voice: [+1 312 566-8910] Macro Computer Solutions, Inc. "Quality Solutions at a Fair Price"
peter@ficc.uu.net (Peter da Silva) (07/10/89)
In article <1989Jul9.191033.8620@utzoo.uucp>, henry@utzoo.uucp (Henry Spencer) writes: > I regret to point out that your program was already broken: "Lines:" has > always been optional according to the specs (RFC1036 is the current spec). Can you say "quality of implementation issue"? I knew you could. -- Peter da Silva, Xenix Support, Ferranti International Controls Corporation. Business: peter@ficc.uu.net, +1 713 274 5180. | "WHAT HAPPENED TO ALL Personal: peter@sugar.hackercorp.com. `-_-' | THE WOMEN IN TEXAS?" Quote: Have you hugged your wolf today? 'U` | -- ACS1W@jane.uh.edu (meesh)
karl@ddsw1.MCS.COM (Karl Denninger) (07/10/89)
In article <1989Jul9.191033.8620@utzoo.uucp> henry@utzoo.uucp (Henry Spencer) writes: >In article <1989Jul9.043700.11769@ddsw1.MCS.COM> karl@ddsw1.MCS.COM (Karl Denninger) writes: >>1) "C" news articles don't have a "Lines: xxx" header. This stinks, and >> broke one of our local interface programs... > >I regret to point out that your program was already broken: "Lines:" has >always been optional according to the specs (RFC1036 is the current spec). Hmmm... Ok. So why the gratuitous omission? Believe it or not, that was a USEFUL header line; you could tell at a glance how long the posting that follows is. Now that useful piece of information is gone. >>2) "C" news has little or no documentation. For example, "expire" flags had >> to be discerned by reading the code... > >Uh, last time I looked, expire's flags were fully documented in man/expire.8. >Did you not get that file for some reason? The documentation is admittedly >a bit sketchy for total novices, but for experienced news administrators >the contents of the "doc", "man", and "notebook" directories should be >reasonably sufficient. We got a bad copy here first time out. I have already pointed this error on my part out. What about my other complaint (C) - constructions of commands in the sys file which used to work (specifically, parenthesis encasing shell statements) which now don't do what you would expect. While this was a minor hassle for us, it would be a major one for someone with a 2000-line sys file like uunet! -- Karl Denninger (karl@ddsw1.MCS.COM, <well-connected>!ddsw1!karl) Public Access Data Line: [+1 312 566-8911], Voice: [+1 312 566-8910] Macro Computer Solutions, Inc. "Quality Solutions at a Fair Price"
henry@utzoo.uucp (Henry Spencer) (07/11/89)
In article <1989Jul10.145737.4605@ddsw1.MCS.COM> karl@ddsw1.MCS.COM (Karl Denninger) writes: >>>1) "C" news articles don't have a "Lines: xxx" header... >> >>... "Lines:" has always been optional... > >Hmmm... Ok. So why the gratuitous omission? Believe it or not, that was a >USEFUL header line; you could tell at a glance how long the posting that >follows is. Now that useful piece of information is gone. Opinions vary on how useful it is. The size of the article in bytes is a better measure of the length of the posting for most purposes; that's what rn uses. It's also a lot less likely to be wrong, which "Lines:" often was. Personally I have no strong feelings either way about "Lines:", but Geoff loathes it. >What about my other complaint (C) - constructions of commands in the sys >file which used to work (specifically, parenthesis encasing shell >statements) which now don't... Geoff says, roughly: "tsk; this is a real bug; fix, probably one line, is coming". -- $10 million equals 18 PM | Henry Spencer at U of Toronto Zoology (Pentagon-Minutes). -Tom Neff | uunet!attcan!utzoo!henry henry@zoo.toronto.edu