blair@obdient.chi.il.us (Doug Blair) (07/18/90)
We have a problem with b news 2.11.19 when compiled with the dbz package. inews and expire run fast, but duplicate copies of articles (with the same article id in <>'s) are not rejected and are instead inserted twice into our spooldir. I added dmbclose() calls in inews.c and expire.c in the (I think) appropriate places. Has anyone else seen this problem? Hardware: 386 clone box, Microport System V/386 3.0e.1, 4 meg/ram, 160 meg/hd, Computone 8-port, Trailblazers, HSTs and a couple of generic 2400 modems. Please respond via email, I'll post a summary if appropriate. Thanks! Doug ___ _ _ _ _ | || |_ ___ _| ||_| ___ __ _| |_ Doug Blair Obedient Software Corp. | | || .\/ ._\/. || |/ ._\| \|_ _| 1007 Naperville Rd, Wheaton IL 60187 |___||___/\___/\___||_|\___/|_|_| |_| 708-653-5527 blair@obdient.chi.il.us
cpcahil@virtech.uucp (Conor P. Cahill) (07/19/90)
In article <5662@obdient.chi.il.us> blair@obdient.chi.il.us (Doug Blair) writes: >We have a problem with b news 2.11.19 when compiled with the dbz package. >inews and expire run fast, but duplicate copies of articles (with the >same article id in <>'s) are not rejected and are instead inserted twice >into our spooldir. I don't have any info on you problem, but I will make a suggestion. Upgrade to C news. We upgraded in 10/89 and have been very happy ever since. News processing is much more efficient (I can do other work while the news is unbatching), the expire program works much better, etc, etc. Highly recommended. -- Conor P. Cahill (703)430-9247 Virtual Technologies, Inc., uunet!virtech!cpcahil 46030 Manekin Plaza, Suite 160 Sterling, VA 22170
karl@ficc.ferranti.com (Karl Lehenbauer) (07/22/90)
In article <5662@obdient.chi.il.us> blair@obdient.chi.il.us (Doug Blair) writes: >We have a problem with b news 2.11.19 when compiled with the dbz package. >inews and expire run fast, but duplicate copies of articles (with the >same article id in <>'s) are not rejected and are instead inserted twice >into our spooldir. I have determined the cause of this problem. I had to learn way more about the internals of news than I ever wanted to, but the problem is the optimizer of the C compiler. I forget which news source file it is, but when used with dbz (System V/386 3.0 and 3.2) it screws up a calling argument and causes trashed article ID's to be written into the dbz database. Then, whenever the same duplicate article is presented, it isn't found in the database, so news treats it as a new article. The lazy fix, at least for a test, is to compile news without optimization. -- -- uunet!ficc!karl "I am here by the will of the people and I won't leave uunet!sugar!karl until I get my raincoat back." -- Metrophage
zeeff@b-tech.ann-arbor.mi.us (Jon Zeeff) (07/24/90)
>I have determined the cause of this problem. I had to learn way more about the >internals of news than I ever wanted to, but the problem is the optimizer of the >C compiler. I forget which news source file it is, but when used with dbz Or you could avoid the optimizer problem by compiling with gcc. -- Jon Zeeff (NIC handle JZ) zeeff@b-tech.ann-arbor.mi.us
src@scuzzy.mbx.sub.org (Heiko Blume) (07/28/90)
zeeff@b-tech.ann-arbor.mi.us (Jon Zeeff) writes: >Or you could avoid the optimizer problem by compiling with gcc. don't be too sure! there *are* bugs in the gcc (1.36 at least) optimizer, although i compiled at least ~40MB sources with it without any problems. but the 41st MB (:-) did it. i tried to compile pbmplus and in this one file of about 30KB i got a SIGIOT from gcc. well, i compiled that one without -O, and all went well. sure enough that won't keep me from using gcc as well as any other GNU software, it's just great stuff. -- Heiko Blume c/o Diakite blume@scuzzy.mbx.sub.org FAX (+49 30) 882 50 65 Kottbusser Damm 28 blume@netmbx.UUCP VOICE (+49 30) 691 88 93 D-1000 Berlin 61 blume@netmbx.de TELEX 184174 intro d "Have you bugged your source today?"
blair@obdient.chi.il.us (Doug Blair) (08/02/90)
A few weeks ago I described a condition in which news 2.11.19 with the dbz code included did not properly reject duplicate articles received from a second news site. Thanks to all of you who replied and added your thoughtful insights. Summary follows: The most helpful comments came from karl@ficc.ferranti.com (Karl Lehenbauer): I have determined the cause of this problem. I had to learn way more about the internals of news than I ever wanted to, but the problem is the optimizer of the C compiler. I forget which news source file it is, but when used with dbz (System V/386 3.0 and 3.2) it screws up a calling argument and causes trashed article ID's to be written into the dbz database. Then, whenever the same duplicate article is presented, it isn't found in the database, so news treats it as a new article. The lazy fix, at least for a test, is to compile news without optimization. To which Jon Zeeff, dbz author, added: Or you could avoid the optimizer problem by compiling with gcc. I don't have gcc, but I did recompile without the -O flag and it seems to be working just fine these days! Several (noteable Conor Cahill) recommended cnews. sneaky.lonestar.org!gordon (Gordon Burditt) also sent: There is a define in the dbz routines which controls the size of the database file. It's supposed to be prime and several times the number of articles in the history file. I had problems with duplicate articles being accepted, and discovered that this number was about half of the number of articles in the history file - so half of the articles would be "not there" for the purpose of duplicate checking. I would expect dbz performance to start getting real terrible around the time file size <= 1.05 * # of articles in history file. Whether you have this problem depends on what groups you receive, and how long you keep history, but I expect my site to be a bit low on both of these compared with the bigger sites. Raise the number in the dbz source file. Recompile expire, inews, and anything else that uses it (rn?). REBUILD THE DATABASE using expire -R after installing these, and before letting any articles arrive. And m2c.m2c.org!jjmhome!jjm (Jim Murray) sent: You said that you put dbmclose in both expire and inews. You only need dbmclose in expire if you did things correctly. You should only link the copy of dbz compiled with INCORE for expire. The other programs should be linked with a version of dbz compiled without the INCORE option. Also you should be using at least version 1.9 of dbz. This is the earliest version that I believe had a working version of INCORE. Again, thanks to all who replied. Wait til I tell you about my next problem!! -- Doug ___ _ _ _ _ | || |_ ___ _| ||_| ___ __ _| |_ Doug Blair Obedient Software Corp. | | || .\/ ._\/. || |/ ._\| \|_ _| 1007 Naperville Rd, Wheaton IL 60187 |___||___/\___/\___||_|\___/|_|_| |_| 708-653-5527 blair@obdient.chi.il.us
henry@zoo.toronto.edu (Henry Spencer) (08/04/90)
In article <5693@obdient.chi.il.us> blair@obdient.chi.il.us (Doug Blair) writes: >I would expect dbz performance to start getting real terrible around >the time file size <= 1.05 * # of articles in history file. Note that the current dbz -- version 3.0, shipped with C News -- resizes itself automatically if used right, and also handles overflow (in between resizings, or if the software doesn't invoke it properly for resizing) gracefully and fairly efficiently. -- The 486 is to a modern CPU as a Jules | Henry Spencer at U of Toronto Zoology Verne reprint is to a modern SF novel. | henry@zoo.toronto.edu utzoo!henry