blair@obdient.chi.il.us (Doug Blair) (07/18/90)
We have a problem with b news 2.11.19 when compiled with the dbz package. inews and expire run fast, but duplicate copies of articles (with the same article id in <>'s) are not rejected and are instead inserted twice into our spooldir. I added dmbclose() calls in inews.c and expire.c in the (I think) appropriate places. Has anyone else seen this problem? Hardware: 386 clone box, Microport System V/386 3.0e.1, 4 meg/ram, 160 meg/hd, Computone 8-port, Trailblazers, HSTs and a couple of generic 2400 modems. Please respond via email, I'll post a summary if appropriate. Thanks! Doug ___ _ _ _ _ | || |_ ___ _| ||_| ___ __ _| |_ Doug Blair Obedient Software Corp. | | || .\/ ._\/. || |/ ._\| \|_ _| 1007 Naperville Rd, Wheaton IL 60187 |___||___/\___/\___||_|\___/|_|_| |_| 708-653-5527 blair@obdient.chi.il.us
blair@obdient.chi.il.us (Doug Blair) (08/02/90)
A few weeks ago I described a condition in which news 2.11.19 with the dbz code included did not properly reject duplicate articles received from a second news site. Thanks to all of you who replied and added your thoughtful insights. Summary follows: The most helpful comments came from karl@ficc.ferranti.com (Karl Lehenbauer): I have determined the cause of this problem. I had to learn way more about the internals of news than I ever wanted to, but the problem is the optimizer of the C compiler. I forget which news source file it is, but when used with dbz (System V/386 3.0 and 3.2) it screws up a calling argument and causes trashed article ID's to be written into the dbz database. Then, whenever the same duplicate article is presented, it isn't found in the database, so news treats it as a new article. The lazy fix, at least for a test, is to compile news without optimization. To which Jon Zeeff, dbz author, added: Or you could avoid the optimizer problem by compiling with gcc. I don't have gcc, but I did recompile without the -O flag and it seems to be working just fine these days! Several (noteable Conor Cahill) recommended cnews. sneaky.lonestar.org!gordon (Gordon Burditt) also sent: There is a define in the dbz routines which controls the size of the database file. It's supposed to be prime and several times the number of articles in the history file. I had problems with duplicate articles being accepted, and discovered that this number was about half of the number of articles in the history file - so half of the articles would be "not there" for the purpose of duplicate checking. I would expect dbz performance to start getting real terrible around the time file size <= 1.05 * # of articles in history file. Whether you have this problem depends on what groups you receive, and how long you keep history, but I expect my site to be a bit low on both of these compared with the bigger sites. Raise the number in the dbz source file. Recompile expire, inews, and anything else that uses it (rn?). REBUILD THE DATABASE using expire -R after installing these, and before letting any articles arrive. And m2c.m2c.org!jjmhome!jjm (Jim Murray) sent: You said that you put dbmclose in both expire and inews. You only need dbmclose in expire if you did things correctly. You should only link the copy of dbz compiled with INCORE for expire. The other programs should be linked with a version of dbz compiled without the INCORE option. Also you should be using at least version 1.9 of dbz. This is the earliest version that I believe had a working version of INCORE. Again, thanks to all who replied. Wait til I tell you about my next problem!! -- Doug ___ _ _ _ _ | || |_ ___ _| ||_| ___ __ _| |_ Doug Blair Obedient Software Corp. | | || .\/ ._\/. || |/ ._\| \|_ _| 1007 Naperville Rd, Wheaton IL 60187 |___||___/\___/\___||_|\___/|_|_| |_| 708-653-5527 blair@obdient.chi.il.us
henry@zoo.toronto.edu (Henry Spencer) (08/04/90)
In article <5693@obdient.chi.il.us> blair@obdient.chi.il.us (Doug Blair) writes: >I would expect dbz performance to start getting real terrible around >the time file size <= 1.05 * # of articles in history file. Note that the current dbz -- version 3.0, shipped with C News -- resizes itself automatically if used right, and also handles overflow (in between resizings, or if the software doesn't invoke it properly for resizing) gracefully and fairly efficiently. -- The 486 is to a modern CPU as a Jules | Henry Spencer at U of Toronto Zoology Verne reprint is to a modern SF novel. | henry@zoo.toronto.edu utzoo!henry