[news.software.b] Expire is mangling the history database

boehme@unvax.union.edu (Eric M. Boehm) (01/07/91)

I am running cnews (patchdate 15-Dec-1990) with nntp.1.5.10 on a
DECsystem 5000/Model 200 running Ultrix 4.1. When doexpire runs (and
thus expire), the history database is corrupted.

This is my diagnosis because:
1. After expire rebuilds the history database, nntp fails on the next
connect after only 1 article.
2. Examining the coredump for nntpd shows that a Segmentation
violation occurred when trying to lookup the article in the database.
3. If I run "mkhistory" the problem goes away (that is, there are no
more core dumps and news runs okay).

For the time being, I have modified the doexpire script to run expire
with "-r" and run "mkhistory" to rebuild the database.

Is this normal behavior? Is anyone else having the same problem? One
possible explanation is that I selected dbz when building cnews but
selected ndbm for nntpd. I also selected fast stdio for cnews. I had
to copy /usr/include/stdio.h to cnews/libstdio so that rdwr.c would
compile.

I don't think that the dbz is a problem for nntpd since it works okay
after mkhistory does its task.

Thanks in advance for your assistance.

-- 
Eric M. Boehm
boehme@unvax.Union.EDU
BOEHME@UNION.BITNET

henry@zoo.toronto.edu (Henry Spencer) (01/08/91)

In article <1991Jan7.021551.16323@unvax.union.edu> boehme@unvax.union.edu (Eric M. Boehm) writes:
>1. After expire rebuilds the history database, nntp fails on the next
>connect after only 1 article.
>2. Examining the coredump for nntpd shows that a Segmentation
>violation occurred when trying to lookup the article in the database.
>3. If I run "mkhistory" the problem goes away (that is, there are no
>more core dumps and news runs okay).

A significant issue here is that expire tries to size the dbz hash table
based on the actual volume of material, while mkhistory is not yet smart
enough to do that -- it just uses a fixed size, chosen for compatibility
with the old dbz.  I would suspect that C News and NNTP have been compiled
with different versions of the dbm/dbz code.

>Is this normal behavior?

No.

>Is anyone else having the same problem?

First we'd heard of it.

>One
>possible explanation is that I selected dbz when building cnews but
>selected ndbm for nntpd. ...
>I don't think that the dbz is a problem for nntpd since it works okay
>after mkhistory does its task.

Are you sure that mkhistory and expire were compiled with the same
database routines?  If you've just recently installed C News this should
not arise, but if you've updated an old one it is possible.

>I also selected fast stdio for cnews. I had
>to copy /usr/include/stdio.h to cnews/libstdio so that rdwr.c would
>compile.

That's odd.  You might possibly want to try not using the fast stdio,
although I don't see how it could cause the problem offhand.

Apart from the faint possibility that it's a stdio problem, my diagnosis
would indeed be database incompatibility between mkhistory, expire, and nntpd.
-- 
If the Space Shuttle was the answer,   | Henry Spencer at U of Toronto Zoology
what was the question?                 |  henry@zoo.toronto.edu   utzoo!henry

rsnider@xrtll.uucp (Richard Snider) (01/09/91)

In article <1991Jan7.021551.16323@unvax.union.edu>
   boehme@unvax.union.edu (Eric M. Boehm) writes:
>I am running cnews (patchdate 15-Dec-1990) with nntp.1.5.10 on a
>DECsystem 5000/Model 200 running Ultrix 4.1. When doexpire runs (and
>thus expire), the history database is corrupted.
>1. After expire rebuilds the history database, nntp fails on the next
>connect after only 1 article.
>2. Examining the coredump for nntpd shows that a Segmentation
>violation occurred when trying to lookup the article in the database.
>3. If I run "mkhistory" the problem goes away (that is, there are no
>more core dumps and news runs okay).

>possible explanation is that I selected dbz when building cnews but
>selected ndbm for nntpd. I also selected fast stdio for cnews. I had

I had a variant of this same problem when I compiled relaynews with 
the dbz that comes with C News and expire with the dbm that comes with
the system (Don't ask why, you don't want to know :-)

Be warned that the dbm that comes with your system is probabbly not at
all compatible with the dbz that comes with C News or any other software
when it gets down to actually accessing {.dir|.pag} files.

I suggest picking one that has everything that all the software needs to
use and sticking to it.

I also remember that copying the history database files using normal
means tends to upset some flavors of dbm in this way.

                                        Richard Snider
------------------------------------------------------------------------
Where: ..uunet!mnetor!yunexus!xrtll!rsnider    Also:  rsnider@xrtll.UUCP
If a hammer dosn't work, get a bigger hammer.

huntting@csn.org (Brad Huntting) (01/13/91)

boehme@unvax.union.edu (Eric M. Boehm) writes:

> One possible explanation is that I selected dbz when building
> cnews but selected ndbm for nntpd.

I had problems because I compiled nntp (1.5.10) with dbm while cnews
programs where using dbz...  nntpd ran fine except it could never find
articles by message-id's.  After defining DBZ (and DBM) in
common/conf.h and linking with -lcnews, the problem stopped.

brad
	huntting@csn.org
	huntting@colorado.edu