[news.software.b] Expire corrupts active file

robbinj@molihp.UUCP (Robbin W Johnson) (11/28/86)

First the configuration:

Hardware: Hewlett Packard HP9000/550 (1 CPU + 4MB Ram + 404 MB Disk)
Software: HP-UX release 5.11 (SYS V rel 2.2 + BSD enhancements)
          2.11 news - patchlevel 0

The problem:

Expire is called nightly by the cron entry:

	30 1 * * * exec /usr/lib/news/expire

After it has run, the LIBDIR/active file is broken.  The minimum article
number and maximum article number are both equal in value and set to what
should be the value of the maximum article.  That is if the directory
SPOOLDIR/news/software/b (as an example, this applies to ALL newsgroups)
contains articles 1 through 10, both the minimum and maximum article numbers
in LIBDIR/active are set to 10.  None of the articles are older than 2 weeks.
The value of DFLTEXP in defs.h is 2*WEEKS.  The value of HISTEXP is 4*weeks.
I did no hand customizing of defs.h outside of localize.sh (I have appended
the contents of localize.sh below).  If I don't run expire, active is not
corrupted.

What gives?

I cannot run expire while this is going on.  If I do so then anyone reading
the news sees one and only one article unread and all the others that may
be in the spool directories invisible.

I have to run expire since we are limited in the amount of disk storage
availible for the netnews.

If anyone has had a similiar problem I would appreciate pointers/advise on
solving this.  I have looked at expire.c and have a couple of ideas since all
the playing with active is in one area, but I am unfamiliar with the algorithm
and cannot spend a lot of time on this for the next couple of weeks.

WHAT HAVE I DONE WRONG?

Please reply by mail and I will post a summary to this group if anything
obviously wrong or HP(-UX) sensitive appears.

Thank-you,
Robbin.

...!ubc-vision!van-bc!molihp!robbinj

---This is the contents of my localize.sh---
rm -f Makefile
cp Makefile.dst Makefile
chmod u+w Makefile
ed - Makefile  <<'EOF'
/^UUXFLAGS/s/-r -z/-r -z -n/
g/^#USG /s///
g/^#V7 /d
g/^#VMS /d
g/^#BSD4_[123] /d
g/^VFORK/s/=-Dvfork=fork/ =/
g/^LFLAGS/s/-i //
g/termlib/s//curses/
w
q
EOF
rm -f defs.h
cp defs.dist defs.h
chmod u+w defs.h
ed - defs.h <<'EOF'
/ROOTID/s/10/3/
/N_UMASK/s/000/022/
/TMAIL/s;^;/\* ;
/PAGE/s;/usr/ucb/more;/bin/more;
/UXMIT/s;^;/\* ;
/NONEWGROUPS/s;/\* ;;
/INTERNET/s;/\* ;;
/GHNAME/s;/\* ;;
/DOXREFS/s;/\* ;;
/ALWAYSALIAS/s;/\* ;;
/SENDMAIL/s;/\* ;;
/HIDDENNET/s;frooz;moli;
/MYORG/s/Frobozz Inc., St. Louis/Moli Energy Ltd., Burnaby, BC, CANADA/
/NICENESS/s;/\* ;;
/ORGDISTRIB/s;froozum;moli;
w
q
EOF

rick@seismo.CSS.GOV (Rick Adams) (12/02/86)

The problem sounds like readdir() is returning garbage. I've heard
things about the 9000/500 directory structure being "interesting".

In a posting talking about unipress emacs mg@unirot.UUCP (Mike Gallaher) says:

	However, I have spent many hours of my life compensating for the
	differences between HPUX on the 500 and on everything else that also
	claims to be Unix System V.  (There are more "special case"
	conditionals in the Emacs code for the 500 than for any other
	machine...)  When I wrote that flame I had just wrestled with the
	following differences:

		- the file system is neither berkeley or v7/SysV, so the bsd
		  directory emulator didn't work until I taught it
		  how.  (HP does supply its own BSD directory
		  emulation, but it wasn't documented for the 500, at
		  least that I could find).
		 - The tty driver does not support VMIN/VTIME.  This *is*
		  documented, but it depends on which hardware
		  interface you have, which is not easy to detect (even
		  by looking at the box!).


I suspect your problems are related. You should contact whoever sold
you the system and ask  about reading the filesystem.

---rick

robbinj@molihp.UUCP (Robbin W Johnson) (12/02/86)

Having found some time to look into this, I found the problem and a solution.

As distributed, 2.11 news assumes that if it is being brought up upon a USG
flavour of UN*X there is no directory(3) library (/usr/lib/libndir.a) and
that the structure of a directory in the file system is vanilla SYS V.
Believing this to be true it then provides its own limited version of the
directory package, making several assumptions about the format of a directory
as it lives in the file system.  When dealing with Hewlett-Packards's HP-UX,
this is a fatal mistake.

HP-UX on a series 500 machine sits on top of other layers of software,
including the "SDF file system".  Because of this, the format of a directory
is slightly different and totally incompatible with the SYS V model.

Fortunately Hewlett-Packard has provided the directory(3) library with HP-UX
and so solving the problem is very straight forward.  I solved it in the
Makefile by removing all dependencies upon "ndir.o" and adding a search of
libndir.a for those programs which might require it.  This can be done from
within localize.sh with one exception.  The "ndir.h" file to be included
must be changed to "/usr/include/ndir.h".  I took the easy way out by
mv'ing the origional "ndir.h" to "ndir.h.rel" to get it out of the way and
then ln'ing "/usr/include/ndir.h" to "ndir.h".  This could cause problems if
"ndir.h" is ever changed in a minor revision.  There may be far more elegant
ways, if anyone has suggestions I would appreciate hearing them.  As far as
the "ndir.o" dependencies were concerned, I corrected that by adding the
following two lines to the "ed" script in localize.sh (if you reference my
origional posting, place them anywhere before the "g/termlib/s//curses" line):

g/^LIBS/s/=/= -lndir/
g/ ndir.o/s///
  ^watch that space...its important

You might also have to add:

g/#NOTVMS/s///

to prevent the compiler from breaking on the "cc" line for inews.

When rn is configured, it knows enough to check for libndir.a no matter
what version of UN*X it is being brought up on.  This would have been a
nice idea for 2.11 news.

Robbin.

dennis@rlgvax.UUCP (Dennis Bednar) (12/28/86)

In article <253@molihp.UUCP>, robbinj@molihp.UUCP (Robbin W Johnson) writes:
> As distributed, 2.11 news assumes that if it is being brought up upon a USG
> flavour of UN*X there is no directory(3) library (/usr/lib/libndir.a) and
> that the structure of a directory in the file system is vanilla SYS V.

I had the same problem with installing 2.11 news on a CCI Power 6/32
machine running SVR2.  Although I used localize.usg as a starting
point for localize.sh, I had to modify it slightly to have the CFLAGS
pass a new flag down to the ndir.c routine, so that ndir.c is commented
out, and the 4.2 style directory(3) routines in libc.a (or wherever)
are used instead.  If anybody out in net land wants a copy of localize.cci632,
please reply to me.
-- 
-Dennis Bednar
{decvax,ihnp4,harpo,allegra}!seismo!rlgvax!dennis	UUCP