[comp.unix.shell] Rebuild a .newsrc file

davidsen@crdos1.crd.ge.COM (Wm E Davidsen Jr) (10/03/90)

  Someone asked about rebuilding a .newsrc file. I have often wanted to
build one from scratch which wouldn't take me all day. I started
thinking about it and knocked out this little shell script which build a
file.

  This subscribes to everything and allows you to mark things new in the
last N days as unread. I find it a lot less painful to type 'u' than to
have to add each group by hand.

  This script is SLOW. I didn't put much effort into making it fast,
because the ability to read the recent articles requires a scan of the
whole spool directory (or database, and that's no fun in shell).

#!/bin/sh
# rebuild .newsrc
#    - all groups will be subscribed
#    - articles for the last three days will be marked unread
#      (or use options to pick another date)
#
# author: Bill Davidsen - Oct 2, 1990

# initial values
SPOOLDIR=/usr/spool/news
AGE_LIMIT=3
LIBDIR="/usr/lib/news"

# look for arguments
#    -aNN   is age in days
#    -sXXX  is alternate spool directory
#    -lXXX  is alternate dir for active file

while [ $# -gt 0 ]
do	case $1 in
	-s)	SPOOLDIR=$2
		shift;shift
		;;
	-s*)	SPOOLDIR=`expr $1 : '..\(.*\)'`
		shift
		;;
	-a)	AGE_LIMIT=$2
		shift;shift
		;;
	-a*)	AGE_LIMIT=`expr $1 : '..\(.*\)'`
		shift
		;;
	-l)	LIBDIR=$2
		shift;shift
		;;
	-l*)	LIBDIR=`expr $1 : '..\(.*\)'`
		shift
		;;
	*)	echo "Unknown option: $1"
		exit 1
		;;
	esac	#
done	#

# pipe the directory names into a loop to build the values
cd $SPOOLDIR
SELECT="-type f -name '[1-9]*' -mtime -${AGE_LIMIT}"

sed 's/ .*$//' ${LIBDIR}/active | tr '.' '/' |
while read newsgroup
do	# drop any group starting with .
	char1=`expr "$newsgroup" : '\(.\).*'`
	[ -z "$char1" -o "$char1" = "." ] && continue
	[ ! -d $newsgroup ] &&
	{	# just the group name, nothing read
		echo "$newsgroup:" | tr '/' '.'
		continue
	}
	file1=`eval find $newsgroup ${SELECT} -print | sort -n | head -1`
	[ -z "$file1" ] && 
	{	# just the group name, nothing read
		echo "$newsgroup:" | tr '/' '.'
		continue
	}
	file1=`basename $file1`
	last1=`expr $file1 - 1`
	echo "$newsgroup: 1-$last1" | tr '/' '.'
done	#
-- 
bill davidsen	(davidsen@crdos1.crd.GE.COM -or- uunet!crdgw1!crdos1!davidsen)
    VMS is a text-only adventure game. If you win you can use unix.

lvc@cbnews.att.com (Lawrence V. Cipriani) (10/03/90)

In article <2713@crdos1.crd.ge.COM>, davidsen@crdos1.crd.ge.COM (Wm E Davidsen Jr) writes:
> 
>   Someone asked about rebuilding a .newsrc file.

Here's a pretty quick one in awk.

# this will create a .newsrc for folks showing that they've suscribed
# to all groups and haven't read anything.  This keeps rn from bothering
# the new user with hundereds of "... not in .newsrc.  Add[y/n]? " messages.
awk '{ 
        if($3 > 0) {
                printf("%s: 1-%d\n", $1, $3-1);
        } else {
                printf("%s: \n", $1);
        }
}' /usr/lib/news/active > .newsrc
-- 
Larry Cipriani, att!cbvox!lvc or lvc@cbvox.att.com
           "Save a logger, eat an owl"

dylan@ibmpcug.co.uk (Matthew Farwell) (10/04/90)

In article <1990Oct3.014039.27275@cbnews.att.com> lvc@cbnews.att.com (Lawrence V. Cipriani) writes:
>In article <2713@crdos1.crd.ge.COM>, davidsen@crdos1.crd.ge.COM (Wm E Davidsen Jr) writes:
>>   Someone asked about rebuilding a .newsrc file.
>Here's a pretty quick one in awk.
> [ solution deleted ]

or
sed -e 's/ .*/:/' /usr/lib/news/active > $HOME/.newsrc

Dylan.
-- 
Matthew J Farwell                 | Email: dylan@ibmpcug.co.uk
The IBM PC User Group, PO Box 360,|        ...!uunet!ukc!ibmpcug!dylan
Harrow HA1 4LQ England            | CONNECT - Usenet Access in the UK!!
Phone: +44 81-863-1191            | Sun? Don't they make coffee machines?

Dan_Jacobson@ATT.COM (10/05/90)

Mine takes regular expressions of your liked/disliked groups and sorts
the .newsrc.  Since it depends on sed/awk, etc, I noticed it poops out
on long lines, so I stopped using it.  Plus now that I'm using GNUS, I
hope to find a more high tech solution.

[Cheapest shar, and signature at end...]

echo x - newsrc-rx
sed 's/^X//' >newsrc-rx <<'*-*-END-of-newsrc-rx-*-*'
X# regular expressions for my home-made newsrc sorter.  Dan_Jacobson@ATT.COM
X# this file holds the "loved" newsgroups regular expressions
X# `#' is a comment character at beginning of line.
X# [Was] Always being refined, sloppy, order not too signifigant, especially
X# further down.
X
X#if .newsrc lines are too long, screws up.
X
X# Loacl issues, and emergency stuff, etc.
X^general|important
X^ih
X^att
X^chi
X
X# Things that are going to make my job easier, or pointers to job-easers
X#^gnu\\.(announce|bash|misc|utils|ghost|emacs|gdb)
Xgnu
Xemacs|editors
X\\.tex[^a-z]
Xcomp\\.(fonts|text)
X\\.sun
Xblit|tty5620
X#comp\\.dcom\\.(modems|telecom)
Xalt\\.sources
Xcomp\\.sources(:|\\.d|\\.misc|\\.unix|\\.wanted)
Xarchive
Xcomp\\.unix
X#bugs\\.(sys5|bsd|sun)
Xbugs
X^news\\.announce\\.new
X^news\\.(lists|newusers)
Xsecurity|\\.doc
X\\.x[.:]
X
X#Things that make life easier
X
Xtelecom
Xcomp\\.risks
Xsci\\.(med|lang)
Xrec\\.travel|driving
Xastro
X#comp\\.org
Xrec\\.food\\.veg
Xconsumers
X#backcountry
X\\.mail
X#asian|china|culture|korean|africa
Xnews
Xculture
Xmusic
Xlegal
Xenvironment
X\\.invest|\\.taxes
Xgnu
X#^comp
Xconference
X^sci
X#^rec
X^soc
X^list
X#^alt
X#^misc
X#^talk
*-*-END-of-newsrc-rx-*-*
echo x - newsrc-rx-kill
sed 's/^X//' >newsrc-rx-kill <<'*-*-END-of-newsrc-rx-kill-*-*'
X# Dan_Jacobson@ATT.COM
X# this file holds the "hated" newsgroups regular expressions
X#these go to the bottom of $NEWSRC
X\\.amiga|\\.atari|\\.ibm|\\.binaries|\\.test|sport|games|^ak\\.
X^al\\.|^alc\\.|^an\\.|^atl\\.|^bionet\\.
X^biz\\.|^cb\\.|^control|^dr\\.|^er\\.|^essug\\.|^ga\\.|^hl\\.|^ho\\.|^in\\.
X^junk|^lc\\.|^mh\\.|^ne\\.|^nj\\.|^oh\\.|^pk\\.|^pr\\.|^princeton\\.|^rd\\.
X^sf\\.|^unix-pc\\.|^wb\\.|^wh\\.|^wi\\.|^inh\\.|philosophy|\\.cray|\\.msdos
X\\.xenix|\\.sco[.:]|\\.vms:|^att\\.(lib|jobdigest)|physics|ps-maps|\\.aix
X\\.aux|[234]86|\\.ultrix
*-*-END-of-newsrc-rx-kill-*-*
echo x - rcsrt
sed 's/^X//' >rcsrt <<'*-*-END-of-rcsrt-*-*'
X: Dan_Jacobson@ATT.COM. Sort .newsrc file my style. Last edit: 8/1990
Xtrap '' 1 15 #best?
Xset -xe #quit on error
X: ${NEWSRC=$HOME/.newsrc}
Xreg_exp_file=${1-$HOME/newsrc-rx} #a file full of regular expression strings
Xin=/tmp/$$in out=/tmp/$$out
X#backup=$HOME/NEWSRC$$
Xbackup=/usr/tmp/.NEWSRC$$
Xkill_file=$HOME/newsrc-rx-kill
Xtrash=/tmp/$$trash
Xtest -f "$reg_exp_file" -a -f "$NEWSRC" -a $# -lt 2 -a ! -f $in -a ! -f \
X	$out -a ! -f $backup -a ! -f $trash
Xcp $NEWSRC $backup && compress $backup #be thrifty on disk space
Xcp $NEWSRC $in
Xset +e
X#for reg_exp in `cat $reg_exp_file` #but *blanks* & newlines separate words
X
X#put the junk at bottom
Xtest -f $kill_file &&
X{
Xexec 3<$kill_file
Xwhile read reg_exp <&3
Xdo
X	case "$reg_exp" in \#*|'') continue;; esac # "#" is comment character
X		#blank lines with no spaces ok too.
X	egrep "$reg_exp" $in >> $trash
X	egrep -v "$reg_exp" $in > $in.tmp
X	mv $in.tmp $in
Xdone
X}
X
Xexec 3<"$reg_exp_file"
Xwhile read reg_exp <&3
Xdo
X	case "$reg_exp" in \#*|'') continue;; esac
X	egrep "$reg_exp" $in | sort >> $out
X	egrep -v "$reg_exp" $in > $in.tmp
X	mv $in.tmp $in
Xdone
Xset -e
X{ cat $out; sort $in; cat $trash;} > $NEWSRC
Xrm $out $in $trash
X
X
X
Xexit
X#to make a test file...
Xset -ex
Xcp .newsrc NEWSRC-REAL
Xsed 's/\([:!]\).*/\1/' NEWSRC-REAL > .newsrc
Xrcsrt
Xsed 's/\([:!]\).*/\1/' NEWSRC-REAL | diff - .newsrc
*-*-END-of-rcsrt-*-*
exit
-- 
Dan_Jacobson@ATT.COM  Naperville IL USA  +1 708-979-6364