[news.sysadmin] news group automatic creation

neil@zardoz.UUCP (Neil Gorsuch) (04/13/89)

Does anyone know how to convince B2.11 patchlevel 14 news software how to
automatically create a news group when an article comes in that is not in
the current "active" file?  Or if one of the new flavors of news coming
out such as TMN supports this?  I accept ALL news groups and do not like
to have to watch the junk group to create the newly found groups.  And I
especially do not like to have articles in "junk" that should have been
in a properly created group.  The problem will become more severe in
the near future, since I am about to connect to the internet and plan
on getting FULL feeds from some large sites that have many obscure groups.

Neil Gorsuch
neil@cpd.com
uunet!zardoz!neil

pda@stiatl.UUCP (Paul Anderson) (04/15/89)

In article <610@zardoz.UUCP> you write:
>Does anyone know how to convince B2.11 patchlevel 14 news software how to
>automatically create a news group when an article comes in that is not in
>the current "active" file?

Neil-
I run the following script out of my crontab with the -y option.  It
automatically creates the new newsgroups and then redistributes the news
in junk to those new newsgroups.  It was a quick hack I did one day because
I got tired of doing the redistribution myself...  

Paul
------------snip,snip-----------


# /bin/sh
# This script redistributes news articles in junk.  It creates
# new newsgroups as necessary.  It is interactive and prompts for
# group creation. That is, unless you specify the -y switch, in which
# case, it will run in the background...

# Make sure you change SPOOLD and LIBD to your appropriate directories.

# Note: the 'active' file is sorted as a result of running this script.

# Author:	Paul Anderson   10/1/88		
# Rev:		@(#) Redistribute 1.2

# 3/7/89 pda
#	added support for -y switch: create newsgroups without prompting


export PATH || (echo "You didn't use sh, you clunch." ; kill $$)

SPOOLD=/news/spool/news
LIBD=/sti/news/bin

INEWS=$LIBD/inews
EXPIRE=$LIBD/expire

LOOKUP=$SPOOLD/tmp.$$

# setup default variables
autocreate="no"

if [ $# != 0 ]
then
    case $1 in 
    -y)		autocreate="yes"
		;;
    esac
fi

cd $SPOOLD/junk

# check to see if there is any news in junk.  if not then exit.

if [ `ls | wc -w` -eq 0 ]
then
    exit 0
fi

# Process the active file and the aliases file, getting only the
# newsgroup name, one name per line.  This is so that we can do
# an exact match on the name by using ^name$.

awk '{ print $1 }' $LIBD/aliases $LIBD/active |sort -u >$LOOKUP

# fgrep: Find out what newsgroups news was being sent to.
# awk: throw away any duplicate Newsgroup lines from the file (only use first)
# awk2: print out the (space separated) newsgroup names
# awk3: print out the (comma separated) newsgroup names
# 'sort -u' is used in case there is more than one 
#	newsgroup by the same name that came in.  if there is, then this
#	will keep us from creating the newsgroup more than once (by throwing
#	away the duplicate lines).
# Then see if the newsgroups exist (from either active or aliases)
# If the newsgroup doesn't exist, then create it.  If it does exist, 
# then this article was posted orignially to junk or there is a bug
# in the system that is causing articles to valid newsgroups to be
# posted to junk. 

fgrep Newsgroups: /dev/null * |\
awk -F: 'BEGIN { last = "foo.foo" } \
	last !=  $1 { print ; last = $1 }' |\
awk 'BEGIN { FS=" " } \
	{ for (i=2; i<=NF; ++i) print $i }' |\
awk 'BEGIN { FS="," } \
	{ for (i=1; i<=NF; ++i) print $i }' |\
sort -u |\
while read group
do
    # look for the group in either aliases or active. if not found, then
    # create the group.

    if [ `grep "^${group}\$" $LOOKUP | wc -l` -eq 0 ]
    then
	if [ $autocreate != "yes" ]
	then
	    echo Going to create $group
	    echo "Do you want to do this? [yn] \c"
	    read ans </dev/tty
	    if [ `echo $ans | grep -i n | wc -w` -eq 0 ]
	    then
		echo Creating $group ...
		$INEWS -C $group < /dev/null
	    fi
	else
		echo Creating $group ...
		$INEWS -C $group < /dev/null
	fi
    fi
done

# After creating the new newsgroups, sort the original active file

cd $LIBD 
cp active oactive 
sort <active >tmpactive.$$ 
cp tmpactive.$$ active 
rm tmpactive.$$

# OK, now that the newsgroups have been created, create a temp dir,
# copy the articles that were not posted to junk into the temp dir, 
# expire all news out of junk,
# then redistribute the news in the temp dir to the appropriate news
# groups.

cat <<EOF

Newsgroups created.  News is now being delivered to appropriate
groups.  This will take a while.  Please standby.

EOF

cd $SPOOLD ; rm -rf temp ; mkdir temp
cd $SPOOLD/junk
for file in *
do
    if [ `fgrep Newsgroups: $file | fgrep -v junk | wc -l` -ne 0 ]
    then
	cp $file $SPOOLD/temp
    fi
done

$EXPIRE -e 0 -E 0 -p -n junk

cd $SPOOLD/temp
for file in *
do
	rnews < $file
done

cd $SPOOLD ; rm -rf temp

rm $LOOKUP
-- 
Paul Anderson                           "But Why?"
gatech!stiatl!pda                       "Because they tried to make it
(404) 841-4000                           secure..."

eric@snark.uu.net (Eric S. Raymond) (04/21/89)

In <610@zardoz.uucp> Neil Gorsuch wrote:
> Does anyone know how to convince B2.11 patchlevel 14 news software how to
> automatically create a news group when an article comes in that is not in
> the current "active" file?  Or if one of the new flavors of news coming
> out such as TMN supports this?

Indeed we do! Just set the FLEXGROUPS attribute on the subscription "all"
via a one-line change to TMN-netnews's `admin' file and you'll get the
behavior you want. Old flexgroups quietly disappear after a period of
inactivity.
-- 
      Eric S. Raymond                     (the mad mastermind of TMN-Netnews)
      Email: eric@snark.uu.net                       CompuServe: [72037,2306]
      Post: 22 S. Warren Avenue, Malvern, PA 19355      Phone: (215)-296-5718