[news.sysadmin] Unpacking comp.mail.maps files

ron@vsedev.VSE.COM (Ron Flax) (02/09/88)

This is really a query to determine the best, if there is one, way to
unpack comp.mail.maps files, since they seem to come in every day or
so now.

I used to simply type '1000-1050w | (cd /v/hostnames/map; unshar)' at
rn when the maps came in once a month or so... but now since the
distribution is split across the entire month, this is really a pain!
Plus the fact that I can't seem to figure out when the damned cut off
is :-)

--
ron@vsedev.vse.com	(Ron Flax)
uucp:	..!uunet!vsedev!ron
inet:	vsedev!ron@uunet.uu.net

lyndon@ncc.UUCP (Lyndon Nerenberg) (02/12/88)

Manually unpacking the maps is becoming a bit unreasonable these
days. Actually, it's quite simple to automate this. First, add the
following line to your sys file:

MAPS:world,comp.mail.maps::/usr/local/lib/getmaps

Next, install the following script in /usr/local/lib/getmaps (
or wherever is appropriate):

#!/bin/sh
#
# getmaps - unpack the usenet maps
#
umask 2
cd /usr/spool/uucppublic/uumap
sed -e '1,/^$/d' | sh


Not too fancy, but it's been working here for over a year :-)

--lyndon   {alberta,utzoo}!ncc!lyndon

mikel@codas.att.com (Mikel Manitius) (02/12/88)

In article <1059@vsedev.VSE.COM> ron@vsedev.VSE.COM (Ron Flax) writes:
> 
> This is really a query to determine the best, if there is one, way to
> unpack comp.mail.maps files, since they seem to come in every day or
> so now.

This definetly isn't the best, but it's simple and clean. Get yourself
in the right directory, and feed anything in comp.mail.maps through:

	sed -e '1,/^[:#]/d' | sh

And beware of trojan horses!
-- 
					Mikel Manitius @ AT&T
					mikel@codas.att.com

gerry@syntron.UUCP (G. Roderick Singleton) (02/13/88)

In article <1059@vsedev.VSE.COM> ron@vsedev.VSE.COM (Ron Flax) writes:
>This is really a query to determine the best, if there is one, way to
>unpack comp.mail.maps files, since they seem to come in every day or
>so now.
>
>I used to simply type '1000-1050w | (cd /v/hostnames/map; unshar)' at
>rn when the maps came in once a month or so... but now since the
>distribution is split across the entire month, this is really a pain!
>Plus the fact that I can't seem to figure out when the damned cut off
>is :-)
>
>--
>ron@vsedev.vse.com	(Ron Flax)
>uucp:	..!uunet!vsedev!ron
>inet:	vsedev!ron@uunet.uu.net


Some time ago a friend gave me a map handler that does a marvelous job.  It
even takes care of supersedes.  I mangled it a bit of course but because it's
so useful here (we are currently at the end of a feed) I am including it for
yours and anyone else's use:

#! /bin/sh
# This is a shell archive.  Remove anything before this line, then unpack
# it by saving it into a file and typing "sh file".  To overwrite existing
# files, type "sh file -c".  You can also feed this as standard input via
# unshar, or by typing "sh <file", e.g..  If this archive is complete, you
# will see the following message at the end:
#		"End of shell archive."
# Contents:  xmaps
# Wrapped by gerry@syntron on Fri Feb 12 20:22:11 1988
PATH=/bin:/usr/bin:/usr/ucb ; export PATH
if test -f xmaps -a "${1}" != "-c" ; then 
  echo shar: Will not over-write existing file \"xmaps\"
else
echo shar: Extracting \"xmaps\" \(1146 characters\)
sed "s/^X//" >xmaps <<'END_OF_xmaps'
X#! /bin/sh
X
X# Moves the UUCP maps to the appropriate directory, un-shars them, and
X# pitches the originals.
X
X# Written 30 December 1987 by Stanley F. Quayle
X# modified to automatically execute pathalias and pathproc to generate and
X# install paths -- 14 January 1988 by G. Roderick Singleton
X
Xumask 002
X
X# Exit if news is executing (and probably unbatching)
Xif [ -r /usr/spool/uucp/LCK.XQT ] ; then
X	echo "News is being received, cannot execute."
X	exit 0
Xfi
X
X# Exit if no maps have been received recently
Xgrep -s "comp.mail.maps" /usr/lib/news/log || exit 0
X
X# Create temporary directory
Xmkdir /usr/u/Maps/tmp
X
X# Change directory
Xcd /usr/u/Maps/tmp
X
X# Move the maps
Xmv /usr/spool/news/comp/mail/maps/* .
X
X# Unshar them
X/usr/local/bin/unshar [0-9]* >/dev/null
X
X# Zap the originals, and any shar messages
Xrm -f [0-9]* s.log s.tmp
X
X# Compress the maps
X/usr/lib/news/compress *
X
X# Move everyone to the Maps directory, replacing as needed
Xmv * ..
X
X# Zap the temporary directory
Xcd /usr/u/Maps
Xrmdir /usr/u/Maps/tmp
X
X# now create the new paths file
X# /usr/local/bin/pathalias -f * | /usr/local/bin/pathproc >newpaths
X# cp newpaths /usr/lib/uucp
Xexit 0
END_OF_xmaps
if test 1146 -ne `wc -c <xmaps`; then
    echo shar: \"xmaps\" unpacked with wrong size!
fi
chmod +x xmaps
# end of overwriting check
fi
echo shar: End of shell archive.
exit 0
-- 
G. Roderick Singleton              |  "ALL animals are created equal,
   <gerry@syntron.uucp>,           |   BUT some animals are MORE equal
or <gerry@geac.uucp>,              |   than others." a warning from
or <gerry@eclectic.uucp>           |  "Animal Farm" by George Orwell

pleasant@rutgers.rutgers.edu (Mel Pleasant) (02/13/88)

In article <1059@vsedev.VSE.COM> ron@vsedev.VSE.COM (Ron Flax) writes:

> This is really a query to determine the best, if there is one, way to
> unpack comp.mail.maps files, since they seem to come in every day or
> so now.
> 
> I used to simply type '1000-1050w | (cd /v/hostnames/map; unshar)' at
> rn when the maps came in once a month or so... but now since the
> distribution is split across the entire month, this is really a pain!
> Plus the fact that I can't seem to figure out when the damned cut off
> is :-)

FLAME ON:

	Click your heals and repeat three times after me:

	there is no `damned cutoff'

	There is no `damned cutoff'

	*THERE* *IS* *NO* `*DAMNED* *CUTOFF*'

I've answered this a couple of times.  There was even an announcement
about this.  Once again, in an effort to turn map updates around
quickly, we now post the map files as we receive new entries and
updates.  There is no such thing as a monthly posting anymore.  There
is no such thing as a DONE file anymore.  I kid you not.  Really!!!
Truly!!!  Gone for good (hopefully) are these.  Once a month we post
every file that has not been posted, due to not being updated, in the
last 30 days.  This guarantees that a new site will eventually receive
all of the files.

The official software that will help you receive and use these files
are Bnews 2.11 (patch #14), pathalias version 9, and uuhosts.  All can
be had by writing to Rich Salz (salz@pineapple.bbn.com).  If you
choose to use other software to deal with the postings, you're on your
own.  I don't want to get into things like guaranteeing that a certain
string will appear a certain way everytime.  The only guarantees I'd
like to make is that the software mentioned above does the job.
That's it.  That's all.

 :-)  :-)  :-)  :-)  :-)  :-)  :-)  :-)  :-)  :-)  :-)  :-)  :-)  :-)

FLAME OFF:


A couple of folks have sent/posted suggestions about posting shortened
versions of our updates.  I haven't responded to any of these yet
because I've been rather busy.  However, I'd like to say that I like
the idea of using `ar' and posting map entries individually.  Besides
the bashing the net would take if/when a switchover to this mode
occurred, does any see any problems with this type of setup?
-- 

                                  Mel Pleasant
 {backbone}!rutgers!pleasant   pleasant@rutgers.edu     mpleasant@zodiac.bitnet

rsalz@bbn.com (Rich Salz) (02/13/88)

In news.sysadmin (<8283@rutgers.rutgers.edu>), pleasant@rutgers.rutgers.edu (Mel Pleasant) writes:
>  The official software you need to unpack the maps
>are Bnews 2.11 (patch #14), pathalias version 9, and uuhosts.  All can
>be had by writing to Rich Salz (salz@pineapple.bbn.com).

It's rsalz, not salz, and "sources@uunet.uu.net" is a better address;
mail there doesn't interfere with my work as much (your tax dollars...),
and please don't ask me for 2.11 news -- get it from an upstream site.

:-),
	/r$
-- 
For comp.sources.unix stuff, mail to sources@uunet.uu.net.

kutz@bgsuvax.UUCP (Kenneth Kutz) (02/16/88)

In article <1059@vsedev.VSE.COM>, ron@vsedev.VSE.COM (Ron Flax) writes:
> This is really a query to determine the best, if there is one, way to
> unpack comp.mail.maps files, since they seem to come in every day or
> so now.
> 
Here at BGSU we have an unbatching scheme as follows:

In "/usr/lib/crontab" we have the following line:

30 5 2,9,16,23,30 * *	news	/usr/local/uuhosts -unbatch

"uuhosts" is a script which will automatically unbatch the map files among
other things.  This script looks for a file called "Batch" in the
"/usr/local/lib/news/maps/comp.mail.maps" directory.  In this file is a
list of maps which have been received and not unbatched yet.  "uuhosts"
then uses this list to unbatch the files.  Another file which comes in
to play is /usr/lib/news/sys which on our system looks like:

tut:oh,net,world,comp,news,sci,rec,misc,soc,talk,usa,na,mod,to,to.tut:LF:/usr/spool/batch/tut.cis.ohio-state.edu
#osu-cis:oh,net,world,comp,news,sci,rec,misc,soc,talk,usa,na,mod,to,to.osu-cis:LB:
bgsuvax:net,world,comp,news,sci,rec,misc,soc,talk,oh,usa,na,mod,local,alt,to,to.bgsuvax::
bgsu-stu:local,to,to.bgsu-stu::
maps:world,comp.mail.maps:F:/usr/local/lib/news/maps/comp.mail.maps/Batch

This file is the SYSTEM subscription list.  Only those newsgroups that you 
specify will be the ones that your site receives.  For example, bgsu-stu
only receives local stuff since we don't want students reading news on
this system. 

Once the files are "unbatched" (which uuhosts does by
calling itself with the "-x" option), uuhosts calls /usr/local/undo_mail
which looks like this:

----------------- Start of undo_mail ---------------------
#! /bin/sh 
cd /usr/local/lib/news/maps/comp.mail.maps
../pathalias -dcbatt u* d* 2> /tmp/path_error | sort > paths 
cp /usr/local/lib/nmail.paths /usr/local/lib/opaths
mv ./paths /usr/local/lib/nmail.paths
cd /usr/local/lib
chmod 644 nmail.where
./uuwhere
chmod a-w nmail.where
chmod a+r nmail.paths
mail -s "Pathalias errors" someuser < /tmp/path_error
----------------- End of undo_mail -----------------------

I hope this helps some.  You should be able to get these scripts
and executables off the net.  They should be archived in one of the
sources news groups.

Any questions e-mail to me.  Good luck.


Kenneth J. Kutz         	CSNET kutz@bgsu.edu
Unix Systems Programmer		UUCP  ...!osu-cis!bgsuvax!kutz
Bowling Green State Univ.       PHONE (419) 372-7493