[news.misc] D-D-Duplicated Articles

sewilco@mecc.UUCP (Scot E. Wilcoxon) (11/25/86)

[What is the regular expression for an empty line?  Wanted for csplit.]

Some sites are renaming news articles with different Message-IDs.  After
reading four copies of the same article in the same newsgroup, and the same
day a net neighbor complained of getting articles he'd already seen, I scanned
for renumbered articles.

My theory was that some gateways were relabeling some articles.  If a specific
gateway or network was having repeated problems with an article it might show
up with several IDs from that gateway.

This information needs interpretation by people with better knowledge of the
gateways and the networks being served.

The Path of several duplicated articles contained ucbvax.berkely.edu several
times.  I assume it is an ARPANET artifact, but don't know if it indicates
a genuine problem (might it be a mail loop?).

====

The first-draft script for this is at the end of the article.  It assumes that
articles with the same Subject and article body checksum are the same article.

About 24 days of news took 13 hours in a Tower 32 (12 hours in Subjecter,
which can be sped up with 'for Path in $*' and removing the option on xargs
in IDerr).  Continued reports could be done by adding to IDlist instead of
rescanning all files.

====

Here is a list of mangled IDs.  They are in groups representing each
retransmitted article.  One of the IDs might be the original (correct) one
for the article.

Message-ID:_<1006@brl-adm.ARPA>
Message-ID:_<1017@brl-adm.ARPA>
Message-ID:_<1024@brl-adm.ARPA>

Message-ID:_<837@brl-adm.ARPA>
Message-ID:_<844@brl-adm.ARPA>
Message-ID:_<845@brl-adm.ARPA>

Message-ID:_<4960@brl-smoke.ARPA>
Message-ID:_<8610280935.AA07453@ucbvax>

Message-ID:_<8474ZSR@PSUVMB>
Message-ID:_<8475ZSR@PSUVMB>

Message-ID:_<8610292329.AA25412@EDDIE>
Message-ID:_<8610300249.AA28699@EDDIE>

Message-ID:_<472@pedsgo.UUCP>
Message-ID:_<473@pedsgo.UUCP>

Message-ID:_<734@svax.cs.cornell.edu>
Message-ID:_<735@svax.cs.cornell.edu>

Message-ID:_<6235@decwrl.DEC.COM>
Message-ID:_<6237@decwrl.DEC.COM>

Message-ID:_<595@mtfmp.UUCP>
Message-ID:_<596@mtfmp.UUCP>
Message-ID:_<597@mtfmp.UUCP>

Message-ID:_<150@tiger.Princeton.EDU>
Message-ID:_<151@tiger.Princeton.EDU>

Message-ID:_<6278@decwrl.DEC.COM>
Message-ID:_<6280@decwrl.DEC.COM>
Message-ID:_<6281@decwrl.DEC.COM>
Message-ID:_<6282@decwrl.DEC.COM>

Message-ID:_<233@wolf.UUCP>
Message-ID:_<234@wolf.UUCP>

Message-ID:_<2657@princeton.UUCP>
Message-ID:_<2658@princeton.UUCP>
Message-ID:_<2659@princeton.UUCP>
Message-ID:_<2660@princeton.UUCP>

Message-ID:_<8611010231.AA15050@ucbvax.berkeley.edu>
Message-ID:_<8611040133.AA23867@ucbvax.Berkeley.EDU>

Message-ID:_<8611170029.AA22642@ucbvax.Berkeley.EDU>
Message-ID:_<8611170136.AA23725@ucbvax.Berkeley.EDU>

Message-ID:_<8611180352.AA17075@ucbvax.Berkeley.EDU>
Message-ID:_<8611181349.AA24614@ucbvax.Berkeley.EDU>

Message-ID:_<8611060337.AA05896@ucbvax.Berkeley.EDU>
Message-ID:_<8611061506.AA14652@ucbvax.Berkeley.EDU>

Message-ID:_<1738@iuvax.UUCP>
Message-ID:_<1739@iuvax.UUCP>

Message-ID:_<2199@orca.TEK.COM>
Message-ID:_<2201@orca.TEK.COM>

Message-ID:_<5349@brl-smoke.ARPA>
Message-ID:_<5352@brl-smoke.ARPA>

Message-ID:_<5112@brl-smoke.ARPA>
Message-ID:_<5119@brl-smoke.ARPA>

Message-ID:_<176@unc.unc.UUCP>
Message-ID:_<178@unc.unc.UUCP>

Message-ID:_<5183@brl-smoke.ARPA>
Message-ID:_<5184@brl-smoke.ARPA>

Message-ID:_<990@hounx.UUCP>
Message-ID:_<992@hounx.UUCP>

Message-ID:_<2506@cisunx.UUCP>
Message-ID:_<2507@cisunx.UUCP>

Message-ID:_<1004@brl-adm.ARPA>
Message-ID:_<1018@brl-adm.ARPA>

Message-ID:_<8611090224.AA07999@ucbvax.Berkeley.EDU>
Message-ID:_<8611090224.AA07>99@ucbvax.Berkeley.EDU>

Message-ID:_<290@rutgers.RUTGERS.EDU>
Message-ID:_<291@rutgers.RUTGERS.EDU>

Message-ID:_<8611190128.AA06181@ucbvax.Berkeley.EDU>
Message-ID:_<8611190605.AA09925@ucbvax.Berkeley.EDU>

Message-ID:_<6694@topaz.RUTGERS.EDU>
Message-ID:_<6695@topaz.RUTGERS.EDU>

Message-ID:_<1986Nov12.194941.17785@utfyzx.uucp>
Message-ID:_<1986Nov12.195205.17851@utfyzx.uucp>

Message-ID:_<1005@brl-adm.ARPA>
Message-ID:_<1019@brl-adm.ARPA>
Message-ID:_<1560@batcomputer.tn.cornell.edu>

Message-ID:_<519@instable.UUCP>
Message-ID:_<520@instable.UUCP>

Message-ID:_<1016@brl-adm.ARPA>
Message-ID:_<1026@brl-adm.ARPA>
Message-ID:_<8611211445.aa17743@SEM.BRL.ARPA>

Message-ID:_<4935@brl-smoke.ARPA>
Message-ID:_<4936@brl-smoke.ARPA>

Message-ID:_<4938@brl-smoke.ARPA>
Message-ID:_<4939@brl-smoke.ARPA>

Message-ID:_<148@tiger.Princeton.EDU>
Message-ID:_<149@tiger.Princeton.EDU>

Message-ID:_<12321@alice.UUCP>
Message-ID:_<12321@placeholder.UUCP>

Message-ID:_<5085@brl-smoke.ARPA>
Message-ID:_<5122@brl-smoke.ARPA>

Message-ID:_<8610271726.AA12550@ucbvax.Berkeley.EDU>
Message-ID:_<8610302158.AA11259@ucbvax.berkeley.edu>

Message-ID:_<8610271727.AA12580@ucbvax.Berkeley.EDU>
Message-ID:_<8610300648.AA27256@ucbvax.berkeley.edu>

Message-ID:_<8610271726.AA12558@ucbvax.Berkeley.EDU>
Message-ID:_<8610300552.AA26636@ucbvax.berkeley.edu>

Message-ID:_<8610271727.AA12570@ucbvax.Berkeley.EDU>
Message-ID:_<8610300646.AA27231@ucbvax.berkeley.edu>

Message-ID:_<8610271725.AA12539@ucbvax.Berkeley.EDU>
Message-ID:_<8610300549.AA26609@ucbvax.berkeley.edu>

Message-ID:_<217@ritcv.UUCP>
Message-ID:_<218@ritcv.UUCP>
Message-ID:_<219@ritcv.UUCP>

Message-ID:_<8611211847.AA19294@jade.Berkeley.Edu>
Message-ID:_<8611211848.AA19314@jade.Berkeley.Edu>

Message-ID:_<1358@utastro.UUCP>
Message-ID:_<1359@utastro.UUCP>

Message-ID:_<305@rutgers.RUTGERS.EDU>
Message-ID:_<309@rutgers.RUTGERS.EDU>

Message-ID:_<1820001@wdl1.UUCP>
Message-ID:_<2790002@wdl1.UUCP>
Message-ID:_<3240004@wdl1.UUCP>

Message-ID:_<306@rutgers.RUTGERS.EDU>
Message-ID:_<307@rutgers.RUTGERS.EDU>

Message-ID:_<8611020421.AA01029@ucbvax.berkeley.edu>
Message-ID:_<8611081158.AA28068@ucbvax.Berkeley.EDU>

Message-ID:_<8610310414.AA18796@ucbvax.berkeley.edu>
Message-ID:_<8610310518.AA20050@ucbvax.berkeley.edu>

Message-ID:_<269@humu.UUCP>
Message-ID:_<270@humu.UUCP>

Message-ID:_<928@brl-adm.ARPA>
Message-ID:_<929@brl-adm.ARPA>

Message-ID:_<8611191732.AA06820@ucbvax.Berkeley.EDU>
Message-ID:_<8611191753.AA25892@Zarathustra.Think.COM>

Message-ID:_<904@brl-adm.ARPA>
Message-ID:_<913@brl-adm.ARPA>

Message-ID:_<8611021901.AA09382@ucbvax.berkeley.edu>
Message-ID:_<8611041230.AA05003@ucbvax.Berkeley.EDU>

Message-ID:_<8611071210.AA06323@ucbvax.Berkeley.EDU>
Message-ID:_<8611110805.AA15533@ucbvax.Berkeley.EDU>

----- Joined.sites -----

This is the above list grouped by site.  It was produced by rev | sort | rev.

Message-ID:_<8611211445.aa17743@SEM.BRL.ARPA>
Message-ID:_<4960@brl-smoke.ARPA>
Message-ID:_<5112@brl-smoke.ARPA>
Message-ID:_<5122@brl-smoke.ARPA>
Message-ID:_<5352@brl-smoke.ARPA>
Message-ID:_<5183@brl-smoke.ARPA>
Message-ID:_<5184@brl-smoke.ARPA>
Message-ID:_<4935@brl-smoke.ARPA>
Message-ID:_<5085@brl-smoke.ARPA>
Message-ID:_<4936@brl-smoke.ARPA>
Message-ID:_<4938@brl-smoke.ARPA>
Message-ID:_<5119@brl-smoke.ARPA>
Message-ID:_<4939@brl-smoke.ARPA>
Message-ID:_<5349@brl-smoke.ARPA>
Message-ID:_<913@brl-adm.ARPA>
Message-ID:_<1004@brl-adm.ARPA>
Message-ID:_<904@brl-adm.ARPA>
Message-ID:_<1024@brl-adm.ARPA>
Message-ID:_<844@brl-adm.ARPA>
Message-ID:_<1005@brl-adm.ARPA>
Message-ID:_<845@brl-adm.ARPA>
Message-ID:_<1006@brl-adm.ARPA>
Message-ID:_<1016@brl-adm.ARPA>
Message-ID:_<1026@brl-adm.ARPA>
Message-ID:_<1017@brl-adm.ARPA>
Message-ID:_<837@brl-adm.ARPA>
Message-ID:_<1018@brl-adm.ARPA>
Message-ID:_<928@brl-adm.ARPA>
Message-ID:_<1019@brl-adm.ARPA>
Message-ID:_<929@brl-adm.ARPA>
Message-ID:_<8474ZSR@PSUVMB>
Message-ID:_<8475ZSR@PSUVMB>
Message-ID:_<8610292329.AA25412@EDDIE>
Message-ID:_<8610300249.AA28699@EDDIE>
Message-ID:_<6280@decwrl.DEC.COM>
Message-ID:_<6281@decwrl.DEC.COM>
Message-ID:_<6282@decwrl.DEC.COM>
Message-ID:_<6235@decwrl.DEC.COM>
Message-ID:_<6237@decwrl.DEC.COM>
Message-ID:_<6278@decwrl.DEC.COM>
Message-ID:_<2201@orca.TEK.COM>
Message-ID:_<2199@orca.TEK.COM>
Message-ID:_<8611191753.AA25892@Zarathustra.Think.COM>
Message-ID:_<1820001@wdl1.UUCP>
Message-ID:_<2790002@wdl1.UUCP>
Message-ID:_<3240004@wdl1.UUCP>
Message-ID:_<176@unc.unc.UUCP>
Message-ID:_<178@unc.unc.UUCP>
Message-ID:_<12321@alice.UUCP>
Message-ID:_<520@instable.UUCP>
Message-ID:_<519@instable.UUCP>
Message-ID:_<233@wolf.UUCP>
Message-ID:_<234@wolf.UUCP>
Message-ID:_<2660@princeton.UUCP>
Message-ID:_<2657@princeton.UUCP>
Message-ID:_<2658@princeton.UUCP>
Message-ID:_<2659@princeton.UUCP>
Message-ID:_<472@pedsgo.UUCP>
Message-ID:_<473@pedsgo.UUCP>
Message-ID:_<1358@utastro.UUCP>
Message-ID:_<1359@utastro.UUCP>
Message-ID:_<595@mtfmp.UUCP>
Message-ID:_<596@mtfmp.UUCP>
Message-ID:_<597@mtfmp.UUCP>
Message-ID:_<12321@placeholder.UUCP>
Message-ID:_<270@humu.UUCP>
Message-ID:_<269@humu.UUCP>
Message-ID:_<217@ritcv.UUCP>
Message-ID:_<218@ritcv.UUCP>
Message-ID:_<219@ritcv.UUCP>
Message-ID:_<1738@iuvax.UUCP>
Message-ID:_<1739@iuvax.UUCP>
Message-ID:_<990@hounx.UUCP>
Message-ID:_<992@hounx.UUCP>
Message-ID:_<2506@cisunx.UUCP>
Message-ID:_<2507@cisunx.UUCP>
Message-ID:_<290@rutgers.RUTGERS.EDU>
Message-ID:_<291@rutgers.RUTGERS.EDU>
Message-ID:_<305@rutgers.RUTGERS.EDU>
Message-ID:_<306@rutgers.RUTGERS.EDU>
Message-ID:_<307@rutgers.RUTGERS.EDU>
Message-ID:_<309@rutgers.RUTGERS.EDU>
Message-ID:_<6694@topaz.RUTGERS.EDU>
Message-ID:_<6695@topaz.RUTGERS.EDU>
Message-ID:_<150@tiger.Princeton.EDU>
Message-ID:_<151@tiger.Princeton.EDU>
Message-ID:_<148@tiger.Princeton.EDU>
Message-ID:_<149@tiger.Princeton.EDU>
Message-ID:_<8611191732.AA06820@ucbvax.Berkeley.EDU>
Message-ID:_<8610271726.AA12550@ucbvax.Berkeley.EDU>
Message-ID:_<8610271727.AA12570@ucbvax.Berkeley.EDU>
Message-ID:_<8610271727.AA12580@ucbvax.Berkeley.EDU>
Message-ID:_<8611190128.AA06181@ucbvax.Berkeley.EDU>
Message-ID:_<8611170029.AA22642@ucbvax.Berkeley.EDU>
Message-ID:_<8611061506.AA14652@ucbvax.Berkeley.EDU>
Message-ID:_<8611041230.AA05003@ucbvax.Berkeley.EDU>
Message-ID:_<8611071210.AA06323@ucbvax.Berkeley.EDU>
Message-ID:_<8611110805.AA15533@ucbvax.Berkeley.EDU>
Message-ID:_<8611181349.AA24614@ucbvax.Berkeley.EDU>
Message-ID:_<8611170136.AA23725@ucbvax.Berkeley.EDU>
Message-ID:_<8611190605.AA09925@ucbvax.Berkeley.EDU>
Message-ID:_<8611180352.AA17075@ucbvax.Berkeley.EDU>
Message-ID:_<8611060337.AA05896@ucbvax.Berkeley.EDU>
Message-ID:_<8611040133.AA23867@ucbvax.Berkeley.EDU>
Message-ID:_<8610271726.AA12558@ucbvax.Berkeley.EDU>
Message-ID:_<8611081158.AA28068@ucbvax.Berkeley.EDU>
Message-ID:_<8610271725.AA12539@ucbvax.Berkeley.EDU>
Message-ID:_<8611090224.AA07999@ucbvax.Berkeley.EDU>
Message-ID:_<8611090224.AA07>99@ucbvax.Berkeley.EDU>
Message-ID:_<1986Nov12.195205.17851@utfyzx.uucp>
Message-ID:_<1986Nov12.194941.17785@utfyzx.uucp>
Message-ID:_<8611211848.AA19314@jade.Berkeley.Edu>
Message-ID:_<8611211847.AA19294@jade.Berkeley.Edu>
Message-ID:_<1560@batcomputer.tn.cornell.edu>
Message-ID:_<734@svax.cs.cornell.edu>
Message-ID:_<735@svax.cs.cornell.edu>
Message-ID:_<8610310518.AA20050@ucbvax.berkeley.edu>
Message-ID:_<8611010231.AA15050@ucbvax.berkeley.edu>
Message-ID:_<8610300646.AA27231@ucbvax.berkeley.edu>
Message-ID:_<8611021901.AA09382@ucbvax.berkeley.edu>
Message-ID:_<8610300552.AA26636@ucbvax.berkeley.edu>
Message-ID:_<8610300648.AA27256@ucbvax.berkeley.edu>
Message-ID:_<8610310414.AA18796@ucbvax.berkeley.edu>
Message-ID:_<8610300549.AA26609@ucbvax.berkeley.edu>
Message-ID:_<8611020421.AA01029@ucbvax.berkeley.edu>
Message-ID:_<8610302158.AA11259@ucbvax.berkeley.edu>
Message-ID:_<8610280935.AA07453@ucbvax>

----- Joined -----

This is the original output from the script.

Subject:_A_strictly_legal_question...	47572_2_item01	Message-ID:_<1006@brl-adm.ARPA>
Subject:_A_strictly_legal_question...	47572_2_item01	Message-ID:_<1017@brl-adm.ARPA>
Subject:_A_strictly_legal_question...	47572_2_item01	Message-ID:_<1024@brl-adm.ARPA>

Subject:_Addition_to_the_distribution_list_and_help_needed	37449_2_item01	Message-ID:_<837@brl-adm.ARPA>
Subject:_Addition_to_the_distribution_list_and_help_needed	37449_2_item01	Message-ID:_<844@brl-adm.ARPA>
Subject:_Addition_to_the_distribution_list_and_help_needed	37449_2_item01	Message-ID:_<845@brl-adm.ARPA>

Subject:_Apple_CP/M_and_RAMdrives	31217_2_item01	Message-ID:_<4960@brl-smoke.ARPA>
Subject:_Apple_CP/M_and_RAMdrives	31217_2_item01	Message-ID:_<8610280935.AA07453@ucbvax>

Subject:_BITBLT_Line_'A'_Use	63104_2_item01	Message-ID:_<8474ZSR@PSUVMB>
Subject:_BITBLT_Line_'A'_Use	63104_2_item01	Message-ID:_<8475ZSR@PSUVMB>

Subject:_Bill_Hsu,_I_got_your_message	20778_1_item01	Message-ID:_<8610292329.AA25412@EDDIE>
Subject:_Bill_Hsu,_I_got_your_message	20778_1_item01	Message-ID:_<8610300249.AA28699@EDDIE>

Subject:_CZ-101_&_MIDI_Programming	51417_2_item01	Message-ID:_<472@pedsgo.UUCP>
Subject:_CZ-101_&_MIDI_Programming	51417_2_item01	Message-ID:_<473@pedsgo.UUCP>

Subject:_Cycling_in_Italy	8116_1_item01	Message-ID:_<734@svax.cs.cornell.edu>
Subject:_Cycling_in_Italy	8116_1_item01	Message-ID:_<735@svax.cs.cornell.edu>

Subject:_Epson_MX80_graphics_chips	57764_1_item01	Message-ID:_<6235@decwrl.DEC.COM>
Subject:_Epson_MX80_graphics_chips	57764_1_item01	Message-ID:_<6237@decwrl.DEC.COM>

Subject:_Experience_with_4GLs?	39298_1_item01	Message-ID:_<595@mtfmp.UUCP>
Subject:_Experience_with_4GLs?	39298_1_item01	Message-ID:_<596@mtfmp.UUCP>
Subject:_Experience_with_4GLs?	39298_1_item01	Message-ID:_<597@mtfmp.UUCP>

Subject:_Help:_Bomb_Recovery_Applications_needed.	26528_1_item01	Message-ID:_<150@tiger.Princeton.EDU>
Subject:_Help:_Bomb_Recovery_Applications_needed.	26528_1_item01	Message-ID:_<151@tiger.Princeton.EDU>

Subject:_How_to_do_it_?	5418_1_item01	Message-ID:_<6278@decwrl.DEC.COM>
Subject:_How_to_do_it_?	5418_1_item01	Message-ID:_<6280@decwrl.DEC.COM>
Subject:_How_to_do_it_?	5418_1_item01	Message-ID:_<6281@decwrl.DEC.COM>
Subject:_How_to_do_it_?	5418_1_item01	Message-ID:_<6282@decwrl.DEC.COM>

Subject:_I'm_beginning_to_like_Leo	39037_12_item01	Message-ID:_<233@wolf.UUCP>
Subject:_I'm_beginning_to_like_Leo	39037_12_item01	Message-ID:_<234@wolf.UUCP>

Subject:_I/O_benchmarks	28369_1_item01	Message-ID:_<2657@princeton.UUCP>
Subject:_I/O_benchmarks	28369_1_item01	Message-ID:_<2658@princeton.UUCP>
Subject:_I/O_benchmarks	28369_1_item01	Message-ID:_<2659@princeton.UUCP>
Subject:_I/O_benchmarks	28369_1_item01	Message-ID:_<2660@princeton.UUCP>

Subject:_INFO-ATARI16_Digest_V86_#01	51873_31_item01	Message-ID:_<8611010231.AA15050@ucbvax.berkeley.edu>
Subject:_INFO-ATARI16_Digest_V86_#01	51873_31_item01	Message-ID:_<8611040133.AA23867@ucbvax.Berkeley.EDU>

Subject:_INFO-MAC_Digest___V5_#11	49846_29_item01	Message-ID:_<8611170029.AA22642@ucbvax.Berkeley.EDU>
Subject:_INFO-MAC_Digest___V5_#11	49846_29_item01	Message-ID:_<8611170136.AA23725@ucbvax.Berkeley.EDU>

Subject:_Info-IBMPC_Digest_V5_#101	26156_40_item01	Message-ID:_<8611180352.AA17075@ucbvax.Berkeley.EDU>
Subject:_Info-IBMPC_Digest_V5_#101	26156_40_item01	Message-ID:_<8611181349.AA24614@ucbvax.Berkeley.EDU>

Subject:_Info-IBMPC_Digest_V5_#98	43663_27_item01	Message-ID:_<8611060337.AA05896@ucbvax.Berkeley.EDU>
Subject:_Info-IBMPC_Digest_V5_#98	43663_27_item01	Message-ID:_<8611061506.AA14652@ucbvax.Berkeley.EDU>

Subject:_Laser_Printer_Deal?	64766_1_item01	Message-ID:_<1738@iuvax.UUCP>
Subject:_Laser_Printer_Deal?	64766_1_item01	Message-ID:_<1739@iuvax.UUCP>

Subject:_Laser_communications	16337_1_item01	Message-ID:_<2199@orca.TEK.COM>
Subject:_Laser_communications	16337_1_item01	Message-ID:_<2201@orca.TEK.COM>

Subject:_N*_Advantage_=_Dumb_terminal?	830_2_item01	Message-ID:_<5349@brl-smoke.ARPA>
Subject:_N*_Advantage_=_Dumb_terminal?	830_2_item01	Message-ID:_<5352@brl-smoke.ARPA>

Subject:_New_program_to_read_MSDOS_disks_on_Commodore_C128_CP/M	43526_1_item01	Message-ID:_<5112@brl-smoke.ARPA>
Subject:_New_program_to_read_MSDOS_disks_on_Commodore_C128_CP/M	43526_1_item01	Message-ID:_<5119@brl-smoke.ARPA>

Subject:_OSSI:_SIConversions	51274_12_item01	Message-ID:_<176@unc.unc.UUCP>
Subject:_OSSI:_SIConversions	51274_12_item01	Message-ID:_<178@unc.unc.UUCP>

Subject:_Please_add_us	15304_1_item01	Message-ID:_<5183@brl-smoke.ARPA>
Subject:_Please_add_us	15304_1_item01	Message-ID:_<5184@brl-smoke.ARPA>

Subject:_Pogo	22291_1_item01	Message-ID:_<990@hounx.UUCP>
Subject:_Pogo	22291_1_item01	Message-ID:_<992@hounx.UUCP>

Subject:_Post-Machel_Mozambique	26194_2_item01	Message-ID:_<2506@cisunx.UUCP>
Subject:_Post-Machel_Mozambique	26194_2_item01	Message-ID:_<2507@cisunx.UUCP>

Subject:_Re:_A_strictly_legal_question...	15912_2_item01	Message-ID:_<1004@brl-adm.ARPA>
Subject:_Re:_A_strictly_legal_question...	15912_2_item01	Message-ID:_<1018@brl-adm.ARPA>

Subject:_Re:_Cable_TV_info_requested	4698_1_item01	Message-ID:_<8611090224.AA07999@ucbvax.Berkeley.EDU>
Subject:_Re:_Cable_TV_info_requested	4698_1_item01	Message-ID:_<8611090224.AA07>99@ucbvax.Berkeley.EDU>

Subject:_Re:_Canonical_SF_Music_List	21636_4_item01	Message-ID:_<290@rutgers.RUTGERS.EDU>
Subject:_Re:_Canonical_SF_Music_List	21636_4_item01	Message-ID:_<291@rutgers.RUTGERS.EDU>

Subject:_Re:_EGP_madness_(net_127)	39485_1_item01	Message-ID:_<8611190128.AA06181@ucbvax.Berkeley.EDU>
Subject:_Re:_EGP_madness_(net_127)	39485_1_item01	Message-ID:_<8611190605.AA09925@ucbvax.Berkeley.EDU>

Subject:_Re:_God's_use_of_his_power	54472_9_item01	Message-ID:_<6694@topaz.RUTGERS.EDU>
Subject:_Re:_God's_use_of_his_power	54472_9_item01	Message-ID:_<6695@topaz.RUTGERS.EDU>

Subject:_Re:_HP-UX_5.1_on_HP9000_Series_500	1143_3_item01	Message-ID:_<1986Nov12.194941.17785@utfyzx.uucp>
Subject:_Re:_HP-UX_5.1_on_HP9000_Series_500	1143_3_item01	Message-ID:_<1986Nov12.195205.17851@utfyzx.uucp>

Subject:_Re:_Is_this_Mail_Fraud?	9095_3_item01	Message-ID:_<1005@brl-adm.ARPA>
Subject:_Re:_Is_this_Mail_Fraud?	9095_3_item01	Message-ID:_<1019@brl-adm.ARPA>
Subject:_Re:_Is_this_Mail_Fraud?	9095_3_item01	Message-ID:_<1560@batcomputer.tn.cornell.edu>

Subject:_Re:_Shocking,_just_shocking......._(Iran)	54910_1_item01	Message-ID:_<519@instable.UUCP>
Subject:_Re:_Shocking,_just_shocking......._(Iran)	54910_1_item01	Message-ID:_<520@instable.UUCP>

Subject:_Re:__A_strictly_legal_question...	9461_2_item01	Message-ID:_<1016@brl-adm.ARPA>
Subject:_Re:__A_strictly_legal_question...	9461_2_item01	Message-ID:_<1026@brl-adm.ARPA>
Subject:_Re:__A_strictly_legal_question...	9461_2_item01	Message-ID:_<8611211445.aa17743@SEM.BRL.ARPA>

Subject:_Re:__NS32032_and_UNIX	62328_1_item01	Message-ID:_<4935@brl-smoke.ARPA>
Subject:_Re:__NS32032_and_UNIX	62328_1_item01	Message-ID:_<4936@brl-smoke.ARPA>

Subject:_Re:__Source_for_4.3BSD	33437_1_item01	Message-ID:_<4938@brl-smoke.ARPA>
Subject:_Re:__Source_for_4.3BSD	33437_1_item01	Message-ID:_<4939@brl-smoke.ARPA>

Subject:_Red_Rider_Host_and_modem<->XL_hookup_question.	50756_1_item01	Message-ID:_<148@tiger.Princeton.EDU>
Subject:_Red_Rider_Host_and_modem<->XL_hookup_question.	50756_1_item01	Message-ID:_<149@tiger.Princeton.EDU>

Subject:_Rescued_puppies_and_adult_dogs_need_homes_-_URGENT!	441_1_item01	Message-ID:_<12321@alice.UUCP>
Subject:_Rescued_puppies_and_adult_dogs_need_homes_-_URGENT!	441_1_item01	Message-ID:_<12321@placeholder.UUCP>

Subject:_SIGMA_Prolog_from_LPA	25960_1_item01	Message-ID:_<5085@brl-smoke.ARPA>
Subject:_SIGMA_Prolog_from_LPA	25960_1_item01	Message-ID:_<5122@brl-smoke.ARPA>

Subject:_Searle,_Turing,_Symbols,_Categories_(Reply_to_Cugini)	16956_8_item01	Message-ID:_<8610271726.AA12550@ucbvax.Berkeley.EDU>
Subject:_Searle,_Turing,_Symbols,_Categories_(Reply_to_Cugini)	16956_8_item01	Message-ID:_<8610302158.AA11259@ucbvax.berkeley.edu>

Subject:_Searle,_Turing,_Symbols,_Categories_(Reply_to_Danny_Paul)	27088_2_item01	Message-ID:_<8610271727.AA12580@ucbvax.Berkeley.EDU>
Subject:_Searle,_Turing,_Symbols,_Categories_(Reply_to_Danny_Paul)	27088_2_item01	Message-ID:_<8610300648.AA27256@ucbvax.berkeley.edu>

Subject:_Searle,_Turing,_Symbols,_Categories_(Reply_to_Kalish)	43319_5_item01	Message-ID:_<8610271726.AA12558@ucbvax.Berkeley.EDU>
Subject:_Searle,_Turing,_Symbols,_Categories_(Reply_to_Kalish)	43319_5_item01	Message-ID:_<8610300552.AA26636@ucbvax.berkeley.edu>

Subject:_Searle,_Turing,_Symbols,_Categories_(Reply_to_Krulwich)	44864_2_item01	Message-ID:_<8610271727.AA12570@ucbvax.Berkeley.EDU>
Subject:_Searle,_Turing,_Symbols,_Categories_(Reply_to_Krulwich)	44864_2_item01	Message-ID:_<8610300646.AA27231@ucbvax.berkeley.edu>

Subject:_Searle,_Turing,_Symbols,_Categories_(reply_to_E._Mozes,_reconstructed)	9800_8_item01	Message-ID:_<8610271725.AA12539@ucbvax.Berkeley.EDU>
Subject:_Searle,_Turing,_Symbols,_Categories_(reply_to_E._Mozes,_reconstructed)	9800_8_item01	Message-ID:_<8610300549.AA26609@ucbvax.berkeley.edu>

Subject:_Slides/Photographs_made_from_Your_Amiga_Pictures!	45286_2_item01	Message-ID:_<217@ritcv.UUCP>
Subject:_Slides/Photographs_made_from_Your_Amiga_Pictures!	45286_2_item01	Message-ID:_<218@ritcv.UUCP>
Subject:_Slides/Photographs_made_from_Your_Amiga_Pictures!	45286_2_item01	Message-ID:_<219@ritcv.UUCP>

Subject:_Software_for_use_in_China	25100_1_item01	Message-ID:_<8611211847.AA19294@jade.Berkeley.Edu>
Subject:_Software_for_use_in_China	25100_1_item01	Message-ID:_<8611211848.AA19314@jade.Berkeley.Edu>

Subject:_StarDate:_October_26_Regulus_and_the_Moon	31655_2_item01	Message-ID:_<1358@utastro.UUCP>
Subject:_StarDate:_October_26_Regulus_and_the_Moon	31655_2_item01	Message-ID:_<1359@utastro.UUCP>

Subject:_Star_Wars	8691_2_item01	Message-ID:_<305@rutgers.RUTGERS.EDU>
Subject:_Star_Wars	8691_2_item01	Message-ID:_<309@rutgers.RUTGERS.EDU>

Subject:_State_of_the_Net_Address	52044_3_item01	Message-ID:_<1820001@wdl1.UUCP>
Subject:_State_of_the_Net_Address	52044_3_item01	Message-ID:_<2790002@wdl1.UUCP>
Subject:_State_of_the_Net_Address	52044_3_item01	Message-ID:_<3240004@wdl1.UUCP>

Subject:_Time_Travel_and_related_problems	24483_2_item01	Message-ID:_<306@rutgers.RUTGERS.EDU>
Subject:_Time_Travel_and_related_problems	24483_2_item01	Message-ID:_<307@rutgers.RUTGERS.EDU>

Subject:_compiler_validation	56555_1_item01	Message-ID:_<8611020421.AA01029@ucbvax.berkeley.edu>
Subject:_compiler_validation	56555_1_item01	Message-ID:_<8611081158.AA28068@ucbvax.Berkeley.EDU>

Subject:_events_in_Stanford_sky	51700_1_item01	Message-ID:_<8610310414.AA18796@ucbvax.berkeley.edu>
Subject:_events_in_Stanford_sky	51700_1_item01	Message-ID:_<8610310518.AA20050@ucbvax.berkeley.edu>

Subject:_programmer/analyst	64764_5_item01	Message-ID:_<269@humu.UUCP>
Subject:_programmer/analyst	64764_5_item01	Message-ID:_<270@humu.UUCP>

Subject:_programming_languages	49711_1_item01	Message-ID:_<928@brl-adm.ARPA>
Subject:_programming_languages	49711_1_item01	Message-ID:_<929@brl-adm.ARPA>

Subject:_re:_Prevalence_of_RDP_implementations?	58667_1_item01	Message-ID:_<8611191732.AA06820@ucbvax.Berkeley.EDU>
Subject:_re:_Prevalence_of_RDP_implementations?	58667_1_item01	Message-ID:_<8611191753.AA25892@Zarathustra.Think.COM>

Subject:_reply-to	50498_2_item01	Message-ID:_<904@brl-adm.ARPA>
Subject:_reply-to	50498_2_item01	Message-ID:_<913@brl-adm.ARPA>

Subject:_st10.x_tech_reports	59549_20_item01	Message-ID:_<8611021901.AA09382@ucbvax.berkeley.edu>
Subject:_st10.x_tech_reports	59549_20_item01	Message-ID:_<8611041230.AA05003@ucbvax.Berkeley.EDU>

Subject:_st9.x_tech_reports	23330_20_item01	Message-ID:_<8611071210.AA06323@ucbvax.Berkeley.EDU>
Subject:_st9.x_tech_reports	23330_20_item01	Message-ID:_<8611110805.AA15533@ucbvax.Berkeley.EDU>

----- Shar -----
#! /bin/sh
# This is a shell archive, meaning:
# 1. Remove everything above the #! /bin/sh line.
# 2. Save the resulting text in a file.
# 3. Execute the file with /bin/sh (not csh) to create the files:
#	IDerr
#	Subjecter
#	Joiner.awk
# This archive created: Sun Nov 23 12:05:00 1986
export PATH; PATH=/bin:$PATH
if test -f 'IDerr'
then
	echo shar: will not over-write existing file "'IDerr'"
else
cat << \SHAR_EOF > 'IDerr'
# IDerr : Shell script to find articles being issued with 
# different Message-IDs

find /usr/spool/news -type f -print >NEWSlist
rm -f IDlist
xargs -l1 Subjecter <NEWSlist

# IDlist now has a list of Subject, Body, and Message-ID.
# Now to eliminate duplicates due to articles in multiple groups.

sort IDlist >IDlist.sort
rm IDlist
uniq IDlist.sort >IDlist.uniq
rm IDlist.sort

# Next report all articles with the same Subject and Body but different
# Message-ID, as these are probably duplicate articles with mangled IDs.

awk -f Joiner.awk IDlist.uniq >Joined

rm NEWSlist
rm IDlist.uniq
SHAR_EOF
chmod +x 'IDerr'
fi # end of overwriting check
if test -f 'Subjecter'
then
	echo shar: will not over-write existing file "'Subjecter'"
else
cat << \SHAR_EOF > 'Subjecter'
csplit -s -f item $1 '/Lines:/+1'
if [ -s item01 ]
then
	# item00 has headers
	# item01 has body of article
	grep "^Subject:" item00 >xSubject
	grep "^Message-ID:" item00 >xID
	# something to decide uniqeness of body..wc and sum?
	sum item01 >xBODY
	sed 's/ /_/g' xSubject >xsubj
	sed 's/ /_/g' xID >xid
	sed 's/ /_/g' xBODY >xbody
	if [ ! -s xsubj ]
	then
		echo ">none<" >xsubj
	fi
	if [ ! -s xbody ]
	then
		echo ">none<" >xbody
	fi
	paste xsubj xbody xid >>IDlist
fi
rm -f item00 item01 xBODY xSubject xID xsubj xbody xid
SHAR_EOF
chmod +x 'Subjecter'
fi # end of overwriting check
if test -f 'Joiner.awk'
then
	echo shar: will not over-write existing file "'Joiner.awk'"
else
cat << \SHAR_EOF > 'Joiner.awk'
BEGIN	{
	flagdiff=0
	}
{	IDENT=$1 $2
	if(IDENT==prevIDENT) {
		print prevLINE
		flagdiff=1
		}
	else	if(flagdiff==1) {
			print prevLINE
			printf "\n"
			flagdiff=0
			}
	prevLINE=$0
	prevIDENT=IDENT
}
END	{	if(flagdiff==1) {
		print prevLINE
		printf "\n"
		flagdiff=0
		}
	}
SHAR_EOF
fi # end of overwriting check
#	End of shell archive
exit 0
-- 
Scot E. Wilcoxon   Minn Ed Comp Corp  {quest,dayton,meccts}!mecc!sewilco
(612)481-3507           sewilco@MECC.COM       ihnp4!meccts!mecc!sewilco
   "Can you stop yourself from hitting the dashboard unbelted at 30 MPH?
    Can you fall off a three-story building and land safely? 32 ft/sec."