emv@mailrus.cc.umich.edu (Edward Vielmetti) (10/31/88)
I'm too lazy to polish this up for a real distribution, so I thought I'd dump it out on alt.sources. These are perl sources cobbled from Joe Buck's posting of several months ago to comp.sources.misc that help in the process of making usenet maps that correspond to the reality of how articles are transported around, not the officially sanctioned view of who is supposed to be connected to who. The biggest difference between my stuff and Joe's is that I have split up the single program of his into several filters that do different things. Here's a short summary: hiarts takes history file, emits article names artpaths takes article names, emits Path: lines pairs takes path lines, emits pair and hop info plength takes path lines, emits hop aggregate report plength is kind of dumb about things, and I would like to go back and redo at least some of them in something other than perl, since it's kind of hard to convince people to compile perl just so you can look at their connectivity. The usual way I run this is like so: hipaths -100 history | pairs | sort +nr which shows a nice summary of the last hour's traffic. Here's one from now: 41 4.17 mailrus!ames 40 4.08 ames!pasteur 28 4.21 pasteur!ucbvax 21 4.90 mailrus!tut.cis.ohio-state.edu 20 4.55 tut.cis.ohio-state.edu!rutgers 17 4.94 mailrus!ncar 14 6.29 ncar!tank 13 6.38 tank!uxc 13 6.38 uxc!uxc.cso.uiuc.edu 10 3.50 mailrus!cornell 10 6.00 uxc.cso.uiuc.edu!m.cs.uiuc.edu 8 3.38 cornell!batcomputer 7 3.14 pasteur!cory.Berkeley.EDU 7 4.57 rutgers!mcnc 6 4.83 ncar!ames only the top 15 lines of this one for brevity. If you have Joe's stuff you'll see that the new feature is a hop count indication. This is a metric of the average # of hops that an article passing through any particular connection takes to reach your site. Long paths around here are 10-12, ordinary ones are about what you see here. Some creative awkery can make this even more interesting. Grep all of yesterday's articles out of the history file, run them through hiarts | artpaths to get the paths, then take that and run it through "awk -F\! 'NF>15'". You'll see only the paths that articles that are generated very far away from you. Wish list: turn this into input for netmap(1) get a per-article delay metric generate some sort of postscript, netmap or no netmap Watching these snapshots can provide very interesting information about network connectivity, like noticing that a lot of news goes through sun!pitstop!sundc!seismo and also through hal!nic.MR.NET!xanth!mcnc. Anyway, if anyone want to look at some raw data, check the anonymous FTP area on mailrus.cc.umich.edu for some paths. --Ed # This is a shell archive. # Remove everything above and including the cut line. # Then run the rest of the file through sh. -----cut here-----cut here-----cut here-----cut here----- #!/bin/sh # shar: Shell Archiver # Run the following text with /bin/sh to create: # hiarts # artpaths # hipaths # pairs # plength # This archive created: Sun Oct 30 20:53:28 1988 cat << \SHAR_EOF > hiarts #!/usr/local/bin/perl while(<>) { ($id, $date, $time, $file) = split (' '); next if ($file eq 'cancelled' || $file eq '') ; ($ng, $n) = split (/\//, $file); $file =~ tr%.%/%; print "$file\n"; } SHAR_EOF cat << \SHAR_EOF > artpaths #!/usr/local/bin/perl die "Can't cd to news spool directory!\n" unless chdir ("/usr/spool/news/news"); $np = $nlocal = 0 ; while (<>) { next unless open (art, $_); while (<art>) { ($htype, $hvalue) = split (' '); if ($htype eq "Path:") { print "$hvalue\n" ; last ; } } } SHAR_EOF cat << \SHAR_EOF > hipaths #!/bin/csh -f set history=/usr/spool/news/lib/news/$2 tail $1 $history | hiarts | artpaths SHAR_EOF cat << \SHAR_EOF > pairs #!/usr/local/bin/perl while(<>) { @path = split (/!/) ; if ($#path < 2) { $nlocal++ ; last;} for ($i = 0 ; $i < $#path - 1 ; $i++) { $pair = $path[$i] . "!" . $path[$i+1]; $pcount{$pair} += 1; $plength{$pair} += $#path - 1 ; } } while (($pair, $n) = each (pcount)) { printf ("%6d %5.2f %s\n", $n, ($plength{$pair}/$n), $pair); } close(sort); SHAR_EOF cat << \SHAR_EOF > plength #!/usr/local/bin/perl while(<>) { @path = split (/!/) ; $plength[$#path-1]++ ; } for ($i = 1 ; $i < 40 ; $i++) { printf ("%2d %6d\n",$i, $plength[$i]); } SHAR_EOF # End of shell archive exit 0