fitz@wang.com (Tom Fitzgerald) (07/31/90)
I'm trying to write a perl script that crunches the output of pathalias
and produces a paths file with lots of shortcuts. Some of the things I
definitely want to cut are:
reduce:
...!site1!...!dom.ain!...
to:
...!site1!dom.ain!...
if site1 is known to be on the Internet and at least 2 hops can be removed,
or if 1 hop can be removed and dom.ain is known to be directly on the Internet.
reduce:
...!site1!...!site2!...
to:
...!site1!site2.dom.ain!...
if site2 is known to be site2.dom.ain from inspection of the paths file, and
at least 2 hops can be removed, etc.
remove:
dom.ain smarthost!....
lines from the paths file entirely, since this is what smail and company
will do anyway.
All shortcuts will be done so as to remove the largest number of hops from
each path, and so no mail passes through machines that it wouldn't have
passed through anyway.
Does anyone have any other suggestions about things that could be excised
from the paths, without any chance of getting things broken?
Also, both these optimizations assume a knowledge of which sites are
directly on the Internet. Right now I'm building this list by hand, from
a map of NEARnet and some comments in the UUCP paths files. If there's
a better source of information, I could use it. (I don't want hosts.txt,
though).
Any other suggestions would be welcome.
---
Tom Fitzgerald Wang Labs fitz@wang.com
1-508-967-5278 Lowell MA, USA ...!uunet!wang!fitzlyndon@cs.athabascau.ca (Lyndon Nerenberg) (07/31/90)
fitz@wang.com (Tom Fitzgerald) writes: >Some of the things I >definitely want to cut are: >reduce: > ...!site1!...!dom.ain!... >to: > ...!site1!dom.ain!... >if site1 is known to be on the Internet and at least 2 hops can be removed, NO! If you are going to use the UUCP transport, please use the published UUCP transport routing tables (aka comp.mail.maps). If you use the above method when sending mail to *.athabascau.ca, your message will cost MORE to send, and will take LONGER to get here? Why? Let me explain ... [ Note: this only applies to UUCP only sites. If you're on the Internet as well a UUCP site, feel free to give priority to whatever transport you like. ] We are one of the *many* registered domains that are not connected to the Internet. Our MX record points at wrl.dec.com (aka decwrl). If you knew about Canadian long distance phone rates, you would know why we only poll them once per day, in the wee hours of the morning. Even then, it costs us around $.45/minute to connect. On the other hand, an excellent UUCP path to us can be had via the ...!ubc-cs!alberta!atha path. A number of sites advertise connections to ubc-cs. Most of these actually travel over the Internet. ubc-cs->alberta is an Internet link, and alberta->atha goes over X.25. The latter path is significantly cheaper for all the sites, and means the difference between me getting your message the day you sent it (usually within a couple of hours), or getting it the next day via our MX forwarder. If you use the map information as it's published, mail will flow smoothly and quickly. If you start second guessing the information the system administrators publish, you'll only cause problems. -- Lyndon Nerenberg VE6BBM / Computing Services / Athabasca University {alberta,cbmvax,mips}!atha!lyndon || lyndon@cs.athabascau.ca Practice Safe Government Use Kingdoms
peter@ficc.ferranti.com (Peter da Silva) (08/01/90)
In article <11@aupair.cs.athabascau.ca> lyndon@cs.athabascau.ca (Lyndon Nerenberg) writes: > If you use the map information as it's published, mail will flow > smoothly and quickly. If you start second guessing the information > the system administrators publish, you'll only cause problems. I'll second that. Mail to ferranti.com via the DNS costs us a bundle (enough that the folks on the third floor are likely to drop UUNET in reaction), but Houston (and Texas in general) has excellent UUCP connectivity. -- Peter da Silva. `-_-' +1 713 274 5180. 'U` <peter@ficc.ferranti.com>
emv@math.lsa.umich.edu (Edward Vielmetti) (08/01/90)
In article <11@aupair.cs.athabascau.ca> lyndon@cs.athabascau.ca (Lyndon Nerenberg) writes:
We are one of the *many* registered domains that are not connected to
the Internet. Our MX record points at wrl.dec.com (aka decwrl)....we
only poll them once per day, in the wee hours of the morning. Even then,
it costs us around $.45/minute to connect. On the other hand, an excellent
UUCP path to us can be had via the ...!ubc-cs!alberta!atha path.
This would argue that you should point your MX forwarder at alberta, if
they are able to do the proper header magic to get them to you unscathed.
(Better for the Canadian content laws too.) In that way you will get
mail the cheapest and best way possible.
--Ed
Edward Vielmetti, U of Michigan math dept <emv@math.lsa.umich.edu>I.G.Batten@fulcrum.bt.co.uk (Ian G Batten) (08/02/90)
lyndon@cs.athabascau.ca (Lyndon Nerenberg) writes: > the Internet. Our MX record points at wrl.dec.com (aka decwrl). If you [...] > it costs us around $.45/minute to connect. On the other hand, an excellent > UUCP path to us can be had via the ...!ubc-cs!alberta!atha path. A number > of sites advertise connections to ubc-cs. Most of these actually travel over > the Internet. ubc-cs->alberta is an Internet link, and alberta->atha goes So why isn't alberta your MX? ian
lyndon@cs.athabascau.ca (Lyndon Nerenberg) (08/03/90)
emv@math.lsa.umich.edu (Edward Vielmetti) writes: >This would argue that you should point your MX forwarder at alberta, if >they are able to do the proper header magic to get them to you unscathed. >(Better for the Canadian content laws too.) In that way you will get >mail the cheapest and best way possible. Yes, it does. At the time we registered our domain, "alberta" was not an Internet site. The "closest" site we could find (at the time) who was willing to be our forwarder was decwrl. (Believe me, I pounded on the door of every Canadian Internet connected site that I could find.) I hope that the U of Alberta would be willing to provide an MX forwarding service for us. We haven't really discussed it yet. With the current load on "alberta" I wouldn't blame them if they said no. That doesn't change the fact that there are a lot of sites out there still who will get poorer service if sites arbitrarily reroute mail. What kills me is that rabid rerouters assume they know all. One of the main proponents on rabid rerouting also maintains the comp.mail.maps database. Given the following two map entries, I would like to know how rutgers plans to guarantee to do the "right thing" if I send mail to rutgers!csb!postmaster ??? Or how does Elloit Lear's software deal with this? === From u.can.ab.1 === #N csb #S Convergent Technologies S/320; CTIX 5.11 #O Alberta Public Works Supply & Services #C Curtis Quigley #E curtisq@nexus.ca #T +1 403 421 8181 #P #L 53 24 N / 113 29 W city #R #U #W auvax!lyndon (Lyndon Nerenberg); Thu Feb 9 16:47:00 MST 1989 # csb ncc(HOURLY) === From u.ita.1 === #N csb #S #O CSB - Centro Software Brescia #C Marco Massardi #E csb!root #T +39 30 3530571 #P Via Cacciamali 63, I-20125 Brescia #L #W i2unix!ab 881103 # csb .csb.it -- Lyndon Nerenberg VE6BBM / Computing Services / Athabasca University {alberta,cbmvax,mips}!atha!lyndon || lyndon@cs.athabascau.ca Practice Safe Government Use Kingdoms
fitz@wang.com (Tom Fitzgerald) (08/03/90)
> fitz@wang.com (Tom Fitzgerald) writes: >> Some of the things I >> definitely want to cut are: >> reduce: >> ...!site1!...!dom.ain!... >> to: >> ...!site1!dom.ain!... >> if site1 is known to be on the Internet and at least 2 hops can be removed, lyndon@cs.athabascau.ca (Lyndon Nerenberg) writes: > NO! If you are going to use the UUCP transport, please use the published > UUCP transport routing tables (aka comp.mail.maps). If you use the above > method when sending mail to *.athabascau.ca, your message will cost MORE Sorry, but it vastly improves 95% of our paths, at the expense of increasing the cost of the last 5%. In your own news articles, you're putting "...@cs.athabasca.ca" in your return address, so return mail from here will get to you via your Internet forwarder regardless. Only mail to lyndon@atha.UUCP would have tried to take the UUCP route. Doesn't 90% of your mail reach you via your forwarder anyway? Since all Internet sites, and all the UUCP sites that don't maintain complete paths files will use it, it seems like only a small fraction of the mail will get there through Canadian sites. I really am trying to minimize the harm, that's why I'm only rewriting addresses that save 2 hops - rewrites that only save a single hop will be skipped. That doesn't help mail from me to you though, since I can save 2 hops by delivering the mail to you via the Internet. By doing this, I can crunch addresses like these, drawn from our paths file: orac2 bu.edu!harvard!rutgers!ksuvax1!phobos.cis.ksu.edu!orac2!%s pur-phy bu.edu!harvard!ames!pur-ee!newton.physics.purdue.edu!%s caesar bu.edu!harvard!ames!indri!uakari!caesar.cs.montana.edu!%s into: orac2 bu.edu!phobos.cis.ksu.edu!orac2!%s pur-phy bu.edu!newton.physics.purdue.edu!%s caesar bu.edu!caesar.cs.montana.edu!%s and about 100 more like it. It's worth it to me. How about if I create a stop-list for addresses to skip rewriting, and put your node in it, as well as anyone else who complains? Rewriting for your address will be explicitly disabled. This sounds like the best of both worlds. It won't fix mail to ...@cs.athabasca.ca, though. It'd be better if my filter knew which sites really are on the Internet, and which are on the other end of an MX link, but the paths don't tell me that. --- Tom Fitzgerald Wang Labs fitz@wang.com 1-508-967-5278 Lowell MA, USA ...!uunet!wang!fitz
peter@ficc.ferranti.com (Peter da Silva) (08/03/90)
In article <aqry5r.kr6@wang.com> fitz@wang.com (Tom Fitzgerald) writes: > It'd be better if my filter knew which sites really are on the Internet, > and which are on the other end of an MX link, but the paths don't tell me > that. Perhaps the paths need to have that information added? There is support for it... perhaps a "u.internet" that simply has: internet = { uu.net psi.com rice.edu uh.edu ... } (DIRECT) Or you can cull your own copy of this and add it to your pathalias run. Much smarter than hacking the output of pathalias, no? Anyone want to build a canonical u.internet? -- Peter da Silva. `-_-' +1 713 274 5180. 'U` <peter@ficc.ferranti.com>
lear@turbo.bio.net (Eliot) (08/04/90)
Peter da Silva writes: > internet = { > uu.net > psi.com > rice.edu > uh.edu > ... > } (DIRECT) > > Or you can cull your own copy of this and add it to your pathalias run. > Much smarter than hacking the output of pathalias, no? > > Anyone want to build a canonical u.internet? One of the programs that used to be distributed with pathalias would generate just such a file based on the system's host table. Unsurprisingly enough, it's a maintenance headache. What's more, it's unnecessary. The information is already there in the maps. -- Eliot Lear [lear@turbo.bio.net]
schoff@uu.psi.com (Martin Schoffstall) (08/04/90)
In article <74:4H-G@xds13.ferranti.com> peter@ficc.ferranti.com (Peter da Silva) writes: > uu.net > psi.com > rice.edu > uh.edu > ... > } (DIRECT) > Without commenting on the rest of the message it would be uu.psi.com Marty
pcg@cs.aber.ac.uk (Piercarlo Grandi) (08/05/90)
"fitz" == Tom Fitzgerald writes:
Something about munging the output of pathalias to do aggressive
rerouting on the routes generated by it. Note that he does not want to
reroute aggressively the routes he finds in mail envelopes passing thru
his site; he is (hopefully) leaving fully specified mail routes alone,
and does not second guess the sender.
What he wants to do is to optimize the output of pathalias. My first
observation is that instead of using perl, as he is thinking about, he
could use and/or improve "pathprune", a free program designed to reduce
the size of pathalias maps, mostly by removing redundant entries.
He then continues:
fitz> I really am trying to minimize the harm, that's why I'm only rewriting
fitz> addresses that save 2 hops - rewrites that only save a single hop will
fitz> be skipped.
As long as you do this for addresses originating at your site, munging
the maps is perfectly kosher. Well, almost. If you have better
information on routing than other people, you should share it by
submitting a map update request.
fitz> That doesn't help mail from me to you though, since I can
fitz> save 2 hops by delivering the mail to you via the Internet.
fitz> By doing this, I can crunch addresses like these, drawn from our paths file:
fitz> orac2 bu.edu!harvard!rutgers!ksuvax1!phobos.cis.ksu.edu!orac2!%s
fitz> pur-phy bu.edu!harvard!ames!pur-ee!newton.physics.purdue.edu!%s
fitz> caesar bu.edu!harvard!ames!indri!uakari!caesar.cs.montana.edu!%s
fitz> into:
fitz> orac2 bu.edu!phobos.cis.ksu.edu!orac2!%s
fitz> pur-phy bu.edu!newton.physics.purdue.edu!%s
fitz> caesar bu.edu!caesar.cs.montana.edu!%s
fitz> and about 100 more like it. It's worth it to me.
This simply means that your pathalias database is wrongly built, that
means that either you are supplying the wrong link costs or there is a
bug somewhere. Incidentally, the intermediate UUCP forwarders in the
three examples above are nearly all on the Internet themselves (one of
them is Rutgers, and they will aggressively rewrite your path,
incidentally), so your three lines above are true monstrosities, which
simply mean that your maps (or maps in general) are ludicrously
specified.
You should be specifying that the Internet is fully connected, so that
if bc.edu and cd.montana.edu are both on it, there should be no
intermediate step.
Now the problem is "How I do this with pathalias?". Well, you could
start by listing as fully connected all .edu subdomains that you know
are on the Internet, in the local additions to the maps.
I do not see any reason to munge the output of pathalias; you should
correctly specify its input.
--
Piercarlo "Peter" Grandi | ARPA: pcg%cs.aber.ac.uk@nsfnet-relay.ac.uk
Dept of CS, UCW Aberystwyth | UUCP: ...!mcsun!ukc!aber-cs!pcg
Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@cs.aber.ac.ukperand@admin.kth.se (Per Andersson) (08/06/90)
In article <XE+4ZF3@ggpc2.ferranti.com> peter@ficc.ferranti.com (Peter da Silva) writes: > >I'll second that. Mail to ferranti.com via the DNS costs us a bundle Peter, I'm gonna ask you a simple stupid (probably) question. If you think the DNS holds information which is wrong, why don't you get it fixed. You have a real domain name. Have your MX-record point to foo.bar.whatever.edu instead of uunet. Or maybe you wan't to unsubscribe from uunet. That I do not know how to do. Per -- --- Per Andersson Royal Institute of Technology, Stockholm, Sweden perand@admin.kth.se, @nada.kth.se
peter@ficc.ferranti.com (Peter da Silva) (08/07/90)
In article <Aug.3.14.10.17.1990.657@turbo.bio.net> lear@turbo.bio.net (Eliot) writes: > One of the programs that used to be distributed with pathalias would > generate just such a file based on the system's host table. What system's host table? I don't have a host table. > Unsurprisingly enough, it's a maintenance headache. What's more, it's > unnecessary. The information is already there in the maps. No, it's not. Whether a given domain uses an A record or an MX record on the Internet is not in the maps. And in fact... unless things have changed recently... internet links aren't even supposed to be in the maps, or are in them only as terminals. -- Peter da Silva. `-_-' +1 713 274 5180. 'U` <peter@ficc.ferranti.com>
peter@ficc.ferranti.com (Peter da Silva) (08/07/90)
In article <1990Aug5.205619.9277@kth.se> perand@admin.kth.se (Per Andersson) writes: > Peter, I'm gonna ask you a simple stupid (probably) question. If you think > the DNS holds information which is wrong, why don't you get it fixed. Because it doesn't support the data structures necessary to put the right information in: namely a set of varying-priority connections. The local net is all UUCP, and the flat internet name space does not address that very well. -- Peter da Silva. `-_-' +1 713 274 5180. 'U` <peter@ficc.ferranti.com>
heiby@mcdchg.chg.mcd.mot.com (Ron Heiby) (08/08/90)
One of the questions in the original posting asked whether it was ok to eliminate from the pathalias output all of the paths that began with the system acting as the "smart host". The idea being that if the destination did not appear in the paths, it would end up going to the smart host, anyway. I'm also interested in the answer to this question. I take the output of the pathalias run and end up putting it into a dbm database. Elimination of the paths that specify going to my "smart host" would trim about 34% of the paths that would have to be inserted into that dbm database, a goodly savings of cpu time and disk space. Thinking about it some, the only objection I can think of is related to possible order dependancies in where smail 3.1 looks to try to resolve an address. I can't point to a problem, though. Is there one? Thanks. -- Ron Heiby, heiby@chg.mcd.mot.com Moderator: comp.newprod "Mandatory Drug Testing? Just Say NO!!!"
Makey@Logicon.COM (Jeff Makey) (08/08/90)
In article <74:4H-G@xds13.ferranti.com> peter@ficc.ferranti.com (Peter da Silva) writes: >Anyone want to build a canonical u.internet? I have been thinking of doing just the opposite: building a UUCP zone file (which would contain just CNAME and MX records) from the UUCP Map, and feeding that to my nameserver. A u.internet file (or its equivalent) could easily be built from all of the CNAME records. :: Jeff Makey Department of Tautological Pleonasms and Superfluous Redundancies Department Disclaimer: All opinions are strictly those of the author. Internet: Makey@Logicon.COM UUCP: {nosc,ucsd}!logicon.com!Makey
urlichs@smurf.sub.org (Matthias Urlichs) (08/09/90)
In comp.mail.uucp, article <42693@mcdchg.chg.mcd.mot.com>,
heiby@mcdchg.chg.mcd.mot.com (Ron Heiby) writes:
< One of the questions in the original posting asked whether it was ok
< to eliminate from the pathalias output all of the paths that began
< with the system acting as the "smart host". [...]
< Thinking about it some, the only objection I can think of is related
< to possible order dependancies in where smail 3.1 looks to try to
< resolve an address. I can't point to a problem, though. Is there
< one?
<
Yes.
Consider the following connectivity:
(<-> : directly connected, ---> : knows path to)
you <-> smart
smart ---> .some.domain
you <-> somebody
somebody ---> .domain
Now suppose that mail from smarthost to .domain (if != .some.domain) goes
through you.
Now, if you delete all "smarthost" entries, and if "somebody" thinks that the
best connection to "some.domain" goes through you and smarthost, then you
obviously have a mail loop.
It seems that to be safe, you can only drop paths from your table which point
to UUCP names, but not to subdomains or subdomainized hosts, viz.
anyone smart!bang!there (dropped)
some.domain smart!bang!here!%s (not dropped)
.domain smart!bing!bang!%s (also not dropped)
--
Matthias Urlichs -- urlichs@smurf.sub.org -- urlichs@smurf.ira.uka.de
Humboldtstrasse 7 - 7500 Karlsruhe 1 - FRG -- +49+721+621127(Voice)/621227(PEP)fitz@wang.com (Tom Fitzgerald) (08/10/90)
heiby@mcdchg.chg.mcd.mot.com (Ron Heiby) writes: > One of the questions in the original posting asked whether it was ok > to eliminate from the pathalias output all of the paths that began > with the system acting as the "smart host". [...] > Thinking about it some, the only objection I can think of is related > to possible order dependancies in where smail 3.1 looks to try to > resolve an address. I can't point to a problem, though. Is there > one? I don't know about smail 3.1, but smail 2.5 gets wierd if you strip out the root domains: .com smarthost!%s .edu smarthost!%s etc. Stripping these makes smail 2.5 kick into rabid-rerouting in cases where it shouldn't, so mail to "host!user@somewhere.edu" can get misrouted. Leaving the root domains in the paths keeps this from happening. The reason why this happens is long and complicated, and I'll only type it in if someone's interested. Smail (both 2.5 and 3.1) might get wierd if there's a path to a subdomain that passes through smarthost, but the path to the parent domain passes through a site other than smarthost. I wish I could say that it never happens, but I just checked the paths here, and it does in a number of cases. So I'll probably change the path-fixer here to handle this, and to only remove domains on the other side of smarthost if the parent domain is also on the other side of smarthost. This is gonna be a pain. --- Tom Fitzgerald Wang Labs fitz@wang.com 1-508-967-5278 Lowell MA, USA ...!uunet!wang!fitz
les@chinet.chi.il.us (Leslie Mikesell) (08/11/90)
In article <2r-/e2.s9@smurf.sub.org> urlichs@smurf.sub.org (Matthias Urlichs) writes: >In comp.mail.uucp, article <42693@mcdchg.chg.mcd.mot.com>, > heiby@mcdchg.chg.mcd.mot.com (Ron Heiby) writes: >< One of the questions in the original posting asked whether it was ok >< to eliminate from the pathalias output all of the paths that began >< with the system acting as the "smart host". [...] >Yes. >Consider the following connectivity: >(<-> : directly connected, ---> : knows path to) >you <-> smart >smart ---> .some.domain >you <-> somebody >somebody ---> .domain >Now suppose that mail from smarthost to .domain (if != .some.domain) goes >through you. >Now, if you delete all "smarthost" entries, and if "somebody" thinks that the >best connection to "some.domain" goes through you and smarthost, then you >obviously have a mail loop. This shouldn't happen, because smail 3 doesn't re-route beyond the next hop specified in the path. If you get mail for you!smarthost!some.domain!user you will hand it to smarthost without looking farther down the path, and smarthost will deliver it without sending it back. What's the problem? Even if you hand off something to smarthost that is destined for .domain, smarthost should expand the full path to you!somebody!whatever before sending it back through you. Loops only become a problem when the expansion that others perform is not honored or 2 machines try to use each other as smart-hosts. The only difference that I can think of that might happen because you delete paths starting with smarthost is that you will run "uuname" for every host name that you don't find in the paths file before using the smart-host route. This is, of course, something that can be changed in the configuration of smail. If you know that all direct neighbors are in the paths file, you could eliminate the "uuname" router. However, you normally should only have to route locally generated mail unless someone else is using you as a smart-host. Mail passing through should already have a next-hop of one of your direct neighbors unless someone else made a mistake. Les Mikesell les@chinet.chi.il.us
urlichs@smurf.sub.org (Matthias Urlichs) (08/14/90)
In comp.mail.uucp, article <1990Aug10.221547.13301@chinet.chi.il.us>, les@chinet.chi.il.us (Leslie Mikesell) writes: < In article <2r-/e2.s9@smurf.sub.org> urlichs@smurf.sub.org (Matthias Urlichs) writes: < >< One of the questions in the original posting asked whether it was ok < >< to eliminate from the pathalias output all of the paths that began < >< with the system acting as the "smart host". [...] < < >Yes. [...] < < [...] If you get mail for you!smarthost!some.domain!user < you will hand it to smarthost without looking farther down the path, and < smarthost will deliver it without sending it back. What's the problem? Not everyone generates full!bang!paths!to!destination!user. Some people beliebe in generating next!destination!user, i.e. handing off "destination!user" or "user@dest.dom.ain" to machine "next". The latter also happens if you're using BSMTP as your transport, which is a very good idea especially if someone is using weird addresses and/or if you or someone near you is expanding a mailing list. < Mail passing through should already have a next-hop of one of your < direct neighbors unless someone else made a mistake. < I don't quite think so. The view that my site can and should find out for itself where passing-through mail should be going next is, to me, equally valid. (Perhaps even more so, since, according to Pathalias, I'm closer.) -- Matthias Urlichs -- urlichs@smurf.sub.org -- urlichs@smurf.ira.uka.de Humboldtstrasse 7 - 7500 Karlsruhe 1 - FRG -- +49+721+621127(Voice)/621227(PEP)
les@chinet.chi.il.us (Leslie Mikesell) (08/15/90)
In article <g!$_e2.[_1@smurf.sub.org> urlichs@smurf.sub.org (Matthias Urlichs) writes: >< Mail passing through should already have a next-hop of one of your >< direct neighbors unless someone else made a mistake. >I don't quite think so. The view that my site can and should find out for >itself where passing-through mail should be going next is, to me, equally >valid. (Perhaps even more so, since, according to Pathalias, I'm closer.) The original article was in the context of one site using another as a "smart-host". The idea of a "smart-host" is that you hand off mail to all hosts that you can't identify over to another machine that hopefully has more complete information for address resolution. It would obviously not be wise to use a site that did not perform complete route expansion as a "smart-host" if there is any chance that any route chosen will be back through your site. Still, it doesn't make any sense to me to let each hop recompute the route based on information that might be inconsistant between sites. The original calculation is based on the paths cost of the total route which only the first site will consider. If you feel that your site's path data is not up to date, then you might consider arranging for some other site to route everything for you, but you better hope it has a complete route specified if it comes back at you. Les Mikesell les@chinet.chi.il.us