mike@BRL.MIL (Mike Muuss) (07/26/90)
This is one of the most painful trips from Maryland to Boston I have ever seen! -Mike From: Phil Dykstra <phil@BRL.MIL> For ha ha's check this out: traceroute to expo.lcs.mit.edu (18.30.0.212), 30 hops max, 40 byte packets 1 ext328 (128.63.4.22) 21 ms 21 ms 26 ms 2 MOFFETT-FLD-MB.DDN.MIL (26.20.0.16) 586 ms 632 ms 272 ms 3 Palo_Alto.CA.NSS.NSF.NET (192.52.195.254) 238 ms * 200 ms 4 Salt_Lake_City.UT.NSS.NSF.NET (129.140.79.13) 426 ms 497 ms 261 ms 5 Ann_Arbor.MI.NSS.NSF.NET (129.140.81.15) 275 ms 263 ms 367 ms 6 Princeton.NJ.NSS.NSF.NET (129.140.72.17) 303 ms 257 ms 248 ms 7 zaphod-gateway.jvnc.net (192.12.211.65) 271 ms 294 ms 250 ms 8 hotblack-gateway.jvnc.net (130.94.0.67) 271 ms 384 ms 287 ms 9 capital1-gateway.jvnc.net (130.94.1.9) 556 ms 472 ms 300 ms 10 cheesesteak2-gateway.jvnc.net (130.94.33.250) 459 ms 294 ms 666 ms 11 * * cheesesteak1-gateway.jvnc.net (130.94.32.1) 306 ms 12 * beantown2-gateway.jvnc.net (130.94.27.250) 335 ms 311 ms 13 near-gateway.jvnc.net (130.94.27.10) 335 ms 328 ms 385 ms 14 ihtfp.mit.edu (192.54.222.1) 774 ms * 353 ms 15 SEWAGE.MIT.EDU (18.68.0.8) 576 ms * 303 ms 16 * MOSS.LCS.MIT.EDU (18.10.0.11) 344 ms 365 ms The MILNET throws it as far away as it possibly can, NSFNET spends four (nice orderly) hops getting it to Princeton, and then JVNC spends 7(!) hops getting it to Boston. - Phil
heker@NISC.JVNC.NET (Sergio F. Heker) (07/27/90)
Phil, While the number of hops is large over JvNCnet (don't forget that the network extends over eight states) I don't see a "serious routing loop" on your trace, in fact I don't see any routing loop. For your information, the JvNCnet network is a ring of T1s connecting backbone nodes colocated at the phone company point of presence with two cisco routers at each point of presence to add capacity and for reliability purposes. The backbone nodes are located at Trenton NJ, Newark, NJ, New York City, New Haven, CT, Providence, RI, Boston, Mass and Philadelphia, PA. On a test performed just now and attached below, 5 packets were lost of 1000 with an average round trip of 51 ms on the path from one of our hosts at the new JvNCnet headquarters at Princeton University and MIT. This is quite reasonable. We have been monitoring access to NEARnet due to the importance of NEARnet as a whole and specially since the last three weeks we have experienced excellent performance. This was also observed by the NSFNET NOC. Previously we were affected by a limitation to the amount of traffic carried by the JvNCnet together with the routing table size (of about 300 networks) which accounted for memory limitations. Two major changes took place three weeks ago on the JvNCnet cisco routers, the first one was an upgrade of all firmware (400 PROMS) to cisco 8.1 which improved the way cisco 7.1 handled changes of routing information. The second change was a reduction of the cisco image running on the routers. We are very happy with the results and the cooperation of cisco and the JvNCnet members as well as RCI our long distance provider to make these changes expedioustly. In anticipation of the installation of the new NSS for NEARnet, JvNCnet will work with NEARnet in the installation of a temporary direct access T1 line connecting NEARnet and JvNCnet directly at the JvNCnet NSS in Princeton. The JvNCnet and NEARnet NOCs are working together to make this new NSS transition become transparent to the users. We always welcome comments on the network, however you should probably send them to a list more dedicated to this purpose than "tcp-ip". If you do see a loop or a performance problem could you please send a message to our NOC. We can be reached at "noc@nisc.jvnc.net". Thank you very much. -- Sergio -- begin forwarded message -- From owner-tcp-ip Thu Jul 26 18:24:56 1990 Received: by nisc.jvnc.net (5.61/1.34) id AA01801; Thu, 26 Jul 90 18:24:54 -0400 Received: from jvncb.csc.org by nisc.jvnc.net (5.61/1.34) id AA01797; Thu, 26 Jul 90 18:24:52 -0400 Received: from NIC.DDN.MIL by jvncb.csc.org (5.61/1.34) id AA02895; Thu, 26 Jul 90 18:29:44 -0400 Received: from WOLF.BRL.MIL by NIC.DDN.MIL with TCP; Thu, 26 Jul 90 08:42:22 PDT Received: by WOLF.BRL.MIL id aa14006; 26 Jul 90 11:33 EDT Date: Thu, 26 Jul 90 11:25:38 EDT From: Mike Muuss <mike@BRL.MIL> To: TCP-IP@nic.ddn.mil Subject: Serious Routing Problems Message-Id: <9007261125.aa13838@WOLF.BRL.MIL> Status: RO This is one of the most painful trips from Maryland to Boston I have ever seen! -Mike From: Phil Dykstra <phil@BRL.MIL> For ha ha's check this out: traceroute to expo.lcs.mit.edu (18.30.0.212), 30 hops max, 40 byte packets 1 ext328 (128.63.4.22) 21 ms 21 ms 26 ms 2 MOFFETT-FLD-MB.DDN.MIL (26.20.0.16) 586 ms 632 ms 272 ms 3 Palo_Alto.CA.NSS.NSF.NET (192.52.195.254) 238 ms * 200 ms 4 Salt_Lake_City.UT.NSS.NSF.NET (129.140.79.13) 426 ms 497 ms 261 ms 5 Ann_Arbor.MI.NSS.NSF.NET (129.140.81.15) 275 ms 263 ms 367 ms 6 Princeton.NJ.NSS.NSF.NET (129.140.72.17) 303 ms 257 ms 248 ms 7 zaphod-gateway.jvnc.net (192.12.211.65) 271 ms 294 ms 250 ms 8 hotblack-gateway.jvnc.net (130.94.0.67) 271 ms 384 ms 287 ms 9 capital1-gateway.jvnc.net (130.94.1.9) 556 ms 472 ms 300 ms 10 cheesesteak2-gateway.jvnc.net (130.94.33.250) 459 ms 294 ms 666 ms 11 * * cheesesteak1-gateway.jvnc.net (130.94.32.1) 306 ms 12 * beantown2-gateway.jvnc.net (130.94.27.250) 335 ms 311 ms 13 near-gateway.jvnc.net (130.94.27.10) 335 ms 328 ms 385 ms 14 ihtfp.mit.edu (192.54.222.1) 774 ms * 353 ms 15 SEWAGE.MIT.EDU (18.68.0.8) 576 ms * 303 ms 16 * MOSS.LCS.MIT.EDU (18.10.0.11) 344 ms 365 ms The MILNET throws it as far away as it possibly can, NSFNET spends four (nice orderly) hops getting it to Princeton, and then JVNC spends 7(!) hops getting it to Boston. From "picasso.jvnc.net" to 18.10.0.11, 7/27/90 8:00am EST ..... 18 bytes from 18.10.0.11: icmp_seq=997. time=29. ms 18 bytes from 18.10.0.11: icmp_seq=998. time=31. ms 18 bytes from 18.10.0.11: icmp_seq=999. time=27. ms ----18.10.0.11 PING Statistics---- 1000 packets transmitted, 995 packets received, 0% packet loss round-trip (ms) min/avg/max = 26/51/932 ______________________________________________________________________________ Sergio Heker Phone # (609)520-2000 (until 8/18) Director, JvNCnet Phone # (609)258-2400 (after 8/18) Email "heker@nisc.jvnc.net" ______________________________________________________________________________
hsw@SPARTA.COM (Howard Weiss) (07/27/90)
Here is another very strange route from sparta.com (137.39.6.2) connected to Alternet/UUnet to brl.mil (McLean, Va. to Aberdeen, Md - roughly under 100 miles). This path (Houston, San Diego, Palo Alto, and then back across the county) seems to be the "normal" path taken for traffic from Alternet via Suranet destined for the Milnet! Howie ------------------------------------------------------------ 1 annex.UU.NET (137.39.6.1) 340 ms 360 ms 300 ms 2 janus.UU.NET (192.48.96.1) 300 ms 360 ms 360 ms 3 sura6.sura.net (128.167.59.1) 360 ms 360 ms 360 ms 4 192.80.214.254 (192.80.214.254) 360 ms 360 ms 360 ms 5 Houston.TX.NSS.NSF.NET (129.140.75.9) 420 ms 420 ms 420 ms 6 San_Diego.CA.NSS.NSF.NET (129.140.70.11) 480 ms 480 ms 420 ms 7 Palo_Alto.CA.NSS.NSF.NET (129.140.77.6) 480 ms 480 ms 480 ms 8 Palo_Alto.CA.NSS.NSF.NET (129.140.13.130) 480 ms 540 ms 480 ms 9 BRL.MIL (26.2.0.29) 900 ms 1000 ms 1260 ms
hwb@MERIT.EDU (Hans-Werner Braun) (07/29/90)
Any packet injected into the NSFNET backbone and destined towards MILNET
is currently only going via the only remaining NSFNET/DDN connection
at NASA Ames (FIX-West). This is in result of the FIX-East move at the
east coast two weeks or so ago and the MILNET connectivity there has not
been restored yet. It should be restored soon, I hope. To my knowledge
the regional networks as well as other agencies were aware of this. You
should really talk to the people who are responsible for your network
connectivity. I only quickly scan TCP-IP mailing stuff and am only seeing
these things by chance. Talking to your direct network provider should yield
faster and reliable responses.
-- Hans-Werner
PS: To my knowledge, as soon as DCA signs off on their T1 link beween FIX-East
and the Mailbridge site we can activate the east coast NSFNET/DDN
connection again. DCA is very cooperative here and is also keeping
NSFNET staff well informed about the progress. Give it a few more days...phil@BRL.MIL (Phil Dykstra) (07/30/90)
My original note was sent to an internal list; I didn't expect to see it here. Nonetheless, it has led to some interesting discussion. To Sergio, note that no one used the term "loop". Mike called it a "problem" which I agree is debatable. JvNC did get the packets there, which one comes to appreciate when they are stuck behind a Buttergate. Thanks for the informative reply. I am somewhat aware of JvNCnet's design. Given the number of hops, I have to wonder how well the Cisco routing protocol is doing, but then I don't know the net topology. [Is an on-line map available? I would very much like to see one.] Thanks to Dave and Hans-Werner for the update on the FIX-East connection. It will be nice when that is back on line. And yes, every route I have traced through the NSFNET Backbone has taken a very logical route. The Merit folks are doing a good job. As of late last week up to now, the route from MOFFET-FLD to MIT has started going via FEBA-WEST over WIDEBAND to BBN to MIT - a few less hops. This is perhaps even better than a direct Princeton to NEARnet link, for the west coast, but that new link will be nice. From my perspective (the MILNET), the number one problem is with the Mailbridges (the "M" word). Routes still flop around, and when they do, you can get temporary net unreachables (which of course break connections). Some routes are hard to keep open for even ten minutes. We have had numerous complaints from our own users and people comming in from "outside". We used to complain about this periodically last Fall (to DCA or the folks at BBN), and while we grew tired of complaining, the problem remains. Looking forward to switching over to our ASNET connection to the world. - Phil
kwe@bu-it.bu.edu (Kent England) (07/31/90)
In article <9007300339.aa00505@SPARK.BRL.MIL>, phil@BRL.MIL (Phil Dykstra) writes: > ... Given the number of hops, I have to wonder how well the Cisco > routing protocol is doing, but then I don't know the net topology. Not too well in my opinion. Chuck Hedrick from Rutgers posted some very informative material in comp.dcom.sys.cisco about setting timers in IGRP to speed convergence. The timers are now settable in cisco configs. When lines flap for whatever reason, routes in IGRP go away and get held down for many minutes, breaking TCP connections. While caution must be used, in many cases, shortening up the IGRP reaction times will prevent line flaps from turning into big route lossage. It all depends on how you have your net set up. In my opinion, this will become a thing of the past with link state protocols. I'm anxious to see results from the OSPF early deployments in NASA and SURAnet. Milo, Dave; How does OPSF do when you have routes flapping? Instant convergence? --Kent
medin@NSIPO.ARC.NASA.GOV ("Milo S. Medin", NASA ARC NSI Project Office) (08/01/90)
Kent, I'm a little confused here. Line flapping should be fixed by having hystersis in the up/down line protocol. It's been our experience that OSPF in our system reroutes completely after about 3 seconds on average after the dead link is detected. OSPF has it's timers set up in a way as to avoid link flap causing a lot of simultaneous LSP flooding, due to deliberate jitter in the timers (LSP's being flooded from all the gateways connected to a down net). As for link up/downs going on every few seconds, we haven't had experience with that sort of situation, but in the case of a link that flaps every couple of minutes, that seems to work fine. Because link-state protocols flood the topology information, and then the routers recompute the routing tables more or less in parallel, convergence can happen very quickly, and you also can do without the complicated holddowns and such needed to prevent route looping in vector-distance protocols. This should be true for any reasonably well implemented link-state protocol. It's just easy to do this part right. You can certainly screw up in other ways, but if you want fast convergance, fast enough to avoid lining up hundreds of user TCP connections against a wall and killing them in rapid sucession, you really need to use a link-state protocol. This is one reason why noone has bothered to build a new vector distance IGP in quite some time. Even the new proprietary IGP's these days are link-state based... BARRNET also runs OSPF now, and those guys can tell you more about how it's performance has been. Thanks, Milo
kwe@bu-it.bu.edu (Kent England) (08/01/90)
In article <9007311833.AA22548@nsipo.arc.nasa.gov>, medin@NSIPO.ARC.NASA.GOV ("Milo S. Medin", NASA ARC NSI Project Office) writes: > > Kent, I'm a little confused here. Line flapping should be fixed by > having hystersis in the up/down line protocol. Yes, I was just wanting to point out that using the cisco recommended settings for keepalive and update/holddown timers, a 20 second line outage will stimulate a route lossage of 4.5 minutes. I'm not faulting cisco for this, they have to recommend *something* to their customers without knowing all details of a particular situation. There are trade-offs that I won't elaborate. As I said, Chuck Hedrick has already posted details on issues involved in shortening timers in IGRP for the bold-hearted network managers out there that would like faster convergence. > Because link-state protocols flood the topology information, and then > the routers recompute the routing tables more or less in parallel, > convergence can happen very quickly, and you also can do without > the complicated holddowns and such needed to prevent route looping > in vector-distance protocols. Right, I was just looking for demonstrated corroboration. I must be belaboring the obvious, so let's move on to other topics like expanded network addressing or open routing or S.I.N/dual IS-IS. --Kent
efb@suned1.Nswses.Navy.MIL (Everett Batey) (08/09/90)
In article <9007311833.AA22548@nsipo.arc.> ("Milo S. Medin") writes:
from all the gateways connected to a down net).
As for link up/downs going on every few seconds, we haven't had
experience with that sort of situation, but in the case of a link
that flaps every couple of minutes, that seems to work fine.
Milo .. wish we didnt spend so much time 15min to days at a time unable to
get routing off our 137.24 net thru our net 26 VAX .
--
+ efb@suned1.nswses.Navy.MIL efb@elroy.JPL.Nasa.Gov efb@voodoo.bitnet +
+ efb@suned1.UUCP | Gold Coast Sun Users | Vta-SB-SLO DECUS | WA6CRE +
+ Statements, Opinions, MINE, NOT Uncle Sam_s | News Postmaster DNS guy +vaf@Valinor.Stanford.EDU (Vince Fuller) (08/10/90)
----- Kent, To follow-up on Milo's comments about routing protocols - BARRNet has been running OSPF for several months now and we are pleased with it for a variety of reasons, one of which being its ability to quickly re-route around failures. When a line flap occurs somewhere in the system, the OSPF part of BARRNet recovers quickly and things stabilize within a matter of seconds. Unfortunately, the non-OSPF part of BARRNet is not so fortunate - the non-SPF protocols that it runs immediately go into hold-down when a line flap occurs, partitioning the network for the duration of the hold-down period (several minutes). We find this very annoying and is one of the (many) reasons that we have been pushing so hard for widespread implementation of OSPF. Vince Fuller, BARRNet technical coordinator (BARRNet is an equal-opportunity vendor basher)