dove@savax.UUCP (webster dove) (01/06/87)
We are just beginning to setup a TCP/IP network here. We have several facilities separated by 5-20 miles, and most facilities have several distinct ethernet cables. I am looking for explanations of the tradeoffs between using filtering repeaters to convey traffic between wires and between facilities (e.g. VITALINK) and using TCP routers for the same purpose (e.g. PROTEON). I need clear lucid explanations that can be used to guide management decisions. Please mail me the responses instead of posting them. addr: "..!decvax!savax!dove". Thanks in advance.
root@topaz.RUTGERS.EDU (Charles Hedrick) (01/07/87)
I'm responding to your message in public as well as by mail because I've found recently that a number of people don't understand the dangers to using bridges, which you refer to as filtering repeaters. Bridges include the DEC LANbridge, Vitalink, Applitek broadband and fiber bridges, and point to point selective repeaters from companies such as Ungermann-Bass. Routers for IP are primarily from Bridge, Cisco, Proteon, and, Ungermann-Bass. Of course the main conceptual difference is that they operate at a different level in the OSI level hierarchy, and thus use different classes of address. The bridges decide whether to forward a packet based on its Ethernet address, while the routers use the protocol address (normally the IP address, but there are routers for other protocols, and some that will handle multiple protocol types). The bridge has the obvious advantage that it will work with any protocol or combination of protocols. Because it is based on only the Ethernet address, in theory it can handle any network services capable of passing over Ethernet. This can be a big advantage for organizations that use many protocols, or that are not sure what they may be using in the future. In addition, because the bridges don't know anything about the protocols, the software in them is less likely to need continual upgrades as protocols and needs change. You can think of a bridge as a piece of hardware. You should think of a router as routing software that happens to run on a particular kind of box. You are going to be conscious of that software. You are likely to need new features as time goes on, and changes will likely be needed as protocols and your needs evolve. You may even run into bugs and crashes. In theory, once a bridge works there would be no need to update it. (One suspects that in practice there will be new releases of software with them as well, particularly the more sophisticated ones. At Rutgers we actually have more crashes from our bridges than our routers, but I think that is because we have unusually unreliable bridges.) The advantages of routers include the following. More detailed explanations will follow: - better isolation from misbehaviors on the other Ethernets - a slight increase in security - more routing flexibility - the ability to handle busy networks - network monitoring ISOLATION Most of the new bridges work in roughly the same way: They watch all packets on the local Ethernet, and make a list of every Ethernet address that has appeared. When they see a packet addressed to an address not on the list, they know it is intended for another Ethernet, and forward it. The problem is, what should they do about broadcasts and multicasts. For various good reasons, it turns out that they have to forward them. The problem is that certain kinds of configuration errors can result in "broadcast storms". We have seen such things several times on Ethernets that contain diskless Suns. (They were a result of development work. Properly configured Suns don't just suddenly decide to start a broadcast storm.) During such a storm, the rate of broadcasts is large enough to bring every machine on the network to its knees. By its nature, a broadcast must be processed by every machine that sees it. It takes a certain amount of CPU time to look at the packet and figure out whether it is for you. If the packet rate is high enough, this can take over your machine. In practice this seems to happen before the Ethernet is physically saturated. With bridges, these broadcasts will be passed on to all of the other Ethernets. It's not clear how serious this problem would be in practice. It is possible in principle to put various safety features in the bridge to prevent complete disaster on all of the Ethernets. But I'm not at all sure whether actual bridges use these safety measures. I'm also not sure how often a broadcast storm would happen to other sites. It happened to us because of changes we were making to the Sun network handling. But I am nervous about a design where one misbehaving workstation can bring down the entire campus. SECURITY No Ethernet is very secure, if you have people who can run their own software on a machine attached to the Ethernet. However routers do provide some help. Normally when a router is in use, each Ethernet is a separate subnet. No matter what a user does with his machine, he can't pretend to be a machine on a different subnet. With bridges, in theory a student could program his PC to look like any other machine on campus, and then take advantage of any .rhost files to break into files on other machines. A network constructed with bridges is logically a single large Ethernet. If he does this on a network built with routers, he can only imitate other machines on his own Ethernet. ROUTING It is very hard to construct complex networks using bridges. Some implementations are better than others. With the simplest kinds, it is not feasible to use redundant connections to get faster throughput or insurance against failure. DEC's technology (which I believe is also licensed by Vitalink) seems to be the most sophisticated. They can handle configurations with redundancy, but they can't use the redundant lines. That is, when they detect a loop, they put a link into standby mode. Should a failure occur, that link can be brought into play. This helps reliability, but still can be a limit on configuration. Suppose you have a large network A ---- B ----- C ----- D ------ E A and E find that they talk to each other a lot, and want to install a direct line. That line would cause a loop. With most vendors, turning on a line from A to E would get the system into a loop and cause trouble. With the DEC LANbridge, it's OK, but the line wouldn't be used unless one of the others fails. Note by the way that there is a routing standard for IP. There is some hope that you can build a network with both Proteon and Cisco routers, because they both (I think) will support RIP routing. There is no such standard for bridges, so you will want to construct networks from just one vendor's product. BUSY NETWORKS Note that bridges must look at every packet on the network. This is because the machines that use them don't know they are there. The bridge must be prepared to forward a packet addressed to any Ethernet address on the other end. With existing technology, the only way to do that is to run its Ethernet interface in "promiscuous mode", and examine every packet. This means that one could saturate a bridge by attaching it to a busy network, even though little if any traffic actually passes through the bridge. Again, how serious this problem is depends upon the vendor. Based on vendor specs, at least some bridges would have problems coping with out networks. However boxes built with special-purpose high-speed hardware (as the LANbridge is reputed to be) may be able to cope with fairly high packet rates. A router only needs to look at the packets that actually go through it. NETWORK MONITORING Again, this depends upon the vendor. A simple selective repeater may cause you trouble with network diagnosis. There is no easy way to tell whether it is working. We have a program that periodically issues pings (echo requests) to all of our routers. If a router is down, it will stop answering pings. But a repeater has no address of its own. There is no way to ping it. You have to ping a bunch of hosts, and deduce that if you suddenly can't reach any host on network X, probably the bridge going to that network is down. However the more sophisticated bridges have their own network management software. GENERAL COMMENTS If you do decide to use bridges (because of cost, simplicity, or because you need to support protocols that you can't find routers for), I'd be very careful how they are used. For a network of any size, I'd pick a system that can do some routing and that has network monitoring capabilities. The DEC technology comes to mind here, though we haven't actually used it. I'd also put at least some routers in the system at points where you think you need isolation. You have to be insane to use bridges between different companies or universities. You also probably want to isolate student from administrative portions of the network. You will also want to use enough routers to allow you to put in redundant connections where you want them. But my personal preference would be to use routers exclusively.
ron@brl-sem.ARPA (Ron Natalie <ron>) (01/08/87)
In article <476@savax.UUCP>, dove@savax.UUCP (webster dove) writes: > > We are just beginning to setup a TCP/IP network here. We have several > facilities separated by 5-20 miles, and most facilities have several > distinct ethernet cables. > > I am looking for explanations of the tradeoffs between using filtering > repeaters to convey traffic between wires and between facilities (e.g. > VITALINK) and using TCP routers for the same purpose (e.g. PROTEON). > First, your terminology is off. The VITALINK TRANSLAN is not a repeater. A repeater passes through the ethernet signal between two points. The translan actually receives the packet into it's memory and then retransmits it on the other cable. The community is not overly sure what to call this but the word "bridge" in lower case seems to be gaining popularity. The Proteon is not a TCP router, it is an IP router. BRIDGE: VitaLink TransLan, DEC LANBridge ADVANTAGES- - Will pass data without regard to protocol. - Maintains your ethernet address space so every host is direcly accessable to every other host. - Broadcast messages are received everywhere. DISADVANTAGES- - Problems on one ethernet cable such as transceiver jabbering will be propagated to other segments. - Broadcast messages are receieved everywhere. - All segments of the cable will receieve a large percentage of the total traffic, hence you will run into the maximum throughput limits faster. ROUTER: Proteon, CISCO, Bridge, among others ADVANTAGES: - Isolates the cable segments from each other for better performance and fault tolerance. - Multiple routes can be provided in the system to provide for increased performance and redundancy. - Can interface to things other than Ethernet. DISADVANTAGES: - Some level of IP routing must be done (generally handled automatically though). - Must be designed to handle each protocol that they will route (multi-protocol routers have been built, or just use a different router for each protocol). ================================== Frankly, the performance and fault tolerance problems are enough to scare anyone into routers. There is an ethernet spread through eight states by satellite and TRANSLAN boxes and while it is amusing, I'm sure that any of the parties can tell you it is no way to run a network. -Ron
phil@amdcad.UUCP (Phil Ngai) (01/08/87)
Having some experience with the DEC LANBridge 100, aka DEBET, I wanted to talk about how DEC has addressed some of the problems with bridges mentioned by Charles Hedrick. Note that the DEBET is not applicable to the original poster's application as it can not use leased lines. It can operate over fiber optic cable but there is a 2000 meter length limit. >The advantages of routers include the following. More detailed >explanations will follow: > - better isolation from misbehaviors on the other Ethernets > - a slight increase in security > - more routing flexibility > - the ability to handle busy networks > - network monitoring A router can also provide slightly better communications efficiency because it need only send the level 3 packet, not the level 2 (MAC) packet. That is, a bridge has to send the Ethernet physical source and destination address and the Ethernet type number. A router does not. >SECURITY > >No Ethernet is very secure, if you have people who can run their own >software on a machine attached to the Ethernet. However routers do >provide some help. Normally when a router is in use, each Ethernet is >a separate subnet. No matter what a user does with his machine, he >can't pretend to be a machine on a different subnet. With bridges, in >theory a student could program his PC to look like any other machine >on campus, and then take advantage of any .rhost files to break into >files on other machines. A network constructed with bridges is >logically a single large Ethernet. If he does this on a network built >with routers, he can only imitate other machines on his own Ethernet. This may not be possible, but could a cracker imitate a router? First, he could try to wipe out the real router and take its place. Most routers have some kind of network management virtual terminal port and it may be possible to get in this way to take it down. Second, if this approach failed, could he come up as a new router advertising via RIP packets a shorter way to a host he wished to imitate? The original point still stands, that it is easier to do evil on a bridge type network. However, if people use telnet or ftp, it's already pretty easy to grab passwords as they login and this is independent of the bridge/router issue. >Note by the way that there is a routing standard for IP. There is >some hope that you can build a network with both Proteon and Cisco >routers, because they both (I think) will support RIP routing. >There is no such standard for bridges, so you will want to construct >networks from just one vendor's product. Agreed, but I don't see how the "routing" standard would be a problem. The routing technique shouldn't vary. If you get a packet with a destination address you know is on the other side, forward it. If you get a packet with a destination address you know is on the same side, ignore it. If you haven't seen the address, forward it. Seems simple enough to me. > With existing technology, the only way to >do that is to run its Ethernet interface in "promiscuous mode", and >examine every packet. This means that one could saturate a bridge by >attaching it to a busy network, even though little if any traffic >actually passes through the bridge. Again, how serious this problem >is depends upon the vendor. The DEBET is claimed to be able to handle full Ethernet speed. DEC has done some clever things to make this possible. See the DEBET technical manual for more details. Other people's bridges do have bandwidth limitations. I won't name names in public. >down, it will stop answering pings. But a repeater has no address of >its own. There is no way to ping it. You have to ping a bunch of >hosts, and deduce that if you suddenly can't reach any host on network >X, probably the bridge going to that network is down. However the >more sophisticated bridges have their own network management software. A "repeater" does not have a network address. A bridge need not. The DEBET does have a network management address. In addition, it broadcasts "I am here" type messages periodically. A network of DEBETs configured in a redundant topology (a good one is a ring, as you get N+1 redundancy. That is, if you need 7 DEBETs and put 8 in a ring, any one of the 8 can fail without taking down the network. Much better than the usual "one spare for each unit in service".) automatically reconfigures when one DEBET does down. A minimum distance spanning tree topology is always maintained. >The DEC technology comes to mind here, >though we haven't actually used it. We used one for several months and had good results. Throughput and delay looked good. We only had one so we didn't get to try the redundancy but I have no reason to believe it doesn't work. We did not buy any due to budget constraints. They are about $8,000 or so. The DEBET is excellent for an "extended" Ethernet type application. That is, if you've run out of repeater allowances or collision propagation time limit, the DEBET allows you to "start over", in effect. If you are using thinwire Ethernet, for example, you start getting lots of repeaters all of a sudden. The DEBET is also good for isolating a segment with heavy local traffic, say between a file server and its diskless workstations. A router would work also but the network administration is more involved and the bandwidth through a router will probably be less. One of the important issues with bridges vs routers which has been much discussed in other forums but not here as far as I know is the issue of routing. The bridges make everything look like one big Ethernet. If it is not, if you have skinny wires(serial lines, say 9,600 or even 56,000) between sites, then you may want to take into account the bandwidth of the hops you are considering routing over. With a bridge, this is impossible. With a router, it is possible in theory although I don't believe any versions of TCP/IP are that sophisticated. But at least with a router, the infrastructure exists. Usage of redundant communications lines has already been covered by Charles but I want to reiterate that for us, redundancy is vital and waste is frowned upon. Systems that do not provide redundancy are unacceptable. Systems that fail to use the backup resources during normal operation to provide higher throughput seem badly designed. -- Butter and salt can make almost anything taste good. Phil Ngai +1 408 749 5720 UUCP: {ucbvax,decwrl,hplabs,allegra}!amdcad!phil ARPA: amdcad!phil@decwrl.dec.com
kik@cernvax.UUCP (kik) (01/15/87)
Sender: In the "ROUTING" section of Charles Hendrick's article on gateways and bridges, he says "It is very hard to construct complex networks using bridges". This restriction comes from shortcomings in current products, not from the concept of bridging. Current offerings are nearer to stepping-stones than bridges, in that they rely on loop-free point-to-point links for their interconnection. Computers also used to do this before the advent of the communications subnet: it is clear that the next step in bridge technology is for the bridges to use a separate, high-speed backbone network for their interconnection. In fact, we have developed such a system at CERN, and it works very well: * ease of addition of new bridges into the full topology, * automatic reconfiguration in case of problems in the backbone, * optimal route automatically chosen, * monitoring included in the backbone management procedures, * no Ethernets forced to carry third-party traffic. Note that the last point underlines one other disadvantage of current offerings that Charles Hendrick missed: in the example A -------- B --------- C B will have to carry all of the A to C traffic. So, in summary, the current topological shortcomings of bridges are due to the technology (and some manufacturer ignorance?); they are not inherent in the concept. Crispin Piney CERN 1211, Geneva, 23 Switzerland