[comp.protocols.tcp-ip] How to set up subnets where logical subnet != physical subnet

dcm@baldur.dell.com (Dave McCracken) (04/10/91)

I am trying to solve a problem we are having trying to set up a
subnet in our corporate network.

The current environment is an ethernet with MAC-layer bridges
between all subnets, which are necessary because we need to support
IP, Netware IPX, and 3com 3+share packets.

What we would like to do is split off the Unix development group
as an IP-only subnet, with the rest of the network as the other subnet.
What we would end up with is a small subnet with 30-40 machines
connected to a large subnet with 500-800 machines, of which probably
100-300 will use IP at least some of the time.

Our IP address numbering scheme is a class B network address, and
we would like to run with a 6-8 bit subnet mask.  We have already
assigned subnet numbers based on department for administrative reasons.

The real problem is when I try to set up a router for this arrangement.
Since most of the network is being routed by a layer below IP, it 
is necessary for several logical subnets, based on the subnet
mask, to be on the same physical network.  The routing code in the IP
driver will cheerfully accept that the other subnets are local when
I specify 0 hops to the route command, but it absolutely refuses
to let me specify an IP address for the router that is not in the
same logical subnet.  We are currently running mostly System V Release 4,
but the same problem exists on our Suns and in the straight BSD4.3
code (I looked in the source).

What I would like to know from the collected wisdom of Usenet is
why the restriction is there, and if you think anything would break
if I changed the IP driver in SVR4 to accept a router address outside
the subnet.  I would also like to know is there is a simple way in the
router to present miltiple IP addresses without plugging in extra
network cards.  This would be an alternate solution that would not
require changing all clients.

Thanks,

--
Dave McCracken      dcm@dell.dell.com      (512) 343-3720
Dell Computer       9505 Arboretum Blvd    Austin, TX 78759-7299

ji@cs.columbia.edu (John Ioannidis) (04/10/91)

In article <dcm.671220071@baldur.dell.com> dcm@baldur.dell.com (Dave McCracken) writes:
>I am trying to solve a problem we are having trying to set up a
>subnet in our corporate network.
>
>
 [ Description of their corporate network with MAC-layer bridges and
   multiple IP subnets on the same wire delted. ]
	

>is necessary for several logical subnets, based on the subnet
>mask, to be on the same physical network.  The routing code in the IP
>driver will cheerfully accept that the other subnets are local when
>I specify 0 hops to the route command, but it absolutely refuses
>to let me specify an IP address for the router that is not in the
>same logical subnet.  We are currently running mostly System V Release 4,
>but the same problem exists on our Suns and in the straight BSD4.3
>code (I looked in the source).

Let us be specific. Assume your campus network is 182.95 (nice unused
Class-B network), and that "subnets" 182.95.20, 182.95.21 and
182.95.22 are all on the same wire.

Your hostname is host-20-19 and its address is 182.95.20.19.
Your ethernet inteface has been configured as

# ifconfig le0 182.95.20.19 up netmask 255.255.255.0 -trailers

Your routing table looks something like:

# netstat -r -n
Routing tables
Destination          Gateway              Flags    Refcnt Use        Interface
127.0.0.1            127.0.0.1            UH       0      0          lo0
182.95.20            182.95.20.19         U        0      5          le0

Now, in order to access machines on the subnets .21. and .22., you add
static routes like this:

# route add net 182.95.21 182.95.20.19 0
# route add net 182.95.22 182.95.20.19 0

So that your routing table now looks like:

# netstat -r -n
Routing tables
Destination          Gateway              Flags    Refcnt Use        Interface
127.0.0.1            127.0.0.1            UH       0      0          lo0
182.95.20            182.95.20.19         U        0      5          le0
182.95.21            182.95.20.19         U        0      0          le0
182.95.22            182.95.20.19         U        0      0          le0

Now, for reasons that I'd rather not know (!), there exists a router
(call it router-21-1, address 182.95.21.1) to some other net(s), that
you want to use from subnets .20. and .22.. Evidently, you cannot say

# route add default 182.95.21.1 1 

on host-20-19; the route command will say (and with good reason):

add net default: gateway 182.95.21.1: Network is unreachable

>
>What I would like to know from the collected wisdom of Usenet is
>why the restriction is there, and if you think anything would break
>if I changed the IP driver in SVR4 to accept a router address outside

The "restriction" is there because of the way routes are set up with
SIOCADDRT. For gatewaying through another machine (metric > 0), the
code checks whether that gateway is on the same subnet as yourself. If
it is not, it gives you a "network is unreachable" error.

Conceivably, you may want to check whether the subnet of the gateway
you are trying to route through already has a route through yourself
(which is your case), and thus allow the addition of routes to
machines not on your subnet but still on the same physical network.
There is no reason this should create any problems, unless someone
deletes those static routes.

Of course, the whole reason for these network gymnastics is that you
need the *ethernet* address of a gateway to send the packets through.
The gateway may be on the same wire as you are (so you can send it the
packets), but the routing code will not allow you to add it.

Instead, you can fool your code into thinking it's using a gateway on
its subnet in the following (hacky) way: Assign a dummy IP address on
sunet 20 to your router and a machine on the same physical to
proxy-arp for it: We've already said that your router is 182.95.21.1.
Now, reserve the address 182.95.20.254 for it. On some machine on the
wire, add the following ARP entry:

# arp -s 182.95.20.254 <router-21-1's ethernet address>

and on all machines on subnet .20. add the routing entry:

# route add default 182.95.20.254 1

On router-21-1, add the following static routes:

# route add net 182.95.20 182.95.21.1 0
# route add net 182.95.22 182.95.21.1 0

Now, every time you want to send something out that would have to go
through router-21-1, host-20-19 will arp for 182.95.20.254. The
machine with the static ARP entry will respond with .21.1's ethernet
address, and your host will send the IP packet to that ethernet
address. Now, the router does not care what the source is; it only
cares what the destination is. Upon receipt of a packet, if it can
route the packet, it will do so. So this takes care of routing packets
out. The static routes we set up on router-21-1 will take care of
routing packets back to hosts on .20. and .22. 

>the subnet.  I would also like to know is there is a simple way in the
>router to present miltiple IP addresses without plugging in extra
>network cards.  This would be an alternate solution that would not

CISCOs can do that. On BSD-derived Unixes, although there is a linked
list of addresses for each interface, there are no ioctl's that will
allow you to bind multiple addresses to an interface. The ifnet
structure has a pointer to a linked list of addresses for the
interface, but I suspect that too much code just assumes that there is
only one address per interface. I haven't looked at the multicast code
lately, but I don't think it uses the linked list of addresses;
someone please correct me if I'm wrong (I hope I am; I'll be needing
the ability to have the same interface have multiple addresses very soon!)


>require changing all clients.
>
>Thanks,
>
>--
>Dave McCracken      dcm@dell.dell.com      (512) 343-3720
>Dell Computer       9505 Arboretum Blvd    Austin, TX 78759-7299

Hope this helps

/ji

In-Real-Life: John "Heldenprogrammer" Ioannidis
E-Mail-To: ji@cs.columbia.edu
V-Mail-To: +1 212 854 8120
P-Mail-To: 450 Computer Science \n Columbia University \n New York, NY 10027

gah@hood.hood.caltech.edu (Glen Herrmannsfeldt) (04/14/91)

More than one subnet on a physical cable....

This is possible.  I understand that cisco routers actually know
how to do this.  Otherwise, the two interface method.
THe two interface method does not work for machines that assign
the same ethernet (hardware) address to all ports on the machine.
Suns do this, I don't know about others.

The next possible problem is that some machines (suns, again)
complain if they see routing packets for the wrong subnet.
This can probably be fixed in syslog.conf, but is something to
know about.  THe messages appear on the console, ordinarily.

Here, we have more than one subnet on one cable, in anticipation
of splitting it when a new router arrives.  All except one are
not broadcasting routes, so that static routing must be used.u!

barrett@Daisy.EE.UND.AC.ZA (Alan P. Barrett) (04/15/91)

In article <1991Apr10.063716.9725@cs.columbia.edu>,
ji@liberty.columbia.edu (John Ioannidis) writes:
> >router to present miltiple IP addresses without plugging in extra
> >network cards.  This would be an alternate solution that would not
> 
> CISCOs can do that. On BSD-derived Unixes, [you probably can't do it].

PC-Route can do that, if you get the declare.inc file right, which is
not very difficult.  However, the documentation doesn't even mention
that this is possible, much less explain how to do it.

OK, now a question for the standards folk.

Section 2.3 of RFC791 (the IP standard) has this to say:

rfc791> Care must be taken in mapping internet addresses to local net
rfc791> addresses; a single physical host must be able to act as if it were
                                          ^^^^
rfc791> several distinct hosts to the extent of using several distinct
rfc791> internet addresses.  Some hosts will also have several physical
rfc791> interfaces (multi-homing).

If the word "must" here really means MUST in the Host-Requirements
sense, then shouldn't all IP implementations allow a host to present
multiple IP addresses without having multiple interfaces?

--apb
Alan Barrett, Dept. of Electronic Eng., Univ. of Natal, Durban, South Africa
Internet: barrett@ee.und.ac.za           UUCP: m2xenix!quagga!undeed!barrett

ckollars@deitrick.East.Sun.COM (Chuck Kollars - Sun Technical Marketing - Boston) (04/16/91)

In article <gah.671608756@hood> gah@hood.hood.caltech.edu (Glen Herrmannsfeldt) writes:
>More than one subnet on a physical cable....
>
>This is possible.  I understand that cisco routers actually know
>how to do this.  Otherwise, the two interface method.
>THe two interface method does not work for machines that assign
>the same ethernet (hardware) address to all ports on the machine.
>Suns do this, I don't know about others.

Some machines default to using the same Ethernet address on all
interfaces, some default to using different Ethernet addresses on all
interfaces, and some don't allow you to override the default with
whatever you really want.  Sun machines have flip-flopped their default
behavior, since half the world is unhappy if the default is one way,
and the other half the world is unhappy if the default is the other
way.

More important than the default is that Sun machines don't _force_ you
do do it either way -- you're welcome to override the default to meet
your needs.

cheers!
---
chuck kollars    <ckollars@East.Sun.COM>
Sun Technical Marketing, located in Sun's Boston Development Center