geoff@FERNWOOD.MPK.CA.US (the terminal of Geoff Goodfellow) (09/01/89)
THE CRUCIBLE INTERNET EDITION
an eleemosynary publication of the August, 1989
Anterior Technology IN MODERATION NETWORK(tm) Volume 1 : Issue 1
Geoff Goodfellow
Moderator
In this issue:
A Critical Analysis of the Internet Management Situation
THE CRUCIBLE is an irregularly published, refereed periodical on the
Internet. The charter of the Anterior Technology IN MODERATION NETWORK
is to provide the Internet and Usenet community with useful, instructive
and entertaining information which satisfies commonly accepted standards
of good taste and principled discourse. All contributions and editorial
comments to THE CRUCIBLE are reviewed and published without attribution.
Cogent, cohesive, objective, frank, polemic submissions are welcomed.
Mail contributions/editorial comments to: crucible@fernwood.mpk.ca.us
------------------------------------------------------------------------------
A Critical Analysis of the Internet Management Situation:
The Internet Lacks Governance
ABSTRACT
At its July 1989 meeting, the Internet Activities Board made some
modifications in the management structure for the Internet. An outline of
the new IAB structure was distributed to the Internet engineering community
by Dr. Robert Braden, Executive Director. In part, the open letter stated:
"These changes resulted from an appreciation of our successes, especially
as reflected in the growth and vigor of the IETF, and in rueful
acknowledgment of our failures (which I will not enumerate). Many on
these lists are concerned with making the Internet architecture work in
the real world."
In this first issue of THE INTERNET CRUCIBLE we will focus on the failures
and shortcomings in the Internet. Failures contain the lessons one often
needs to achieve success. Success rarely leads to a search for new
solutions. Recommendations are made for short and long term improvements
to the Internet.
A Brief History of Networking
The Internet grew out of the early pioneering work on the ARPANET. This
influence was more than technological, the Internet has also been
significantly influenced by the economic basis of the ARPANET.
The network resources of the ARPANET (and now Internet) are "free". There
are no charges based on usage (unless your Internet connection is via an
X.25 Public Data Network (PDN) in which case you're well endowed, or better
be). Whether a site's Internet connection transfers 1 packet/day or a 1M
packets/day, the "cost" is the same. Obviously, someone pays for the
leased lines, router hardware, and the like, but this "someone" is, by and
large, not the same "someone" who is sending the packets.
In the context of the Research ARPANET, the "free use" paradigm was an
appropriate strategy, and it has paid handsome dividends in the form of
developing leading edge packet switching technologies. Unfortunately,
there is a significant side-effect with both the management and technical
ramifications of the current Internet paradigm: there is no
accountability, in the formal sense of the word.
In terms of management, it is difficult to determine who exactly is
responsible for a particular component of the Internet. From a technical
side, responsible engineering and efficiency has been replaced by the
purchase of T1 links.
Without an economic basis, further development of short-term Internet
technology has been skewed. The most interesting innovations in
Internet engineering over the last five years have occurred in resource
poor, not resource rich, environments.
Some of the best known examples of innovative Internet efficiency
engineering are John Nagle's tiny-gram avoidance and ICMP source-quench
mechanisms documented in RFC896, Van Jacobsen's slow-start algorithms and
Phil Karn's retransmission timer method.
In the Nagle, Jacobsen and Karn environments, it was not possible or cost
effective to solve the performance and resource problems by simply adding
more bandwidth -- some innovative engineering had to be done.
Interestingly enough, their engineering had a dramatic impact on our
understanding of core Internet technology.
It should be noted that highly efficient networks are important when
dealing with technologies such as radio where there is a finite amount of
bandwidth/spectrum to be had. As in the Nagle, Jacobsen and Karn cases,
there are many environments where adding another T1 link can not be used
to solve the problem. Unless innovation continues in Internet
technology, our less than optimal protocols will perform poorly in
bandwidth or resource constrained environments.
Developing at roughly the same time as Internet technology have been the
"cost-sensitive" technologies and services, such as the various X.25-based
PDNs, the UUCP and CSNET dial-up networks. These technologies are all
based on the notion that bandwidth costs money and the subscriber pays for
the resources used. This has the notable effect of focusing innovation to
control costs and maximize efficiency of available resources and bandwidth.
Higher efficiency is achieved by concentrating on sending the most amount
of information through the pipe in the most efficient manner thereby making
the best use of available bandwidth/cost ratio.
For example, bandwidth conservation in the UUCP dial-up network has
multiplied by leaps and bounds in the modem market with the innovation of
Paul Baran's (the grandfather of packet switching technology) company,
Telebit, which manufactures a 19.2KB dial-up modem especially optimized for
UUCP and other well known transfer protocols. For another example,
although strictly line-at-a-time terminal sessions are less "user friendly"
than character-oriented sessions, they make for highly efficient use of
X.25 PDN network resources with echoing and editing performed locally on
the PAD.
While few would argue the superiority of X.25 and dial-up CSNET and UUCP,
these technologies have proved themselves both to spur innovation and to be
accountable. The subscribers to such services appreciate the cost of the
services they use, and often such costs form a well-known "line item" in
the subscriber's annual budget.
Nevertheless, the Internet suite of protocols are eminently successful,
based solely on the sheer size and rate of growth of both the Internet and
the numerous private internets, both domestically and internationally. You
can purchase internet technology with a major credit card from a mail order
catalog. Internet technology has achieved the promise of Open Systems,
probably a decade before OSI will be able to do so.
Failures of the Internet
The evolution and growth of Internet technology have provided the basis for
several failures. We think it is important to examine failures in detail,
so as to learn from them. History often tends to repeat itself.
Failure 1:- Network Nonmanagement
The question of responsibility in todays proliferated Internet is completely
open. For the last three years, the Internet has been suffering from
non-management. While few would argue that a centralized czar is necessary
(or possible) for the Internet, the fact remains there is little to be done
today besides finger-pointing when a problem arises.
In the NSFNET, MERIT is incharge of the backbone and each regional network
provider is responsible for its respective area. However, trying to debug
a networking problem across lines of responsibility, such as intermittent
connectivity, is problematic at best. Consider three all too true refrains
actually heard from NOC personal at the helm:
"You can't ftp from x to y? Try again tomorrow, it will
probably work then."
"If you are not satisfied with the level of [network]
service you are receiving you may have it disconnected."
"The routers for network x are out of table space for routes,
which is why hosts on that network can't reach your new
(three-month old) network. We don't know when the routers will
be upgraded, but it probably won't be for another year."
One might argue that the recent restructuring of the IAB may work towards
bringing the Internet under control and Dr. Vinton G. Cerf's recent
involvement is a step in the right direction. Unfortunately, from a
historical perspective, the new IAB structure is not likely to be
successful in achieving a solution. Now the IAB has two task forces, the
Internet Research Task Force (IRTF) and the Internet Engineering Task Force
(IETF). The IRTF, responsible for long-term Internet research, is largely
composed of the various task forces which used to sit at the IAB level.
The IETF, responsible for the solution of short-term Internet problems, has
retained its composition.
The IETF is a voluntary organization and its members participate out of
self interest only. The IETF has had past difficulties in solving some of
the Internet's problems (i.e., it has taken the IETF well over a year to
not yet produce RFCs for either a Point-To-Point Serial Line IP or Network
Management enhancements). It is unlikely that the IETF has the resources
to mount a concerted attack against the problems of today's ever expanding
Internet. As one IETF old-timer put it: "No one's paid to go do these
things, I don't see why they (the IETF management) think they can tell us
what to do" and "No one is paying me, why should I be thinking about the
these things?"
Even if the IETF had the technical resources, many of the Internet's
problems are also due to lack of "hands on" management. The IETF
o Bites off more than it can chew;
o Sometimes fails to understand a problem before making a solution;
o Attempts to solve political/marketing problems with technical
solutions;
o Has very little actual power.
The IETF has repeatedly demonstrated the lack of focus necessary to
complete engineering tasks in a timely fashion. Further, the IRTF is
chartered to look at problems on the five-year horizon, so they are out of
the line of responsibility. Finally, the IAB, per se, is not situated to
resolve these problems as they are inherent to the current structure of
nonaccountability.
During this crisis of non-management, the Internet has evolved into a
patch quilt of interconnected networks that depend on lots of
seat-of-the-pants flying to keep interoperating. It is not an unusual
occurrence for an entire partition of the Internet to remain disconnected
for a week because the person responsible for a key connection went on
vacation and no one else knew how to fix it. This situation is but one
example of an endemic problem of the global Internet.
Failure 2:- Network Management
The current fury over network management protocols for TCP/IP is but a
microcosm of the greater Internet vs. OSI debate going on in the
marketplace. While everyone in the market says they want OSI, anyone
planning on getting any work done today buys Internet technology. So it
is with network management, the old IAB made the CMOT an Internet
standard despite the lack of a single implementation, while the only
non-proprietary network management protocol in use in the Internet is the
SNMP. The dual network management standardization blessings will no
doubt have the effect of confusing end-users of Internet
technology--making it appear there are two choices for network
management, although only one choice, the SNMP has been implemented. The
CMOT choice isn't implemented, doesn't work, or isn't interoperable.
To compound matters, after spending a year trying to achieve consensus on
the successor to the current Internet standard SMI/MIB, the MIB working
group was disbanded without ever producing anything: the political climate
prevented them from resolving the matter. (Many congratulatory notes were
sent to the chair of the group thanking him for his time. This is an
interesting new trend for the Internet--congratulating ourselves on our
failures.)
Since a common SMI/MIB could not be advanced, an attempt was made to
de-couple the SNMP and the CMOT (RFC1109). The likely result of RFC1109
will be that the SNMP camp will continue to refine their experience
towards workable network management systems, whilst the CMOT camp will
continue the never-ending journey of tracking OSI while producing demo
systems for trade shows exhibitions. Unfortunately the end-user will
remain ever confused because of the IAB's controversial (and technically
questionable) decision to elevate the CMOT prior to implementation.
While the network management problem is probably too large for the SNMP
camp to solve by themselves they seem to be the only people who are
making any forward progress.
Failure 3:- Bandwidth Waste
Both the national and regional backbone providers are fascinated with T1
(and now T3) as the solution towards resource problems. T1/T3 seems to
have become the Internet panacea of the late 80's. You never hear anything
from the backbone providers about work being done to get hosts to implement
the latest performance/congestion refinements to IP, TCP, or above.
Instead, you hear about additional T1 links and plans for T3 links. While
T1 links certainly have more "sex and sizzle" than efficient technology
developments like slow-start, tiny gram avoidance and line mode telnet, the
majority of users on the Internet will probably get much more benefit from
properly behaving hosts running over a stable backbone than the current
situation of misbehaving and semi-behaved hosts over an intermittent catenet.
Failure 4:- Routing
The biggest problem with routing today is that we are still using phase I
(ARPANET) technology, namely EGP. The EGP is playing the role of routing
glue in providing the coupling between the regional IGP and the backbone
routing information. It was designed to only accommodate a single point of
attachment to the catenet (which was all DCA could afford with the PSNs).
However with lower line costs, one can build a reasonably inexpensive
network using redundant links. However the EGP does not provide enough
information nor does the model it is based upon support multiple
connections between autonomous systems. Work is progressing in the
Interconnectivity WG of the IETF to replace EGP. They are in the process
of redefining the model to solve some of the current needs. BGP or the
Border Gateway Protocol (RFC1105) is an attempt to codify some of the ideas
the group is working on.
Other problems with routing are caused by regionals wanting a backdoor
connection to another regional directly. These connections require some
sort of interface between the two routing systems. These interfaces are
built by hand to avoid routing loops. Loops can be caused when information
sent into one regional network is sent back towards the source. If the
source doesn't recognize the information as its own, packets can flow until
their time to live field expires.
Routing problems are caused by the interior routing protocol or IGP. This
is the routing protocol which is used by the regionals to pass information
to and from its users. The users themselves can use a different IGP than
the regional. Depending on the number of connections a user has to the
regional network, routing loops can be an issue. Some regionals pass
around information about all known networks in the entire catenet to their
users. This information deluge is a problem with some IGPs. Newer IGPs
such as the new OSPF from the IETF and IGRP from cisco attempt to provide
some information hiding by adding hierarchy. OSPF is the internets first
attempt at using a Dykstra type algorithm as an IGP. BBN uses it to route
between their packet switch nodes below the 1822 or X.25 layer.
Unstable routing is caused by hardware or hosts software. Older BSD
software sets the TTL field in the IP header to a small number. The
Internet today is growing and its diameter has exceed the software's
ability to reach the other side. This problem is easily fixed by
knowledgeable systems people, but one must be aware of the problem before
they can fix it.
Routing problems are also perceived when in fact a serial line problem or
hardware problem is the real cause. If a serial line is intermittent or
quickly cycles from the up state into the down state and back again,
routing information will not be supplied in a uniform or smooth manner.
Most current IGPs are Bellman-Ford based and employ some stabilizing
techniques to stem the flow of routing oscillations due to "flapping"
lines. Often when a route to a network disappears, it may take several
seconds for it to reappear. This can occur at the source router who waits
for the route to "decay" from the system. This pause should be short
enough so that active connections persist but long enough that all routers
in the routing system "forget" about routes to that network. Older host
software with over-active TCP retransmission timers will time out
connections instead of persevering in the face of this problem. Also
routers, according to RFC1009, must be able to send ICMP unreachables when
a packet is sent to a route which is not present in its routing database.
Some host products on the market close down connections when a single ICMP
reachable is received. This bug flies in the face of the Internet parable
"be generous in what you accept and rigorous in what you send".
Many of the perceived routing problems are really complex multiple
interactions of differing products.
Causes of the Failures
The Internet failures and shortcomings can be traced to several sources:
First and foremost, there is little or no incentive for efficiency and/or
economy in the current Internet. As a direct result, the resources of the
Internet and its components are limited by factors other than economics.
When resources wear thin, congestion and poor performance result. There is
little to no incentive to make things better, if 1 packet out of 10 gets
through things "sort of work". It would appear that Internet technology
has found a loophole in the "Tragedy of The Commons" allegory--things get
progressively worse and worse, but eventually something does get through.
The research community is interested in technology and not economics,
efficiency or free-markets. While this tack has produced the Internet
suite of protocols, the de facto International Standard for Open Systems,
it has also created an atmosphere of intense in-breeding which is overly
sensitive to criticism and quite hardened against outside influence.
Meanwhile, the outside world goes on about developing economically viable
and efficient networking technology without the benefit of direct
participation on the part of the Internet.
The research community also appears to be spending a lot of its time
trying to hang onto the diminishing number of research dollars available
to it (one problem of being a successful researcher is eventually your
sponsors want you to be successful in other things). Despite this, the
research community actively shuns foreign technology (e.g., OSI), but,
inexplicably has not recently produced much innovation in new Internet
technology. There is also a dearth of new and nifty innovative
applications on the Internet. Business as usual on the Internet is mostly
FTP, SMTP and Telnet or Rlogin as it has been for many years. The most
interesting example of a distributed application on the Internet today is
the Domain Name System, which is essentially an administrative facility,
not an end-user service.
The engineering community must receive equal blame in these matters.
While there have been some successes on the part of the engineering
community, such as those by Nagel, Jacobsen and Karn mentioned above, the
output of the IETF, namely RFCs and corresponding implementations, has
been surprisingly low over its lifetime.
Finally, the Internet has become increasingly dependent on vendors for
providing implementations of Internet technology. While this is no doubt
beneficial in the long-term, the vendor community, rather than investing
"real" resources when building these products, do little more than
shrink-wrap code written primarily by research assistants at universities.
This has lead to cataclysmic consequences (e.g., the Internet worm
incident, where Sendmail with "debug" command and all was packaged and
delivered to customers without proper consideration). Of course, when
problems are found and fixed (either by the vendor's customers or software
sources), the time to market with these fixes is commonly a year or longer.
Thus, while vendors are vital to the long-term success of Internet
technology, they certainly don't receive high marks in the short-term.
Recommendations
Short-term solutions (should happen by year's end):
In terms of hardware, the vendor community has advanced to the point where
the existing special-purpose technologies (Butterfly, NSSs) can be replaced
by off-the-shelf routers at far less cost and with superior throughput and
reliability. Obvious candidates for upgrade are both the NSFNET and
ARPANET backbones. Given the extended unreliability of the mailbridges,
the ARPA core is an immediate candidate (even though the days of net 10 are
numbered).
In terms of software, ALL devices in the Internet must be network
manageable. This is becoming ever more critical when problems must be
resolved. Since SNMP is the only open network management protocol
functioning in the Internet, all devices must support SNMP and the Internet
standard SMI and MIB.
Host implementations must be made to support the not-so-recent TCP
enhancements (e.g., those by Nagle, Jacobsen and Karn) and the more
recent linemode TELNET.
The national and regional providers must coordinate to share network
management information and tools so that user problems can be dealt with in
a predictable and timely fashion. Network management tools are a big help,
but without the proper personnel support above this, the benefits can not
be fully leveraged.
The Internet needs leadership and hands-on guidance. No one is seemingly
in charge today, and the people who actually care about the net are pressed
into continually fighting the small, immediate problems.
Long-term solutions:
To promote network efficiency and a free-market system for the delivery of
Internet services, it is proposed to switch the method by which the network
itself is supported. Rather than a top-down approach where the money goes
from funding agencies to the national backbone or regional providers, it is
suggested the money go directly to end-users (campuses) who can then select
from among the network service providers which among them best satisfies
their needs and costs.
This is a strict economic model: by playing with the full set of the laws of
economics, a lot of the second-order problems of the Internet, both present
and on the horizon, can be brought to heel. The Internet is no longer a
research vehicle, it is a vibrant production facility. It is time to
acknowledge this by using a realistic economic model in the delivery of
Internet services to the community (member base).
When Internet sites can vote with their pocketbooks, some new regionals
will be formed; some, those which are non-performant or uncompetitive, will
go away; and, the existing successful ones will grow. The existing
regionals will then be able to use their economic power, as any consumer
would, to ensure that the service providers (e.g., the national backbone
providers) offer responsive service at reasonable prices. "The Market" is
a powerful forcing function: it will be in the best interests of the
national and regional providers to innovate, so as to be more competitive.
Further, such a scheme would also allow the traditional telecommunications
providers a means for becoming more involved in the Internet, thus allowing
cross-leverage of technologies and experience.
The transition from top-down to economic model must be handled carefully,
but this is exactly the kind of statesmanship that the Internet should
expect from its leadership.
-------henry@utzoo.uucp (Henry Spencer) (09/07/89)
In article <8909010312.AA03885@fernwood.MPK.CA.US> geoff@FERNWOOD.MPK.CA.US (the terminal of Geoff Goodfellow) writes: >While few would argue the superiority of X.25 and dial-up CSNET and UUCP, >these technologies have proved themselves both to spur innovation and to be >accountable. The subscribers to such services appreciate the cost of the >services they use, and often such costs form a well-known "line item" in >the subscriber's annual budget. This is undoubtedly true for X.25 and dialup CSNET, but for UUCP the exact reverse is often the case: a network connection exists precisely because, unlike the alternatives, a UUCP connection does *not* require a line item in the budget. All it requires is suitable software (present in most versions of Unix already), dialup modems on both ends (usually present anyway), and a bit of setup work. If no obscure problems intervene, such a connection can be up and running in fifteen minutes, with *no* paperwork. If traffic remains modest -- where the exact semantics of "modest" depend on modem type and the cost (if any) of the calls -- often no formal justification of the connection is ever required. When communication is a means to an end rather than a research topic, this can be a major asset. -- V7 /bin/mail source: 554 lines.| Henry Spencer at U of Toronto Zoology 1989 X.400 specs: 2200+ pages. | uunet!attcan!utzoo!henry henry@zoo.toronto.edu