Ira@CIS.UPENN.EDU.UUCP (06/04/87)
This Spring I evaluated seven commerical implementations of TCP/IP for
VAX/VMS for use on the University of Pennsylvania's campus network, which
is Ethernet based. I have included the results of my evaluation in this
message. Neither the University of Pennsylvania nor I guarantee the
correctness of the results obtained. However, the results were given
to each vendor and they were each given an opportunity to correct
any incorrect statements.
Note that CMU/TEK TCP/IP was not tested because we haven't received
our copy yet.
Ira Winston
University of Pennsylvania
ira@cis.upenn.edu
===============================================================================
All products were tested on a Microvax 2 in a BA23 pedestal cabinet, with 2
megabytes of memory, RQDX3 controller, RD53 disk drive, TK50 tape drive, VMS
4.4 and DEQNA ethernet interface (for software-only implementations).
A DEC DELNI was used as the transceiver.
FTP tests were performed with a VAX 8650 running Ultrix 1.2. The outbound
tests transferred a 2.7 megabyte file ten times in binary mode to /dev/null on
the 8650. The inbound tests transferred a 2.7 megabyte file ten times in
binary mode to nla0: on the microvax.
CPU overhead was measured by observing how much CPU time the NULL process
used during the then 2.7 mb transfers and subtracting this from the
elapsed time to perform the transfer. The result was the total amount of
CPU time, including interrupt stack time, used in peforming the
transfer.
Memory utilization was measured by observing the combined size
of the free list and the modified page list before and after the
TCP/IP software was running. Automatic working set adjustment
was disabled for the process doing the measurements to obtain
more consistent results.
Interoperability was tested with Ultrix 1.2, Wollongong WIN/VX 2.3, SunOS 3.2,
Bridge CS/200, Bridge CS/1, and Wiscnet/DACU 1.3
-------------------------------------------------------------------------------
Product
Network Research Corporation - Fusion Network Software
Version
3.1
Ethernet Interface(s)
Shared DEUNA/DEQNA
Address Resolution Protocol
supported
IP trailer encapsulation
not supported
IP Broadcast Technique
all ones or all zeros on input, all ones on output,
option for sending to both addresses on output
Subnets
supported
IP Fragmentation
supported
ICMP Redirects
?
Name to IP address Translation
text file in NRC format, synonyms on multiples lines
Routing
static routes
Program interface
socket library
User FTP
batch supported, several file modes, multiple file transfers
Server FTP
slow startup, ignores DISUSER, no anonymous FTP
User telnet
slow connections, fails with terminals set for even parity
many network control characters set by default
Server telnet
2 processes per session, fixed number of sessions,
Set Term/Unknown prevents further input
SMTP
optional, didn't test
Berkeley "R" services
optional, didn't test
Disk space required
4.5 megabytes
Installation procedure
a little confusing
Documentation
acceptable
Network statistics
nstat program
Interoperability with other campus products
no problems
Ability to shutdown and restart
available, long startup time
Performance
16 kilobytes/second - outbound FTP, 50 kilobytes/second - inbound FTP
Overhead
47 kbytes/cpu second - outbound FTP, 51 kbytes/cpu second - inbound FTP
Character echo time
acceptable
Process and memory utilization
400Kbytes memory, 2 permanent processes
VMS help file
provided
Comments
Ethernet address isn't read from board
Packet monitor provided
Network test utility provided
Finger server provided
Shared DEUNA uses ALTSTART
Processes running FTP and telnet tend to hang
-------------------------------------------------------------------------------
Product
The Wollongong Group - WIN/VX
Version
2.3 and 3.0 (beta test)
Ethernet Interface(s)
DEUNA, DEQNA, Shared DEUNA/DEQNA, Interlan NI1010, 3COM
Address Resolution Protocol
supported
IP trailer encapsulation
optional
IP Broadcast Technique
all zeros, configurable in 3.0
Subnets
not supported until 3.0
IP Fragmentation
supported
ICMP Redirects
supported - one route per connection until 3.0
Name to IP address Translation
/etc/hosts format, named in 3.0
Routing
static routes
Program interface
socket library (VAX C version in 3.0) and qio interface
User FTP
Batch supported, .exe files can been tranferred, multiple file transfers
Server FTP
ignores DISUSER, no anonymous login
User telnet
user option negotiation
Server telnet
no fixed number of sessions
SMTP
MAILER!, PMDF
Berkeley "R" services
clients, servers with eunice
Disk space required
6.5mb - although many files can be deleted
Installation procedure
good
Documentation
good
Network statistics
netstat
Interoperability with other campus products
can't telnet to a Bridge CS/1 in 'reverse' mode without
negotiating "WONT BIN" - fixed in 3.0
Ability to shutdown and restart
shared DEUNA/DEQNA can be stopped and restarted
Performance
version 2.3
33 kilobytes/second - outbound FTP, 40 kilobytes/second - inbound FTP
version 3.0
70 kilobytes/second - outbound FTP, 56 kilobytes/second - inbound FTP
Overhead
version 2.3
36 kbytes/cpu second - outbound FTP, 41 kbytes/cpu second - inbound FTP
version 3.0
77 kbytes/cpu second - outbound FTP, 73 kbytes/cpu second - inbound FTP
Character echo time
acceptable
Process and memory utilization
240Kbytes and 2 permanent processes (3 if named is used)
VMS help file
man pages
Comments
user and server finger provided, shared deuna uses ALTSTART
-------------------------------------------------------------------------------
Product
MICOM-Interlan - NP200/NP221
Version
1.1
Ethernet Interface(s)
NP200 protocol processor
quad height board
4.5 amps +5vdc
0.5 amps +12vdc - 3 amp surge
status LED's
diagnostics
802.3 compatible physical level
Address Resolution Protocol
supported
IP trailer encapsulation
optional
IP Broadcast Technique
none
Subnets
not supported until next release
IP Fragmentation
supported
ICMP Redirects
supported - one route per connection
Name to IP address Translation
/etc/hosts format, named in next release
Routing
static routes
Program interface
socket library
User FTP
Batch supported, multiple file transfers
Server FTP
ignores DISUSER, no anonymous login
User telnet
problems with echoing line-feeds
Server telnet
fixed number of sessions
SMTP
in next release
Berkeley "R" services
not supported
Disk space required
3.5mb
Installation procedure
good, must be installed in sys$sysroot:[sysmgr.np221]
Documentation
good
Network statistics
next release
Interoperability with other campus products
no problems
Ability to shutdown and restart
available but board emits 10 loopback packets per second when not running
Performance
43 kilobytes/second - outbound FTP, 63 kilobytes/second - inbound FTP
Overhead
486 kbytes/cpu second - outbound FTP, 138 kbytes/cpu second - inbound FTP
Character echo time
a little slow, rubout generates 3 packets (BS SP BS)
Process and memory utilization
50Kbytes and 1 permanent process
VMS help file
not provided
Comments
ICMP no such port sent in response to every IP broadcast
telnet keepalives used
link layer interface provided
-------------------------------------------------------------------------------
Product
Excelan EXOS 203
Version
Hardware 5.3, Software 3.3
Ethernet Interface(s)
quad board
status LED's
5 amps +5vdc
.6 amps +12vdc
Address Resolution Protocol
supported
IP trailer encapsulation
no trailers on output, accepted on input
IP Broadcast Technique
all ones or all zeros on output - configurable
all ones or all zeros on input
Subnets
supported
IP Fragmentation
supported
ICMP Redirects
supported
Name to IP address Translation
/etc/hosts format with one minor difference, named in next release
Routing
static routes
Program interface
qio socket library
User FTP
batch supported, .exe files can be transferred, recursive copy
multiple connections
Server FTP
observes DISUSER and /NETWORK, anonymous login supported
User telnet
no problems
Server telnet
limited number of sessions - 32
SMTP
EXOS% interface provided, mail routing not flexible enough
Berkeley "R" services
user rsh
Disk space required
1.5mb
Installation procedure
good
Documentation
good
Network statistics
utility provided
Interoperability with other campus products
no problems
Ability to shutdown and restart
yes
Performance
88 kilobytes/second - outbound FTP, 96 kilobytes/second - inbound FTP
Overhead
270 kbytes/cpu second - outbound FTP, 182 kbytes/cpu second - inbound FTP
Character echo time
a little slow
Process and memory utilization
215Kbytes and 2 permanent processes
VMS help file
provided
Comments
non-IP arp packets available
link layer interface provided
so_large option
-------------------------------------------------------------------------------
Product
SRI Multinet
Version
4.3
Ethernet Interface(s)
Shared VAX/VMS, Interlan NI1010, 3COM
Address Resolution Protocol
supported
IP trailer encapsulation
negotiated
IP Broadcast Technique
all zeros or all ones - configurable
Subnets
supported
IP Fragmentation
supported
ICMP Redirects
supported
Name to IP address Translation
hosts.txt with utility to translate to binary file for fast lookup
named coming
Routing
static routes
Program interface
socket library, qio interface - wollongong compatible
User FTP
Batch supported via ftp.ini, .exe files can been transferred
multiple file transfers, tops-20 interface
Server FTP
ignores DISUSER, anonymous login with world access
User telnet
tops-20 interface, tells you the host type on connect
Server telnet
no fixed number of sessions
SMTP
pony express, PMDF
Berkeley "R" services
supplied to EUNICE sites
Disk space required
3mb binary, 5mb source
Installation procedure
good
Documentation
none yet
Network statistics
netstat
Interoperability with other campus products
no problems
Ability to shutdown and restart
interfaces can be shutdown and restarted
Performance
43 kilobytes/second - outbound FTP, 87 kilobytes/second - inbound FTP
Overhead
64 kbytes/cpu second - outbound FTP, 113 kbytes/cpu second - inbound FTP
Character echo time
acceptable
Process and memory utilization
245Kbytes and 1 permanent process
VMS help file
none
Comments
user and server finger, chaos and pup, source to all utilities provided
shared DEUNA uses FFI
-------------------------------------------------------------------------------
Product
CMC ENP-50
Version
1.1.4
Ethernet Interface(s)
quad board
status LED
5.5 amps +5vdc
0.8 amps +12vdc
256 bytes of Qbus i/o space used
diagnostics
Address Resolution Protocol
supported
IP trailer encapsulation
not supported
IP broadcast technique
none required
Subnets
not supported until next release
IP Fragmentation
supported
ICMP Redirects
supported
Name to IP address Translation
/etc/hosts format - one name per line
Routing
one default gateway
Program interface
qio library
User FTP
batch not supported, multiple file modes, multiple file transfers
goes into an access violation loop in response to a bad filename on a get
command
Server FTP
ignore DISUSER and /NETWORK, 2 processes per session
User telnet
attention character not programmable
Server telnet
2 processes per session, set term/unknown prevents further input
SMTP
not supported until later release
Berkeley "R" services
not supported
Disk space required
1mb
Installation procedure
good
Documentation
good
Network statistics
utility provided - no ip statistics
Interoperability with other campus products
no problems
Ability to shutdown and restart
yes
Performance
51 kilobytes/second - outbound FTP, 137 kilobytes/second - inbound FTP
Overhead
112 kbytes/cpu second - outbound FTP, 179 kbytes/cpu second - inbound FTP
Character echo time
good
Process and memory utilization
60Kbytes and 1 permanent process
VMS help file
providedgkn@Sdsc.Arpa (06/05/87)
From: Ira Winston <Ira@cis.upenn.edu> Subject: Evaluation of seven implementations of TCP/IP for VAX/VMS Date: Wed, 3 Jun 87 17:53 EDT [much omitted text] Product SRI Multinet [more omitted text] Server FTP ignores DISUSER, anonymous login with world access You've got the old version; I added code to the FTP server which honors DISUSER, expired accounts, expired passwords, network access restrictions and primary/secondary day restrictions months ago. These changes have been incorporated in the last couple of releases from SRI. ANONYMOUS FTP is governed by the existence of a GUEST_FTP account with a fixed password (also subject to the above restrictions). The current version of the server also inherits the account's identifiers, which is very nice (courtesy of someone at MCC, I think). gkn -------------------------------------- Arpa: GKN@SDSC.ARPA Bitnet: GKN@SDSC Span: SDSC::GKN (27.1) USPS: Gerard K. Newman San Diego Supercomputer Center P.O. Box 85608 San Diego, CA 92138 AT&T: 619.534.5076
KASHTAN@IU.AI.SRI.COM.UUCP (06/06/87)
As GKN@SDSC.ARPA has already listed some of the things that have changed
since the tests were run on SRI MultiNet -- I would like to fill in the rest
of the relevent changes since March (tapes for which have already begun
shipping to all MultiNet sites):
1) Host name lookup: BIND ("named" built into master server process)
Binary host table is used as backup when
when BIND is either disabled or cannot
reach any server for the desired domain.
2) Routing: Cornell's "gated" Gateway Daemon
(built into master server process)
supports EGP, RIP (routed) and Fuzzball
HELLO. Will gateway routing information between
these protocols. Route gatewaying is
configurable.
3) User FTP no longer uses the VAX-11 "C" runtime library but
access RMS directly. This has a significant effect on FTP
performance, as the I/O system in the VAX-11 "C" runtime library
is terribly slow. Although I do not have the same configuration
as Ira Winston -- my tests show something like:
80Kb/second OUT
90Kb/second IN
overhead
110kb/CPU-second OUT
115kb/CPU-second IN
David
-------