martillo@cpoint.UUCP (Joacim Martillo) (01/12/89)
I wrote this summary description of XNET and would appreciate comments.
Development Memorandum 0001
To: XNET Developers
From: Joachim Martillo
Subject: XNET Essentials
Date: January 11, 1989
____________________________________________________________
I. Introduction
Originally, communications networks were used only for
remote login and file transfer. Thus, kermit is actually
the modern incarnation of the original network programs.
Fairly quickly people realized that more sophisticated
resource sharing is desirable. The most lucrative
sophisticated network application is probably remote
database systems. The most flashy sophisticated network
application is probably network transparent window systems.
The most ubiquitous is probably network file systems.
Network file systems being an obvious generalization of
remote file transfer is one of the earliest developed of the
genuine resource sharing systems. When I was a student in
1978, we already had remote file systems for the PDP-10
working.
Remote file systems are apparently the most complicated and
least standardized of genuine resource sharing systems.
Important network file systems (sometimes actually remote
virtual disk systems) include NFS, RFS, XNET (Xenix Net,
basically equivalent to MS-Net), Novell and Vines. The
Berkeley Unix developers have recently implemented a new
network file system, and proprietary remote file systems
like Primenet and DECNET remote disks have been around for
years. The plethora of remote file systems and the
complication in network file systems probably originates in
the inherent difficulty of joining communications to such a
basic operating system service like file service. Unlike
the implementer of a network transparent window system, the
implementer of a remote file system has to know lots about
networking and the target operating system. Data caching,
concurrency and security issues are really complicated, and
there is no particular consensus on handling these issues.
When I was a student, a computer scientist proved his
machismo by implementing a debugger. Nowadays, implementing
a network file system seems to be the current proof of
machismo.
II. TCP/IP
Most current popular network file systems take TCP/IP
virtual circuits (or UDP/IP datagram service) as a starting
point. TCP/UDP/IP was developed by DARPA as a means to
interconnect different physical networking systems. (It is
not obvious that most commercial TCP/IP based file systems
actually make use of the ability to interconnect different
physical networks.) IP datagrams provide a network
independent means for gateways to move data packets from one
physical network to another. Gateways are allowed to
fragment IP datagrams when the maximum packet size on the
next subnetwork is smaller than the maximum packet size on
the current subnetwork. TCP and UDP identify specific
source and target applications on the source and target
machines. TCP, UDP and IP are typically implemented as
protocol processing finite state automata in an operating
system kernel. While about 10 years old, TCP/IP is one of
the few network protocol suites which effectively makes of
network bandwidth and avoids thrashing and hysteresis
problems which arise in packet counting based flow control.
A system interface is necessary for user applications to
actually use TCP or UDP services. In Berkeley Unix, the
system designers developed a generalization of files to
provide the interface. The process file table contains
entries associated with file or socket descriptor indices
pointing either to entries in the global kernel file table
or to entries in a global network data connection table.
The double table formalism allows various forms of file or
connection sharing. The details are straightforward but not
so important at this point. The essentials are:
1. operations on file descriptors, pipe descriptors
and socket descriptors are essentially identical
and
2. sockets are a generalization of pipes which can
handle multiplexed data streams and non-
simultaneous rendezvous between several processes.
There is no requirement that the interface to TCP/IP look
like sockets. Inside the unix kernel there is actually
another less general interface to TCP/IP which is used by
kernel processes. In the local environment, both TCP/IP and
the socket interface are provided by Excelan.
III. NetBIOS
NetBIOS is both a protocol suite and user interface defined
in the IBM PC Net Technical Reference. The protocol suite
does not provide internetting (connectivity between
different networks) but does provide a device independent
way of using network services which in the case of netbios
consists of reliable point-to-point virtual circuits.
NetBIOS can handle media providing only broadcast
communications or media providing only point-to-point
service or media providing both types of service.
In NetBIOS, a machine (in the DOS case) or a process (in the
XENIX case) allots itself a name with the local NetBIOS Name
Server (NBSS) and then sends a packet to the NetBIOS
Datagram Distribution Server (NBDDS) which connects to the
NBNS to find the address of the destination host and then
redirects the connection to the destination host. In a pure
broadcast system the connection setup is slightly different.
NetBIOS is a much more chatty communications system than
basic TCP/IP. It seems to use either single-frame-
acknowledged or packet-counting flow control. In the Xenix
case, both the NBNS and NBDDS are implemented on each node
so that many of the steps of connection setup do not
actually require the transmission of network data.
In one LAN environment, Excelan implements NetBIOS on top of
TCP/IP. This sort of implementation immediately provides
internetting capabilities to NetBIOS. Yet, NetBIOS does not
provide any service that TCP/IP does not already provide.
In fact, NetBIOS provides less than TCP/IP and in terms of
network bandwidth utilization actually provides this service
much less efficiently. One might legitimately ask why
anyone would implement NetBIOS on top of TCP/IP and thereby
cripple network performance. In fact, many companies have
done the inverse by implementing TCP/IP on top of NetBIOS so
that a NetBIOS network is simply treated as another type of
physical network. The reason for implementing NetBIOS on
top of TCP/IP is that there are many applications like XNET
or MS-NET which assume a NetBIOS interface to the network.
Implementing NetBIOS on top of TCP/IP gives the ability to
make quick and dirty ports of NetBIOS applications to a
TCP/IP environment.
The major complexity of implementing NetBIOS on top of
TCP/IP seems to be naming. A NetBIOS name (or a group of
names in the case of a multiprocessing Xenix system) needs
to be mapped to a TCP/IP domain name which is then mapped to
an IP address. NetBIOS names and Domain Names have
different sets of legal characters. The high and low nibble
of each byte of the NetBIOS name is added to 'A' and put
into a separate byte which produces an all caps name twice
as long as the original NetBIOS name. The local domain
suffix is then added to the new name. This new name plus
suffix is used as the domain name. These names can be seen
with the Excelan Lanalyzer.
The NetBIOS application interface uses the int 5c DOS system
calls. Excelan provides an int5c interface for Xenix so
that application developers can actually make direct calls
and write NetBIOS applications (like xfserv -- the XNET file
server daemon) under Xenix. The manuals do not document
this interface very well. Really no more information is
provided than can be found in the system include file.
There seems to be no way to wait on several NetBIOS virtual
circuits at the same time using the int5c interface, and
there seems to be no interface from select() to the NetBIOS
virtual circuits. As a consequence, each client process
gets associated with its own xfserv server process on the
file server machine.
IV. Local and Remote File Systems
Understanding local file systems helps understanding remote
file systems. I suppose it might be possible to define a
network representation of files and file systems which is
completely independent of the source and target operating
system file systems, but operating systems tend to consider
different types of information important. XNET is
influenced by the MS-DOS file system while NFS is influenced
by the Unix file system.
A. The DOS File System
A hard disk which contains a DOS partition starts off with a
partition table which identifies DOS partitions and how to
get to them. A DOS partition starts off with a system block
which contains boot and partition parameter information.
The system block is followed by N (specified in the system
block) FATs (File Allocation Tables). The root directory
follows the partition table. The root directory contains a
list of files (including directories) in the root directory.
With each file entry is associated the files first cluster.
(Clusters are sequentially numbered groups of disk sectors)
To get the next cluster, DOS looks at this cluster index in
the FAT. If the cluster index references 0xFFFF (for a 16
bit FAT), the file has but one cluster. Otherwise the FAT
entry referenced by the first cluster index contains the
index for the second cluster. The FAT entry for the Nth
cluster contains the cluster index for the Nth+1 cluster of
the file or 0xFFFF if the current cluster is the last
cluster in the file.
B. The UNIX File System
A hard disk which contains a Unix partition starts off which
a partition table which identifies Unix partitions and how
to get to them. A Unix partition starts off with a boot
block, a system block (which parameterizes the file system),
an inode list which includes the root inode and then data
blocks. Every file has an inode which identifies the first
N blocks of a file (which can be a directory). The N+1
block entry in the inode actually is an indirect block which
contains M+1 entries identifying M indirect blocks and one
indirect block. The amount of permitted indirection depends
on the Unix implementation.
Unix file system access tends to be faster than DOS file
access because inode access is simply faster than FAT
chaining and because Unix maintains read and write inode and
disk block caches. DOS can only be enhanced to contain a
write through cache. Because Unix maintains a write cache,
disk write strategy can be optimized. Read strategy is
harder to optimize and database programs for this reason
prefer to treat disks as raw devices and do their own
caching. Directories contain a list of pairs of the files
contained in the directory and the associated inodes.
Reading a unix file involves opening the directory which
contains the file to get the file inode (namei routine),
getting the inode and then getting data out of the disk
blocks. Each process maintains its own fixed-length process
file table which contains N file read and write buffers and
a pointer to an entry in the fixed-length global file table
which points to an entry in the fixed-length global inode
table. An inode can only have one entry in the global inode
table, while several global file table entries can point to
the same inode. If two processes open the same file the
process file table entries point to two different global
file table entries which point to the same inode table
entry. If the two processes are related by a fork, the
process file table entries point to the same global file
table entry which obviously points to but one global inode
table entry.
One file system can be mounted on another file system at an
empty directory inode which is then marked as a "mounted"
inode and references an entry in the fixed-length mount
table. The mount table identifies the disk device which
contains the second file system. The Unix user unlike the
DOS, VMS or PRIMOS user generally does not have to worry
about how the file system is laid out on the underlying disk
partitions.
C. XNET and NFS
XNET uses the SMB protocol to convey information from the
server machine so that a client DOS process when invoking
DOS system calls can read remote files as if they were
local. That means virtual disk FATs, cluster indices and
clusters can be built out of SMB (XNET) protocol blocks.
SMB protocol blocks are passed over NetBIOS virtual
circuits. Cluster caching is on the remote server. The SMB
protocol provide an unacceptable form of passworded
directory security, and is a stated protocol which means
that it is possible for bad things to happen to a file
system if one side of an XNET connection goes away and the
other side then gets out of synchronization in terms of what
it believes about the file system. Fortunately, there is
not much in the way of state associated with the DOS file
system, so that if both client and server are DOS machines,
there is not much problem with crashes. Aborted circuits on
Xenix clients are probably a symptom of loss of
synchronization and are probably the equivalent of the DOS
critical error Abort, Ignore, Retry? prompt which is always
a pain when it occurs.
NFS uses the NFS protocol to convey information from the
server machine so that a client Unix process when invoking
Unix system calls can read remote files as if they were
local. This means that virtual disk inodes and virtual disk
blocks are built out of NFS protocol blocks. NFS protocol
blocks are passed via UDP datagrams. Generally inode and
block caching is on the remote server. NFS also lacks for
security. NFS is a stateless protocol in which a remote
read requires the remote file system to be instantaneously
mounted with a request to the remote mount daemon, followed
immediately by a remote open request, then a remote lseek
request, then a remote read and then a remote close all done
more or less atomically. Other file system operations are
handled similarly so that it is next to impossible for
server and client file system states to get out of
synchronization. Since mount tables are not file system
structures but rather are in-core kernel structure, it is
not possible on a client machine to access file systems
which are mounted on remote file systems.
In many respects NFS is more work than XNET, but the NFS and
UDP/IP protocol suites actually make a much more efficient
use of bandwidth. Further a Xenix XNET server basically
takes Unix file information translates it into the SMB
protocol in anticipation that the remote client will be
converting this information will be converted by the client
into DOS file system information. It is possible that such
conversion does happen, and then the DOS information is
converted into Unix inode/disk block information. With NFS,
the NFS protocol is optimized to carry information to be
converted back into inodes and disk blocks.
V. Conclusion
Generally, one would expect a Unix-based remote file system
like NFS to work better in the Unix environment than a
system like XNET. If someone has the time to look into the
int5c() interface, it may be possible to clean up aborted
circuits via direct NetBIOS calls. Unfortunately, it might
not be possible to get sufficient information from SCO
actually to write such applications. I am still looking for
the IBM PC Net Technical Reference which would probably be
helpful. Another way to handle the problem might be to run
the file servers as DOS machines. XNET under DOS is
probably stabler. Also, with such a configuration effective
Unix/DOS conversion contortions would only occur at the
communications servers and not also at the file servers. It
might be worth a try just for the sake of curiosity.