[comp.sys.amiga] Software-only network protocol standards

bryan@geo-works.UUCP (Bryan Ford) (06/26/89)

(Notice that I'm cross-posting to several newsgroups.  Please reply only on
comp.sys.amiga or by E-mail, preferrably by E-mail.)

I'm writing a software-only networking package for the Amiga computer, to
work over standard serial lines.  I would like to use a standard,
already-existing protocol, if possible (to avoid inventing Yet Another
Protocol, and to make the work easier).  Of course, PD source code of some
kind would also be nice, but not required.

Someone here mentioned SLIP (Serial Line Internet Protocol).  Does anybody
have any information about this?

I understand AppleTalk is quite popular.  Does this require extra hardware?
What is its architecture like?

I can use anything, even if it isn't exactly 'ideal', just for the ideas if
nothing else.

Thanks in advance.

				Bryan

P.S.  My site isn't fully registered yet, so you might have to use a UUCP
path starting a 'higher' site, such as 'caeco!i-core!geo-works!bryan'.

--

     ____________________________________________
   _/ Bryan Ford - bryan@geo-works.geo-works.com \_
 _/    ..!utah-cs!caeco!i-core!geo-works!bryan     \_
/   ..!uunet!iconsys!caeco!i-core!geo-works!bryan    \

walker@sas.UUCP (Doug Walker) (07/03/89)

What's the matter with DNET?  Seems to be a perfectly workable network protocol
standard over serial lines.  Write a few DNET servers, and you can do anything.

--Doug

bryan@geo-works.UUCP (Bryan Ford) (07/14/89)

In article <1089@sas.UUCP>, walker@sas.UUCP (Doug Walker) said:
>What's the matter with DNET?  Seems to be a perfectly workable network protocol
>standard over serial lines.  Write a few DNET servers, and you can do anything.

First, DNET was built as a two-computer network, and what I'm doing
requires that you can hook up many computers in a mesh-type network just by
using standard null-modem cables.  Of course, you could just add relay
servers and things like that, but there are problems with that explained
below.

DNET is written completely in C, and doesn't have enough OSI layering built
into the code to make it easy to convert part to assembly language.  When
you're running at over 200KBPS, something in C just can't cut it.  And,
even if the low-level parts were rewritten, the relay servers would still
basically be way on top in the Application layer, in a different process no
less, when they're supposed to be way down in the Network layer.  As a
result, to just relay a packet, DNET has to do a bunch of message-passing
stuff to get the packet to the relay server, then the server sends a
possibly modified version of the packet back to DNET, which then sends the
packet to another server.  Though the Amiga's message-passing system is
very efficient, they're just not efficient *enough* for this.

The protocol was not written for high-speed relaying.  It requires that
each packet be gotten entirely and correctly by one machine before it sends
it off to the next.  To get really good speed, you've got to relay each
*byte* as soon as it is received, then do your checking later.

Please note that this isn't a flame against DNET or Matt Dillon, and I
think it's a wondeful program.  It's just not right for my purpose.

Any discussion?

				Bryan

--

     ____________________________________________
   _/ Bryan Ford - bryan@geo-works.geo-works.com \_
 _/    ..!utah-cs!caeco!i-core!geo-works!bryan     \_
/   ..!uunet!iconsys!caeco!i-core!geo-works!bryan    \

papa@pollux.usc.edu (Marco Papa) (07/17/89)

In article <1719.AA1719@geo-works> bryan@geo-works.UUCP (Bryan Ford) writes:
>DNET is written completely in C, and doesn't have enough OSI layering built
>into the code to make it easy to convert part to assembly language.  When
>you're running at over 200KBPS, something in C just can't cut it.

Oh, yea? What do you think is the language in which UNIX device drivers (serial
or network, take your pick), TCP/IP, sockets, etc ... have been written in?
Hint: the name of the language is one-letter long :-) And I can tell you that 
that code supports data transfers > 200KBPS.  Don't blame the language. You got
the wrong culprit.

-- Marco Papa 'Doc'
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
uucp:...!pollux!papa       BIX:papa       ARPAnet:pollux!papa@oberon.usc.edu
"There's Alpha, Beta, Gamma, Diga and Caligari!" -- Rick Unland
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

bryan@geo-works.UUCP (Bryan Ford) (07/19/89)

In article <18559@usc.edu>, papa@pollux.usc.edu (Marco Papa) writes:
>In article <1719.AA1719@geo-works> bryan@geo-works.UUCP (Bryan Ford) writes:
>>DNET is written completely in C, and doesn't have enough OSI layering built
>>into the code to make it easy to convert part to assembly language.  When
>>you're running at over 200KBPS, something in C just can't cut it.
>
>Oh, yea? What do you think is the language in which UNIX device drivers (serial
>or network, take your pick), TCP/IP, sockets, etc ... have been written in?
>Hint: the name of the language is one-letter long :-) And I can tell you that 
>that code supports data transfers > 200KBPS.  Don't blame the language. You got
>the wrong culprit.

Maybe you're right, and I have overestimated the difference in speed of C
and assembly language.  In any case, let's not get into a 'C vs Assembly'
war.  However, consider this:

This network has to run on a "lowest-level" 8 MHz Amiga (or is that 7.xx?).
This means I must program for 125 nanoseconds per clock cycle, and no 68000
instruction takes less than two cycles (maybe more, I can't remember).  I'm
assuming that most Unix boxes run at much higher speeds, and since they
inherently *require* at least a 68020 (if memory protection is supported),
many of the instructions will also execute in less clock cycles (the 68020
often uses less clock cycles per instruction).  At 200,000 baud, you have 5
milliseconds per bit and 50 milliseconds per byte (at 8N1).  That means
there are approximately 400 clock cycles per byte.  This may seem like a
lot, but remember that, in any single host, there at least *four* processes
going at this speed:  one sending and one receiving for each of the two
required ports.  And if there are more than two ports, the overhead is even
greater.  Since this is a multitasking machine, I don't want to bog down
the system too much because of network handling.

Only the raw transfer of raw data is handled in hardware in this case,
while (for example) most Ethernet cards often handle things like error
checking and resending bad packets.  In addition, with an Ethernet network,
packets go straight to the destination host without needing to be processed
and rerouted by other hosts.  In a mesh network, many hosts will have lots
of data going through them, so there will be more traffic through each
host.

What I'll probably do is implement the network and data-link layers in
assembly language, and do the rest in C.  This way, only the end source and
destination hosts of packets use C, while packets which just go 'through' a
host never get to the C part (the Network layer rerouts them first.)

Now, one question:  How many overhead cycles do the Amiga's interrupt
routines use (on a plain 68000)?  I know it's not much, but I'd like to
know exactly *how* much.  How about for software interrupts?

Thanks!
	
				Bryan

--

     _______________________________________
   _/   Bryan Ford - bryan@geo-works.uucp   \_
 _/  ..!utah-cs!caeco!i-core!geo-works!bryan  \_
/ ..!uunet!iconsys!caeco!i-core!geo-works!bryan \

cmcmanis%pepper@Sun.COM (Chuck McManis) (07/25/89)

[I've rearranged this a bit, for clarity, apologies if I changed the meaning]

In article <1789.AA1789@geo-works> bryan@geo-works.UUCP (Bryan Ford) writes:
> I'm
>assuming that most Unix boxes run at much higher speeds, and since they
>inherently *require* at least a 68020 (if memory protection is supported),
>many of the instructions will also execute in less clock cycles (the 68020
>often uses less clock cycles per instruction).

This is a bogus assumption. The Sun-1 was a 68000 and the Sun-2 was a 68010.
The latter was running at 12Mhz I believe. But it did have memory management.

>This network has to run on a "lowest-level" 8 MHz Amiga (or is that 7.xx?).
>This means I must program for 125 nanoseconds per clock cycle, and no 68000
>instruction takes less than two cycles (maybe more, I can't remember). 
>At 200,000 baud, you have 5 milliseconds per bit and 50 milliseconds per 
>byte (at 8N1).  That means there are approximately 400 clock cycles per 
>byte.

This says that for any stream of bytes at 200,000 baud, we should be able
stuff them from the serial port into a buffer. With some time to spare
something like (my assembler for the 68K sucks, please forgive):

Serial_interrupt:
	move.l	#SERDATA,a0
	move.l 	#bufaddr,a1
	move	#PACKETSIZE, d0
$10	tst.b	SERFLAGS,#DATA_AVAILABLE  ; guarunteed to be ~400 clocks
	bz.s	$10			  ; so busy waiting is ok.
	movem	(a0), (a1)+
	dbf	$10
	movem	#1,packet_available
	rts

This simple routine, is supposed to just activate when it gets the first
byte, suck the packet out of the serial device into some memory somewhere,
and then return. At which point some other routine will process the packet.

>This may seem like a lot, but remember that, in any single host, there 
>at least *four* processes going at this speed:  one sending and one 
>receiving for each of the two required ports.  And if there are more 
>than two ports, the overhead is even greater.  Since this is a 
>multitasking machine, I don't want to bog down the system too much 
>because of network handling.

But sending processes can hold off for long periods of time when the
receiving process is fetching data.

Plus if you want to do both (send and receive) at the same time you
should put them both into the same thread such they use the same cycles.
The simplest way to do this is to have one tight loop check for both
the Transmitter Buffer Empty flag and the Receive Buffer Full flag
and branch to the appropriate mini routine until you ran out of stuff
to do (send or receive) and then exit. 

What you will never get away from without new hardware is that the
CPU has to do everything, and no matter how you code it if you get
100% network utilization your CPU won't have time to do anything else.

>Only the raw transfer of raw data is handled in hardware in this case,
>while (for example) most Ethernet cards often handle things like error
>checking and resending bad packets.  In addition, with an Ethernet network,
>packets go straight to the destination host without needing to be processed
>and rerouted by other hosts.  In a mesh network, many hosts will have lots
>of data going through them, so there will be more traffic through each
>host.

This makes for some interesting real time code. When you use the same
threads/buffer approach that I mentioned, you can use a trick whereby
if the packet you are receiving isn't for you, you can just dump it into
the transmit buffer. By doing so the transmit part of the thread will 
begin sending it back out immediately. 

>Now, one question:  How many overhead cycles do the Amiga's interrupt
>routines use (on a plain 68000)?  I know it's not much, but I'd like to
>know exactly *how* much.  How about for software interrupts?

Get Carl Sassenrath's book on Interrupts. (Guru's Guide #1). It's 
available from HT Electronics in Sunnyvale for sure. (Or direct from
Carl). 


One project I worked on long ago realized that once the processor
had responded they wouldn't have any trouble receieving the packet
but latency was a problem. The solution they chose was to send a couple
of 0xFF bytes first, which would start the interrupt proccessing part
of the receiver who would always get an overrun error because it couldn't
receive the first byte fast enough, but that was ok, it would clear the 
serial chip and wait for the 0xAA which signaled the start of the packet.



--Chuck McManis
uucp: {anywhere}!sun!cmcmanis   BIX: cmcmanis  ARPAnet: cmcmanis@sun.com
These opinions are my own and no one elses, but you knew that didn't you.
"A most excellent barbarian ... Genghis Kahn!"

bryan@geo-works.UUCP (Bryan Ford) (07/29/89)

In article <117428@sun.Eng.Sun.COM>, cmcmanis%pepper@Sun.COM (Chuck McManis) writes:
>> I'm
>>assuming that most Unix boxes run at much higher speeds, and since they
>>inherently *require* at least a 68020 (if memory protection is supported),
>>many of the instructions will also execute in less clock cycles (the 68020
>>often uses less clock cycles per instruction).
>
>This is a bogus assumption. The Sun-1 was a 68000 and the Sun-2 was a 68010.
>The latter was running at 12Mhz I believe. But it did have memory management.

I stand corrected.  Please clarify 'memory management':  You don't include
memory protection, do you?  How could a pre-68020 provide memory protection
unless it has some special protection hardware?  (Did it?)

>This says that for any stream of bytes at 200,000 baud, we should be able
>stuff them from the serial port into a buffer. With some time to spare
>something like (my assembler for the 68K sucks, please forgive):
>
>...
>
>$10	tst.b	SERFLAGS,#DATA_AVAILABLE  ; guarunteed to be ~400 clocks
>	bz.s	$10			  ; so busy waiting is ok.

So, you're saying that interrupts are too slow to do anything but busy-wait
after the beginning of the packet has been detected?  Too bad.  I was
hoping that some needy task could get some precious CPU between the bytes,
but it looks like this was a hopeless hope.  :-)

> [lots of other good stuff]

Thanks very much for the tips and the routine!  They are going to help a
lot!

				Bryan

--

     _______________________________________
   _/   Bryan Ford - bryan@geo-works.uucp   \_
 _/  ..!utah-cs!caeco!i-core!geo-works!bryan  \_
/ ..!uunet!iconsys!caeco!i-core!geo-works!bryan \
\_____________Author: Chroma Paint______________/

lphillips@lpami.wimsey.bc.ca (Larry Phillips) (08/01/89)

In <1993.AA1993@geo-works>, bryan@geo-works.UUCP (Bryan Ford) writes:
>In article <117428@sun.Eng.Sun.COM>, cmcmanis%pepper@Sun.COM (Chuck McManis) writes:
>>This is a bogus assumption. The Sun-1 was a 68000 and the Sun-2 was a 68010.
>>The latter was running at 12Mhz I believe. But it did have memory management.

>I stand corrected.  Please clarify 'memory management':  You don't include
>memory protection, do you?  How could a pre-68020 provide memory protection
>unless it has some special protection hardware?  (Did it?)

One of the founders of Sun (Andy Bechtelscheim I believe), developed an MMU,
implemented in hardware.  In fact, you could say that the development of the
MMU was a very important factor in the decision to start the company.  If you
squint just a little bit, and look at Sun's MMU, and if the light is just
right, you will see remarkable similarities to the Motorola MMU, both the 68851
and the built-in one in the 68030.

-larry

--
"So what the hell are we going to do with a Sun?" - Darlene Phillips -
+-----------------------------------------------------------------------+ 
|   //   Larry Phillips                                                 |
| \X/    lphillips@lpami.wimsey.bc.ca -or- uunet!van-bc!lpami!lphillips |
|        COMPUSERVE: 76703,4322  -or-  76703.4322@compuserve.com        |
+-----------------------------------------------------------------------+

cmcmanis%pepper@Sun.COM (Chuck McManis) (08/03/89)

In article <1993.AA1993@geo-works> bryan@geo-works.UUCP (Bryan Ford) writes:
>I stand corrected.  Please clarify 'memory management':  You don't include
>memory protection, do you?  How could a pre-68020 provide memory protection
>unless it has some special protection hardware?  (Did it?)

Yes, most Sun products have custom MMU's that provide both page relocation
and protection. [not sure about the 386i or 3/80] The Sun-1's dual 68000 
design allowed it to fake instruction restart by keeping the state in the 
"other" CPU. The 68010 of course provided the required state and so the 
redundant CPU was removed. 

>So, you're saying that interrupts are too slow to do anything but busy-wait
>after the beginning of the packet has been detected?  Too bad.  I was
>hoping that some needy task could get some precious CPU between the bytes,
>but it looks like this was a hopeless hope.  :-)

Yes, given the design constraint of 200,000 baud. But from a systems standpoint
the if the packets are small, say 1K bytes, then and relatively infrequent
say 1 per second. Then the packet takes about .06 seconds to receive and
stuff into memory and that leaves you with 94% of the CPU to play around
with. If the packets were 10/sec on average then you get down to something
like 40% of the CPU left and that might suggest that you needed to invest
in some additional CPU horsepower (like a DMA based network interface). 
It is all a question of system design and allocating your resources to
the where they are most profitably used.

--Chuck McManis
uucp: {anywhere}!sun!cmcmanis   BIX: cmcmanis  ARPAnet: cmcmanis@sun.com
These opinions are my own and no one elses, but you knew that didn't you.
"A most excellent barbarian ... Genghis Kahn!"

thad@cup.portal.com (Thad P Floryan) (08/05/89)

Re: Bryan Ford's comments about UNIX "requiring" at least a 68020 ...

Besides the Sun-2 (with a 68010 and custom MMU), we find:

1) the AT&T UNIXPC (aka 3B1 aka PC7300) with a 10 MHz 68010 with custom VLSI
   providing zero wait state MMU with full protection and demand-paged virtual
   memory, and this machine is architecturally identical to the Convergent
   Technologies' Miniframe and Safari UNIX boxes and to Motorola's 6300 UNIX box
   (in fact, all these are binary code compatible since they were all built by
   Convergent :-).  I have 4 of these and the MMU operates quite nicely; just a
   "core" file left if an errant program runs amock ... no "Guru Meditations"
   and no reboots required.  These are running a variant of SVR2 with some SVR3
   stuff, shared libs, dynamically loaded device drivers, etc. and some BSD
   extensions, and are expected to be capable of running the GNU-kernel (aka
   "Mach" from CMU) sometime next year.  The new kernel release from AT&T is
   expected within a few weeks, and there's already POSIX compliance with GNU
   cc and c++ and the recently-posted POSIX-compliant library sources (which may
   even work on the Amiga; hmmm, something to try this weekend (the libraries,
   that is)).

2) the Concurrent (Perkin-Elmer) XF-210 (68000 with MMU running a (Berkeley)
   Unisoft UNIX) which I have flanking one of my office Amigas.

3) the CADMUS being a 68000-based UNIX box with an MMU. 

I'm sure one can find other examples of non-68020/-68030 680x0-based systems
with MMUs.  Someone must also have used the 68012 in a system(s).

Thad Floryan [ thad@cup.portal.com (OR) ..!sun!portal!cup.portal.com!thad ]