[comp.unix.xenix] Ethernet math

hwajin@wrs.wrs.com (Hwa Jin Bae) (05/29/90)

In article <2841@crash.cts.com> jca@pnet01.cts.com (John C. Archambeau) writes:
>Anything can affect the throughput of an ethernet.  If one machine that is
>part of the network chatter is not up to the speed in processing data (i.e. a
>slow ethernet card) then it can and often times will affect the through put of
>the entire network.  Overload the Appletalk/ethernet gateway with a large
>number of Macs telnet'ing into one or more Suns and that will slow things down
>well since the gateway has to service all of these Macs as well as handle all
>of the ethernet traffic going into the Appletalk network.

Duh.  Tell me something new.

Is it just me or are you trying to change the story here?  The story being:

In article <2755@crash.cts.com> jca@pnet01.cts.com (John C. Archambeau) writes:
> Yes, but that's a quiet ethernet.  What I'm referring to is a real noisy
> ethernet that is being used.
> Let me setup a real network of the order of 10+ machines with varying
> performance on the ethernet and I will bet that the performance will be about
> 3 Mbit per second on a good day.

Where I replied:

Not true at all.  Even with "a real network" I have here with 30+ various sun
workstations (sparcStations, 3/60's, etc.) and servers (plenty of NFS 
traffic), burst throughput performance of around 600 MegBytes/sec can be
sustained without causing meltdowns.  This is using severly hacked version
of 4.3 BSD Tahoe TCP/IP running within our realtime OS on a 16Mhz 68020
VME SBC with an onboard LANCE chip.

Now you say:
>
>The issue is that a lot of things can happen that will make ethernet slower
>than molasses and it can be in either hardware or software.
>
>I've found that from experience that Macs and PC's are the biggest offenders
>to mucking up ethernet throughput.

Duh.  What a revelation!

Which point are you trying to make?

Let me help:

Your point #1: a "real" ethernet with more than 10 machines will have at most
3Mbit/sec performance rate on a good day.   --- My data indicates this is
wrong.  Give me some of your data that proves otherwise.  No rhetorics, please.

Your point #2: Ethernet throughput varies and can be affected by ill-behaved
hosts.  --- What a brilliant analysis!  Gee... I'd have never guessed that.

Do you see a smooth transition from Point #1 to Point #2?  I don't.

This is getting boring, quick.  Enough from me.  Carry on JCA.
-- 
hwajin@daahoud.wrs.com

jca@pnet01.cts.com (John C. Archambeau) (05/29/90)

hwajin@wrs.wrs.com (Hwa Jin Bae) writes:
>Which point are you trying to make?
>
>Let me help:
>
>Your point #1: a "real" ethernet with more than 10 machines will have at most
>3Mbit/sec performance rate on a good day.   --- My data indicates this is
>wrong.  Give me some of your data that proves otherwise.  No rhetorics, please.

My data is my findings via experience in hooking machines to an ethernet.  I
have yet to see an ethernet perform a consistant (key word here) throughput of
more than 3 Mbit per second.  Last October I added a Sun 386i/250 to an
existing ethernet of less than 10 machines.  The best throughput I've seen on
that wire was 4 Mbit per second.  You tell me why it was dropping packets like
crazy.  The customer we sold the Sun 386i has talked to Sun about it and they
can't even tell him why his ethernet has a sloppy throughput.  Again, there's
no such thing as an expert.  And the ethernet is primarily Suns with a pair of
VAX'es.  If you have this magic formula for getting 6+ Mbits per second on an
ethernet, I'd sure as hell love to hear it.

When *I* start seeing consistant throughputs of more than 3 or 4 Mbits per
second on an ethernet then I'll agree with you, but until then I write all of
this off as the overhead of ethernet.  I don't care what your throughput is or
what AST gets on his pair of Sun 3/60's, but what I see when I go and add or
setup a network.
 
     // JCA

 /*
 **--------------------------------------------------------------------------*
 ** Flames  : /dev/null                     | Small memory model only for
 ** ARPANET : crash!pnet01!jca@nosc.mil     | Unix?  Get the (*bleep*) out
 ** INTERNET: jca@pnet01.cts.com            | of here!
 ** UUCP    : {nosc ucsd hplabs!hd-sdd}!crash!pnet01!jca
 **--------------------------------------------------------------------------*
 */

davin@me.utoronto.ca (Davin Yap) (05/30/90)

In article <2871@crash.cts.com> jca@pnet01.cts.com (John C. Archambeau) writes:

>My data is my findings via experience in hooking machines to an ethernet.  I
>have yet to see an ethernet perform a consistant (key word here) throughput of
>more than 3 Mbit per second.  Last October I added a Sun 386i/250 to an
>existing ethernet of less than 10 machines.  The best throughput I've seen on
>that wire was 4 Mbit per second.  You tell me why it was dropping packets like
>crazy.  

Are you sure it was dropping packets?  What were the other machines on
the net and how were you testing the ethernet _interface_.  A lot depends
on the combination of computer/ethernet controller, the smarter the
controller, the less the cpu has to do, the greater your throughput.  If
the controller doesn't handle the IP and the transport layers, the cpu
must; if the card isn't a bus master, you lose anyway.  3 - 4 Mbit/s
sounds right if you've got a bunch of Sun 3s and pc's on your net, the
individual ethernet interfaces of these machines max out around here; the
Suns because of the nonoptimal drivers that came with SunOS, the pc's
more likely due to hardware limits.  I know about the Sun 3 numbers, I'm
speculating on the pc numbers.  
   On the other hand, I've come across machines/ethernet controllers
that'll handle 10 Mbit/s easy.  A MIPS RC3260 with a block mode ethernet
controller comes to mind: over NFS (i.e. UDP) I was transfering 1 MByte/s.
On a Sun 4/490 I was getting 800+ kBytes/s.  Over TCP on a cluster of
Sparcstation 1(s) I was getting over 700 kBytes/s.  These were all casual
tests, I wasn't out to find the ultimate performance, just close to it :-).
The tests were constructed so that the ethernet interface was always the
rate limiting step, the data to be transfered was always from memory not
from disk.
   BTW, I'm told the 'le' (lance ethernet) controller has been enhanced
in SunOS 4.1 (apparently they've incorporated Van Jacobsons improvements)
so that a Sparcstation 1 can now saturate the ethernet.  Putting multiple
ethernet cards in an SS1 leads to a truly inspiring router :-)
   I wonder, though, about putting slow machines on the same ethernet as
these faster machines, it'll be difficult for them to get a word in
edgewise :-).  A collision of wills (like wit) where the quickest
prevails :-).

>When *I* start seeing consistant throughputs of more than 3 or 4 Mbits per
>second on an ethernet then I'll agree with you, but until then I write all of
>this off as the overhead of ethernet.

It isn't ethernet overhead that's bogging you down, depending on you
packet size there's precious little of that.  Your problem is that your
ethernet interfaces suck, the combination of computer, ethernet
controller and driver.  Unfortunately, you're in the same boat as most
people out there, and for you 3-4 Mbit/s is what you're gonna see.

Davin
--
Davin Yap, Center for Computer Integrated Engineering, University of Toronto
davin@me.utoronto.ca  davin@me.utoronto.bitnet | 5 King's College Rd., Toronto
  ...{pyramid,uunet}!utai!utme!davin           |    Ontario, CANADA, M5S 1A4

hwajin@daahoud.wrs.com (Hwa Jin Bae) (05/30/90)

In article <2871@crash.cts.com> jca@pnet01.cts.com (John C. Archambeau) writes:
   When *I* start seeing consistant throughputs of more than 3 or 4 Mbits per
   second on an ethernet then I'll agree with you, but until then I write all of
   this off as the overhead of ethernet.

Kinda like saying: I don't care if there are bunch of people out in the real
world running 4.3 Tahoe TCP/IP on Sun 3/60's and getting ~8Mbits/sec on 
ethernets and I personally tell you that I have a system that can do
~6Mbits/sec on a busy ethernet.

   I don't care what your throughput is or
   what AST gets on his pair of Sun 3/60's, but what I see when I go and add or
   setup a network.

Like: if I can't figure out how to make my network go faster, it's not
my fault, but everyone else's for having done just what I can't seem
to figure out.  I can always just blame the "ethernet overhead" and
keep saying "I don't care."

Note, the max throughput limit on ethernet is a fixed value: 10 Mbits/sec.  
It is not something that changes.  What do you mean by "throughput" here
anyway?  The raw data transfer rate capacity of any task that talks to
your ethernet board will depend heavily on the way you wrote your ethernet
driver and the way your ethernet board itself is designed/constructed.
Sure, if you have some brain-dead ethernet implementation running with a
not-so-smart ethernet driver software, and your application is written
sloppily, your task-to-task data transfer rate will suffer.  This has nothing
to do with the channel capacity of a given segment of ethernet.  It has
everything to do with the quality of local implementation: the ethernet board,
the driver, the application/protocol code.

And, I'm telling you that if you have a nicely designed hardware (like an
onboard LANCE with enough dual-ported memory or fully arbited on-board
bus that lets you share main memory between CPU and LANCE) and an ethernet
driver that utilizes all these hardware design features by "loaning-out"
LANCE ring buffers to elimite extraneous copying, and a good protocol
implmenenation like 4.3 Tahoe BSD TCP/IP with all of Van Jacobsen's
optimizations, and a tightly coded application, ethernet will provide
you with ample bandwidth to guarantee ~6Mbits/sec even on a "real ethernet",
and more.  I have this type of systems running here.  Oh, I forgot: you don't
care.

What you tell me sounds like: well, I've plugged in all these fancy Sun
workstations and things into my ethernet and they're just pretty slow, so
don't tell me otherwise, I don't care, etc.  Hey, whatever makes you happy.
Enuf said.

hwajin
--
hwajin@daahoud.wrs.com

cgs@umd5.umd.edu (Chris G. Sylvain) (05/30/90)

In article <2871@crash.cts.com> jca@pnet01.cts.com (John C. Archambeau) writes:
> [...]  The best throughput I've seen on that wire was 4 Mbit per second.
> You tell me why it was dropping packets like crazy. [...]

Let's begin with a simple reason before we blame the I/O chips or the network
protocol:

The interface cards you were using were terminated ON THE CARD, but you went
and added a T-connector and a terminating resistor anyway.

Don't laugh, I've seen it happen more times than would be funny.
-- 
--==---==---==--
Outgrabe: Like a bellowing and whistling, with a kind sneeze in the middle
--   ARPA: cgs@umd5.UMD.EDU     BITNET: cgs%umd5@umd2   --
--   UUCP: ..!uunet!umd5.umd.edu!cgs                    --

jca@pnet01.cts.com (John C. Archambeau) (06/01/90)

davin@me.utoronto.ca (Davin Yap) writes:
>It isn't ethernet overhead that's bogging you down, depending on you
>packet size there's precious little of that.  Your problem is that your
>ethernet interfaces suck, the combination of computer, ethernet
>controller and driver.  Unfortunately, you're in the same boat as most
>people out there, and for you 3-4 Mbit/s is what you're gonna see.

That's the problem, I can not see a consistant throughput of more than maybe 4
4 Mbits per second at best on the entire network.  The lack of consistancy in
throughput.
 
     // JCA

 /*
 **--------------------------------------------------------------------------*
 ** Flames  : /dev/null                     | Small memory model only for
 ** ARPANET : crash!pnet01!jca@nosc.mil     | Unix?  Get the (*bleep*) out
 ** INTERNET: jca@pnet01.cts.com            | of here!
 ** UUCP    : {nosc ucsd hplabs!hd-sdd}!crash!pnet01!jca
 **--------------------------------------------------------------------------*
 */

jca@pnet01.cts.com (John C. Archambeau) (06/01/90)

hwajin@daahoud.wrs.com (Hwa Jin Bae) writes:
>In article <2871@crash.cts.com> jca@pnet01.cts.com (John C. Archambeau) writes:
>   When *I* start seeing consistant throughputs of more than 3 or 4 Mbits per
>   second on an ethernet then I'll agree with you, but until then I write all of
>   this off as the overhead of ethernet.
>
>Kinda like saying: I don't care if there are bunch of people out in the real
>world running 4.3 Tahoe TCP/IP on Sun 3/60's and getting ~8Mbits/sec on 
>ethernets and I personally tell you that I have a system that can do
>~6Mbits/sec on a busy ethernet.
>
>   I don't care what your throughput is or
>   what AST gets on his pair of Sun 3/60's, but what I see when I go and add or
>   setup a network.
>
>Like: if I can't figure out how to make my network go faster, it's not
>my fault, but everyone else's for having done just what I can't seem
>to figure out.  I can always just blame the "ethernet overhead" and
>keep saying "I don't care."
>
>Note, the max throughput limit on ethernet is a fixed value: 10 Mbits/sec.  
>It is not something that changes.  What do you mean by "throughput" here
>anyway?  The raw data transfer rate capacity of any task that talks to
>your ethernet board will depend heavily on the way you wrote your ethernet
>driver and the way your ethernet board itself is designed/constructed.
>Sure, if you have some brain-dead ethernet implementation running with a
>not-so-smart ethernet driver software, and your application is written
>sloppily, your task-to-task data transfer rate will suffer.  This has nothing
>to do with the channel capacity of a given segment of ethernet.  It has
>everything to do with the quality of local implementation: the ethernet board,
>the driver, the application/protocol code.
>
>And, I'm telling you that if you have a nicely designed hardware (like an
>onboard LANCE with enough dual-ported memory or fully arbited on-board
>bus that lets you share main memory between CPU and LANCE) and an ethernet
>driver that utilizes all these hardware design features by "loaning-out"
>LANCE ring buffers to elimite extraneous copying, and a good protocol
>implmenenation like 4.3 Tahoe BSD TCP/IP with all of Van Jacobsen's
>optimizations, and a tightly coded application, ethernet will provide
>you with ample bandwidth to guarantee ~6Mbits/sec even on a "real ethernet",
>and more.  I have this type of systems running here.  Oh, I forgot: you don't
>care.
>
>What you tell me sounds like: well, I've plugged in all these fancy Sun
>workstations and things into my ethernet and they're just pretty slow, so
>don't tell me otherwise, I don't care, etc.  Hey, whatever makes you happy.

I shouldn't have to be an engineer at Novell  or 3 Com to get decent
throughput on an ethernet, but it seems to me that you're saying that I have
to be.  I don't build the networking boards nor do I write the drivers for
them and if you have to spend time going in, ripping apart the ethernet
driver, then that's more work that has to be done in the name of performance
that should've been done by the vendor of the product.

I'm not going to go and take apart everything on the ethernet at both the
hardware and software level since it's not my job for one.  If you want to
go and do it for me, be my guest.

This wouldn't be the first time a vendor screwed up.  And it certainly
won't be the last.

Keep in mind that people often times can't optimize what they have because of
time and/or money.  It's easy for you to say what needs to be done, but hand
the customer the bill on what it costs to do it or change over to the optimal
set up and they very quickly change their mind about upgrading.

Or another even more important limitation, availability.  What if it just
isn't plain available yet?  There isn't much you can do then.  A lot of people
faced this problem when they wanted to upgrade from Netware SFT to Netware 386
just after it came out.  A lot of hardware wasn't compatable with Netware 386
since there were no drivers.  The driver structure in Netware 386 was redone
completely and it was not compatable with its successors.

Go ahead and preach about the optimal configuration, but when you're in that
situation when you're in between a rock, a time limit, and a hard place you
aren't so much concerned with getting it running in the optimal configuration,
you're concerned with getting it running, period.
 
     // JCA

 /*
 **--------------------------------------------------------------------------*
 ** Flames  : /dev/null                     | Small memory model only for
 ** ARPANET : crash!pnet01!jca@nosc.mil     | Unix?  Get the (*bleep*) out
 ** INTERNET: jca@pnet01.cts.com            | of here!
 ** UUCP    : {nosc ucsd hplabs!hd-sdd}!crash!pnet01!jca
 **--------------------------------------------------------------------------*
 */

hwajin@daahoud.wrs.com (Hwa Jin Bae) (06/02/90)

To: uunet!pnet01.cts.com!jca
Subject: Re: Ethernet math (Was Re: MWC's Coherent - A Lemon...)
Newsgroups: comp.sys.ibm.pc,comp.os.minix,comp.unix.xenix
References: <2932@crash.cts.com>

In comp.sys.ibm.pc you write:

>I shouldn't have to be an engineer at Novell  or 3 Com to get decent
>throughput on an ethernet, but it seems to me that you're saying that I have
>to be.  I don't build the networking boards nor do I write the drivers for
>them and if you have to spend time going in, ripping apart the ethernet
>driver, then that's more work that has to be done in the name of performance
>that should've been done by the vendor of the product.

>I'm not going to go and take apart everything on the ethernet at both the
>hardware and software level since it's not my job for one.  If you want to
>go and do it for me, be my guest.

>This wouldn't be the first time a vendor screwed up.  And it certainly
>won't be the last.

>Keep in mind that people often times can't optimize what they have because of
>time and/or money.  It's easy for you to say what needs to be done, but hand
>the customer the bill on what it costs to do it or change over to the optimal
>set up and they very quickly change their mind about upgrading.

>Or another even more important limitation, availability.  What if it just
>isn't plain available yet?  There isn't much you can do then.  A lot of people
>faced this problem when they wanted to upgrade from Netware SFT to Netware 386
>just after it came out.  A lot of hardware wasn't compatable with Netware 386
>since there were no drivers.  The driver structure in Netware 386 was redone
>completely and it was not compatable with its successors.

>Go ahead and preach about the optimal configuration, but when you're in that
>situation when you're in between a rock, a time limit, and a hard place you
>aren't so much concerned with getting it running in the optimal configuration,
>you're concerned with getting it running, period.
> 
>     // JCA


Here, you're talking about yet another point, keep avoiding the initial issue.
You were asserting that "in a real ethernet" throughput of more than 3Mbit/sec
is almost impossible, and that's the point people were trying to correct you
on.   You don't seem to have enough experience with ethernet, at least not
enough to make such definitive assertions, but still very unwilling to 
admit the obvious flaw in your statement, and thus insist on haranguing on 
your diatribe about various different issue completely unrelated to the
technical issue at hand: that of ethernet cable and boards actually being 
able to support 10Mbit/sec, with reasonable amount of engineering.  First,
you've tried to divert the issue by saying that PC's and Mac's connected
to the ethernet will certainly prevent having more than 3Mbit/sec throughput
on "any good day", and when proven wrong, you start talking "I don't
care whether you can, I can't here, so I don't care", and now the latest
strategy of yours is this point about "time limit" and "customer point of
view".  All I want is concrete data from you that will convince me that
your initial statement is correct.  That's the only point I'm arguing about.
Not the realities of the customer's situation and vendors screwing up their
products or having incompetent network administrators who don't know anything
about their networks while making some loud, noisy statements about something
they know nothing about.

Hwajin

hwajin@daahoud.wrs.com

jca@pnet01.cts.com (John C. Archambeau) (06/03/90)

hwajin@daahoud.wrs.com (Hwa Jin Bae) writes:

> [ Lot's of bull deleted... ]

Would you please make up your mind if you want to take this to e-mail or keep
it on the net?

I'm getting fed up of carbon copying everything to both e-mail and on the net.
I much prefer being proven wrong and getting caught by the more experienced
than repeating myself.  
 
     // JCA

 /*
 **--------------------------------------------------------------------------*
 ** Flames  : /dev/null                     | Small memory model only for
 ** ARPANET : crash!pnet01!jca@nosc.mil     | Unix?  Get the (*bleep*) out
 ** INTERNET: jca@pnet01.cts.com            | of here!
 ** UUCP    : {nosc ucsd hplabs!hd-sdd}!crash!pnet01!jca
 **--------------------------------------------------------------------------*
 */