jr@amanue.UUCP (Jim Rosenberg) (02/07/88)
I will be making a decision soon on a DP machine for a company with several
remote sites. I've specified UNIX at both ends, with the DP software in a
4GL. (My inclination at the moment is toward Progress, but that's another
matter.) We are transferring files right now by chewing gum and baling wire,
and I would like to be using uucp so it will run unattended and automated from
both ends. At the moment we have only a few remote sites, but we hope to
bring on a new franchising system that will be expanding the number of remote
sites -- perhaps explosively. I've been told to prepare a plan that can
handle 100 remote sites relatively soon, expanding from there to *hundreds* of
sites. I can't give exact figures on how much data needs to be transferred,
but a typical transmission is probably 30K to 90K. The nature of the business
means that there are periods where the demand peaks drastically, i.e. certain
days of the week where we might have to talk to everybody -- perhaps even
during business hours!
I hadn't known about all this planned growth. When I heard about it my
reaction to this kind of demand was to think multiprocessor, such as a
Sequent. (An earlier article I posted on whether a Sun could be used for DP
suggested Sequent.) There is currently *TREMENDOUS* uncertainty about whether
all these franchises will actually happen, and the ability to incrementally
upgrade all the way to "mainframe" power <how I hate that word "mainframe"!>
is extremely attractive. I've got them pretty excited about this idea.
My question is, are any of you out there talking to this many sites *without*
using either a huge machine or a multiprocessor box? How many uucp sites is
anybody handling with say a 68020 or 80386 box with slave I/O processors?
Unisys is trying to tell us a supermarket in our area is talking to 80 sites
with just a Unisys 5000/50, which is a 68020 and 1 slave 68010 I/O processor
per 8 serial lines. I'm pretty skeptical. I asked them for a copy of the
uucp LOGFILE -- so far no response.
I know a machine like a Sequent will do the job (yup, I know they picked a
Sequent for uunet) and with that kind of box I know exactly what the recourse
is if the machine begins bogging down. If anybody's getting a solution any
other way I'd be interested to hear about it. It's not that I'm looking for a
reason not to get a Sequent or Encore (any others I should be looking at??) --
I just want to cover all the bases.
-Thanks,
--
Jim Rosenberg
CIS: 71515,124 decvax!idis! \
WELL: jer allegra! ---- pitt!amanue!jr
BIX: jrosenberg uunet!cmcl2!cadre! /mark@intek01.UUCP (Mark McWiggins) (02/09/88)
I haven't done this, but I have thought about a similar setup. Why put all the load on the host? If you set up a "chain reaction" such that the host calls 2 (or 4 or 10) systems, those systems each call 2, and so forth, then you don't need concentrated processor power anywhere (as far as UUCP goes, anyhow). Why buy a Sequent when you could get by with an AT? :) -- Mark McWiggins UUCP: uunet!intek01!mark DISCLAIMER: I could be wrong. INTERNET: mark@intek01.UUCP (206) 641-5014
grr@cbmvax.UUCP (George Robbins) (02/10/88)
In article <238@intek01.UUCP> mark@intek01.UUCP (Mark McWiggins) writes: > > I haven't done this, but I have thought about a similar setup. Why > put all the load on the host? If you set up a "chain reaction" such > that the host calls 2 (or 4 or 10) systems, those systems each call > 2, and so forth, then you don't need concentrated processor power > anywhere (as far as UUCP goes, anyhow). NONONONONONONONO, this guy will have enough problems assuring that the batches of data from the remote sites are actually transmitted to the central site and received intact. Unless he installs some kind of higher level controls and batch tracking, he's likely to have real problems with the uucp "works right most of the time" transmission behavior. Adding multi-hop or alternate paths would transform the uncontrolled situation form an occasional "missing" batch to endless problems. > Why buy a Sequent when you could get by with an AT? :) Why not? 8-) -- George Robbins - now working for, uucp: {uunet|ihnp4|rutgers}!cbmvax!grr but no way officially representing arpa: cbmvax!grr@uunet.uu.net Commodore, Engineering Department fone: 215-431-9255 (only by moonlite)
davidsen@steinmetz.steinmetz.UUCP (William E. Davidsen Jr) (02/11/88)
[ question about lots of ports on a micro ]
After I saw your question I collected statistics on my machine, and I
think I have a reasonable answer to your question. All times quoted are
for Xenix/286, 8MHz AT, dumb serial ports.
The overhead of starting uucico, locating a remote with data to
transfer, and placing the call is about 1500ms. The steady state cpu
load is about 6% at 9600, about 2% at 2400.
Given that (a) my little machine could theoretically support about 16
dumb lines, (b) my 386 at home is about 300% faster, and (c) the steady
state overhead will be a *lot* lower with smart serial cards, I see no
reason to doubt that you could run 24 lines at up to 9600 baud without
problems. There are a number of good cards out there, including Bell
Technologies 6 port smart card.
I think you can do it with power to spare. Use lots of memory.
--
bill davidsen (wedu@ge-crd.arpa)
{uunet | philabs | seismo}!steinmetz!crdos1!davidsen
"Stupidity, like virtue, is its own reward" -mecsg@pyramid.pyramid.com (Carl S. Gutekunst) (02/11/88)
In article <9499@steinmetz.steinmetz.UUCP> davidsen@crdos1.UUCP (bill davidsen) writes: > The overhead of starting uucico, locating a remote with data to >transfer, and placing the call is about 1500ms. The steady state cpu >load is about 6% at 9600, about 2% at 2400. The amount of time needed to find work using UUCP increases exponentially with the number of jobs queued, and that's mostly time searching and sorting the UUCP work directory. Yes, you can start one job to one site in 1.5 seconds. It won't be anywhere near that when you've got 100 or more jobs queued. It will also be highly dependent on the disk used. Steady state of 6% is for sending only; receiving saturates the 386 boxes I've used to 100% instantly. <csg>
davidsen@steinmetz.steinmetz.UUCP (William E. Davidsen Jr) (02/12/88)
In article <14699@pyramid.pyramid.com> csg@pyramid.UUCP (Carl S. Gutekunst) writes: | In article <9499@steinmetz.steinmetz.UUCP> davidsen@crdos1.UUCP (bill davidsen) writes: | [ result of testing on a 286 box ] | | The amount of time needed to find work using UUCP increases exponentially with | the number of jobs queued, and that's mostly time searching and sorting the | UUCP work directory. Yes, you can start one job to one site in 1.5 seconds. It | won't be anywhere near that when you've got 100 or more jobs queued. It will | also be highly dependent on the disk used. (a) only the user time goes up, not the process initiation time, (b) sorry, the CPU will not be affected much by the disk speed (the clock time will). | | Steady state of 6% is for sending only; receiving saturates the 386 boxes I've | used to 100% instantly. I don't know about "SysV" derivitives, but the times on my 386 running Xenix/386 (I had a chance to look there since my last posting), look like CPU = 1 + (0.015 * clock) for all jobs, at all baud rates, sending or receiving, to both SysV, SysIII, and BSD sites, no matter who initiates the call. Receiving 310k at 2400 baud took 1505 sec, 3.2sec CPU (using a dumb serial port). Could you quote your configuration for hardware and software, what type of machine you were connected to, and how you measured the CPU vs. real time. I would like to repeat your test on my own hardware. I'm sure you measured the 100% under controlled conditions or you wouldn't have mentioned it, therefore I must be missing something. sixhub is the distribution machine for starix, and handles about 200 files/day. -- bill davidsen (wedu@ge-crd.arpa) {uunet | philabs | seismo}!steinmetz!crdos1!davidsen "Stupidity, like virtue, is its own reward" -me
jec@nesac2.UUCP (John Carter ATLN SADM) (02/16/88)
Posting this because mail bounced. But the prime concern should NOT be how many at once, but how much data to be moved, and how quickly. If the sites are remote, the line speed is the biggest factor - a 90K file would take ~5 minutes at 9600, and ~17 minutes at 1200. (Time determined from historic data on several local systems, some on LAN at 9600, some on dialup at 1200, transfers done at various times in the systems load cycle.) At 5 minutes/site, you get 12 sites/hour with one 9600 baud line. If you have four simultaneous 9600 baud transfers, you get 48 sites/hour. This would get the first hundred sites done in two hours. The question then becomes "How soon is 'immediate' or 'simultaneous' updating?" (Even two simultaneous uucp jobs won't complete at the same time.) The tradeoffs are the number of lines a given box can support, the number of sites to update, the speed of the lines used, how closely the 'simultaneous' updates must coincide, the amount of $$$ available initially and for growth. One machine running 8 concurrent uucp jobs at 9600 baud would be capable of doing ~100 sites/hour. If one hour is close enough for the updates of the sites, then the need is for one machine capable of continuous simultaneous I/O at 9600 on 8 ports, and the installation of 9600 baud modems at all sites. If the machine is running just the operating system (including accounting, etc) and the uucp transfers, then it's possible to get a lot of thru-put from a small machine, but it's very architecture dependent. Example: DEC's PDP 11/70 running Unix could handle more I/O with a mature version of an application than an AT&T 3B5 could running the initial port of the application, even though the 3B5 would usually be considered the more powerful of the two machines. How much was the maturity level of the software and how much was hardware dependent? I really don't know, except that the 3B5 application was rated as 50% greater capacity than the 11/70 version. How successful would a developer be in getting a group of suppliers to provide one each of their machines for testing? Maybe if you could get one each of several different Unix boxes for a period of weeks you could do real analysis on the capabilities of the various small Unix machines. -- USnail: John Carter, AT&T, Atlanta RWC, 3001 Cobb Parkway, Atlanta GA 30339 Video: ...ihnp4!cuea2!ltuxa!ll1!nesac2!jec Voice: 404+951-4642 (The above views are my very own. How dare you question them? :-)
csg@pyramid.pyramid.com (Carl S. Gutekunst) (02/22/88)
>| The amount of time needed to find work using UUCP increases exponentially >| with the number of jobs queued, and that's mostly time searching and >| sorting the UUCP work directory. > >(a) only the user time goes up, not the process initiation time, I fail to see your point. Time is time; who cares whether the CPU spends it in user code or process launch? A master uucico starting on a busy system has a lot of work to do before it can make a call, and you cannot begin to extrapo- late the time for running a queue with 100 jobs based on the time for 1. Granted, if you're running HoneyDanBer UUCP (which is in Xenix/386, at least it was in the copy I used) that this phase will be much less time consuming than for any other UUCP version. But startup is still exponential in nature. >| It will also be highly dependent on the disk used. > >(b) sorry, the CPU will not be affected much by the disk speed.... Have you ever benchmarked I/O subsystems? The type of controller, the type of disk, and a large number of other factors directly affect the amount of work the CPU is able to do in parallel. Given identical everything else, and disks that differ only in seek time, you statement is true. But any other bets are off. Anyway, it still takes *time*. With a slow disk, you can timeout the other uucico while your box sorts the work directory. This is really common on overworked 68000 boxes, and happens occasionally to 68020 systems as well, particularly those at university sites, who tend to put more users on them than God ever intended. :-) >I don't know about "SysV" derivitives, but the times on my 386 running >Xenix/386 (I had a chance to look there since my last posting), look like >[equation deleted] for all jobs, at all baud rates, sending or receiving.... Well, yeah, fuuny how it works that way. You're using process time, which does not include interrupt time. And interrupt time is where most of the difference in baud rate and incoming vs. outgoing occurs. This is especially true with HoneyDanBer, which does an excellent job of sleeping until all 70 characters of the 'g' protocol packet have been received. If you include interrupt time, a drastically different situation unfolds. Now, I have been told that a good intelligent serial card can bring the CPU load down to only 25%. But the garden variety of multi-port RS-232 boards don't do that. Also, as many people have opinied, comparing the UNIX process times to wall clock time is an exercise in futily. And BTW, Xenix/386 *is* a System V derivitive. >I'm sure you measured the 100% under controlled conditions or you wouldn't >have mentioned it, therefore I must be missing something. If one user on one Xenix 386 with a Telebit TrailBlazer counts as controlled, then yes. But nothing elaborate. Try vmstat, and just watch the idle time sink to 0%. Or if you don't have that (or the version you have isn't accurate, which several people have told me is the case on Xenix/386), try a totally CPU bound program for which you have precisely measured its wall clock time. Then run it when uucico is running. If it takes twice as long in real time, then you have a 100% saturated CPU. Another test: try two 9600 incoming uucico transfers simultaneously, and measure the individual and aggregate throughput. >sixhub is the distribution machine for starix, and handles about 200 >files/day. 200 a day? I think we have a problem in scale. I thought we were talking an order of magnitude more than that, on the scale of uunet (which transfers in excess of 300 files per hour, many of them news batches), or pyramid (150 files per hour). I would certainly trust a 386 PC for 200 news-sized files a day. 200 an hour, no way. <csg>