[comp.realtime] Streaming tape drives

kevin@ttidca.TTI.COM (Kevin Carothers) (05/06/89)

Greetings

This is a rather whimsical question, but I would like an answer.

I used to work for a company that made a data-acquisition front-end
to a DEC PDP-11/73 (Cyber systems, Anaheim).  At the time, we had a 
requirement to write 50Kwords/sec to a Cipher (tm) 9-track streamer tape
under the RSX O/S.

Well, we never made it. The best we did was maybe 35Kwords/Sec.
The tape drive never went into "streaming" mode, and  we all just 
kind of gave up.

Does anyone out there know if this is even possible? We tried every
conceivable buffering size, I/O mode, double-buffering schemes, etc.
RSX is very (ok, somewhat) similar to VMS in philosophy (ie; DCL), and 
the PDP and VAX have similar Command/Status/DMA register structure, so it 
seems like *someone* in the VAX/realtime world would have come across this 
problem before.

BTW --
Anyone out there use Cyber Systems data acquisition equipment? 
If so, what model(s), and any gossip about how they are/were doing?


Just curious
=========================================================================
Kevin Carothers   !{csun,psivax,rdlvax}!ttidca!kevin

jkrueger@daitc.daitc.mil (Jonathan Krueger) (05/07/89)

>In article <4374@ttidca.TTI.COM>, kevin@ttidca (Kevin Carothers) writes:
>...The best we did was maybe 35Kwords/Sec.
>The tape drive never went into "streaming" mode, and  we all just 
>kind of gave up.

>Does anyone out there know if this is even possible? We tried every
>conceivable buffering size, I/O mode, double-buffering schemes, etc.
>RSX is very (ok, somewhat) similar to VMS in philosophy (ie; DCL), and 
>the PDP and VAX have similar Command/Status/DMA register structure, so it 
>seems like *someone* in the VAX/realtime world would have come across this 
>problem before.

Based on a mild familiarity with a similar problem (getting TU80's to
stream under VMS on an 11/750) I'd say the problem is seeks, due to
directory lookups, possibly updating the date of last backup for each
file, and possible filesystem fragmentation on disk.  An excellent
DECUS presentation of several years back detailsed "the 10 laws of the
current state", several of which pertained to the need to avoid
unnecessary, inefficient, or badly timed I/O.  More generally,
realtime systems tend to require contiguous files to avoid seeks.  VMS
and RSX support contiguous files, but not contiguous filesystems, and
there's still the need to shuffle for directory lookups and possible
backup datestamping.  Some of this can be overlapped with tape writes
by buffering, but even an occasional wait loses the stream mode for a
while, causing more lost throughput than the wait itself.  And
ironically, the more successful you are at keeping the data flowing,
the more often you'll hit a necessary seek.

A brute force solution is a faster processor and asynchronous I/O;
thus the 750 could keep the TU80 in slow stream mode under standalone
backup.  When the 750 was timesharing, you couldn't do this, and the
TU80 in start-stop mode was pitifully slow.  Both the 750 and TU80
were reasonable devices, but they were badly mistuned for each other
for this application.  Another brute force solution was disk-to-disk
backups, then copying the save set to tape.  The save set was one
file, mostly contiguous, with no directory lookups or datestamps to
cut.  The cost of keeping an extra disk around for this purpose
outweighed any savings from buying the TU80.  Moral: a balanced
configuration has adequate memory, bus bandwidth, and disk capacity
for its processor, but also includes adequate backup capacity,
storage, i/o bandwidth, and acceptable degradation of system resources
during backup operations.  Also, realtime systems have learned some
lessons that may be applied to mundane needs of ordinary systems.

-- Jon
-- 

neil@zardoz.UUCP (Neil Gorsuch) (05/08/89)

In article <4374@ttidca.TTI.COM> kevin@ttidca.TTI.COM (Kevin Carothers) writes:
>Greetings
>I used to work for a company that made a data-acquisition front-end
>to a DEC PDP-11/73 (Cyber systems, Anaheim).  At the time, we had a 
>requirement to write 50Kwords/sec to a Cipher (tm) 9-track streamer tape
>under the RSX O/S.
>Well, we never made it. The best we did was maybe 35Kwords/Sec.
>The tape drive never went into "streaming" mode, and  we all just 
>kind of gave up.
>Does anyone out there know if this is even possible? We tried every
>conceivable buffering size, I/O mode, double-buffering schemes, etc.
>RSX is very (ok, somewhat) similar to VMS in philosophy (ie; DCL), and 
>the PDP and VAX have similar Command/Status/DMA register structure, so it 
>seems like *someone* in the VAX/realtime world would have come across this 
>problem before.

Hi kevin, this is Neil (small world, isn't it?).
I have a 25 inches per second tape drive streaming at 1600 bpi off a stock IBM
AT running Microport Unix.  As long as the input data can keep up, and you
use concurrent forked tasks sharing a common memory pool, and you use IPC
for synchronization, and you write your own tape driver with double chained
DMA channels, and you design your own tape interface board with automatic
DMA channel switching when one DMA channel runs into the segment boundary,
there is no problem.  I am feeding the data from the ethernet (the Solbourne
can spew out 300K per second from disk, nothing else here can keep up
with it, so the tape drive has more than enough data coming in to it).  We
have also used the tape drive in 3200 bpi mode, but that isn't reliable because
of the IBM AT's limited DMA bandwidth.  I believe that if we bothered to put
in a PC clone running at a higher clock speed, we could easily stream at
3200 bpi and 25 ips, but I never bothered, since I don't use 9 track for backup
much any more.  So, if we can do 40 Kwords/second using an AT clone, a
PDP 11/73 should be able to do much higher than that.  It's just a matter
of doing everything yourself all the way through.

Neil Gorsuch
neil@cpd.com
uunet!zardoz!neil