gr47@sdcc12.ucsd.EDU (gr47) (11/12/87)
------------------------------------------------------------------------------
12-NOV-1987 12:48:14.90 PST
I am about to write a Cray Fortran program to convert Cray datatypes to VAX
binary representations. This is needed to avoid having to convert very large
binary output datasets to ascii and back to binary for the transfer from Cray
to VAX.
My tentative flow diagram is as follows:
1) Generate large Cray unformatted Fortran sequential model output file
2) Convert to VAX binary and write to a Cray absolute file (absolute =
with no Cray control bytes) -- write VAX record control bytes as
necessary in this step
3) Transport to VAX with NETOUT program -- for binary files, NETOUT
spits out only VAX files with fixed length records (choice of either
512 or 4192 bytes)
4) Diddle with the VAX file header to make the file either VARIABLE,
STREAM or SEGMENTED
In 3) & 4), I have ruled out FIXED-LENGTH type records since use of
these would entail significant changes to VAX application programs already in
place.
NETOUT is the only reliable way to transport files at this point; I may be
able to FTP files later, but I am taking NETOUT limitations (i.e., FIXED
record output) as a given at this point.
I want to do most or all of the necessary conversion on the Cray due to
space/time considerations in transport between the machines and due to
space limitations on the VAX. I want to avoid having to do any record
restructuring, thus file duplicating, on the VAX.
Questions:
Are there any problems with changing VAX file attributes from
FIXED-LENGTH records to VARIABLE, STREAM or SEGMENTED by playing only
with the VAX file header (not reading/writing the file), assuming
appropriate control bytes have been placed in the file beforehand ?
Can someone fill me in on SEGMENTED records? Apparently, for very long
records, these comprise 'segments' of 2044 bytes each. Each segment
begins with 2 control bytes that indicates whether that segment is
the first, last, or only of that logical record, or none of the above.
What tells RMS how long the last or only (this is often less than max
length) segment of a record is ?
I would prefer to use SEGMENTED records since this would entail no changes
to my existing VAX Fortran programs. Additionally, there is a limit on the
size of a VARIABLE record (8191 longwords = 32764 bytes) which may be
surpassed in my application or that of others ... STREAM would seem to
be more suitable for my application than VARIABLE since there is no such
size limit (correct?). However, I have never used STREAM records
(un-VAX-like, :^) ); can they be used for unformatted files and are
there any drawbacks in using them for large binary files?
Thanks for any help.
Steve Piper
scripps inst. of oceanography a-020
la jolla, ca 92093
cdrgmv::piper2 (SPAN/HEPNET)
piper2%cdrgmv.span@sdsc (BITNET I think)
gr47%sdcc12@sdcsvax.ucsd.edu (ARPA/Usenet)
------------------------------------------------------------------------------