marc@rna.UUCP (Marc Johnson) (08/11/88)
Posting this for a friend in Israel.
Any responses would be appreciated and should be e-mailed to:
rna!marc@ROCKVAX or marc%rna@rockefeller.edu.
------------------------------------------------------------------------------
I am trying to write a VMS C program that takes some data files on the
microVax and then converts them into my private binary format. So far this
seemingly trivial task has taken a week. I won't go into all of the details,
but the problem is creating a binary file. VMS doesn't just have files,
it has file types. One must choose a file type, and then put data in the
file according to the rules and limits of that type. What I want to do is
the following:
1. Use the C standard I/O calls (fopen, fwrite, fclose) to
write a stream of binary bytes into a file.
2. Use ftell to find out where I am in the file.
3. Use fseek to move to a new position in the file.
4. Create a file which kermit can access and transmit literally,
meaning that the byte sequence on the VMS system then appears
on the remote system.
I have tried most of the VMS file types, and I have not yet found one that
does what I want (STDIO binary files):
a. Using the default STDIO file type, stmlf (stream lf), the file is created
perfectly, but kermit can't send it, because even when I tell kermit that
the file is binary, it expands LF to CR-LF. Also with some files, their
internal data is such that kermit fails, because the VAX record management
software (rms) doesn't encounter a LF character during a read of reclen
characters.
b. Using the file type fixed, fwrite fails to put anything into the file.
c. Using the file type stm (stream), every four bytes are followed by CR-LF.
d. Using the file type udf (undefined), the file is written correctly, but
ftell works incorrectly, and fwrites following an fseek fail to work.
e. I haven't tried var and vfc file types, because I understand from talking
to DEC support engineers that they put extra stuff in the file to delimit
my "records."
By the way, I am creating these various file types using the extra rms
parameters of fopen: fopen("test.dat", "w+", "rfm=udf").
We have been asking DEC about this problem, and in three days, I have yet to
speak to anyone who has ever programmed in VMS C. They keep asking me what
type of records I want to write, and I keep saying I want to use STDIO (which
they have never heard of) to write a file containing binary bytes.
------------------------------------------------------------------------------
Thanks in advance, netters!
Marc Johnson for Kaare Christiandhesi@bsu-cs.UUCP (Rahul Dhesi) (08/18/88)
In response to the request for a solution to the problem of how to
create a binary file under VAX/VMS that allows arbitrary seeks: There
is no ideal solution (other than switching to UNIX, of course), but you
can get by if you use stream-LF files.
Kermit will not let you transfer a stream-LF file without risking
corrupting it with newline conversions. The standard VMS "Kermit-32"
won't treat a stream-LF file as a binary file and sends it line by line
instead.
Kermit transfer can still be done, however. My bilf.c utility (that is
included with source code for zoo 2.0) will convert between stream-LF
and fixed-length-record (FLR) files. So to do a Kermit transfer, use
bilf.c to convert to FLR format, then transfer with Kermit as a binary
file. Or the other way around.
--
Rahul Dhesi UUCP: <backbones>!{iuvax,pur-ee,uunet}!bsu-cs!dhesiscjones@sdrc.UUCP (Larry Jones) (08/19/88)
In article <3689@bsu-cs.UUCP>, dhesi@bsu-cs.UUCP (Rahul Dhesi) writes: > In response to the request for a solution to the problem of how to > create a binary file under VAX/VMS that allows arbitrary seeks: There > is no ideal solution (other than switching to UNIX, of course), but you > can get by if you use stream-LF files. > > Kermit will not let you transfer a stream-LF file without risking > corrupting it with newline conversions. The standard VMS "Kermit-32" > won't treat a stream-LF file as a binary file and sends it line by line > instead. > > Kermit transfer can still be done, however. My bilf.c utility (that is > included with source code for zoo 2.0) will convert between stream-LF > and fixed-length-record (FLR) files. So to do a Kermit transfer, use > bilf.c to convert to FLR format, then transfer with Kermit as a binary > file. Or the other way around. If you're using Digital's Vax-11 C, it's much easier to just tell it you want fixed length records in the first place. All you need do is add three additional arguments on to the fopen call: "recfm=f", "mrs=512", "ctx=stm" (that's from memory, so I hope I got them right). The first says you want fixed-length records, the seconds says to make them 512 bytes long, and the last tells the C library to ignore the fact that the file is record structured and pretend it's a stream file instead (which works just grand as long as the records are fixed rather than variable-length). It is worthwhile to note that the Vax-11 C documentation contains a typo listing the last argument as "ctx=str". It is still more interesting that the library itself contains a bug which causes it to report some bizzare VMS-specific error if you actually try that rather than reporting "invalid argument value" as it should. In any event, "ctx=stm" is correct. ---- Larry Jones UUCP: uunet!sdrc!scjones SDRC scjones@sdrc 2000 Eastman Dr. BIX: ltl Milford, OH 45150 AT&T: (513) 576-2070 Nancy Reagan on superconductivity: "Just say mho."
egisin@watmath.waterloo.edu (Eric Gisin) (08/19/88)
In article <3689@bsu-cs.UUCP>, dhesi@bsu-cs.UUCP (Rahul Dhesi) writes: > > In response to the request for a solution to the problem of how to > create a binary file under VAX/VMS that allows arbitrary seeks: There > is no ideal solution (other than switching to UNIX, of course), but you > can get by if you use stream-LF files. > You can create the file in C as a stream-LF file, then change the file type to binary stream (UDF, no record attribute). Kermit will be able to handle UDF files unless it goes out of its way to disallow them. I haven't programmed VMS in a long time, but I think changing file types is done with an undocumented RMS function, or with a disk ACP QIO function. (I don't have access to manuals)
leo@philmds.UUCP (Leo de Wit) (08/23/88)
In article <351@sdrc.UUCP> scjones@sdrc.UUCP (Larry Jones) writes: >In article <3689@bsu-cs.UUCP>, dhesi@bsu-cs.UUCP (Rahul Dhesi) writes: [Rahul's reply omitted]... >If you're using Digital's Vax-11 C, it's much easier to just tell it you >want fixed length records in the first place. All you need do is add three >additional arguments on to the fopen call: "recfm=f", "mrs=512", "ctx=stm" >(that's from memory, so I hope I got them right). The first says you want >fixed-length records, the seconds says to make them 512 bytes long, and the >last tells the C library to ignore the fact that the file is record structured >and pretend it's a stream file instead (which works just grand as long as the >records are fixed rather than variable-length). Unless you're *VERY SURE* never having to port this code (and who can nowadays), you should not use this non-portable form of fopen. If you port to a system with ANSI style C compiler (supporting prototypes) you have a problem with the third argument of fopen(); it has only got two parameters on Unix. Besides, it is not really needed, as the default file type for VAX VMS C is stream LF, which amounts to 512 byte records. This will just read 512 byte blocks each time (except for the last one); I checked it. Leo. P.S. Why in the first place couldn't they leave the library functions alone, instead of 'adding all those nice features' (see also extra format types in printf, etc.)? The least they (DEC) should have done is warn inadvertent users of possible portability problems. Lucky me to come from a Unix womb (and as such somewhat better aware of the problems) 8-).
leo@philmds.UUCP (Leo de Wit) (08/24/88)
In article <20448@watmath.waterloo.edu> egisin@watmath.waterloo.edu (Eric Gisin) writes: |You can create the file in C as a stream-LF file, |then change the file type to binary stream (UDF, no record attribute). |Kermit will be able to handle UDF files unless it goes |out of its way to disallow them. |I haven't programmed VMS in a long time, but I think |changing file types is done with an undocumented RMS function, |or with a disk ACP QIO function. (I don't have access to manuals) To convert file types in VMS, use convert ( 8-). From a program the convert utility is a three-stage rocket: conv$pass_files(), conv$pass_options(), conv$convert(). Consult your manuals for the correct parameters. Leo.
mkhaw@teknowledge-vaxc.ARPA (Mike Khaw) (08/25/88)
> P.S. Why in the first place couldn't they leave the library functions > alone, instead of 'adding all those nice features' (see also extra > format types in printf, etc.)? The least they (DEC) should have done is > warn inadvertent users of possible portability problems. Lucky me to The VMS C compiler does have a "/standard=portable" switch that makes the compiler check for non-portable constructs. However, the VMS implementation of stdio.h DOES NOT PASS the portability check, so the signal/noise ratio of the portability warning messages is pretty low for a large class of C programs (is it significant that the default for the compiler is /standard=noportable?). You'd think they would write the "standard" .h files in such a way that they'd pass the portability checks; e.g., via #ifdefs. I'm a lot happier since I stopped programming on VMS! Mike Khaw -- internet: mkhaw@teknowledge.arpa uucp: {uunet|sun|ucbvax|decwrl|uw-beaver}!mkhaw%teknowledge.arpa hardcopy: Teknowledge Inc, 1850 Embarcadero Rd, POB 10119, Palo Alto, CA 94303
scjones@sdrc.UUCP (Larry Jones) (08/25/88)
In article <613@philmds.UUCP>, leo@philmds.UUCP (Leo de Wit) writes: > In article <351@sdrc.UUCP> scjones@sdrc.UUCP (Larry Jones) writes: > [stuff about using DEC extensions to fopen to get the correct record > structure] > > Unless you're *VERY SURE* never having to port this code (and who can > nowadays), you should not use this non-portable form of fopen. If you > port to a system with ANSI style C compiler (supporting prototypes) you > have a problem with the third argument of fopen(); it has only got two > parameters on Unix. Well, I wouldn't say you shouldn't use it, but I will say you should put an "#ifdef vax11c" around it! > Besides, it is not really needed, as the default file type for VAX VMS C is > stream LF, which amounts to 512 byte records. This will just read 512 byte > blocks each time (except for the last one); I checked it. This is wrong - if it wasn't needed, why would people be asking how to do it? The problem is not reading the file with C, the problem is with programs which are written in other languages which read RECORDS - something C has no (or at least very little) concept of. If you try to read a record from a stream-LF file, RMS gives you everything up to the next linefeed. If your buffer isn't big enought to hold it, you get an error and loose data. Record format IS significant. ---- Larry Jones UUCP: uunet!sdrc!scjones SDRC scjones@sdrc 2000 Eastman Dr. BIX: ltl Milford, OH 45150 AT&T: (513) 576-2070 Nancy Reagan on superconductivity: "Just say mho."
egisin@watmath.waterloo.edu (Eric Gisin) (08/26/88)
In article <618@philmds.UUCP>, leo@philmds.UUCP (Leo de Wit) writes: > In article <20448@watmath.waterloo.edu> egisin@watmath.waterloo.edu (Eric Gisin) writes: > |You can create the file in C as a stream-LF file, > |then change the file type to binary stream (UDF, no record attribute). > To convert file types in VMS, use convert ( 8-). The convert utility does not do what they wanted. It will expect the input file to have a valid record structure, which is generally not the case when binary data is written by C programs. Convert will also "drop" '\n's, and start a new record. What I meant is change the file type in the "file header" (I forget the correct term, I'm refering to the on-disk equivalent of the FAB) without changing the file data.
christiansen@CHEWI.CHE.WISC.EDU ("REED CHRISTIANSEN") (08/30/88)
In article <613@philmds.UUCP>, mcvax!hp4nl!philmds!leo@uunet.uunet (Leo de Wit) writes: >In article <351@sdrc.UUCP> scjones@sdrc.UUCP (Larry Jones) writes: >>In article <3689@bsu-cs.UUCP>, dhesi@bsu-cs.UUCP (Rahul Dhesi) writes: > [Rahul's reply omitted]... >> [Larry's text omitted]... > > [Some of Leo's stuff omitted]... >P.S. Why in the first place couldn't they [DEC] leave the library functions >alone, instead of 'adding all those nice features' (see also extra The problem (if there is one) is that the vanilla-C file types aren't adequate to deal with files created outside of the C environment, as these were. DEC could have put a crippled C on VAX/VMS, one that would not have the capabilities of their other languages, but they decided to supplement (NOT SUPPLANT) the Unix-style I/O. You can still do Unix-style I/O... >format types in printf, etc.)? The least they (DEC) should have done is >warn inadvertent users of possible portability problems. They do. Doesn't anybody read the user documentation? Chapter 1 of the VAC C Run-Time Library Information manual dicusses portability issues in painful detail: Section 1 of Chapter 1 discusses Unix I/O (and how to be compatible, if you wish); Section 4 of Chapter 1 discusses many other portability concerns. And, too, they devote an entire chapter (4) to Unix I/O. >Lucky me to come from a Unix womb (and as such somewhat better aware of the > problems) 8-). This isn't the first VMS question that could have been answered by people opening up the manuals and spending a few minutes to spin through them.