[comp.protocols.tcp-ip] FTP "STRU VMS" extension

adelman@TGV.COM (Kenneth Adelman) (01/30/89)

    One of the problems with implementing FTP under VMS is mapping VMS
file types into FTP TYPEs and STRUctures. Within the confines of the
FTP protocol specification, it isn't possible to, for all VMS file
types (e.g. ISAM files), FTP them in a particular fashion (i.e. TYPE I
or STRU R) and between two VMS machines and get the same file you
started with.

    MultiNet has a non-standard extension to FTP to allow
transfer of arbitrary VMS files between consenting VMS machines
called "STRU VMS". We have an agreement with one other VMS TCP
vendor to implement STRU VMS in a compatible fashion.

    STRU VMS works by prefixing the file contexts with a structure
which describes the RMS attributes of the file, and then sending the
contents of the file, uninterpreted, by using VMS block I/O.  The
structure is designed to allow the addition of new RMS attributes,
while still preserving upward and downward compatibility with the
original specification.

    I would be interested in hearing from any other VMS FTP
implementors, including our competitors, for the purpose of starting a
discussion about the proposed extension to FTP. When we reach a
consensus about how STRU VMS should work, we will publish the
resulting specification in RFC form.

						Kenneth Adelman
						TGV, Incorporated

abstine@sun.soe.clarkson.edu (Arthur Stine) (01/31/89)

From article <890129222237.9d@TGV.COM>, by adelman@TGV.COM (Kenneth Adelman):
> 
>     I would be interested in hearing from any other VMS FTP
> implementors, including our competitors, for the purpose of starting a
> discussion about the proposed extension to FTP. When we reach a
> consensus about how STRU VMS should work, we will publish the
> resulting specification in RFC form.
> 
> 						Kenneth Adelman
> 						TGV, Incorporated
Dale Moore & Co at CMU have done this with CMU/TEK FTP. They use 
STRU PAGE to accomplish the same thing between two machines using 
CMU FTP. You might want to talk to him about it
(Dale.Moore@moore.fac.cs.cmu.edu) and find out the internals of it.

art stine
sr network engineer
clarkson u

dcrocker@TWG.COM (Dave Crocker) (01/31/89)

Ken,

Wollongong's WIN/TCP for VMS has had FTP extensions for transmitting
arbitrary file types for quite awhile.  When we first announced this --
probably in the info-vax list -- there was a question about using a
public standard.  At the time, we had our focus on functionality, but
are more than somewhat interested in moving to an inter-operable spec,
as always...

Dave Crocker
VP, Engineering
The Wollongong Group, Inc.

braden@VENERA.ISI.EDU (01/31/89)

Kenneth,

The aproach you suggest works against interoperability and therefore must
be approached with suspicion.  If STRU VMS, then also STRU MVS?  What if
you want to FTP an ISAM file from a VMS to an MVS system? Is this 
infeasible?

Bob Braden

dab@opus.cray.com (Dave Borman) (01/31/89)

Kenneth Adelman writes:

>     MultiNet has a non-standard extension to FTP to allow
> transfer of arbitrary VMS files between consenting VMS machines
> called "STRU VMS". We have an agreement with one other VMS TCP
> vendor to implement STRU VMS in a compatible fashion.
> 
...
>     I would be interested in hearing from any other VMS FTP
> implementors, including our competitors, for the purpose of starting a
> discussion about the proposed extension to FTP. When we reach a
> consensus about how STRU VMS should work, we will publish the
> resulting specification in RFC form.
> 

Rather than extend the STRU command of ftp to add "STRU VMS", how
about doing it through the SITE command instead?  From RFC959, pg 33:

    SITE PARAMETERS (SITE)

	This command is used by the server to provide services
	specific to his sytem that are essential to file transfer
	but not sufficiently universal to be included as commands in
	the protocol.  The nature of these services and the
	specification of their syntax can be stated in a reply to
	the HELP SITE command.

It seems to me that the SITE command is the one to use when adding
non-standard extensions to FTP.  (I've just finished expanding our
ftp server to allow "SITE UMASK", "SITE CHMOD", "SITE IDLE" and
"SITE HELP".  UMASK and CHMOD do the expected UNIX stuff, IDLE can
be used to change the length of the idle timer, which gets reset
everytime something happens, and if it goes off it closes the
connection.  In 4.3BSD this defaults to 15 minutes.  HELP is used
to find out the syntax for using these site specific commands)

Note that any BSD based ftp does not support the "site" command,
but they do support the "quote" command, so you can still make
use of the site command by issuing commands like "quote SITE UMASK".

		-Dave Borman
		Cray Research, Inc.

adelman@TGV.COM (Kenneth Adelman) (01/31/89)

    I was hoping that I wouldn't have to defend the need for a "STRU VMS"
but here goes...

> The aproach you suggest works against interoperability and therefore must
> be approached with suspicion.  If STRU VMS, then also STRU MVS?  What if
> you want to FTP an ISAM file from a VMS to an MVS system? Is this
> infeasible?

    The idea is to provide a 'standard' way of transfering an arbitrary
VMS file UNINTERPRETED through the network. There are two reasons for this:

    1) Performance. Take your nearest VMS FTP and look at the ASCII mode
       performance as compared to the BINARY mode performance. Any
       transformation between VMS disk blocks and something like STRU Page
       is going to have a similar impact. The extra pass you have to make
       at the data can really matter on a small VAX.

    2) The need for such an extension to FTP is already recognized by
       many VMS TCP vendors. What I want is a 'standard' which will
       allow each of these different *VMS* FTP implementations to
       speak the same language.

    There is also another advantages to my proposal:

       Implementation of it is already very close to what most vendors
       provide in a non-interoperable fashion. Hopefully the small
       amount of work required to convert to the standard would
       encourage other vendors to adopt it as well. A standard without
       any implementations isn't any good to the community at large.

    This is *NOT* an attempt to solve a very difference problem such
as FTPing beasts like ISAM files between VMS and MVS. Even if FTP
contained a specification to allow any of today's VMS file types to be
FTPed, we're back in the same boat when DEC adds a NEW file type.

> Rather than extend the STRU command of ftp to add "STRU VMS", how
> about doing it through the SITE command instead?  From RFC959, pg 33:

    I'm aware of the SITE command, but my understanding was that
the SITE command was reserved for `non-standards', whereas this
proposal would only be useful as an extension to the protocol. As I
said above, I'm looking for more than just us to implement this.

    Perhaps what we need to add to FTP is a "STRU O VMS", where O is
short for operating system, and the resulting structure is the
VMS-specific one defined...

    I'm not sure this is appropriate to involve the entire TCP-IP list
in this discussion.  Anyone interested receiving copies of future
correspondance, please send mail to stru-vms-request@tgv.com.

							    Ken

KASHTAN@IU.AI.SRI.COM (David L. Kashtan) (01/31/89)

Just to throw a little more fuel on the STRU VMS fire:

I think you make a valid (though orthogonal) point about interoperability
in the face of "non-traditional" files.  At the moment we have interoperable
ways of moving ASCII text, raw BINARY bytes and (at least in some FTP
implementations) BINARY variable length records.  It would also be nice
to be able to interoperably move files such as ISAM and TEXT with non-imbeded
print-control but we don't even have any standards for the interoperable
representation of the data.

In the meantime, there are 2 issues that the VAX/VMS FTP world needs to deal
with:
	1) The ability to transfer, in a transparent fashion, arbitrary
	   VAX/VMS files to/from other cooperating machines.

	2) The ability to take advantage of the knowledge that both sides in
	   the FTP transfer are VAX/VMS machines to maximize the FTP
	   throughput by not having to do any data conversion before putting
	   the file data "on-the-wire".

The suggestion of a STRU VMS was consistent with semantics of the STRU
command.  Interoperability is not sacrificed as long as non-VAX/VMS
sites continue to (correctly) reject the STRU VMS command. All the
VAX/VMS clients that will adhere to this standard will attempt to use
STRU VMS when first communicating with the server but upon the rejection
of the STRU VMS command will revert to the interoperable behavior.  The
result we have is that clients that support STRU VMS will initially ask
servers for STRU VMS.  If they receive a success, then this "enhanced-VMS"
FTP communication mode will be used.  If they receive a failure, well then
we are back to the current situation (where STRU VMS doesn't exist).

There have been suggestions that the SITE command be used for this extension.
I am not overly enthusiastic about this -- the transfer mode implied by
STRU VMS is really exclusive of STRU [F, R or P] and not something orthogonal.
When you are no longer in STRU VMS then you want to explictly be in
STRU [F, R or P].

There has also been a suggestion of using STRU P for this extension.
Semantically it feels better to me than using the SITE command but, as
Ken Adelman has already pointed out, in conforming to the FTP STRU P
specification we lose a considerable amount of efficiency.

In conclusion, what we are asking for here is NOT anything intended to
be interoperable with non-VMS sites (although the way VMS clients need
to be written will ensure interoperability with non-VMS sites).  We need
some way of allowing consenting sites (I would assume these to be exclusively
VAX/VMS sites) to dynamically "upgrade" their class of FTP service.

David
-------

cpw%sneezy@LANL.GOV (C. Philip Wood) (01/31/89)

We don't need to add vendor propriatary stuff to FTP.  Binary
and network ascii seem sufficient to me.  Encoding and decoding before
and after a binary file transfer can be used when the file is some form of
random access, master indexed, indexed, mostly empty, massively addressable
database.  (In UNIX terms, encoding == dump, decoding == restore).

Phil Wood,  cpw@lanl.gov

braden@VENERA.ISI.EDU (02/01/89)

	
	    I'm aware of the SITE command, but my understanding was that
	the SITE command was reserved for `non-standards', whereas this
	proposal would only be useful as an extension to the protocol. As I
	said above, I'm looking for more than just us to implement this.

Actually, the SITE command is intended for operating-system-specific
extensions.  It is the place I would have put (and DID put, in the UCLA
ACP in 1975) put such operating-system-specific extensions to FTP as
those you propose.  SITE provides a legal way to make system-specific
extensions without breaking the rest of the standard.  Of course, the
multiple vendors of network code for VMS should agree on the SITE
parameters for this purpose!
   ...
   
	    I'm not sure this is appropriate to involve the entire TCP-IP list
	in this discussion.  Anyone interested receiving copies of future
	correspondance, please send mail to stru-vms-request@tgv.com.
	
Actually, issues of interoperability and standards, vs. ad hoc protocol
extensions (nice word), seems like a very appropriate issue for this 
forum.

Bob Braden
	

jqj@HOGG.CC.UOREGON.EDU (02/01/89)

Phil Wood suggests that anything other than binary (and network ascii,
but by extension we could even do without that) is unnecessary for FTP
since we can have encode and decode filters in a pipeline at either end
of an FTP transfer.  The problem with this suggestion is that FTP does
not support in any standard fashion the idea of reading from or writing
to a filter.  Without some such mechanism, I'd have to *log in* on the
remote system, encode the file, then log out, run FTP to get it in
binary mode, then run the decode.  But maybe I just have FTP read
access and not login priviliges on the remote system!

One alternative to the STRU VMS suggestion is to bury a set of
permitted filters in the remote filename space supported by a
particular implementation, e.g. something suitably gross like:
	FTP> bin
	FTP> put "|encode foo.bar;3" "|decode foo.bar" 
VMS implementations would then only have to agree on the syntax and
function of the filters they supported (note that "|decode myfile.txt"
is a legal Unix file name, but not a legal VMS file name, so there is
no ambiguity).

Note that the different VMS FTP implementations already have non-null
mappings from the ftp-allowed file specifications to the local file
names.  For example, TWG's FTP server allows pseudo-Unix remote file
names, and translates them into valid VMS filespecs ("|decode foo.bar"
currently becomes "9cdecode$7afoo.bar;1").

I prefer STRU VMS.

dcrocker@TWG.COM (Dave Crocker) (02/01/89)

Ahem!!

Back to first principles:  If there is a strong constituency and multiple
implementations, a standard should be developed.

When we first came out with our upgrade to FTP, to allow transfer of
arbitrary VMS files, there were no other implementations to talk to and
we STILL had customers demanding that we create/conform to a standard.

Certainly, it is reasonable to try to bury the VMS-ness into a clean
framework, preferably permitting addition of other operating systems.
However, based upon the feedback we got some months ago, there is no doubt
in my mind that we need to make the spec public and certified.

Dave Crocker
The Wollongong Group, In.c

cmills@CCY.BBN.COM (Cyndi Mills) (02/01/89)

> We don't need to add vendor propriatary stuff to FTP.  Binary
> and network ascii seem sufficient to me.  Encoding and decoding before
> and after a binary file transfer can be used ...
> ... (In UNIX terms, encoding == dump, decoding == restore).

In theory, I agree with Phil.  Any type of compression/encryption/encoding
program that includes O/S-specific file information can have as its output a
file.  The contents of the file may be transferred via FTP to another machine
which runs a program to reverse the algorithm to decompress/decrypt/format the
result into a file.

This approach treats FTP as a bulk-transport mechanism, NOT as an intelligent
file-transfer mechanism that converts between file representations, but is
handy for several applications, such as distributing software among like-minded
O/S (the compressed .z files under UNIX come to mind.)

The drawback is that it usually requires a two-step process. How do you
transfer an entire directory tree, for example, within FTP, if a requirement
for that transfer is that the O/S-specific attributes of each file is preserved
across the transfer?  The user must remember to decode each file upon exiting
FTP, but he may not know the names of all the files in the transferred
directory or how many there are.  This can complicate automated procedures.
Also, the intermediate undecoded files can pose problems in naming space and
disk space.

The first purpose of the SITE VMS or STRU VMS command is to allow a
pre-specified two-step process to be performed automatically within the FTP
command to each file AS IT IS TRANSFERRED, where this process is so common that
it might be applied to every file in a directory.  This eliminates the two-step
process, the problems of storing or naming an intermediate file, and the
problems of applying this process in automated scripts or to entire directory
trees.

Second, if you are implementing an O/S-specific command, I applaud the attempt
to have it be the same across multiple TCP/IP implementations, to improve
interoperability.  Since you are trying to specify a useful VMS "protocol
extension" as an ad-hoc standard, you might consider including not only the RMS
headers, but flags to indicate compression and/or encryption that might be
optionally implemented.  Or a field to specify a procedure, command file, or
program to be executed upon completion of each individual file transfer to
perform these and/or other conversions.

Sample command: > SITE VMS ENCRYPT COMPRESS
style		> SENDTREE [LOCALDIR] [REMOTEDIR]

						Cyndi Mills

mrc@SUMEX-AIM.STANFORD.EDU (Mark Crispin) (02/01/89)

Once upon a time, there was an operating system called Tenex (and its greater
child, TOPS-20) which lots of sites ran.  In fact, it was the Standard ARPANET
Operating System.  To accomodate its internal format of files, STRU P was
defined especially for that purpose.  To this day, this is the format that
TOPS-20 systems prefer to exchange files.

I see nothing wrong with the STRU VMS proposal, although it would be nice if
they could use the existing STRU P mechanism if at all possible.

I'm a little surprised there isn't a STRU TAR for better Unix file transfers
yet...

-- Mark --

-------

postel@VENERA.ISI.EDU (02/01/89)

Hey!  What about the STRU PAGE suggestion copied below?  Can someone
explain why it is not an adaquate soultion to the problem?

--jon.

  Date: 30 Jan 89 16:13:33 GMT
  From: sun.soe.clarkson.edu!abstine@tcgould.tn.cornell.edu  (Arthur Stine)
  Subject: Re: FTP "STRU VMS" extension
  To: tcp-ip@sri-nic.arpa

  ....

  Dale Moore & Co at CMU have done this with CMU/TEK FTP. They use 
  STRU PAGE to accomplish the same thing between two machines using 
  CMU FTP. You might want to talk to him about it and find out the 
  internals of it.  (Dale.Moore@moore.fac.cs.cmu.edu)

  art stine
  sr network engineer
  clarkson u

adelman@TGV.COM (Kenneth Adelman) (02/01/89)

> > ... (In UNIX terms, encoding == dump, decoding == restore).
>
> In theory, I agree with Phil.  Any type of compression/encryption/encoding
> program that includes O/S-specific file information can have as its output a
> file.  The contents of the file may be transferred via FTP to another machine
> which runs a program to reverse the algorithm to decompress/decrypt/format the
> result into a file.

> This approach treats FTP as a bulk-transport mechanism, NOT as an intelligent
> file-transfer mechanism that converts between file representations, but is
> handy for several applications, such as distributing software among like-minded
> O/S (the compressed .z files under UNIX come to mind.)

    The problem is that the market recognizes a need to an intelligent
file-transfer mechanism which isn't any harder to use than FTP.

> The first purpose of the SITE VMS or STRU VMS command is to allow a
> pre-specified two-step process to be performed automatically within the FTP
> command to each file AS IT IS TRANSFERRED, where this process is so common that
> it might be applied to every file in a directory.  This eliminates the two-step
> process, the problems of storing or naming an intermediate file, and the
> problems of applying this process in automated scripts or to entire directory
> trees.

    Understand that 'building' the 'filter' program into FTP isn't enough.
We need a way of determining if the receiving FTP will decode the file before
we encode it. Yes, the "SITE" command could be used for this.

> Second, if you are implementing an O/S-specific command, I applaud the attempt
> to have it be the same across multiple TCP/IP implementations, to improve
> interoperability.  Since you are trying to specify a useful VMS "protocol
> extension" as an ad-hoc standard, you might consider including not only the RMS
> headers, but flags to indicate compression and/or encryption that might be
> optionally implemented.  Or a field to specify a procedure, command file, or
> program to be executed upon completion of each individual file transfer to
> perform these and/or other conversions.

    Actualy ENCRYPTION and COMPRESSION should probably be a MODE command.
There is already defined a standard for run-length data compression
(MODE C) which the server on IU.AI.SRI.COM implements for those would
would like to test it against their implementations. Our FTP also has
the Unix 'comress' LZ-compression built in as a non-standard extension
called "MODE LZ" (try picking up a file in MODE LZ from IU.AI.SRI.COM
and uncompressing it on your Unix machine; actually, this will only work
in TYPE I, because otherwise the transfermation LF->CRLF->compressed(CFLF)->
network->decompress(CRLF)->CRLF->LF won't work).

    Note MODE LZ does not conflict with STRU VMS, and they can be used together
to increase performance over most slower-than-ethernet speed lines.

    No flames, I'm not arguing for a standard for this. But, we
implemented it because our customers asked for it.

								Ken

cpw%sneezy@LANL.GOV (C. Philip Wood) (02/03/89)

I think we should either

	attempt to solve the interoperablity problem using:
	
		an eXternal File Representation?
		Pre and post-processing options? (tar; compress/uncompress;
		    tar for example)
		.
		.
		.
		with FTP.

	or, else realize that it is an os specific problem, and
	interoperablity is not the goal. Maybe "we" want a 'vcp' for
	VMS copy, etc.  The vendor and it's third parties could come up
	with the standard for that.  They probably already have one, it
	just needs to operate on top of TCP/IP instead of DECNET.
	
Phil Wood,  cpw@lanl.gov

cpw%sneezy@LANL.GOV (C. Philip Wood) (02/03/89)

One more thing and I'll leave it to the experts.

Rather than STRU VMS, make it STRU {LOCAL|EXOTIC}.  If two VMS hosts
were "inter-operating" than either option would work correctly.

	STRU LOCAL means that files transfered from the local host
	to the "exotic" host (I could not use Foreign or Remote because
	of File and Record) would be in a standard-for-local-operating-system
	format (SFLOS) ('tar;compressed' for UNIX, 'cpio;???' for ATT, ...).
	Conversely, files transfered to the local host would be in this
	"standard" format.  The "Exotic" host would have to understand
	this "standard".

	STRU EXOTIC means that the structure for a "file" transfer
	would be in standard-for-remote-operating-system  (SFROS) format.
	
That way we don't run out of single character mneumonics for all the
"Open" operating systems coming down the line.

Phil Wood,  cpw@lanl.gov

bzs@pinocchio.UUCP (Barry Shein) (02/07/89)

What I don't understand is why this isn't taken care of extra-FTP by
some sort of archiving utility like tar.

How does one transfer ISAM (eg.) files by tape between VMS systems?
Why can't a similar mechanism be used?  The obvious advantage is that
such representations should even work when one wants to push such
files through third-party, non-VMS systems since all the info to
re-create the file gets bundled into the transferred file itself
rather than relying on wire transmission as server/client commands.

It wouldn't really occur to me to ask for an extension to FTP to
transfer an arbitrary file tree between Unix's (for example), I'd just
bundle it up with tar and send *that* (possibly compressing and/or
uuencoding if need be.) In fact, that's SOP.

At best I could imagine some sugar in the VMS server/clients which
might say "Hmm, that's a MUMBLE file, I'll spawn off a FROTZ command,
send the result of that, and if the other side is a VMS site he'll
recognize the header and automatically do the opposite", but none of
that needs a change in the protocol spec since it only affects the OS
interface, not the network interface (the file would just be xferred
binary I'd guess.) Personally I hate that kind of magic.

No FTP extensions should be needed (eg. something like embedding UNIX
magic numbers should do it.) And again, if the other side wasn't a VMS
site it would work also (that is, would store the file image but make
no attempt to unpack it.) A good example of the utility of this
approach is putting VMS files for Anonymous FTP onto a non-VMS system.

Isn't this the scheme that Unix tar, Macintosh PACKIT, MS/DOS ARC etc
have been using for years?

	-Barry Shein, ||Encore||

hwchoy@zpovc.DEC.COM (Heng-Wah Choy DEC Singapore SWS) (02/07/89)

> From:	DECWRL::"encore!pinocchio!bzs@talcott.harvard.edu" "Barry Shein  6-Feb-89 1136 EST"  7-FEB-1989 08:50
> To:	KASHTAN@IU.AI.SRI.COM
> Subj:	 FTP "STRU VMS" extension
> 
> Cc: braden@venera.isi.edu, tcp-ip@sri-nic.arpa
>  
>  
> What I don't understand is why this isn't taken care of extra-FTP by
> some sort of archiving utility like tar.
>  
	There certainly is. Its known as BACKUP.

> How does one transfer ISAM (eg.) files by tape between VMS systems?
> Why can't a similar mechanism be used?  The obvious advantage is that
> such representations should even work when one wants to push such
> files through third-party, non-VMS systems since all the info to
> re-create the file gets bundled into the transferred file itself
> rather than relying on wire transmission as server/client commands.
>  
	It does work.

> It wouldn't really occur to me to ask for an extension to FTP to
> transfer an arbitrary file tree between Unix's (for example), I'd just
> bundle it up with tar and send *that* (possibly compressing and/or
> uuencoding if need be.) In fact, that's SOP.
>  
	The reason is UNIX (I'm sorry) has a very simple file system and
structure. Correct me if I'm wrong but a UNIX file can be safely treated 
as a binary byte stream, isn't that so?

> At best I could imagine some sugar in the VMS server/clients which
> might say "Hmm, that's a MUMBLE file, I'll spawn off a FROTZ command,
> send the result of that, and if the other side is a VMS site he'll
> recognize the header and automatically do the opposite", but none of
> that needs a change in the protocol spec since it only affects the OS
> interface, not the network interface (the file would just be xferred
> binary I'd guess.) Personally I hate that kind of magic.
>  
> No FTP extensions should be needed (eg. something like embedding UNIX
> magic numbers should do it.) And again, if the other side wasn't a VMS
> site it would work also (that is, would store the file image but make
> no attempt to unpack it.) A good example of the utility of this
> approach is putting VMS files for Anonymous FTP onto a non-VMS system.
>  
> Isn't this the scheme that Unix tar, Macintosh PACKIT, MS/DOS ARC etc
> have been using for years?
>  
> 	-Barry Shein, ||Encore||

	What Kenneth was asking for is an intelligent option where a VMS 
implementation of FTP will attempt to establish that fact to its 
counterpart. And if both parties are VAX/VMS then hey we can do things a 
lot faster and cheaper since we have some "mutual understanding". If 
not, then too bad, its back to the "standard FTP". Ken is trying to 
establish a standard for this "mutual understanding" among VMS/FTP ONLY. 

Your suggestion regarding the use of archival facility (ie BACKUP) as a 
pre-post processor is of course valid. But as some one else has 
mentioned, that pre-post processor takes time (cpu) and resources (read 
disk space). On VMS systems, disk quota are tightly controlled and if 
you don't have enough, its No Can Do. Eg if you retrieve a 1000 blocks 
archive file (in VMS its called a backup saveset), you need to have 2500 
blocks (1000 to keep the saveset, at least temporarily, and another 
1000+ for the files that will be extracted from the save set). 

For a long time, I was using Kermit to transfer files between 2 VAXes 
that weren't on the same DECnet. Now to transfer an indexed (read ISAM) 
file or a directory tree, I had to back it up then send it through 
Kermit in binary, and then restore it on my end. As I said, diskquota 
control can be a real restriction. (Most of the place wouldn't simply 
give you more disk space you know, you have to ask and justify for it). 
Besides there's the hassle of backing up the files and then restoring 
it. Not only that, efficiency on the wire suffers because BACKUP 
(unlike TAR) impose substantial overhead as it will keep lots of extra 
information about the files. This is to allow the recovery of data even 
when the saveset has some corruptions. Although I don't use much of 
UNIX, I know TAR's internal very well. It is nowhere compared to BACKUP, 
sort of comparing a Toyota to a Jaguar.

Ken, I am in favour of your STRU O VMS option. In which case, all OS can 
have such an option .. STRU O MVS, STRU O U43, STRU O EMBOS...

It does nothing to harm interoperability, as far as I can see. Its what 
the real world needed, if there's no standard then it'll just be 
non-standard features, so why not standardize it and make everybody 
happy?

Rgds,

Choy Heng Wah
Software Specialist

._._._._._._._.
|d|i|g|i|t|a|l|   Digital Equipment Singapore
._._._._._._._.

Disclaimer : What I express here is purely my own opinion and does not
             represent any policy of Digital Equipment Corp and its
             subsidiaries ... blah blah blah ...

EASYnet  : {zposws|zpovc|zpoact}::hwchoy
InterNET : hwchoy%zpovc.dec@decwrl.dec.com
	   <@decwrl.dec.com:hwchoy@zpovc.dec.com>  <-- source route addressing
	   hwchoy@zpovc.dec.com

amanda@lts.UUCP (Amanda Walker) (02/08/89)

bzs@pinocchio.UUCP (Barry Shein) writes:

    What I don't understand is why this isn't taken care of extra-FTP by
    some sort of archiving utility like tar.

    [...]

    It wouldn't really occur to me to ask for an extension to FTP to
    transfer an arbitrary file tree between Unix's (for example), I'd just
    bundle it up with tar and send *that* (possibly compressing and/or
    uuencoding if need be.) In fact, that's SOP.

    [...]

    Isn't this the scheme that Unix tar, Macintosh PACKIT, MS/DOS ARC etc
    have been using for years?
    
    	-Barry Shein, ||Encore||

I agree with Barry's philospophy.  What our FTP does is to bundle up
Macintosh files into MacBinary format and send them in Image mode.  If the
other machine is anything but another Macintosh, they just get stored as-is.
If the other side is a Macintosh, it says, "gee, this is a MacBinary file;
I'll unpack it."  This lets you FTP applications and so on between two
Macs without any hassle.  The only problem is that MacBinary is a brain-
damaged format (no magic numbers), so we add SITE commands to turn the
automatic unpacking on and off.  This prevents tar files from being
falsely recognized as MacBinary and resulting in several-megabyte
invisible files full of garbage :-).  Using Apple's AppleSingle &
AppleDouble formats will solve this problem in future releases.

Nopw, I must admit that this approach works best on systems where
files are basically "Streams o' Bytes" like UNIX, but I can't imagine
it would be too hard to use a variant of ANSI tape format D or something
under VMS/TOPS-20/whatever.

-- 
Amanda Walker			...!uunet!lts!amanda / lts!amanda@uunet.uu.net
			  InterCon, 11732 Bowman Green Drive, Reston, VA 22090
--
I humans were meant to fly, the airlines wouldn't keep losing our luggage...

KASHTAN@IU.AI.SRI.COM (David L. Kashtan) (02/08/89)

>What I don't understand is why this isn't taken care of extra-FTP by
>some sort of archiving utility like tar.

That has, in fact, been the solution for a LONG time.  Make a program that
can PACK/UNPACK VMS Backup Savesets using files that can be transferred in
FTP IMAGE mode.  Aside from various the system administration problems that
have already been enumerated there is also a psychological implication.  The
fewer steps you have to go through to get a file transfered the happier you
are (witness the preference of "rcp" over FTP when available).  There have
been many times that I have used slower file transfer methods just because
it was less work for me (the difference in transfer times has to be pretty
significant for me to prefer the more laborious method).  There is also the
issue of ensuring that EVERYBODY you might want to exchange VMS files with
has all the appropriate tools (isn't this what standards are for?).  As a
TCP vendor we are very concerned about providing MAXIMUM functionality to our
customers -- and this issue is of pretty serious concern to us.  So, rather
than just DO something so that machines running our software can communicate
in this fashion we have asked the TCP community to discuss the best way to set
a standard so that we can interoperate with other VMS TCPs for this "extended"
service yet still interoperate with the rest of the TCPs in the world.  I
feel that Ken's suggestions are a good solution to the problem (but I am
obviously biased).

>It wouldn't really occur to me to ask for an extension to FTP to
>transfer an arbitrary file tree between Unix's (for example), I'd just
>bundle it up with tar and send *that* (possibly compressing and/or
>uuencoding if need be.) In fact, that's SOP.

But that is just the point -- UNIX files and FTP IMAGE transfer are a good
match (i.e. they give you the functionality that you need).  We are looking
to do something equivalent for VMS, where IMAGE transfers are not sufficient.

>At best I could imagine some sugar in the VMS server/clients which
>might say "Hmm, that's a MUMBLE file, I'll spawn off a FROTZ command,
>send the result of that, and if the other side is a VMS site he'll
>recognize the header and automatically do the opposite", but none of
>that needs a change in the protocol spec since it only affects the OS
>interface, not the network interface (the file would just be xferred
>binary I'd guess.) Personally I hate that kind of magic.

AHA - but now you have implied some "standard" in the FTP protocol that
would allow both sides to recognize that they can do this.  In fact, this
is really what Ken's suggestion amounts to.

>No FTP extensions should be needed (eg. something like embedding UNIX
>magic numbers should do it.) And again, if the other side wasn't a VMS
>site it would work also (that is, would store the file image but make
>no attempt to unpack it.) A good example of the utility of this
>approach is putting VMS files for Anonymous FTP onto a non-VMS system.

But you would still need to know what "mode" the other side is in to
correctly interoperate with it.  What if you assumed that you were in
"include-the-magic-number-and-transfer-the-file-transparently" mode and
were talking to a "standard" FTP on the other side.  What if you were
transferring an ASCII file.  In this case, the correct thing to do is
to use the standard ASCII transfer mode.  How do you recognize automagically
that you should do this.  Once again -- we need some sort of "standard"
specified to allow the recognition that the other side can do what you
want it to.

David
-------

bzs@pinocchio.UUCP (Barry Shein) (02/09/89)

From: David L. Kashtan <IU.AI.SRI.COM!KASHTAN@talcott.harvard.edu>
>That has, in fact, been the solution for a LONG time.  Make a program that
>can PACK/UNPACK VMS Backup Savesets using files that can be transferred in
>FTP IMAGE mode.  Aside from various the system administration problems that
>have already been enumerated there is also a psychological implication.  The
>fewer steps you have to go through to get a file transfered the happier you
>are (witness the preference of "rcp" over FTP when available).  There have
>been many times that I have used slower file transfer methods just because
>it was less work for me (the difference in transfer times has to be pretty
>significant for me to prefer the more laborious method).  There is also the
>issue of ensuring that EVERYBODY you might want to exchange VMS files with
>has all the appropriate tools (isn't this what standards are for?).  As a
>TCP vendor we are very concerned about providing MAXIMUM functionality to our
>customers -- and this issue is of pretty serious concern to us.  So, rather
>than just DO something so that machines running our software can communicate
>in this fashion we have asked the TCP community to discuss the best way to set
>a standard so that we can interoperate with other VMS TCPs for this "extended"
>service yet still interoperate with the rest of the TCPs in the world.  I
>feel that Ken's suggestions are a good solution to the problem (but I am
>obviously biased).

(what's wrong with biased :-)

If it's a matter of saving steps why not just build a DCL wrapper
which does the job? Heck, Interlisp-D had what amounted to a very good
NFS based on vanilla FTP, required no changes to the server (eg. UNIX
or VMS) system as I remember.

>>It wouldn't really occur to me to ask for an extension to FTP to
>>transfer an arbitrary file tree between Unix's (for example), I'd just
>>bundle it up with tar and send *that* (possibly compressing and/or
>>uuencoding if need be.) In fact, that's SOP.
>
>But that is just the point -- UNIX files and FTP IMAGE transfer are a good
>match (i.e. they give you the functionality that you need).  We are looking
>to do something equivalent for VMS, where IMAGE transfers are not sufficient.

You missed my point. Unix directory trees and FTP IMAGE are *not* a
good match. You have to bundle it up with tar (or equivalent) first.

And saving/restoring file modes requires this even for a single file.

Maybe this gets closer to the heart of the matter, perhaps what VMS
really needs is an analog of Unix's RCP which can transfer directory
trees and preserve file modes. Why not a separate VMSCP utility for
VMS? At least that idea extrapolates cleanly to any number of OS's and
we can leave FTP alone to do what it was designed to, zap the bytes
from point A to point B with a minimum of interpretation (I'd be more
in favor of dropping TENEX mode from FTP.)

Just out of curiosity, how does DECNET handle all this (and, to go one
step further, is FTAM powerful enough to satisfy this requirement?)?

	-Barry Shein, ||Encore||

cmf@cisunx.UUCP (Carl M. Fongheiser) (02/10/89)

In article <8902061636.AA13703@pinocchio.UUCP> bzs@pinocchio.UUCP (Barry Shein) writes:
>How does one transfer ISAM (eg.) files by tape between VMS systems?
>Why can't a similar mechanism be used?  The obvious advantage is that
>such representations should even work when one wants to push such
>files through third-party, non-VMS systems since all the info to
>re-create the file gets bundled into the transferred file itself
>rather than relying on wire transmission as server/client commands.

Not a terrific example.  Normally this is done using BACKUP, but
BACKUP adds all kinds of extra gunk (like CRC's and ECC blocks) that
shouldn't be necessary for FTP.  Also, BACKUP savesets happen to
be some of the hardest things to transfer using FTP, since BACKUP likes
to create giant records (32,258 bytes by default for disk savesets),
and unlike tar, can't deal with the file if it shows up with a smaller
record size on the other end.

Naturally, VMS users have ways of dealing with this, but they're ugly
(probably uglier than the code required to deal with STRU VMS!)

>It wouldn't really occur to me to ask for an extension to FTP to
>transfer an arbitrary file tree between Unix's (for example), I'd just
>bundle it up with tar and send *that* (possibly compressing and/or
>uuencoding if need be.) In fact, that's SOP.

Sure, and it works fine and dandy.  But tar came with your system,
and you didn't need any extra gunk to make that all work, did you?
It just happens that Unix files are nice, simple, easy beasts to deal
with.  VMS files aren't.  BACKUP makes it a little easier, but it really
is a big hassle to do something like that, since you can't directly
FTP a BACKUP saveset either.

>At best I could imagine some sugar in the VMS server/clients which
>might say "Hmm, that's a MUMBLE file, I'll spawn off a FROTZ command,
>send the result of that, and if the other side is a VMS site he'll
>recognize the header and automatically do the opposite", but none of
>that needs a change in the protocol spec since it only affects the OS
>interface, not the network interface (the file would just be xferred
>binary I'd guess.) Personally I hate that kind of magic.

OK, I'll bite here.  What if the file I'm transferring contains that
magic string or number which tells the FTP server/client to go into
the magic mode, but doesn't need the interpretation.  How do I turn
that off?

>No FTP extensions should be needed (eg. something like embedding UNIX
>magic numbers should do it.) And again, if the other side wasn't a VMS
>site it would work also (that is, would store the file image but make
>no attempt to unpack it.) A good example of the utility of this
>approach is putting VMS files for Anonymous FTP onto a non-VMS system.

Sure, but as I said above, you'd need some out-of-band way to turn it
off.

>Isn't this the scheme that Unix tar, Macintosh PACKIT, MS/DOS ARC etc
>have been using for years?

Sure it is, but all of those operating systems have comparatively
simple file formats, and all of those programs write nice, simply
formatted files themselves.  VMS is blessed with neither, and unless
someone's volunteering to write a program to do something like that,
I'd rather let the VMS FTP's have a little private magic to themselves.

				Carl Fongheiser
				University of Pittsburgh
				cmf@unix.cis.pittsburgh.edu
				...!pitt!cisunx!cmf
				CMF@PITTVMS.BITNET

KASHTAN@IU.AI.SRI.COM (David L. Kashtan) (02/10/89)

>If it's a matter of saving steps why not just build a DCL wrapper
>which does the job? Heck, Interlisp-D had what amounted to a very good
>NFS based on vanilla FTP, required no changes to the server (eg. UNIX
>or VMS) system as I remember.
  This is a solution -- though not a pretty one.  It loses on several
  fronts:
	1) It doesn't address the issues of system management and disk
	   quotas.   It is definitely not a selling point for the software
	   if you are required to have 2 times the disk space available
	   in order to transparently transfer the files

	2) It is considerably slower than just transferring the files

	3) Even with the wrapping (though I guess we could make the FTP
	   client do it behind your back) it is not as convenient to use
	   as directly using FTP.

	4) What about the destination site where you don't have login
	   access to invoke the wrapping.

>>>It wouldn't really occur to me to ask for an extension to FTP to
>>>transfer an arbitrary file tree between Unix's
>>>	.
>>>	.
>>
>>But that is just the point -- UNIX files and FTP IMAGE transfer are a good
>>match (i.e. they give you the functionality that you need).  We are looking
>>to do something equivalent for VMS, where IMAGE transfers are not sufficient.
>
>You missed my point. Unix directory trees and FTP IMAGE are *not* a
>good match. You have to bundle it up with tar (or equivalent) first.

IMAGE mode exactly matches the transfer characterstics you need for the UNIX
file data.  The LIST/NLST FTP commands provide "almost" everything you need
to implement directory tree copying (in fact -- you just need an extension to
the UNIX wildcard syntax to make that possible).  And the current FTP spec.
is most definitely well suited to copying whole UNIX directories (MGET/MPUT).
So in many cases "tar" is not needed -- and the facilities are already there
for a vendor to implement a full UNIX directory tree copy without having to
resort to an FTP protocol extension.  We are just trying to arrange for the
same situation for VAX/VMS.

>Maybe this gets closer to the heart of the matter, perhaps what VMS
>really needs is an analog of Unix's RCP which can transfer directory
>trees and preserve file modes. Why not a separate VMSCP utility for
>VMS? At least that idea extrapolates cleanly to any number of OS's and
>we can leave FTP alone to do what it was designed to, zap the bytes
>from point A to point B with a minimum of interpretation (I'd be more
>in favor of dropping TENEX mode from FTP.)

I wouldn't argue of this one much -- a VMS equivalent to UNIX's RCP is
a good idea (and we will probably by supplying one sometime in the future)
but it will be even more difficult to get all the VMS TCP vendors to agree
on a whole NEW application protocol than for them to agree to an extension
to an existing protocol (an extension which, by the way, requires very little
effort for VMS TCP vendors to implement -- this goes a LONG way towards
ensuring that all VMS TCP's can interoperate).

>Just out of curiosity, how does DECNET handle all this

DECNET uses a file data transfer protocol called DAP -- which was designed
to address exactly these issues of sharing different "format" files between
heterogeneous DEC systems (and, although the implementations are a bit slow
for my taste, DAP managed to do a pretty decent job of allowing transparent
file transfers between not just VAX/VMS systems but also to/from most other
systems sold by DEC over the years -- like TOPS-20, RSX ...etc).  I would
point out that DAP is complicated enough that few people would WANT to adopt
it as any kind of standard.

Still battling for a "STRU O xxx" standard,
David
-------

chris@GYRE.UMD.EDU (Chris Torek) (02/11/89)

Those who are arguing over the proper predefined standard for VMS
transmission might do well to observe that most of the successful
parts of Unix that are considered `standard' got that way by being
written first instead of defined first.  Moreover, most of those
evolved, so that if they had been standardised without first being
tested, we would have something less useful.

In other words, if you want it done, do it yourself, then hack away
at it until you really like it.  *Then* cry out for standards, if
you still need them---likely most people will have installed your
code already anyway.

(I realise that some of you are doing this already.  Make that,
`I hope that some of you....'  The approach is not perfect, but it
is much more fruitful than sending mail to tcp-ip.)

Chris

abstine@sun.soe.clarkson.edu (Arthur Stine) (02/12/89)

How about if we take a quick vote on the various solutions to this
problem and then someone can author an RFC describing how it would be
done. I think the Un*x types are being somewhat prejudiced in this case
by refusing to recognize that there is a need for this type of facility.
Just because it doesn't apply to them in this case, they con't want to see
it added to the overall FTP protocol. The whole point of this is
interoperability. I don't see what is so wrong in adding something like
this to the protocol to further that end. Lets face it: Unix is NOT the
only OS in the world, nor will it probably ever be. Lets stop quibbling about
this and get something moving here. Otherwise, the various vendors are
going to just going to have to make something non-standard in their particular
implementations to provide this very necessary service and we would not
benefit from that.

Art Stine
Sr Network Engineer
Clarkson U

ABStine@CLVMS.Clarkson.Edu

cpw%sneezy@LANL.GOV (C. Philip Wood) (02/13/89)

I vote for something that provides "interoperability".  But,
interoperability is not the ability of one VMS system to talk to
another.  The mechanism I would vote for would create an eXternal File
Representation (XFR), which would allow for REAL "interoperability"
between HETEROGENEOUS hosts.  It would take care of mapping file
attributes to and from the file attribute scheme of all time.  And, if
a mapping was impossible, it would give the phone number of the nearest
XYZ vendor sales office, so the user could switch to an OS that was
flexible enough to handle the problem.

Phil Wood,  cpw@lanl.gov

casey@gauss.llnl.gov (Casey Leedom) (02/14/89)

| From: cpw%sneezy@LANL.GOV (C. Philip Wood)
| 
| I vote for something that provides "interoperability".  But,
| interoperability is not the ability of one VMS system to talk to
| another.  The mechanism I would vote for would create an eXternal File
| Representation (XFR), which would allow for REAL "interoperability"
| between HETEROGENEOUS hosts.  It would take care of mapping file
| attributes to and from the file attribute scheme of all time.  And, if a
| mapping was impossible, it would give the phone number of the nearest XYZ
| vendor sales office, so the user could switch to an OS that was flexible
| enough to handle the problem.

  Yech.  Just what we need.  Ftp written by AI people ... :-) Will I have
to have 32Mb of swap space to run ftp now?  Not flaming; just asking that
we don't go over board on this one.

Casey

casey@gauss.llnl.gov (Casey Leedom) (02/15/89)

  I don't mind the idea of per vendor extensions to ftp just as long as
they're not part of the official FTP RFC standard.  And I don't mind the
idea of the InterNet community serving as a host for vendor
standardization efforts.  I think that a separate RFC for each separate
vendor extension (using a standard FTP escape mechanism) is the way to go.

  I think that more than enough evidence has been presented to justify
such extensions and their standardization.  But, that evidence also
suggests that, barring AI extensions to FTP, these facilities are totally
vendor specific with no known bound on the number of such.  Therefore in
order to promote standardization within vendor communities but also avoid
rewriting the FTP RFC every time a vendor extension is thought up, it's
fairly clear that each such extension should be described in a separate
RFC.

  Given the InterNet's traditional desire to achieve and enhance
interoperability, I think we should be receptive to hosting vendor
standardization efforts in this manner.  Part of the InterNet's role is
as a standards body after all.

  But, aside from these thoughts, I think Chris is right: you VMS TCP/IP
suppliers ought to get together on your own in a private meeting and
agree on a standard between yourselves.  I think you've received more
than enough commentary about usage of both "SITE" and "STRU".  My
personal choice is "SITE" because it was explicitly put in for such
non-standard extensions, but that's just me.

Casey

jerry@olivey.olivetti.com (Jerry Aguirre) (02/23/89)

In article <603089521.0.KASHTAN@IU.AI.SRI.COM> KASHTAN@IU.AI.SRI.COM (David L. Kashtan) writes:
>>If it's a matter of saving steps why not just build a DCL wrapper
>>which does the job? Heck, Interlisp-D had what amounted to a very good
>>NFS based on vanilla FTP, required no changes to the server (eg. UNIX
>>or VMS) system as I remember.
>  This is a solution -- though not a pretty one.  It loses on several
>  fronts:
>	1) It doesn't address the issues of system management and disk
>	   quotas.   It is definitely not a selling point for the software
>	   if you are required to have 2 times the disk space available
>	   in order to transparently transfer the files

I think there is a basic misunderstanding here brought about by the
different philosophy of Unix and VMS users.  The VMS users are assuming
that they "backup" or whatever into a file, transfer the file, extract
the files, and then cleanup the temporary files used to store the backup
image.  Thus the requirement for the extra space for the backup.  (Don't
they have the equivelent of a /tmp directory?)

A Unix user implementing the same facility would assume that the data
was being "piped" directly from the output of "tar" to the ftp transfer.
No temporary file names or disk space would be needed and also no long
delay creating the backup before the transfer can start.  And of course
the ftp output can also be piped into tar on the receiving end to
achieve the same benifits.

The common usage of pipes in Unix and the ability of tar and dump to use
them make this an unspoken assumption.

jas@proteon.com (John A. Shriver) (02/23/89)

First, there is no /tmp in VMS.  Face it, /tmp is an arcane leftover
from when UNIX ran on machines that did not have virtual memory.

There is nothing really like pipes in VMS, especially at the command
level.  The syntax of the BACKUP command is also very awkward and
restrictive. 

ALso, a pipe would fail to solve the same basic VMS problem.  Files
are not just a sequence of bytes in VMS.  They have a lot of
attributes, attributes that must be preserved or the file is useless.
(It is better than OS/360, since the system always keeps track of the
attributes for you.)  Thus, the output of BACKUP, when sent throught a
pipe (or transferred via FTP) loses a key attribute: it's
extraordinary block size.

Also, as noted before, the output of the BACKUP program is not an
efficient format.  It has a very high degree of redundnacy.  You can
punch a 1/4" hole in a BACKUP tape and still restore from it.
Overkill for what will be transferred over FTP, which we trust more
than a tape drive.

The VMS folks have a real problem.  The binary abstraction of FTP just
does not hack it for their files.  You will find that they are not the
only OS that has addresses this, I think the UCLA MVS TCP/IP FTP
allows you to specify a bit of (moral) JCL before writing a file.

Let's let them agree on something to solve this problem.  Anything to
make the VMS users less anti-TCP.

braden@VENERA.ISI.EDU (02/24/89)

	does not hack it for their files.  You will find that they are not the
	only OS that has addresses this, I think the UCLA MVS TCP/IP FTP
	allows you to specify a bit of (moral) JCL before writing a file.
	
Yes, the UCLA MVS ACP implements the FTP SITE command with keyword
parameters calculated to warm the heart of a JCL-lover -- BLKSIZE(),
RECFM(), LRECL()... all that wonderful stuff! (yukh)

Bob Braden

bzs@ENCORE.COM (Barry Shein) (02/24/89)

From: jas@proteon.com (John A. Shriver)
>The VMS folks have a real problem.  The binary abstraction of FTP just
>does not hack it for their files.  You will find that they are not the
>only OS that has addresses this, I think the UCLA MVS TCP/IP FTP
>allows you to specify a bit of (moral) JCL before writing a file.
>
>Let's let them agree on something to solve this problem.  Anything to
>make the VMS users less anti-TCP.

Ok, I can live with this but...VMS is a proprietary operating system
managed by exactly one corporation. I hear DEC is supporting or soon
to support TCP on VMS, do they have any proposals in this realm?

If DECNET solves this problem then DECNET over IP might be a better
way to deal with this issue as it simply provides a lower layer for
transport and any changes in the file attributes etc by DEC can be
punted to DECNET support, it can be the only layer aware of such
things and as long as the DECNET packets flow all should be well.

We *are* talking VMS<->VMS communications here, right? And at least
one vendor already offers DECNET over IP, so the only real question is
whether this combination already or could in the near future solve the
problem. I could see an encapsulation standard (RFC) for DECNET/IP as
a way to acheive this in a uniform manner among vendors.

At least that model could lead to dealing with other proprietary
systems and network protocols, as it has in the past (eg. RSCS over
IP, CHAOS over IP, XNS over IP, etc.)

	-Barry Shein, ||Encore||

adelman@TGV.COM (Kenneth Adelman) (02/25/89)

> If DECNET solves this problem then DECNET over IP might be a better
> way to deal with this issue as it simply provides a lower layer for
> transport and any changes in the file attributes etc by DEC can be
> punted to DECNET support, it can be the only layer aware of such
> things and as long as the DECNET packets flow all should be well.

    DECnet over IP isn't a replacement for STRU VMS in the Internet
environment. Phase IV DECnet is routing-braindead, limiting the size
of DECnet networks, and requiring great administrative pains to get
large networks to work.

> We *are* talking VMS<->VMS communications here, right? And at least
> one vendor already offers DECNET over IP, so the only real question is
> whether this combination already or could in the near future solve the
> problem. I could see an encapsulation standard (RFC) for DECNET/IP as
> a way to acheive this in a uniform manner among vendors.

    After all the flames I've received about the proposed STRU VMS
extension to FTP, I'm not going to propose our DECnet over IP
encapsulation as a standard. However, I would be more than happy to
share the encapsulation specification with anyone interested in
developing complementary or competing products to MultiNet.

    TGV does not believe that protocols of this nature are
trade-secrets, only the code and associated technology used to
implement them.  This applies to STRU VMS, IP over DECnet, DECnet over
IP, and potentially others.

					      Kenneth Adelman
					      TGV, Incorporated

KASHTAN@IU.AI.SRI.COM (David L. Kashtan) (02/25/89)

>Ok, I can live with this but...VMS is a proprietary operating system
>managed by exactly one corporation. I hear DEC is supporting or soon
>to support TCP on VMS, do they have any proposals in this realm?

  My impression of things is that TCP is very low priority within DEC
  and that they have a long long way to go before they start worrying
  about this kind of problem.

>If DECNET solves this problem then DECNET over IP might be a better
>way to deal with this issue as it simply provides a lower layer for
>transport and any changes in the file attributes etc by DEC can be
>punted to DECNET support, it can be the only layer aware of such
>things and as long as the DECNET packets flow all should be well.

  Yuck!  Now to solve a very straightforward problem you need to ensure
  that all VMS TCP sites:
	1) Buy DECNET (it is NOT cheap!)
	2) Arrange to be connected (DECNET-wise) into a single accessable
	   DECNET network (even if it IS using IP as the transport).
  Not to mention that the users (who might be transferring some files from
  UNIX workstations and some files from other VMS machines) will have to do
  2 completely different things when they want to transfer VAX/VMS files
  with attributes intact vs moving files to/from hosts running any other
  O/S.

>We *are* talking VMS<->VMS communications here, right? And at least
>one vendor already offers DECNET over IP, so the only real question is
>whether this combination already or could in the near future solve the
>problem. I could see an encapsulation standard (RFC) for DECNET/IP as
>a way to acheive this in a uniform manner among vendors.

  I think that an RFC for DECNET over IP (actually UDP) is a good idea, and
  since we are (as far as I can tell) the only ones who have implemented this
  I guess we should get an RFC out right away.  Note that this DOES NOT
  solve the original problem -- extending FTP to transfer O/S-dependent
  files between cooperating systems in a way that still interoperates
  correctly with OTHER systems.  To this end I think that extending the
  proposed STRU VMS to STRU OS xxx (i.e. STRU OS VMS is a particular
  instantiation) is a valuable contribution to the FTP spec.  The SITE command
  might be more appropriate for this extension -- but the semantics entailed
  in this transfer mode most closely mesh with the semantics of the STRU
  command.

>At least that model could lead to dealing with other proprietary
>systems and network protocols, as it has in the past (eg. RSCS over
>IP, CHAOS over IP, XNS over IP, etc.)
  Again -- tunneling through IP is a most worthwhile thing (and we do this
  quite a bit with our software) but it does NOT address the issue we are
  currently arguing.

					David Kashtan
					TGV Inc.
-------

braden@VENERA.ISI.EDU (03/28/89)

Kenneth,

Your original message on this topic triggered a fairly constructive blast
of messages (about 35?).  I think you should plow ahead with your
proposal.  A useful approach might be to set up a "Birds-of-a-Feather"
session of VMS vendors at a future IETF meeting, to draft a spec.

I suggest you leave to the Protocol Czar Jon Postel the decision on
syntactic sugar, "STRU xxx" vs. "STRU O xxx" vs. "SITE STRU xxx", etc.
Don't waste any more time on it.  You (the set of VMS vendors) need to do
the hard work, documenting the contents and format for VMS (analogous to
Appendix I of RFC-959).

It should be noted that STRU P (PAGE structure) is in reality "STRU O
TENEX", or  "STRU O TOPS20."  FTP would have been cleaner as a
protocol and more powerful, I think, if we had adopted your idea years
ago, and recognized that STRU P was just one example of a class of
service that FTP ought to provide -- efficient operating-system-specific
transfers.

Bob Braden

clements@bbn.com (Bob Clements) (03/29/89)

In article <8903271958.AA02597@braden.isi.edu> braden@VENERA.ISI.EDU writes:
> ...
>It should be noted that STRU P (PAGE structure) is in reality "STRU O
>TENEX", or  "STRU O TOPS20."  FTP would have been cleaner as a
>protocol and more powerful, I think, if we had adopted your idea years
>ago, and recognized that STRU P was just one example of a class of
>service that FTP ought to provide -- efficient operating-system-specific
>transfers.

Just as a small historic footnote, that's what the "tenex" thing
started as.  I wrote it, the first time.  It was in the NCP-based
FTP, pre-dating TCP.

As I implemented it (and put in some early RFC [which I can't
find -- I thought it was right between my Totally Awesome
Protocol spec and my SUDS manual]) ... Anyway, as I implemented
it the command was "TYPE XTP" and I proposed that "X" meant it
was reserved for use between consenting adults (er, compatible
systems), "T" was an example of the affected part of the
universe, here "TENEX", and "P" was the specific kludge being
added to the protocol, here a "Paged" file format.

(It was implemented in a panic over a weekend when we decided
to do a full file system transfer over the net for the first
time between two PDP-10s.)

(It also had checksums, for the first time.  NCP didn't have
checksums like TCP does, nor did NCP-based FTP.)

>
>Bob Braden
(Hi, Bob!)

/Rcc
clements@bbn.com