[comp.sys.att] Problems with compressed files from ftp sites

erict@flatline.UUCP (The Evil Mel Fujitsu) (02/05/89)

I'm not sure if this is a popular problem or not, but here goes.

I was trying to get tar sources from an ftp site.  They were compressed,
no problem.  I ftp'd 'em to a local site (uhnix1), then downloaded them
to my 3b1 via kermit.

The file.Z, was about 60K or so.  (Maybe 40K?).  After a compress -d, it
was up in the 2-4 megabyte range.  The first block was a normal shar file,
then there was nothing but garbage characters.  There were no errors of
any sort during either of the file xfers, or the decompression..

Any hints?


-- 
"I think it would be fun to run a newspaper." -- C.F.K.
J. Eric Townsend 511 Parker #2, Houston, Tx, 77007
Inet: COSC5FA@george.uh.edu             UUCP:  uunet!nuchat!flatline!erict
Bitnet: COSC5FA@UHVAX1.BITNET            ..!bellcore!tness1!/
EastEnders Mailing List(!): eastender@flatline.UUCP

gil@limbic.UUCP (Gil Kloepfer Jr.) (02/06/89)

In article <707@flatline.UUCP> erict@flatline.UUCP (j eric townsend) writes:
>I was trying to get tar sources from an ftp site.  They were compressed,
>no problem.  I ftp'd 'em to a local site (uhnix1), then downloaded them
>to my 3b1 via kermit.
[...]
>then there was nothing but garbage characters.  There were no errors of
>any sort during either of the file xfers, or the decompression..

I've seen this problem transfering files from a system running VAX/VMS
and transfering the files *TO* my UNIX-pc.  I don't know whether it was
the kermit programs I used, or whether it was the typical record-oriented
file structure that VMS uses.  In any case, my problem was the same as your's.
Be sure that your kermit programs do true binary-mode transfer at both
ends, and that one of the hosts isn't a VMS system ;-)

The problem is that somewhere in the string of characters an invalid character
is getting in there and messing up the whole encoding scheme used by the LZ
compression.  One character is all you need to make the rest of the file look
like junk.  If it's a VAX/VMS machine, there are an infinite number of ways
to get some kind of record-control byte in the character stream, or to lose
a character.

To date, I haven't come up with a solution except to copy the compressed
files to a messy-dos floppy disk using DECnet-DOS, then reading the floppy
on my UNIX-pc.  Somehow the DECnet-DOS software preserves the contents of
the file properly.

Hope this helps...if not just provides some insight.

-------
Gil Kloepfer, Jr.          U-Net: {decuac,boulder,talcott,sbcs}!icus!limbic!gil
ICUS Software Systems      Voice: (516) 968-6860 [H]   (516) 746-2350 x219 [W]
P.O. Box 1                 Internet:  gil@icus.islp.ny.us
Islip Terrace, NY  11752   "Life's a ...  well, you know..."

erict@flatline.UUCP (The Evil Mel Fujitsu) (02/06/89)

In article <456@limbic.UUCP>, gil@limbic.UUCP (Gil Kloepfer Jr.) writes:
> In article <707@flatline.UUCP> erict@flatline.UUCP (j eric townsend) writes:
> >I was trying to get tar sources from an ftp site.  They were compressed,
> >no problem.  I ftp'd 'em to a local site (uhnix1), then downloaded them
> >to my 3b1 via kermit.
> [...]
> >then there was nothing but garbage characters.  There were no errors of
> >any sort during either of the file xfers, or the decompression..

> I've seen this problem transfering files from a system running VAX/VMS
> and transfering the files *TO* my UNIX-pc.  I don't know whether it was
> the kermit programs I used, or whether it was the typical record-oriented
> file structure that VMS uses.  In any case, my problem was the same as your's.
> Be sure that your kermit programs do true binary-mode transfer at both
> ends, and that one of the hosts isn't a VMS system ;-)

Grr.  This is not an answer I want to hear.

Ok, how about this:


Is there anyway that I can email the files from a vax to my 3b1?   (I
have a one-hop mail link with the machine, so it's pretty damn fast.)

How about other file xfer protocols?  Would x/ymodem work instead?


We're almost into 1990.  We put a human on the moon.  Why can't I file
xfer from a vax to a 3b1?


-- 
"I think it would be fun to run a newspaper." -- C.F.K.
J. Eric Townsend 511 Parker #2, Houston, Tx, 77007
Inet: COSC5FA@george.uh.edu             UUCP:  uunet!nuchat!flatline!erict
Bitnet: COSC5FA@UHVAX1.BITNET            ..!bellcore!tness1!/
EastEnders Mailing List(!): eastender@flatline.UUCP

dwex@mtgzz.att.com (d.e.wexelblat) (02/07/89)

In article <456@limbic.UUCP> gil@limbic.UUCP (Gil Kloepfer Jr.) writes:
>In article <707@flatline.UUCP> erict@flatline.UUCP (j eric townsend) writes:
>>I was trying to get tar sources from an ftp site.  They were compressed,
>>no problem.  I ftp'd 'em to a local site (uhnix1), then downloaded them
>>to my 3b1 via kermit.
>[...]
>>then there was nothing but garbage characters.  There were no errors of
>>any sort during either of the file xfers, or the decompression..
>
>I've seen this problem transfering files from a system running VAX/VMS
>and transfering the files *TO* my UNIX-pc.  I don't know whether it was
>the kermit programs I used, or whether it was the typical record-oriented
>file structure that VMS uses.  In any case, my problem was the same as your's.
>Be sure that your kermit programs do true binary-mode transfer at both
>ends, and that one of the hosts isn't a VMS system ;-)

[...]

>-------
>Gil Kloepfer, Jr.          U-Net: {decuac,boulder,talcott,sbcs}!icus!limbic!gil
>ICUS Software Systems      Voice: (516) 968-6860 [H]   (516) 746-2350 x219 [W]
>P.O. Box 1                 Internet:  gil@icus.islp.ny.us
>Islip Terrace, NY  11752   "Life's a ...  well, you know..."


The problem is with the initial FTP, not the download to the 3b1.  The FTP
must be done in BINARY mode for compressed files.  I can't believe that kermit
could mung the files (assuming the transfer was done in image mode).  If
your kermit implementation screws up files, get another one -- it's hosed.
The solution to the problem is to redo the FTP in binary mode, and be sure
to check that the size of the file is EXACTLY (not just close) the same size
as the original.  If a binary file in transfered in non-binary mode, the
file size will be close, but not exact.

To verify this, attempt the uncompress on the VAX.  The file will come out
several megabytes there, too. 

--
David Wexelblat			dwex@mtgzz.att.com
AT&T Bell Laboratories		...!att!mtgzz!dwex
200 Laurel Ave - 4B-421
Middletown, NJ  07748		EMT's do it all night long

roger@binky.UUCP (Roger Taranto) (02/12/89)

In article <707@flatline.UUCP> erict@flatline.UUCP (j eric townsend) writes:
>I was trying to get tar sources from an ftp site.  They were compressed,
>no problem.  I ftp'd 'em to a local site (uhnix1), then downloaded them
>to my 3b1 via kermit.
[...]
>then there was nothing but garbage characters.  There were no errors of
>any sort during either of the file xfers, or the decompression..

Did you ftp the files in binary mode?  (I don't mean to insult your
intelligence if you did, I'm just suggesting something that might have
caused the problem.

In article <456@limbic.UUCP> gil@limbic.UUCP (Gil Kloepfer Jr.) writes:
>I've seen this problem transfering files from a system running VAX/VMS
>and transfering the files *TO* my UNIX-pc.  I don't know whether it was
>the kermit programs I used, or whether it was the typical record-oriented
>file structure that VMS uses.

I had a problem transferring files via ftp to VMS.  What was happening was
that VMS was truncating the lines in my files to 512 bytes, even the lines
were 2008.  The solution I found was to port uudecode to VMS (it takes about
2 minutes to comment out the UNIX-specific stuff), uuencode the files on
UNIX, transfer them to VMS, and, finally, uudecode them on VMS.  I would
assume that this would work going the other direction, and it would also
eliminate non-ASCII characters from your file while it was being transferred.

-Roger
roger@binky.UUCP	...!{ucbcad,rtech,pacbell}!binky!roger