[comp.unix.questions] Can't copy giant file

jim@crom2.uucp (James P. H. Fuller) (05/04/91)

     I can't copy a 24-meg file without getting it truncated to 16 megs.

My ULIMIT is 73728 512-byte blocks, which should allow me to write a 36 mb
file; and anyway I'm doing this copying as root and root should be able to
write a file as big as the filesystem will hold, no matter what the ULIMIT is.
So it must be some other limit that I'm bumping into.

     The files to be copied are on filesystems that are type DOS -- that's
Interactive-speak for DOS partitions that are mounted as Unix filesystems,
which ISC SysV allows.  They were copied there originally from a CD-ROM that
will only run under true DOS and will only copy to the DOS partition.  Once
the files are on the hard disk I had expected to be able to mount the DOS
partitions under Unix as usual and then copy the files the rest of the way to
their destination in /usr4/genbank.

     This all works for the smaller files, but it doesn't work for the larger
ones like gbpri.seq, which is 24 megs of primate gene sequences in ASCII.
When I (as root) do

    # cp /dos2/genbank/gbpri.seq /usr4/genbank 

it only copies part of the file -- when the prompt comes back there's a file
called gbpri.seq in the proper place on the Unix partition but it's only
(only!) 17,208,832 bytes long where the original is 26,090,050 bytes.  There's
plenty of room (presently 135 megs free) in the /usr4 filesystem and I can't
think of any other reason why a large file shouldn't copy properly.

     Can someone please tell me what's biting me here, and what I can do about
it?  P.S. on a whim I tried cat /dos2/genbank/gbpri.seq >gbpri.seq and that
one croaked at 17,208,832 bytes also.  What's magic about that number?

                                             Thanks very much,
                                             James P. H. Fuller
                                             jim%crom2@nstar.rn.com

                                             

tony@mcrsys.UUCP (Tony Becker) (05/04/91)

From article <1991May04.023545.535@crom2.uucp>, by jim@crom2.uucp (James P. H. Fuller):
> 
>      I can't copy a 24-meg file without getting it truncated to 16 megs.
> 
> My ULIMIT is 73728 512-byte blocks, which should allow me to write a 36 mb
> file; and anyway I'm doing this copying as root and root should be able to
> write a file as big as the filesystem will hold, no matter what the ULIMIT is.
> So it must be some other limit that I'm bumping into.
> 

  1) swap space (for cp) ???

  2) largest contiguous free disk space ???

  what os configuration is this?
-- 
tony ,....

doug@ohenry.UUCP (Richard H. Douglas) (05/05/91)

jim@crom2.uucp (James P. H. Fuller) writes:
> 
>      I can't copy a 24-meg file without getting it truncated to 16 megs.
> 
>      The files to be copied are on filesystems that are type DOS -- that's
> Interactive-speak for DOS partitions that are mounted as Unix filesystems,
> which ISC SysV allows...
> 
> 
> it only copies part of the file -- when the prompt comes back there's a file
> called gbpri.seq in the proper place on the Unix partition but it's only
> (only!) 17,208,832 bytes long where the original is 26,090,050 bytes.  There's
> plenty of room (presently 135 megs free) in the /usr4 filesystem and I can't
> think of any other reason why a large file shouldn't copy properly.
> 
...                                              

I do not have you problem now because I completely eliminated DOS
from my machine and upgraded to full Unix. However, on my older
Xenix kernal, this was some kind of limit based on the 16 bit
addressing that dos had. So not only did you have problems on
large files, but also on large dos partitions (bigger that 20Meg
I believe). Perhaps you can convert the file to pure text, split
is into smaller segments and then copy it into you Unix partition.
The final problem then will be in recombining the file somehow so
that it makes sense to you.

If it were me, I would get on the case if ISC. I would have
thought that this would have been fixed by now. 

BTW, I am told that early Xenix 286 and 386 had this problem too.

Hope this helps.

Regards,
Rich Douglas     doug@ohenry

allbery@NCoast.ORG (Brandon S. Allbery KB8JRR/AA) (05/05/91)

As quoted from <1991May4.162149.9341@mcrsys.UUCP> by tony@mcrsys.UUCP (Tony Becker):
+---------------
| From article <1991May04.023545.535@crom2.uucp>, by jim@crom2.uucp (James P. H. Fuller):
| >      I can't copy a 24-meg file without getting it truncated to 16 megs.
> (to a DOS filesystem)
| 
|   1) swap space (for cp) ???
|   2) largest contiguous free disk space ???
+---------------

Being that he said it was to a DOS filesystem:  isn't 16MB the maximum
file size under DOS 3.3?  (I have no idea about DOS 4.x.)

++Brandon
-- 
Me: Brandon S. Allbery			  Ham: KB8JRR/AA  10m,6m,2m,220,440,1.2
Internet: allbery@NCoast.ORG		       (restricted HF at present)
Delphi: ALLBERY				 AMPR: kb8jrr.AmPR.ORG [44.70.4.88]
uunet!usenet.ins.cwru.edu!ncoast!allbery       KB8JRR @ WA8BXN.OH

weimer@garden.ssd.kodak.com (Gary Weimer (253-7796)) (05/07/91)

In article <1991May04.023545.535@crom2.uucp>, jim@crom2.uucp (James P.
H. Fuller) writes:
|> 
|>      I can't copy a 24-meg file without getting it truncated to 16 megs.
|> 
|> My ULIMIT is 73728 512-byte blocks, which should allow me to write a 36 mb
|> 
|>     # cp /dos2/genbank/gbpri.seq /usr4/genbank 

You could try:

    # dd if=/dos2/genbank/gbpri.seq of=/usr4/genbank/gbpri.seq bs=512

I don't know if it will have the same problem or not...

weimer@ssd.kodak.com ( Gary Weimer )