faulkner%jmullins@husc6.harvard.edu (Don Faulkner) (05/17/89)
I've been working with mmap to create file space. There are some problems
in letting mmap set the size of the file.
>From "man mmap":
It should be noted that the system will always zero-fill any
partial pages at the end of an object. Further, the system
will never write out any modified portions of the last page
of an object which are beyond its end. References to whole
pages following the end of an object will result in the
delivery of a SIGBUS signal. SIGBUS signals may also be
delivered on various filesystem conditions, including quota
exceeded errors.
(the System Services Overview chapter "Memory Management" doesn't
add any info for my question ...)
So --- you can't just "mmap" with a "len" greater than the length
of the file, and let it grow ...
Experimentally, the following DOES work:
fd = open ("file", O_RDWR | O_CREAT, 0666)
lseek (fd, 10000000, L_SET) /* max file size reasonable */
write (fd, "", 1)
This will create a file with an enormous hole.
Then, you can mmap with len=10000000
if you address and write into the middle of a file, those blocks
touched get allocated and filled in automagically ...
(I'm trying to avoid actually POPULATING a huge block if it's not needed...)
Afterwards, you could open a new file, and copy only the elements that
were actually created...
This DOES work; the question is: can one DEPEND on it working?
Is there a better way?
--
Don Faulkner
Building 1, Room 803
Harvard University, School of Public Health
665 Huntington Avenue
Boston, MA 02115
ARPA: faulkner%jmullins@harvard.harvard.edu
BITNET: faulkner@harvard
Telephone: (617) 732-2297guy@uunet.uu.net (Guy Harris) (05/26/89)
>I've been working with mmap to create file space. There are some problems >in letting mmap set the size of the file. Yup, "mmap" doesn't set the size of the file. >Experimentally, the following DOES work: As should anything else that grows the file. >This DOES work; the question is: can one DEPEND on it working? Yes. Not having a pagein of a page backed by a hole work in the proper UNIX fashion (which would be to zero the page out) would have been a Very Bad Idea, so it works. Similarly, modifying that page causes a pageout to work, by filling in the hole. (Besides, since "read" and "write" on UFS and NFS file systems work, in effect, by an in-the-kernel fast path through "mmap", it might well have broken "read" and "write" were it not to work.) >Is there a better way? Well, it depends on your definition of "better", but: Users who pored over the 4.0 TRUNCATE(2) man page with a fine-toothed comb would have noticed that it says DESCRIPTION truncate() causes the file referred to by path (or for ftruncate() the object referred to by fd) to have a size equal to length bytes. If the file was previously longer than length, the extra bytes are removed from the file. If ^^ it was shorter, bytes between the old and new lengths are ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ read as zeroes. With ftruncate, the file must be open for ^^^^^^^^^^^^^^^^ writing. The pre-4.0 TRUNCATE(2), and the 4.xBSD ones, didn't say anything about "(f)truncate" making files larger. The reason why "(f)truncate" was, extended to allow you to, err, umm, *extend* the file (no pun intended) was to let you grow a file to which you were writing via "mmap" rather than via "write". I don't know if this was mentioned in the Read (Me|This) First (one is for software, one is for hardware, I can't remember which is which), so it may have been somewhat obscure....
david@elroy.jpl.nasa.gov (David Robinson) (06/14/89)
In article <3551@kalliope.rice.edu>, auspex!guy@uunet.uu.net (Guy Harris) writes:
< Well, it depends on your definition of "better", but:
<
< Users who pored over the 4.0 TRUNCATE(2) man page with a fine-toothed
< comb would have noticed that it says
<
< DESCRIPTION
< truncate() causes the file referred to by path (or for
< ftruncate() the object referred to by fd) to have a size
< equal to length bytes. If the file was previously longer
< than length, the extra bytes are removed from the file. If
< ^^
< it was shorter, bytes between the old and new lengths are
< ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
< read as zeroes. With ftruncate, the file must be open for
< ^^^^^^^^^^^^^^^^
< writing.
<
< The pre-4.0 TRUNCATE(2), and the 4.xBSD ones, didn't say anything about
< "(f)truncate" making files larger. The reason why "(f)truncate" was,
< extended to allow you to, err, umm, *extend* the file (no pun intended)
< was to let you grow a file to which you were writing via "mmap" rather
< than via "write".
This is a potential "gotcha" is if you are using NFS. Many (currently most)
NFS servers, including SunOS 3.X, do not handle this correctly, SunOS 4.0
of course does. These "broken" servers will not extend a file using
NFS_SETATTR() which is the SunOS 4.0 NFS equivilant of truncate().
Personally I think they should have implemented it as a write of a zero
byte for backwards bug compatibility. This is the primary reason you
cannot run a 4.0 diskless client off a 3.X server.
--
David Robinson elroy!david@csvax.caltech.edu ARPA
david@elroy.jpl.nasa.gov ARPA
{cit-vax,ames}!elroy!david UUCP
Disclaimer: No one listens to me anyway!