millerje@mozart.cs.colostate.edu (Jeff Miller) (04/08/91)
I am writing a program in C, and just to make life easy, I want to malloc (actually calloc) a data space > 64k. I noticed that if I malloc <65535 bytes (<64k) I can fread directly into that buffer. What I'd like to do is be able to fread into a buffer about 256k in size. (The reason for this is speed). I have tried the large memory model as well as huge pointers, but to no avail. Are there any special versions of malloc &/or calloc that will let me do this? _____________________________________________________________________________ | | | "NUKE THE UNBORN GAY WHALES!" | Jeff Miller | | - graffiti | millerje@handel.CS.ColoState.Edu | |_____________________________________________________________________________|
hollen@megatek.UUCP (Dion Hollenbeck) (04/08/91)
In article <14049@ccncsu.ColoState.EDU> millerje@mozart.cs.colostate.edu (Jeff Miller) writes: > I am writing a program in C, and just to make life easy, I want to malloc > (actually calloc) a data space > 64k. I noticed that if I malloc <65535 > bytes (<64k) I can fread directly into that buffer. What I'd like to do > is be able to fread into a buffer about 256k in size. (The reason for this > is speed). I have tried the large memory model as well as huge pointers, but > to no avail. Are there any special versions of malloc &/or calloc that > will let me do this? Well, you can and you can't. There are either library functions available to get a DOS memory block, or you can do an interrupt to DOS directly to allocate it. However, the problem comes with the fread. You finally get down to the disk controller level, no matter whether you do a read or fread and the disk controller uses the 80x86 instruction set. There is no instruction that can operate on >64k. To do that, you need to break it down into 64k chunks at some level and do them one at a time. I think the best that you can do is to allocate the huge chunk, and read the file in 64k chunks (or go to a 68000 or SPARC running UNIX with a 32 bit linear address space). Your problem is running smack up against the hardware architecture and there is not much you can do about it. BTW, in the old 8088's you had to watch out for floppy disk reads that crossed 64k boundaries. This was of course handled in the BIOS transparently, but if you read as little as 512 bytes from the floppy and asked it to be put at 2000:fff0, it would transfer the first 10h bytes, increment the segment pointer and transfer the rest of the 512. This is because the DMA controller would not cross 64k boundaries. Be thankful you don't write BIOS's and only have applications to worry about. -- ----- Dion Hollenbeck (619) 455-5590 x2814 Megatek Corporation, 9645 Scranton Road, San Diego, CA 92121 uunet!megatek!hollen or hollen@megatek.uucp
boba@netcom.COM (Bob Amstadt) (04/10/91)
In article <14049@ccncsu.ColoState.EDU> millerje@mozart.cs.colostate.edu (Jeff Miller) writes: >I am writing a program in C, and just to make life easy, I want to malloc >(actually calloc) a data space > 64k. I noticed that if I malloc <65535 >bytes (<64k) I can fread directly into that buffer. What I'd like to do >is be able to fread into a buffer about 256k in size. (The reason for this >is speed). I have tried the large memory model as well as huge pointers, but >to no avail. Are there any special versions of malloc &/or calloc that >will let me do this? Actually you have another problem. Once you allocate the buffer, fread is incapable of reading more than 64k at a time. You have to go through DOS with fread, and it does not allow reads larger than 64k. However, this is not a major problem. I once did some tests, and found that reads in blocks of 16k or better all had similar performance. 64k offers almost no advantage over 32k, and the difference between 32k and 16k is also minimal. If you have the memory, 32k is probably the best size to use. It is the largest convenient size to use because the blocksize still fits in a 16-bit unsigned int. -- Bob Amstadt boba@netcom.com 408-738-2479
mgphl@msa3b.UUCP (Michael Phillips) (04/10/91)
millerje@mozart.cs.colostate.edu (Jeff Miller) writes: >I am writing a program in C, and just to make life easy, I want to malloc >(actually calloc) a data space > 64k. I noticed that if I malloc <65535 >bytes (<64k) I can fread directly into that buffer. What I'd like to do >is be able to fread into a buffer about 256k in size. (The reason for this >is speed). I have tried the large memory model as well as huge pointers, but >to no avail. Are there any special versions of malloc &/or calloc that >will let me do this? Try using farcalloc -or- farmalloc.