[comp.os.msdos.programmer] Virtual Memory for the Heap

a35@mindlink.UUCP (Bruce Atherton) (07/25/90)

I'm looking for the best method of implementing virtual memory on 80x86
machines under MSDOS.  Specifically, it has to allow swapping of the
heap, rather than code.  The method that is the simplest to implement
and least buggy would be preferable.

I just received Turbo C++ and was thinking of using VROOMM, but it
appears to only swap overlays of code.  Is there any way to allow it to
swap from the heap?  Do I have to overload the -> operator so that it
works with pages, or can VROOMM take care of that for me (and if so,
how?).

What about using _handle in Zortech C++?  It sounds like it would be
easy for bugs to creep in, from the description I read.  I would also
like to hear about any heap manager toolboxes and how they work.

Finally, is there a speed advantage to any of these methods over using a
disk caching system?

I would be grateful for any information on these questions.


Bruce_Atherton@mindlink.UUCP or| "Lotta people think the spikes on forts was
uunet!van-bc!rsoft!mindlink!a35| to protect 'gainst injuns, but that's stupid.
                                 Injuns knew about doors.  But clams didn't."

6600m00n@ucsbuxa.ucsb.edu (Jihad 'R US) (07/27/90)

In article <2628@mindlink.UUCP> a35@mindlink.UUCP (Bruce Atherton) writes:


   I'm looking for the best method of implementing virtual memory on 80x86
   machines under MSDOS.  Specifically, it has to allow swapping of the
   heap, rather than code.  The method that is the simplest to implement
   and least buggy would be preferable.

   I just received Turbo C++ and was thinking of using VROOMM, but it
   appears to only swap overlays of code.  Is there any way to allow it to
   swap from the heap?  Do I have to overload the -> operator so that it
   works with pages, or can VROOMM take care of that for me (and if so,
   how?).

VROOM will not swap data.  The main problem with swapping data is that 
it is accessed 'asyncronously'. i.e.  You never know when you will need
to be in, since pointers can point anywhere.  Code is always accesed
with calls, and can be routed through a overlay maneger. ( setjmp,
longjmp could cause problems . . .)   One way of dealing with this is to
only use access to memory through 'handles' ( picked the idea up from
_mac_ programing.) and have a byte that says if the info is in main
memory ( in which case you use the pointer in the handle structure) or
not, in which case you use the (ems|xms|ext|disk) handle you have setup
to load the handle into main memory( and load out any needed to make
space). You could even put a byte in for 'locking' the pointer so it
does not get swapped out. 

   What about using _handle in Zortech C++?  It sounds like it would be
   easy for bugs to creep in, from the description I read.  I would also
   like to hear about any heap manager toolboxes and how they work.

   Finally, is there a speed advantage to any of these methods over using a
   disk caching system?

Yes:  disk caching doesn't do diddly for speeding up writing data.
Not so important with code( how often do you write self modifing code)
very important with data.

  I would be grateful for any information on these questions.
happy to reply.

Rob Blair 6600m00n@ucsbuxa.ucsb.edu
'Yea, brother . . .'

bright@Data-IO.COM (Walter Bright) (07/28/90)

In article <2628@mindlink.UUCP> a35@mindlink.UUCP (Bruce Atherton) writes:
<I'm looking for the best method of implementing virtual memory on 80x86
<machines under MSDOS.  Specifically, it has to allow swapping of the
<heap, rather than code.  The method that is the simplest to implement
<and least buggy would be preferable.
<
<What about using _handle in Zortech C++?  It sounds like it would be
<easy for bugs to creep in, from the description I read.

I've used _handles quite a bit in my code. The reason they were implemented
is that the code looked so *ugly* when explicit function calls had to be
done to dereference a handle. Thus, making a handle a special pointer type,
and having the compiler call the function for you, cleaned up the code
a lot. I'd say that the _handles are certainly the easiest to convert your
code into using.

The thing to watch out for is inadvertently converting a handle to a far
pointer, as in stuff like:
	char _handle *h;
	h = (char *) handle_malloc(10);		/* error!	*/
	h = (char _handle *) handle_malloc(10);	/* correct way	*/

a35@mindlink.UUCP (Bruce Atherton) (08/02/90)

6600m00n@ucsbuxa.ucsb.edu (Rob Blair) writes:
> 
> The main problem with swapping data is that
> it is accessed 'asyncronously'. i.e.  You never know when you will need
> to be in, since pointers can point anywhere.  Code is always accesed
> with calls, and can be routed through a overlay maneger.

As long as a function was called every time a pointer was dereferenced,
the time that the data was accessed wouldn't be a problem.  Of course,
that function would somehow have to be able to tell what page the
pointer belonged to, and swap it in if necessary.

You could do that three ways that I can think of off of the top of my
head.  First, you could have a special memory model that had 6 byte
pointers.

Second, you could use the Zortech method of assuming that pointers will
never want to point into the BIOS.  This strikes me as dangerous, since I can
think of reasons why I might want a pointer to point there.

The third method, and one I am sorry they didn't include as part of
VROOMM, would be to use normalized pointers and store the page
information in the unused part.

In Turbo C++, there is a Huge memory model that automatically normalizes
your pointers, so the offset is always between 0 and 15.  That means the
pointers look like this -

Bit FEDCBA9876543210:FEDCBA987654210   s = segment
    ssssssssssssssss:xxxxxxxxxxxoooo   o = offset
                                       x = unused

Why couldn't there be a variant of the Huge memory model called Virtual,
that used those unused bits to store page information for a memory
manager?  Some function must be ensuring that the pointers stay
normalized.  If that function is called whenever a pointer is
dereferenced, a virtual memory manager would be simple to implement and
almost invisible to the programmer.

I know, I know.  That ain't the way Turbo C++ works.  Is there anything
that does, other than Zortech?

a35@mindlink.UUCP (Bruce Atherton) (08/02/90)

bright@Data-IO.COM (Walter Bright) writes:
> 
> I've used _handles quite a bit in my code. The reason they were implemented
> is that the code looked so *ugly* when explicit function calls had to be
> done to dereference a handle. Thus, making a handle a special pointer type,
> and having the compiler call the function for you, cleaned up the code
> a lot. I'd say that the _handles are certainly the easiest to convert your
> code into using.
> 
> The thing to watch out for is inadvertently converting a handle to a far
> pointer, as in stuff like:
>         char _handle *h;
>         h = (char *) handle_malloc(10);         /* error!       */
>         h = (char _handle *) handle_malloc(10); /* correct way  */

I agree that explicit calls for dereferencing is no way to implement
virtual memory.  Zortech seems to use the most elegant solution I've
heard about so far.

Can you use a _handle anywhere you would use a pointer?  Can you declare
a _handle to a _handle to a function that returns a _handle? Does this
have to be a part of the compiler, or could I write an object in any C++
that would do the same thing?

Finally, can you create a pointer into the ROM BIOS area?

Thanks for your help.

Bruce_Atherton@mindlink.UUCP or| Wham bam bam bam bam bam thank you Hymie.
uunet!van-bc!rsoft!mindlink!a35| We drive chicks MAAADD! - The Frantics.

dmurdoch@watstat.uwaterloo.ca (Duncan Murdoch) (08/06/90)

In article <2698@mindlink.UUCP> a35@mindlink.UUCP (Bruce Atherton) writes:
>The third method, and one I am sorry they didn't include as part of
>VROOMM, would be to use normalized pointers and store the page
>information in the unused part.
...
>
>Why couldn't there be a variant of the Huge memory model called Virtual,
>that used those unused bits to store page information for a memory
>manager?  Some function must be ensuring that the pointers stay
>normalized.  If that function is called whenever a pointer is
>dereferenced, a virtual memory manager would be simple to implement and
>almost invisible to the programmer.
>
>I know, I know.  That ain't the way Turbo C++ works.  Is there anything
>that does, other than Zortech?

Turbo Pascal also uses normalized pointers, and there's a package available
(called HEAP55.xxx for TP 5.5) that calls an interrupt every time a pointer
is dereferenced.  

Duncan Murdoch

bright@Data-IO.COM (Walter Bright) (08/07/90)

In article <2698@mindlink.UUCP> a35@mindlink.UUCP (Bruce Atherton) writes:
>6600m00n@ucsbuxa.ucsb.edu (Rob Blair) writes:
<< The main problem with swapping data is that
<< it is accessed 'asyncronously'. i.e.  You never know when you will need
<< to be in, since pointers can point anywhere.  Code is always accesed
<< with calls, and can be routed through a overlay maneger.
<As long as a function was called every time a pointer was dereferenced,
<the time that the data was accessed wouldn't be a problem.  Of course,
<that function would somehow have to be able to tell what page the
<pointer belonged to, and swap it in if necessary.
<Second, you could use the Zortech method of assuming that pointers will
<never want to point into the BIOS.  This strikes me as dangerous, since I can
<think of reasons why I might want a pointer to point there.

You *can* have pointers pointing into the BIOS. They just cannot be _handle
pointers, they must be regular far pointers.

bright@Data-IO.COM (Walter Bright) (08/07/90)

In article <2699@mindlink.UUCP> a35@mindlink.UUCP (Bruce Atherton) writes:
<I agree that explicit calls for dereferencing is no way to implement
<virtual memory.  Zortech seems to use the most elegant solution I've
<heard about so far.
<Can you use a _handle anywhere you would use a pointer?

Anywhere you would use a far pointer, except if you want a far pointer
that points into the BIOS.

<Can you declare a _handle to a _handle to a function that returns a _handle?

You cannot have _handle pointers to functions (there is no point to that!).
But you can do things like:

	char _handle *(_handle **)(void);

which is a _handle to a pointer to a function returning a _handle.

<Does this have to be a part of the compiler, or could I write an object
<in any C++ that would do the same thing?

You can do it in standard C++ on a class-by-class basis by overloading the
* [] and -> operators. The disadvantage is that the optimizer cannot tell
when the dereferencing function calls are redundant and cannot eliminate
them. For instance:
	struct A __handle *h;
	h->a = 1;
	h->b = 2;
On the second assignment, the function call to dereference h can be
eliminated if handles are built in to the compiler, but cannot be if
it's a user-defined function in C++.

You cannot do it with pointers to something-besides-a-class in standard C++.

<Finally, can you create a pointer into the ROM BIOS area?

Yes. Simply use a far pointer:
	char far *p = MK_FP(0xFFFF,0);