[comp.sys.ibm.pc.misc] Extended and expanded memory?

salmassr@oiscola.Columbia.NCR.COM (Samer M. Almassri) (07/31/90)

Hi.  I always get confused between expanded and extended memory.
Could anyone post or mail me a good explanation.

1- What is the difference? (Technically & from a user point of view)
2- Why there are two kinds?
3- Which one you should use?
4- Can the computer use expanded and extended memory at the same time?
and other related information that you can think of...

Thanks in advance...

 __________________________________________________________________________
|   _____   ______           _____  _____     Imagine Life in a Peacefull  |
|  /     | /     / /\    // /      /    / _  _       United World...       | 
| |____   /_____/ /  \ / / /---   /____/  O  O                             | 
|      | /     / /      / /      /   |    \__/       Samer M. Almassri     | 
|\____/ /     / /      / /____  /    |   salmassr@oiscola.columbia.ncr.com | 
|__________________________________________________________________________|

troy@mr_plod.cbme.unsw.oz.au (Troy Rollo, ) (08/01/90)

From article <260@oiscola.Columbia.NCR.COM>, by salmassr@oiscola.Columbia.NCR.COM (Samer M. Almassri):
salmassr> Hi.  I always get confused between expanded and extended memory.
salmassr> Could anyone post or mail me a good explanation.

salmassr> 1- What is the difference? (Technically & from a user point of view)
salmassr> 2- Why there are two kinds?
salmassr> 3- Which one you should use?
salmassr> 4- Can the computer use expanded and extended memory at the same time?
salmassr> and other related information that you can think of...

Alright, but remember - you asked for this.

	Expanded memory was the first kind of memory beyond the 1Meg boundary
of the original PC - it works by swapping parts of memory on an expanded
memory card into the 1Meg address space at segment C000 (or thereabouts). More
recent implementations can splatter it all over the place like a dog's
breakfast. This is done in the same way it was done in the old Apple days -
bank switching - Yuk.

	Only programs which are specifically written for expanded memory can
use it. This means that you can travel to the ends of the Earth and still not
find one that does (Nethack 3.0i is an example of a program which can use it.
I would mention others, but that would entail getting commercial).

	Then there's Extended. Extended memory only exists on AT class machines
or better. Elsewhere it is meaningless. Even on those machines where it exists,
DOS programs can't use it directly. It's beyond the 1Meg address space. In
fact, it occupies the area from 1Meg to 16Meg, which is an altogether
civilised place for it to go, otherwise it may clash with 0Meg-1Meg.
	The catch is, Extended memory can only be used in the protected mode
of the 286/386/486. Dos cannot be used in protected mode. For this reason,
Ms/PCDOS should be taken out and shot. The problem is, that in order to use
the Extended memory properly, you need to use a protected mode operating
system, like the latest version of MsWindows, or Xenix/Unix, or OS/2 (bleuch).
W{hich one then? I suggest you toss a three sided coin, because when DOS is,
as it deserves to be, taken out and shot, we could end up on any of those three.
And I don't care to guess which.

	While DOS is still on death row, however, you can get access to some
of Expanded memory through XMS drivers like HIMEM.SYS. Unfortunately
somebody appears to have cut the balls off HIMEM.SYS, and it only offers
64K. Some software will use Extended memory anyway, by slipping into
protected mode while DOS isn't watching, and slipping out again. This is a
reasonable approach, especially if you don't want to talk to DOS very often.
If I had my way I would send DOS to Tasmania so nobody important would ever
have to talk to it at all.
	There are also some drivers somewhere out there wich will turn the
Extended memory into a RAM disk. A reasonable approach too, but it locks other
uses out.

	Which is the best? Well, if you're running Windows, Unix or OS/2,
you're laughing. Throw in a couple of meg of Extended. If you're using a 386
as well, throw in a gig and have some real fun. If you're using DOS, you're
probably better off with Expanded, because more programs use it. Of course,
DOS could be obsolete before long (with any luck).

___________________________________________________________
troy@mr_plod.cbme.unsw.oz.au	Make our greenies useful!
The Resident Fascist		Put them in the army!

ant@brolga.cc.uq.oz.au (Anthony Murdoch) (08/02/90)

salmassr@oiscola.Columbia.NCR.COM (Samer M. Almassri) writes:

>Hi.  I always get confused between expanded and extended memory.
>Could anyone post or mail me a good explanation.

I guess I can't complain about the number of times this has been discussed,
I asked exactly the same questions when I first started using the net.  So
here goes the regurgitation.

>1- What is the difference? (Technically & from a user point of view)
Extended memory (XMS) is directly addressable by the CPU.  Only 80?86 cpu's
are capable of accessing XMS (ie if you have an 8086 or 8088 then forget
about XMS).
Expanded memory (EMS) is a cludge used to give the poor old (decrepid sp?)
8086 and 8088 processors access to more than the 640K they normally have.
NOTE - the 808? processors are able to address 1024K directly, but the
memory above 640K is used for things like video cards and other configuration
stuff.  64K of this stuff is used for EMS.  Basically all that happens is
that you have all of this EMS memory sitting on its own card and you have
a driver that swaps it in and out of this 64K block.
As you can imagine, this makes EMS a little slower.

>2- Why there are two kinds?
XMS because as processors got bigger, they wanted to address more memory.
EMS because as programs got bigger, they wanted to address more memory, and
people didn't want to get a better machine with a better OS.

>3- Which one you should use?
This depends.  If you have a 808? then EMS is all you can have.  If you have
an 80?86 then it still depends.  EMS is slower than XMS, but appears to have
a lot more support than XMS.  There is a trend I think of late, to support
both EMS and XMS.  For example, windows 3 supports both (in fact it's XMS
support is better than its EMS support methinks).
You can get cards which can be configured both as EMS and XMS or a bit of
both, which brings me to ;

>4- Can the computer use expanded and extended memory at the same time?
>and other related information that you can think of...
Yes you can.  The boards mentioned above make this quite easy.  In fact some
80?86 mother-boards have switches on them that allow you to configure your
memory in this way.

I hope that this all helps.

>Thanks in advance...

> __________________________________________________________________________
>|   _____   ______           _____  _____     Imagine Life in a Peacefull  |
>|  /     | /     / /\    // /      /    / _  _       United World...       | 
>| |____   /_____/ /  \ / / /---   /____/  O  O                             | 
>|      | /     / /      / /      /   |    \__/       Samer M. Almassri     | 
>|\____/ /     / /      / /____  /    |   salmassr@oiscola.columbia.ncr.com | 
>|__________________________________________________________________________|

ant

-- 
  V   ant                       "It's great to be young and insane"
 \o/  ant@brolga.cc.uq.oz.au                    - Dream Team
 -O-  Anthony Murdoch           Prentice Computer Centre
 /0\  Phone (07) 3774078        University of Qld

ray@philmtl.philips.ca (Ray Dunn) (08/14/90)

>In referenced article, salmassr@oiscola.Columbia.NCR.COM (Samer M. Almassri) writes:
> Hi.  I always get confused between expanded and extended memory.
> Could anyone post or mail me a good explanation.

Seems perhaps a good time to repost this.....



THE MEMORY ARCHITECTURE OF THE "INDUSTRY STANDARD" PERSONAL COMPUTER
--------------------------------------------------------------------

Memory up to 1 Megabyte:
-----------------------

The "Industry Standard Architecture" (ISA) of the IBM PC is designed around the
8088 CPU which can address up to 1 megabyte of memory.

Within this architecture, the memory is organized in the following way:


      1M +-------------+ FFFFF
         |             |
         |             |          System BIOS ROM
         |             |
    960K +-------------+ F0000
         |             |          BASIC Interpreter (some IBM systems only)
         |             |          or
         |             |          VGA BIOS (Some Planar VGA implementations
         |             |                    including PS/2's)
    960K +-------------+ E0000
         |             |
         |             |           Option card ROM and RAM
         |             |
    816K +-------------+ CC000
         |             |           XT Hard Disk Controller BIOS ROM
    800K +-------------+ C8000
         |             |           Video BIOS ROM
         |             |
    768K +-------------+ C0000
         |             |           Video RAM space - various video types
         |             |           have different requirements and starting
         |             |           locations.
         |             |
    640K +-------------+ A0000
         |             |
         |             |
         |             |           Base system RAM
         |             |
         |             |
    1286 +-------------+ 506
         |             |           Interrupt vectors and BIOS Data Area
       0 +-------------+ 0


MSDOS expects this architecture, and assumes that the *base memory* is its
resource to manage.  The amount of base memory available is communicated to
MSDOS at boot time by the BIOS, which automatically sizes the available
memory and sets the amount found in location 413H, a location in the BIOS
Data Area.

Most BIOS'es will not detect more than 640K as the base memory, because
their systems do not support more than that, and memory found in the A0000
page is usually part of the video memory and unavailable to MSDOS.

Some machines however can be configured so that when there is no video card
occupying the A0000 page, system RAM can be allocated there.  In this case
the BIOS will report that extra memory in location 413H, and MSDOS *will*
then use this memory.  MSDOS does not have a 640K limit.  That is only the
*usual* limit of the base memory in PC's.

As an example of this, the Philips P3345 80386SX based machine allows the
system to be configured so that 64K is allocated at 640K to give a total of
704K of "base" system RAM.  This is only legal if an EGA/VGA video card
is not present, as those cards have video RAM in that address space.

Option cards can contain EPROMS and proprietary purpose RAM located
anywhere within the address range C8000:0 to E0000:0, although they must
take steps to ensure that they can coexist with each other (using
configuration straps etc).

The system BIOS scans this area during the system initialization sequence
and if it finds an option BIOS *signature* on any 2K boundary, it will pass
control to that BIOS to allow it to initialize itself.

Except for Expanded Memory, which is described below, RAM provided in this
address space is used by applications and option cards in a proprietary
way.  It is not defined in the standard, and is not normally considered a
DOS resource.

An example of a system board which provides RAM here is the Philips P3105
XT system.  It is more convenient on this machine to provide 768K of RAM
using all 256K chips rather than be involved with 64K chips to provide
640K.  This "extra" 128K of RAM can be enabled if required at D0000.  A
RAMDISK driver is provided to use this memory.



EXTENDED AND EXPANDED MEMORY:
----------------------------

*Extended* memory is an intrinsic part of the architecture of the Intel
processor family.

*Expanded* memory is only defined in the architecture of the IBM-PC
standard.

*Extended* memory is memory directly in the CPU memory address space, above
the 1 megabyte address.

It is "extended" because the 8088 can only access 1 meg.  Thus "extended
memory" is part of the "extended memory addressing" of the 80286 & 80386
processors.

*Expanded* memory is paged memory which can be addressed through a Page Frame
within the first 1 megabyte, and is thus accessible on all processors, even the
8088.  Expanded memory can be provided in a number of ways (see below).

The mnemonic is easy, it is in the meaning of the words:

-> -> ->        Extended is longer.        <- <- <-

-> -> ->        Expanded is fatter.        <- <- <-

When you buy an 80286, 80386 or 80386SX based machine which has more than
640K of RAM, that extra memory must be mapped somewhere, because *normally*,
in the architecture of the PC, the area between 640K and 1M is used for
video RAM, option card ROM and RAM, and the BIOS ROM etc, as described
earlier.

Some machines map the 384K from this hole on top of any *other* extended
memory above 1M.  I.e.  if the machine has 1M memory, the 384K is mapped at
physical address 1M.  If the machine has 2M memory, the 384K is mapped at
physical address 2M etc.

Some machines just map the memory away out of trouble at say 16 or 24Meg.

Some machines use all or part of it to "shadow" the system BIOS, and
sometimes a proprietary Video BIOS.  Thus during system initialization,
these BIOSes are copied from ROM into the fast (16-bit or 32-bit) RAM,
write-protected, and the BIOS from then on executes from there.  This
obviously requires motherboard hardware support.

Some machines allow all or part of the extended memory to be used as
expanded memory.  This is achieved either by hardware support or software
emulation as described below.

Some machines, e.g.  the Philips P3345, allow all sorts of permutations of
the above!  This is an area where manufacturers of clones can *add value*.



Extended Memory:
---------------

As *MSDOS* assumes an 8088 and can thus only address the first 1 Megabyte of
the address space, extended memory on a 286 or 386 has limited use with DOS.

It is most commonly used as a RAM disk, because this involves only *block*
access to the memory and is a nicely self-contained application, but it
could also be used for caching, directly by an application, or to implement
some *Expanded* Memory Scheme (EMS).

An 80286 has a 24 bit address space, i.e.  16 megabytes (1 meg plus 15 meg
of extended memory).

An 80386 has a full 32 bit address space (work it out yourself!).  Most AT
architecture implementations using an 80386 use a private memory address bus
which limits the addressing to 16 or 24 or 32 Megabytes.

Extended memory is not generally used in the MSDOS environment other than by
RAM disk programs, disk caches, and expanded memory emulations, for several
reasons:

- It is not available on 8088 machines so *applications* which wish to be
  compatible across the whole product line stay away from using it directly.

- It is not a standard resource, so there is no proper management of it for
  co-existing software (but see MSDOS 4.0 below).

- It is very slow and a pain in the neck to access because MSDOS runs the
  80286 in native mode (i.e.  as an 8088).  To access above 1 meg the 80286
  must be switched into protected mode.  This is fine, but there is no way
  to switch it back to native mode!!  (This is one of the reasons that the
  80286 is regarded as a (cough) less-than-optimal processor).  So, the
  trick used (on the basic PC architecture) is to get the keyboard
  controller to RESET the CPU, and the BIOS reset code recognizes this and
  continues!  Most modern machines have additional direct hardware support
  for the reset to avoid the lengthy keyboard controller operation.

Thus extended memory is really only good for some sort of *block* access to
the memory, and there is an INT15 function in the BIOS which does this for
you.

All of this can be done more easily on an 80386, as it can switch into and
out of protected mode itself, and quite fast.  Thus the extended memory
functions of INT15 should run very much faster on a 386 machine than on a
286 one, but of course this is still a waste of the 80386 which must
continue to run in native mode to be PC and MSDos compatible.

The good news is that Unix and OS/2 make *full* use of extended memory, as
they run the machine in protected mode, as does Windows 3.0, (OS/2 however
does not yet have a version that makes full use of the 32-bit capabilities
of the 80386 - that will be version 2.0, for which don't hold our breath)!

With release 4.0, MSDOS recognizes extended memory for the first time as
more than just a RAM disk resource.  In addition to RAMDRIVE.SYS and a disk
caching program, SMARTDRV.SYS, a device driver, HIMEM.SYS, is provided to
manage the extended memory according to XMS 2.0 specification.

XMS is an eXtended Memory Specification standard defined jointly by
Microsoft, Intel, Lotus and AST (not to be confused with LIM, see below).
This specification is available from them.  XMS 2.0 allows extended memory
to be used by co-resident software, and, for example, allows TSR's to be
positioned in extended memory.  

Very few applications are currently written to make use of this standard -
it is somewhat ironic that the provided RAM disk and disk caching routines
RAMDRIVE.SYS and SMARTDRV.SYS do *not* use the XMS management routines when
using extended memory, but *do* use LIM 4.0 when using expanded memory (see
below)!

HIMEM.SYS also implements the XMS 2.0 feature which formally makes the first
64K of extended memory available as a non protected mode memory resource
using the overflow into A20 trick - if a segment register is set to FFFF,
the segment automatically extends 64K-16 bytes above the 1 megabyte address
(i.e.  into the extended memory), but can still be accessed in native mode.
Thus the largest address an 80286 or 80386 can access in native mode is
actually FFFF:FFFF.

Not only can data be stored in this segment, but code can also be executed
from there in native mode, unlike the rest of the extended memory.

WINDOWS 286 and 3.0 as well as many TSR's, makes use of this feature.

The keyboard controller is involved in this mechanism too, as it is used to
control the A20 address line.  The following explanation goes to a level of
detail you may not want to go to:

   The 8088 processor has no A20 address line, so that any address
   incremented from FFFF:F *wraps around* back to zero.

   Some (sarcastic voice) very clever software, including DOS itself, relies
   on this behaviour.  Because 80286 and 80386 processors do *not* wrap
   around at FFFF:F, but continue to 10000:0, special purpose hardware is
   built into AT's to gate this A20 line.  Using the keyboard controller,
   this gate can be used to disable the A20 line from the CPU and thus mimic
   the 8088 behaviour.

   The normal mode of operation for an AT is to have the A20 line disabled.

   Thus for HIMEM.SYS to take advantage of the fact that it can access the
   first 64K of extended memory in native mode, it must first enable the A20
   line via a command to the keyboard controller, do whatever access is
   required, then again disable the A20 line.




Expanded Memory:
---------------

There were several standards for expanded memory, but this has now stablized
into the LIM 4.0 standard (_L_otus, _I_ntel and _M_icrosoft in agreement).

LIM 4.0 describes the following architecture:

         PC MAIN MEMORY                LIM EXPANDED MEMORY
         --------------                -------------------

         +-------------+
         |             |
         . Extended    .
         . Memory      .
         . up to       .               /+-------------+ 32M
         . 16M on the  .             // |             |
         . 80286,      .           / /  |             |
         . 4096M on    .         /  /   |  Expanded   |
         . the 80386   .       /   /    |  Memory     |
         |             |     /    /     |             |
      1M +-------------+   /     /      |  Divided    |
         |             | /      /       |  into up to |
    960K +-------------+       /        |  2048*16K   |
         | Page Frame  |      /         |  logical    |
         | 12*16K      |     /          |  pages      |
         | Physical    |    /           |             |
         | Pages       |   /            |             |
    768K +-------------+\ /             .             .
         |             | /              .             .
    640K +-------------+/ \             .             .
         | Page Frame  |   \            .             .
         | 24*16K      |    \           |             |
         | Physical    |     \          |             |
         | Pages       |      \         |             |
         |             |       \        |             |
    256K +-------------+        \       |             |
         |             | \       \      |             |
         |             |   \      \     |             |
         |             |     \     \    |             |
       0 +-------------+       \    \   |             |
                                 \   \  |             |
                                   \  \ |             |
                                     \ \|             |
                                       \+-------------+  0

The expanded memory gives you up to 32M to map into up to 36 16K pages in
the positions shown above.

The page frame below 640K which allows the existing base RAM to be paged
is usually regarded as being for the use of operating systems/environments.
The page frame above 640K, in the video RAM and "PROM" area, is for
applications.  Code can be run or data stored interchangeably in any of these
frames.

30 functions are defined using int 67H, to allocate, map, swap etc these
pages.  This functionality is provided, depending on the implementation, as
a loadable driver (usually), as an extension to the motherboard BIOS, or on
a PROM on a LIM 4.0 option card (not usual).

The number of actual pages available in the page frame depends on the
implementation, and how much of the address space is already occupied by
option card ROMS (and perhaps RAM's) in the machine.

Applications which use LIM 4.0 can thus have a very large physical memory
available to them, but must access it by manually switching pages as
required (come back Z80 systems, all is forgiven!!)

LIM 4.0 can be implemented in many ways:

- as an option card with memory and page registers etc as I/O hardware to
  provide the mapping etc, and a loadable software device driver or onboard
  BIOS PROM to provide the functionality.

- using extended memory together with special purpose mapping and page
  register logic built into the motherboard ASICs or logic, on an 80286 or
  80386 machine.  Philips has two machines using this approach, the 80386SX
  based P3345 using the NEAT chipset, and the 80286 based P2230 using the G2
  chipset.

- as a pure software emulation in a .SYS or TSR driver:

   - using extended memory and virtual 86 mode on an 80386 so that the
     memory is mapped using the MMU into any of the physical LIM pages.

   - *copying* into the RAM pages existing below 640K, either from extended
     memory (if other than 8088), or from, say, disk.  This is much slower
     as pages are swapped using copying rather than just updating page
     registers.

- combinations of the above etc.

Because *expanded* memory functionality can be provided by hardware or
software support using physical *extended* memory, the confusion between
expanded and extended memory is made worse (even the description of
HIMEM.SYS in the Philips MSDos 4.01 manual gets it wrong!)

LIM 4.0 management is handled by its BIOS only in so far as the application
can ask it what pages are free, how much memory is free to map etc., and to
perform that mapping.  The actual use of this paging is left up to the
application.  Neither LIM 4.0 nor DOS implements demand paging or anything
remotely like that, nor do they support an application's use of LIM in any
other way.

Two or more co-existing applications can share the expanded memory quite
happily if they are well behaved (i.e.  use the expanded memory *only*
through the LIM 4.0 calls).

Functionality is provided to allow the saving and restoring of the page
registers etc, so that even interrupt routines can safely be written to use
LIM controlled expanded memory, assuming they allocate all their
memory requirements up front.

Whether or not expanded memory will be useful to you is *application*
dependent.  Check to see if the applications you propose to use can take
advantage of it.

Release 4.0x of MSDOS *does* recognize LIM 4.0's presence, and allows
buffers and fastopen file names etc to be allocated in expanded memory
pages, and DEBUG now "understands" LIM.  In no way is LIM *integrated* into
the operating system though.

A driver, EMM386.SYS, is provided in DOS 4.0 to emulate LIM 4.0 using the
extended memory of a 386 based machine.  This emulator provides a fast
efficient implementation of LIM, but slows the total operation of the
machine down marginally (<5%) as it runs the machine in virtual 86 mode.

The RAM disk and disk caching routines RAMDRIVE.SYS and SMARTDRV.SYS can be
configured to use extended memory, *or* expanded memory under the control of
LIM 4.0.

The "Lotus/Intel/Microsoft Expanded Memory Specification Version 4.0"
(published by, you guessed it, Lotus, Intel and Microsoft), is the
definitive manual, and is available from them.

This article is in the public domain.  Please retain this notice and the
attribution below if reprinting.
-- 
Ray Dunn.                    | UUCP: ray@philmtl.philips.ca
Philips Electronics Ltd.     |       ..!{uunet|philapd|philabs}!philmtl!ray
600 Dr Frederik Philips Blvd | TEL : (514) 744-8200  Ext : 2347 (Phonemail)
St Laurent. Quebec.  H4M 2S9 | FAX : (514) 744-6455  TLX : 05-824090

bobmon@iuvax.cs.indiana.edu (RAMontante) (08/14/90)

Thanks, Ray Dunn, for a fascinating and informative article.

Now, a question:  I have an old EMS card which is alleged to be LIM 3.2 and
EEMS compatible.  Alas, the maker has gone to meet its maker, so the driver
I have is the one I must stay with.  My question is, how compatible are LIM
3.2 and EEMS with LIM 4.0?  What am I missing out on?

t-rayc@microsoft.UUCP (Raymond CHEN) (08/15/90)

In article <54348@iuvax.cs.indiana.edu> bobmon@iuvax.cs.indiana.edu (RAMontante) writes:
>Thanks, Ray Dunn, for a fascinating and informative article.

Please note that Mr. Dunn's article has been available on the
comp.sys.ibm.pc mail server for several months now.  Also, the
problems of July and early August have been remedied.  (The
server ran out of disk space, so it couldn't uncompress anything.)
However, due to disappearing disk space, the Berkeley server will
be shutting down.  The Princeton server will remain in operation, however.

psrc@mtunq.ATT.COM (Paul S. R. Chisholm) (08/15/90)

In article <54348@iuvax.cs.indiana.edu> bobmon@iuvax.cs.indiana.edu
(RAMontante) asks, "how compatible are LIM 3.2 and EEMS with LIM 4.0?"

Largely, EEMS is upward compatible with LIM EMS 3.2, and LIM EMS 4.0 is
upward compatible with them both.

If an application asks only for EMS software, any of the three should
do.  If it asks for EMS 4.0, you *may* be able to get by with EEMS,
since most of the big changes in 4.0 were adopted from EEMS.  (EEMS was
created by a competing triumvirate.)

Paul S. R. Chisholm, AT&T Bell Laboratories
att!mtunq!psrc, psrc@mtunq.att.com, AT&T Mail !psrchisholm
I'm not speaking for the company, I'm just speaking my mind.