[comp.sys.ibm.pc] 32-bit addressing on the 80386 in real mode?!?

dhinds@portia.Stanford.EDU (David Hinds) (01/18/90)

    I think I might have found a trick for allowing 32-bit addressing
in real mode on the 80386.  According to a book I have on the 80386,
real mode address calculations pass through the same memory management
hardware as in protected mode.  When the 80386 is reset, it loads the
segment descriptor cache registers with sets of attributes that make
all the segment registers behave as if they were 8086-type segments.
When switching from protected mode BACK to real mode, the book says
that the descriptor cache registers all have to be re-loaded with
attributes appropriate for real mode - i.e., the 80386 does not reset
the attributes on its own.  These attributes include the 64K segment
limit.  So, I thought, it should be possible to switch to protected mode
for a moment, set up the segment descriptor cache to look like real mode
EXCEPT that the limit fields are set to 4 Gbytes, then pop back into real
mode.  The descriptor cache should be locked in real mode, so from that
point on, big offsets should work without causing protection violations.
However, I haven't been able to get it to work.  I can switch into and
out of protected mode OK, and initialize the segment descriptors OK, but
the 64K offset limit still seems to be enforced when I get back to real
mode.  So... is the book wrong?  Am I missing anything trivial?  Do some
real-mode events cause the descriptor cache to be re-initialized, like
interrupts or far jumps?  Or am I just wasting my time?

- David Hinds
  dhinds@portia.stanford.edu

walrus@wam.umd.edu (Udo Karl Schuermann) (01/18/90)

In article <8342@portia.Stanford.EDU> dhinds@portia.Stanford.EDU (David Hinds) writes:
>
>    I think I might have found a trick for allowing 32-bit addressing
>in real mode on the 80386.  According to a book I have on the 80386,
>real mode address calculations pass through the same memory management
>hardware as in protected mode.  When the 80386 is reset, it loads the
>segment descriptor cache registers with sets of attributes that make
>all the segment registers behave as if they were 8086-type segments.
>When switching from protected mode BACK to real mode, the book says
>that the descriptor cache registers all have to be re-loaded with
>attributes appropriate for real mode - i.e., the 80386 does not reset
>the attributes on its own.  These attributes include the 64K segment
>limit.  So, I thought, it should be possible to switch to protected mode
>for a moment, set up the segment descriptor cache to look like real mode
>EXCEPT that the limit fields are set to 4 Gbytes, then pop back into real
>mode.  The descriptor cache should be locked in real mode, so from that
>point on, big offsets should work without causing protection violations.
>However, I haven't been able to get it to work.  I can switch into and
>out of protected mode OK, and initialize the segment descriptors OK, but
>the 64K offset limit still seems to be enforced when I get back to real
>mode.  So... is the book wrong?  Am I missing anything trivial?  Do some
>real-mode events cause the descriptor cache to be re-initialized, like
>interrupts or far jumps?  Or am I just wasting my time?
>
>- David Hinds
>  dhinds@portia.stanford.edu

I don't have any literature on the '386 but let me take a shot at this.

In protected mode, segment and offset registers are 32 bits long.  This
produces a 4 GByte address range.  In real mode, the registers are only
16 bits long, and in addition, the segment and offset registers overlap
such that the total number of bits used is not the sum of 16 + 16 being
32 bits but only 20.  This has the unfortunate (and often confusing)
side-effect of giving you 16 different addresses for each byte in the
real mode address space.

That aside, any existing real mode programs would be utterly unprepared
to have their addresses suddenly interpreted in ways that their authors
didn't intend.

I'm sure I'm not quite right in some of my reasoning, but this is
probably close enough, especially since I've got no docs on the '386 :)
Any gurus to set the record straight(er)?

--------------------------------------------------------------------------
Udo Schuermann           //  "The power to destroy a thing is the
walrus@cscwam.umd.edu  \X/    absolute control over it."  -- Frank Herbert
--------------------------------------------------------------------------

Ralf.Brown@B.GP.CS.CMU.EDU (01/18/90)

In article <1990Jan18.055726.20604@wam.umd.edu>, walrus@wam.umd.edu (Udo Karl Schuermann) wrote:
}produces a 4 GByte address range.  In real mode, the registers are only
}16 bits long, and in addition, the segment and offset registers overlap
}such that the total number of bits used is not the sum of 16 + 16 being
}32 bits but only 20.  This has the unfortunate (and often confusing)
}side-effect of giving you 16 different addresses for each byte in the
}real mode address space.

Actually, that's 4096 different addresses for each byte....
--
UUCP: {ucbvax,harvard}!cs.cmu.edu!ralf -=- 412-268-3053 (school) -=- FAX: ask
ARPA: ralf@cs.cmu.edu  BIT: ralf%cs.cmu.edu@CMUCCVMA  FIDO: Ralf Brown 1:129/46
"How to Prove It" by Dana Angluin              Disclaimer? I claimed something?
14. proof by importance:
    A large body of useful consequences all follow from the proposition in
    question.

toma@tekgvs.LABS.TEK.COM (Tom Almy) (01/19/90)

In article <1990Jan18.055726.20604@wam.umd.edu> walrus@cscwam.umd.edu (Udo Karl Schuermann) writes:
>In article <8342@portia.Stanford.EDU> dhinds@portia.Stanford.EDU (David Hinds) writes:
>>    I think I might have found a trick for allowing 32-bit addressing
>>in real mode on the 80386.  
[long description of "trick" follows]
>>However, I haven't been able to get it to work. 

>I don't have any literature on the '386 but let me take a shot at this.
[shot misses]
>I'm sure I'm not quite right in some of my reasoning, but this is
>probably close enough, especially since I've got no docs on the '386 :)
>Any gurus to set the record straight(er)?

From the Intel 80386 Programmer's Reference Manual, Chapter 14, 80386
Real-Address Mode, "Unlike the 8086 and 80286, 32-bit effective addresses
can be generated (via the address-size prefix); however, the value of a 32-bit
address may not exceed 65535 without causing an exception."

In other words, "forget it."  80386 protected mode is the *only* way to go.

Tom Almy
toma@tekgvs.labs.tek.com
Standard Disclaimers Apply

rodentia@legs.UUCP (Tom Roden) (01/20/90)

in article <8342@portia.Stanford.EDU>, dhinds@portia.Stanford.EDU (David Hinds) says:
> 
> 
>     I think I might have found a trick for allowing 32-bit addressing
> in real mode on the 80386.  According to a book I have on the 80386,
[..part deleted...]
> - David Hinds
>   dhinds@portia.stanford.edu

     This is, in fact, a viable technique.  In an article published in
the Nov/Dec 89 issue of _Programmer's Journal_, I outline how this
is done.  I also have code available for posting.  Just tell me where
you want it.

     The code is written for Microsoft Assembler and C, version 5.1.
It runs only on those processors that support the 386 programming 
model (386-SX, 386-DX, and 486).  As supplied, it does not interfere
with the operation of existing real-mode programs, but may conflict 
with any software accessing extended memory.
 
     One issue not fully addressed is the ability to manipulate other
processor mode control bits in such a way as to greatly enhance the
utility of 386 real mode (while rendering current 86 family real mode
applications useless).  If anyone manages to do research in this area,
I would be greatly interested in the result.

     Text of original article (minus minor changes by the editor)
follows:
--------------------------------------------------------------------------
Using 4GB Segments in Real-Address Mode
Thomas Roden - 11 August, 1989


Until now, all x86 family processors have been limited to 1M 
(or 1M+64K-16 in the case of the 80286(tm)) of immediately available 
linear memory when running under real-address mode (real mode).  
With the advent of the i386DX(tm) and i486(tm) processors, this is no 
longer the case.  With a little manipulation at initialization time, it is 
possible to address all 4G of physical address space in real mode 
without the overhead of virtual 86 mode.  In addition, it is not 
necessary to violate DOS to do so.

Normally, real mode of the i386(tm) family of processors is 
generally viewed as a mode that is totally equivalent to the normal 
operation of the 8086 and 8088 processors.  This is not entirely true.  
The real modes of these processors are intended only to be 
compatible with lesser processors of the x86 family, not identical.  
This is borne out by the fact that extended registers and the 
additional segment registers FS and GS are available in real mode.

It is not illegal to use the extended registers for addressing 
with the additional segment registers as in the following statement:

	MOV		DWORD PTR GS:[EAX], EBX

The only proviso is that EAX must be less than 64K; otherwise a 
general protection fault results.  It seems curious that such an error 
can occur in real mode, a mode where protection is nominally 
unused.

Another curious feature is that when returning to real mode 
from protected mode by clearing the Protection Enabled (PE) bit in 
Control Register 0 (CR0), the system is required to first load all 
segment registers with selectors that reference segments of size 
64K(1).  When this is not done, general protection faults can occur 
upon access of those segments.

This behavior suggests that the limit checking aspects of 
protection are active in real mode, but the default state (set at power 
on or reset) is such that the limits of the segments are all set to 64K-
1, normally obscuring the fact that limit checking is truly in force.

A way to test this is to have a program enter protected mode, 
set a segment register to refer to a selector of size greater than 64K, 
return to real mode, and attempt to access data further than 64k 
from the beginning to the segment.

Tests on the i386DX(tm), i386SX(tm), and i486(tm) have shown that it is 
indeed possible to reference data further than 1M+128K (12000h) 
from the beginning of a segment, and there is no reason to believe 
that this behavior is not consistent for the entire 4G that can be 
referenced by an extended register.

This technique has many potential uses.  Applications that are 
386 aware can run in the faster real mode, and still access all linear 
memory available in the system without disturbing DOS.  System 
software can access normally hidden memory mapped I/O locations 
without the overhead it takes to change processor modes.  In general, 
the i386(tm) and i486(tm) can be used as more than fast 8088's without 
having to support an Operating System other than DOS.

A known danger of using this technique is that software that 
performs a hardware reset (normally from the keyboard controller 
to return from protected mode to real mode) will cause the segment 
limits to return to 64K-1.  This can be handled by an Interrupt 
Service Routine (ISR) that detects when general protection faults are 
caused by an attempt to address in this fashion, and reinstates the 
extended limits.

It should also be noted that this is a system-oriented technique 
and risks collision with other techniques that attempt to allow 
greater memory access.  Most notably, if this is attempted from a 
virtual 86 task, it will definitely cause a general protection fault.  
More generally, any extended memory manager will be in danger of 
colliding with data accessed via the extended registers.

Only data segments have been explored with this technique.  It 
may, however, be possible to perform the same sort of manipulations 
on the Default Bit (D-bit) of code segments to have full use of the 32-
bit Extended Instruction Pointer (EIP) as well.  Similarly, it may also 
be possible to affect the Expand-Down (E-bit) to instill better stack 
underflow recovery in real mode as well.

It is important to keep in mind that the i386(tm) family of 
processors are extremely powerful, and even in real mode, they 
allow new features that have yet to be fully investigated and 
exploited.


[References]
(1)	Intel, 80386 Programmers Reference Manual.  Ahearn-
Wallstrom, 1986

[Biographical Data]
Thomas Roden is a Software Engineer for AST Research, Inc., of 
Irvine California, where he is working on software validation of 
system integrity and general pontification.  Readers may write to 
Thomas in care of AST Research, Inc., P.O. Box 19658, Irvine, CA  
92713-9658.
------------------------------------------------------------------------------
-- 
Thomas A. Roden                     uucp:      ...!uunet!ccicpg!legs!rodentia
AST Research Inc. Dept 659          
P.O. Box 19658                      Voice:     (714) 727-8611
Irvine, CA  92713-9658              Fax:       (714) 727-9358