[comp.arch] 64 bit addresses

mh2f+@andrew.cmu.edu (Mark Hahn) (02/13/91)

Just think of the added overhead in pointer-linked structures!
Big addresses are a major win for typed support though, since
more big-bag-of-pages 'tag' bits are available.  (NB: not 
a portability violation.)  It also makes for a dandy hash function 
for names up to 8 chars!  I guess the whole point is to make large, 
sparse structures feasible, since it doesn't seem realistic 
to do _anything_ in 64bit (~2e19) quantities:

RAM: 1M simms @ $50	5e1/1e6		$1e15
hard disk 1.2G @ $3k	3e3/1e9		$6e13
gigabit net or
fast memory bus 	1/1e8		2e11 sec (millennia)

On the other hand, 32bits (~4e9) is just $2e5 RAM, $1e4 disk, 40 sec.
On the first hand again, perhaps CS needs a superproject, too!

regards, mark

lamaster@pioneer.arc.nasa.gov (Hugh LaMaster) (02/14/91)

In article <cbiAekK00VpINR8nEq@andrew.cmu.edu> mh2f+@andrew.cmu.edu (Mark Hahn) writes:
>Just think of the added overhead in pointer-linked structures!

I tried to think of it, and failed!  Several posters have mentioned this.
At the end of every chain of pointers, there must be some data there somewhere!?
What is the ratio of pointers to other data in typical "small" programs?  
Would doubling the size of addresses add anything significant, considering 
the sizes of main memories on current "small" systems (usu. minimum of 8 MB)?  

(The Cyber 205 used to have 64 bit "addresses" (actually descriptors).
Regardless of whatever other problems the machine had, the addresses were
not a problem.  I note that, though, it only had to do integer arithmetic 
on 48 bits of the address, not 64.)

Caches *could* be a problem, since the contents of caches *might* be more 
address intensive than main memory.  Does anyone have any *data* that shows 
that this is a problem?  I *know* it isn't a problem for typical scientific/
engineering code, but I'm not sure about pointer-intensive object oriented
stuff...

>Big addresses are a major win for typed support though, since

(deleted text)

> .....         I guess the whole point is to make large, 
>sparse structures feasible, since it doesn't seem realistic 
>to do _anything_ in 64bit (~2e19) quantities:

Large sparse structures are *one* of the benefits to "large" addresses, which
we will enjoy for the next few years.  Until what year?  The way I look at it 
(historically), we will *run out* of address bits to address physical
memories somewhere between 2010 and 2020, using 64 bit addresses.  Then, we will
be talking about 128 bit addresses, and how extravagant they are, and how
the only use for them will be large sparse structures.        1/2 :-)

Of course, we have *already* run out of bits in 32 bit addresses.
Obviously!     :-)

>RAM: 1M simms @ $50	5e1/1e6		$1e15
>hard disk 1.2G @ $3k	3e3/1e9		$6e13
>gigabit net or
>fast memory bus 	1/1e8		2e11 sec (millennia)
>
>On the other hand, 32bits (~4e9) is just $2e5 RAM, $1e4 disk, 40 sec.
>On the first hand again, perhaps CS needs a superproject, too!



Looking in my IBM Systems and Products guide (1 year old - early 1990 edition) 
I find that models of the ES/3090 were (then) sporting 512 MBytes of main 
memory, 2 GBytes of extended memory, 128 4.5 MB/sec channels, 
and multi-disk/head storage units with 3.78 GB/head (memory mapped I/O, anyone?)

Off the shelf Computer Science Department workstation servers from various 
vendors already have >= 128 MB of real memory.  Ordinary mass production 
disk drives are in the 1-2 GByte range (in case you might have an application
which needs memory mapped I/O on a large file :-).  You can buy compatible 
RAID disk products for some systems with something like 15 MB/sec throughput...

To put it another way, it won't just be a "superproject" that will need > 32
bit addressing in the next few years.

My congratulations to MIPSCo for facing up to this problem, even though, no 
doubt, it will be a public relations flop with the same people who were telling
us that we didn't need > 16 bit addresses a few years ago, since anything bigger
than that was just evidence of "wasteful" programming.  Actually, I take it
back.  Some of them are *still* saying it     :-)

I argue on historical grounds that it would be counterproductive to
extend addresses to, say, 40 bits, in order to save a few bytes per address,
to last until the year 1999.  In 2000, you will have to do it all over again.
And, a lot of data paths are now already 64 bits wide, anyway, so that isn't 
a problem.  The CDC approach: use 48 bits, and do something else with the
other 16 has *something* to recommend it.  But, I don't.  Better to waste
another whole 64 bit word on other stuff if necessary, than to create such
complications.   8 Bytes is just not that precious anymore.  Only if you
have arrays of millions of objects should you compromise on simplicity.

The only *potential* problem that I can see is that with 64 bit addresses, you
need a 1 cycle 64 bit integer adder, which is no mean feat, as far as I know.
The largest linear address I have previously seen is the aforementioned 48 bits.


Also, with less than 64 bit addresses, you must, in the meantime, deal with 
the hassles of non-power-of-two address sizes.  And, with the size of int not
equal to the size of a pointer (which isn't a problem with *my* code, but it
might be with *yours* :-)    )  And, forgo the benefits of enjoying a large, 
sparse, address space for the next *few* years...     :-)




  Hugh LaMaster, M/S 233-9,  UUCP:                ames!lamaster
  NASA Ames Research Center  Internet:            lamaster@ames.arc.nasa.gov
  Moffett Field, CA 94035    With Good Mailer:    lamaster@george.arc.nasa.gov 
  Phone:  415/604-6117       

mccalpin@perelandra.cms.udel.edu (John D. McCalpin) (02/14/91)

>On 13 Feb 91 21:20:41 GMT,lamaster@pioneer.arc.nasa.gov (Hugh LaMaster) said:

Hugh> (The Cyber 205 used to have 64 bit "addresses" (actually
Hugh> descriptors).  Regardless of whatever other problems the machine
Hugh> had, the addresses were not a problem.  I note that, though, it
Hugh> only had to do integer arithmetic on 48 bits of the address, not
Hugh> 64.)

To clarify, the Cyber 205 used 48-bit *bit* addresses.   Since almost
all addressing was actually done by 64-bit words, the last 6 bits were
almost invariably 0.  Still, 42 bits of word addressing was found to
be adequate for most programs.  (This equals 45 bits of byte addressing).

The upper 16 bits were used to store the vector length for vector
instructions.  Integer instructions operated only on the lower 48 bits.
Although 48-bit addresses would probably handle most applications for
the next few years, the approach taken by the 205 was not RISCy, to
say the least!
--
John D. McCalpin			mccalpin@perelandra.cms.udel.edu
Assistant Professor			mccalpin@brahms.udel.edu
College of Marine Studies, U. Del.	J.MCCALPIN/OMNET

glew@pdx007.intel.com (Andy Glew) (02/14/91)

With 64 bits of space allocated for addresses, but most programs using
far fewer bits (say 40-48) there will be a lot of programs that stick
something extra into that upper few bits - like a tag - and use masks
or shifts to extract the address.

GNU EMACS already does this - a tag in a 32 bit address. It'll be nice to be
able to use GNU on really large files.


---
This is my own opinion. It is not Intel's.

(God, I hate disclaimers)





--
Andy Glew, glew@ichips.intel.com
Intel Corp., M/S JF1-19, 5200 NE Elam Young Parkway, 
Hillsboro, Oregon 97124-6497

tbray@watsol.waterloo.edu (Tim Bray) (02/15/91)

lamaster@pioneer.arc.nasa.gov (Hugh LaMaster) writes:
>mh2f+@andrew.cmu.edu (Mark Hahn) writes:
>>Just think of the added overhead in pointer-linked structures!
>
>What is the ratio of pointers to other data in typical "small" programs?  

Well, the two application domains I've been involved with in the last 5 years
or so are: (1) Symbolic algebra systems, and (2) Text databases.  In the case
of these programs, you tend to find relatively small tight kernels [see PS]
and massive data structures, which are *very* pointer-dense.  At least 50% in
both cases, probably more on average, especially the text database stuff.

Now both of these application domains are starting to crunch into 32-bit
boundaries on non-exotic real-world problems, so the extra addressing bits
will be extremely welcome.  The price will be a dramatic increase in physical 
memory consumption.

Which is maybe (retch) a good trade-off.

Two questions:
 o as a software developer currently bruising my head against 32-bit issues, 
   should I: 
   (1) bite the bullet and write a segmented scheme which will run 
       portably (if suboptimally) on 32-bit systems, or 
   (2) sit tight and wait for rescue-by-recompilation on the R4000 et al?  

   At this point, economics may favour (1) (retch again).

 o a lot of important software has, for a lot of years, cheated and stolen
   a few bits from 32-bit words for tags and typing and so on.  This
   can produce some *really* elegant effects and allow a lot of data 
   manipulation with efficient integer instructions.  While this should be
   unportable, such software has in fact been widely ported, albeit with
   some pain.  With native 64-bit integer primitives, the temptation to cheat
   and steal (as much as a byte's worth) of tag bits and so on will become
   very great again.  C'mon, that leaves ***56 bits***!!! Particularly given 
   the memory growth noted in the point above.  Commentary?

Hugh LaMaster's view on this:
>Better to waste
>another whole 64 bit word on other stuff if necessary, than to create such
>complications.   8 Bytes is just not that precious anymore.  Only if you
>have arrays of millions of objects should you compromise on simplicity.

But we do, sigh, lots of us do.

Cheers, Tim Bray, Open Text Systems, Waterloo, Ontario

[PS] Here's the output of size(1) on the Maple (symbolic algebra) and Pat (text 
search) kernels.  Both of these programs can blow your memory away effortlessly
with data structures, given a really large problem.  (SPARC binaries)

text	data	bss	dec	hex
393216	57344	13632	464192	71540  maple
204800	16384	576	221760	36240  pat

mh2f+@andrew.cmu.edu (Mark Hahn) (02/15/91)

as I think about it more, 64 bits is a really bad idea.
for people like Hugh LaMaster, it's great, because everyone
seems to be seeing it his way: that 64 bits as inevitable
because 32bits as already obsolete.  note that very few
people deal with large arrays as I have to assume he does.

is it true that 32bits is obsolete?  certainly not on workstations, 
where I'd guess the median is only about 24bits of physical memory,
with maybe 29bits of hard disk.  remember that current CPUs 
only have a throughput of about 24bits, no matter whether you like 
FLOPS or MIPS or strcmp's or pointer dereferences.  I suppose
patient people will disagree...

still, my original post on this subject was attempting to find 
a 64bit rationale for the rest of us, who have more diverse
activities like using skiplists or SML or editors whose internals
are lisp or which represent text as a list of chunks.
where does 64bits make sense?  in a relatively small market:
databases, fileservers, numeric analysis and simulation.
in other words 64bit addresses aren't worth the cost to most users,
even ignoring the the unwashed masses using PCs with ~20bits.

so this leads to the question: is MIPS crazy?  It depends on
whether they can make the BigData people happy while not losing
their major market.  perhaps this is the first sign that they
want to split to serve two product lines.  they obviously have to
watch their step to remain competitive in the 32bit market, 
with SPARC and Moto already seeming to pull ahead.

regards, mark

lamaster@pioneer.arc.nasa.gov (Hugh LaMaster) (02/16/91)

In article <1991Feb14.183045.16468@watdragon.waterloo.edu> tbray@watsol.waterloo.edu (Tim Bray) writes:
>lamaster@pioneer.arc.nasa.gov (Hugh LaMaster) writes:
>>mh2f+@andrew.cmu.edu (Mark Hahn) writes:
>>>Just think of the added overhead in pointer-linked structures!
>>What is the ratio of pointers to other data in typical "small" programs?  

(The answer provided).

>... starting to crunch into 32-bit
>boundaries on non-exotic real-world problems, so the extra addressing bits
>will be extremely welcome.  The price will be a dramatic increase in physical 
>memory consumption.

Mea culpa.  My bright-eyed, bushy-tailed optimism punctured by two examples
which need the large address space, and, are *very* pointer dense.


>Two questions:
> o as a software developer currently bruising my head against 32-bit issues, 
>   should I: 
>   (1) bite the bullet and write a segmented scheme which will run 
>       portably (if suboptimally) on 32-bit systems, or 
:


It appears to me that this would benefit from segmentation at some level.


>   (2) sit tight and wait for rescue-by-recompilation on the R4000 et al?  
:

>>complications.   8 Bytes is just not that precious anymore.  Only if you
>>have arrays of millions of objects should you compromise on simplicity.
>
>But we do, sigh, lots of us do.

This brings up another idea that has appeared in the past, and, may again.

"Segments" (warning, segments mean many different things to different folks) 
have developed a deservedly bad reputation as a way to *extend* an
address space which is *too small*. But, another use of segments is as a way of
efficiently providing *compact* access to a nice large address space.  This
method of access has occasionally been provided.  It may be time to explore
it again, particularly as it may be a way to provide *efficient* access to
tagged/object-oriented/capability/etc type functionality.

While I am convinced that segments are no way to extend an address space which
is too small, in fact, I think that alternative methods of generating addresses
using segment number + offset are, in fact, potentially quite useful.  One of
the problems that people experience with this is the assumption that segments
have to be a fixed, architecture-determined size.  In fact, it is possible
for segments to be an arbitrary power of two in length, with hardware
support.   This is a solution, though, which adds hardware
to the TLB (and therefore is potentially "expensive"- nobody likes to add
real estate to the TLB in single-chip designs).

I think you will be able to do this in software with little
overhead, as long as you can put limits on both the number of objects and
the maximum size of each object.  For example, no more than 2^^32 objects of
no more than 2^^32 bytes in size.  If you can, then the space expansion may
be minimal.  On the other hand, if you can't place such limits,
I think that will pay the price of a full 64 bit address for each object.


>Cheers, Tim Bray, Open Text Systems, Waterloo, Ontario


Cheers,



  Hugh LaMaster, M/S 233-9,  UUCP:                ames!lamaster
  NASA Ames Research Center  Internet:            lamaster@ames.arc.nasa.gov
  Moffett Field, CA 94035    With Good Mailer:    lamaster@george.arc.nasa.gov 
  Phone:  415/604-6117                            #include<std.disclaimer> 

DXB132@psuvm.psu.edu (02/18/91)

In article <GLEW.91Feb14074010@pdx007.intel.com>, glew@pdx007.intel.com (Andy
Glew) says:

>GNU EMACS already does this - a tag in a 32 bit address. It'll be nice to be
>able to use GNU on really large files.

I can't believe it; these forward-thinking MIPS people just announce a
64-bit CPU, and you already want to polute it with hacks from the past. :-)

-- Dan Babcock






>--
>Andy Glew, glew@ichips.intel.com
>Intel Corp., M/S JF1-19, 5200 NE Elam Young Parkway,
>Hillsboro, Oregon 97124-6497

robison@brazil (Arch D. Robison) (02/23/91)

64-bit address = 32-bit network address + 32-bit hardware address.  
We should jump to 128 bits now!  Of course, a 256-bit address can 
individually address every atom in the known universe.

Arch D. Robison				robison@shell.com
Bellaire Research Center 		(713)-663-2240
Shell Development Company