[comp.sys.hp] Demand-loaded vs. Shared binaries

hounsell@nmerh53.BNR.CA (Rob Hounsell) (06/21/91)

Folks,

  We are in the middle of a debate about the exact definition of demand-loaded
versus shared binaries on our HP-UX 9000/300 and 9000/800 computers.

  Looking at the manuals on Laserrom, it states that only the HP300 series
support demand-loaded code, and that the series 800 (HP-PA) does not. Both
models support shared (and non-shared?) code.

  This brings up several questions, which I hope the gurus at HP can answer.
The Response Center was of some help...

1. If the 800s don't support demand-loaded paging, why does chatr allow you to
   change the attribute to demand-loaded?

2. If the monitor program is run on the 800s, why does it show paging stats in
   the virtual memory screen, if it doesn't support demand-loading? Does it 
   still use pages to swap in the entire shared code?

3. Can a binary be BOTH demand-loaded and shared? Does setting the demand-load
   flag prevent the code from being shared? Conversely, does setting the shared
   flag imply it is demand-loaded as well??? 

4. What is the result if you set 800 code to be demand-loaded? What does the
   loader do then?

  Based on looking at our machines, it doesn't seem possible that they can run
without doing some kind of paging instead of swapping. Can anyone clear this
up??

Thanks in advance,

Rob
+-----------------------------------------------------------------------------+
| Rob Hounsell                           UUNET:  ...!chekov!hounsell          |
| NT Product Support Technology          BNR WAN:  hounsell@nmerh53           |
|                                        PHONE: (613) 765-2904                |
|                                        ESN: 395-2904                        |
+-----------------------------------------------------------------------------+

mjs@hpfcso.FC.HP.COM (Marc Sabatella) (06/25/91)

>  We are in the middle of a debate about the exact definition of demand-loaded
>versus shared binaries on our HP-UX 9000/300 and 9000/800 computers.

I'll try to clear things up a little:

HP-UX binaries come in three flavors: EXEC_MAGIC, SHARE_MAGIC, and
DEMAND_MAGIC.  This is determined by the magic number in the first few bytes of
the executable, and in some cases by the padding done around the various
sections of the file.  The basic scoop is:

EXEC_MAGIC - not actually directly executable on 800; executable but not shared
             on 300 - in fact, the text segment is not even write-protected.
SHARE_MAGIC - executable and text is shared on both architectures.  On some
              OS versions on the 800, the text segment may actually be demand
              loaded.
DEMAND_MAGIC - executable and text is shared on both architectures; plus text
               segment is demand loaded.  On some OS versions on the 800, the
               data segment may also be loaded on demand.

With 8.0, there is also SHL_MAGIC, which is basically just like DEMAND_MAGIC
except it is not directly executable, but is rather usable as a shared library.

Basically then, the implication that the 800 does not support demand loading is
incorrect.  chatr(1) simply changes the magic number; there is no concept of
separately settable attributes.  But in particular, DEMAND_MAGIC always implies
both demand loading and sharing; SHARE_MAGIC always implies at least sharing.

--------------
Marc Sabatella (marc@hpmonk.fc.hp.com)
Disclaimers:
	2 + 2 = 3, for suitably small values of 2
	Bill and Dave may not always agree with me

jim@tiamat.fsc.com ( IT Manager) (06/25/91)

In article <4670@bnr-rsc.UUCP>, hounsell@nmerh53.BNR.CA (Rob Hounsell) writes:
> Folks,
> 
>   We are in the middle of a debate about the exact definition of demand-loaded
> versus shared binaries on our HP-UX 9000/300 and 9000/800 computers.
> 
>   Looking at the manuals on Laserrom, it states that only the HP300 series
> support demand-loaded code, and that the series 800 (HP-PA) does not. Both
> models support shared (and non-shared?) code.

At the Software Release Planning session in attended at an HP office in
Knoxville (good meeting, but not enough *marketing* info, such as what
pieces 8.0 is divided up into, what all the new pieces will cost, how support
contracts will change as a result, etc), the explanation of shared-mem vs.
demand-loading was:

A program linked only as "shared-library", would have all necessary shared-
library segments linked in at run-time.  Thus, program startup is affected,
and all the linking goes on.  Also, run-time memory usage is affected, since
the program gets on the library code, even if some of it is NEVER used (e.g.
a program that uses printf() only when there's an error message to print).

A program designated as "demand-loaded" will NOT get all required library
routines (or whole libraries, I'm still unclear on exactly what gets linked)
linked at run-time.  Instead, some things will not get linked until it is
actually needed.  This is supposed to reduce program startup time, since
there is less initial linking.

There was no indication given that both would not work on the 800's.  We have
only 800 machines, and I was always asking specific 800 questions (I have
learned that it is good to use the labels "800 HP UX 8.0" and "300/400 HP UX
8.0" since there are several indications that although they are equivalent
functionally, they are very different in terms of the packaging, what is
included in each module, and the licensing/support strategies).  The
presentor never refuted when I asked, specifically, if what he talked about
would work on the 800.
------------- 
James B. O'Connor			jim@tiamat.fsc.com
Ahlstrom Filtration, Inc.		615/821-4022 x. 651

mjs@hpfcso.FC.HP.COM (Marc Sabatella) (06/25/91)

>A program linked only as "shared-library", would have all necessary shared-
>library segments linked in at run-time.  Thus, program startup is affected,
>and all the linking goes on.  Also, run-time memory usage is affected, since
>the program gets on the library code, even if some of it is NEVER used (e.g.
>a program that uses printf() only when there's an error message to print).
>
>A program designated as "demand-loaded" will NOT get all required library
>routines (or whole libraries, I'm still unclear on exactly what gets linked)
>linked at run-time.  Instead, some things will not get linked until it is
>actually needed.  This is supposed to reduce program startup time, since
>there is less initial linking.

These are two entirely different concepts.

Shared executable versus demand magic executables work as I described
previously.  Entirely orthogonal to this is whether you link with shared or
archive libraries - ie, you can link with a shared library and still specify
either shared or demand load for the executable.  The default is to link
against shared libraries, and the default for SHARE_MAGIC / DEMAND_MAGIC is
not specified.  On the 300/400, SHARE_MAGIC is chosen if the executable is
"small enough", where small enough changes from release to release depending on
the characteristics of the VM system.  Demand loaded files contain an average
of 6K of extra padding which can be significant in a file of around 10K or so.
And with shared libraries, most simple programs will be smaller than that.

With shared libraries, the code and data from the library is not copied into
your executable file, so the a.out is much smaller.  The libraries are attached
to your process at run time.  Shared libraries are demand loaded in that the
attach operation merely sets up some pointers and reserves swap space for the
data; no actual code or data is read into memory.  Symbolic linkages into
shared libraries are also resolved at run time, but procedure references are
only bound when the function is actually called.  The actual code and data are
loaded into memory page by page on demand.  The code is shared between
programs that use the library, and the data is copy on write.

Thus there is some startup cost associated with shared libraries (measured in
the tens of milliseconds); but you won't be using more memory except in the
most degenerate of cases.

--------------
Marc Sabatella (marc@hpmonk.fc.hp.com)
Disclaimers:
	2 + 2 = 3, for suitably small values of 2
	Bill and Dave may not always agree with me