[comp.unix.i386] SCO externs ... too many?

davis@jc3b21.UUCP (William J Davis) (04/21/89)

Help!!!!

   I am current developing a LARGE application using SCO Xenix 2.3.0
on a Samsung S800 and have observed the following. During the initial 
port of our package from an IBM RT, as well as an AT&T 3b2, the loader
seems unable to resolve large numbers of external references.
Experimentation further proved that if the number of externs was reduced
past some unknown threshold, suddenly the problem would vanish! ;-(

   This solution is less than desired because our system consists of
over 35 binaries, of which I have spent a week getting 4 of them to
load successfully. I have a call into SCO, but they say they can't
help unless they can replicate the problem, short of me sending them
5 meg of libraries ( yes, I said 5 meg!)  ;-(

   Has ANYBODY experienced this problem before and, if so, WHAT do
I do??? Thanx in advance.

   Bill Davis       Work: 813-785-0583  - M-F 0730-1700 EST

P.S. - My net access is limited, so please E-mail or phone your response.
-------------------------------------------------------------------------

nvk@ddsw1.MCS.COM (Norman Kohn) (04/23/89)

In article <631@jc3b21.UUCP> davis@jc3b21.UUCP (William J Davis) writes:
>
>   I am current developing a LARGE application using SCO Xenix 2.3.0...
>the loader seems unable to resolve large numbers of external references...

I encountered a similar problem under microport: there obviously is
some sort of limit to symbol table size.

I addressed this by putting related externs into structs defined
in appropriate header files: e.g.,
struct {int a,b;
	char c,d,e;
	} SAMPLE;

Presumably SAMPLE.a is coded as an offset off SAMPLE, and everything
in the struct gets linked up using only a single extern entry
in the linker symbol table.  Moreover, this simplifies tracking
large groups of externs.  For ease of management, I favor putting
related programs (related by use of specific header files)
in their own libraries when possible...

-- 
Norman Kohn   		| ...ddsw1!nvk!norman
Chicago, Il.		| days/ans svc: (312) 650-6840
			| eves: (312) 373-0564

fnf@estinc.UUCP (Fred Fish) (05/04/89)

In article <631@jc3b21.UUCP> davis@jc3b21.UUCP (William J Davis) writes:
>
>Experimentation further proved that if the number of externs was reduced
>past some unknown threshold, suddenly the problem would vanish! ;-(
>
>I have a call into SCO, but they say they can't
>help unless they can replicate the problem, short of me sending them
>5 meg of libraries ( yes, I said 5 meg!)  ;-(

This aroused my curiosity about what the limits were for the SCO 2.3.1
compiler and linker (actually the 2.2 development system, which is shipped
with 2.3.1).  This program found that I could have up to 4093 of my own
external symbols before overflowing the internal linker table (this is 
suspiciously close to 4096, but then we don't take into account "main"
and library symbols).

======================================================================

/*
 *	A quick hack to check for limits on the number of external
 *	symbols allowed by the compiler or linker.
 *
 *	Does a binary search from 1 to LIMIT to determine the number
 *	of acceptable symbols.  Set LIMIT higher if necessary.
 *	
 *	This only checks for limits in a single module.  To test for
 *	aggregate limits, it is a straightforward extension to create
 *	and use multiple files once the limit for a single file is
 *	found.
 *
 *	Fred Fish  3-May-89
 */

#include <stdio.h>

#define LIMIT 10000

main ()
{
	int low;
	int high;
	int test;

	low = 1;
	high = LIMIT;
	while (high - low > 1) {
		test = low / 2 + high / 2;
		if (test == low) {
			test++;		/* sneak up on it... */
		}
		if (linkcheck (test) == 0) {
			low = test;
		} else {
			high = test;
		}
	}
	printf ("Maximum of %d syms per file\n", low);
	/* Add code here to write multiple files */
	exit (0);
}

int linkcheck (count)
int count;
{
	register int value;
	FILE *file1;
	FILE *file2;

	printf ("Try %d syms,", count);
	fflush (stdout);
	file1 = fopen ("linktest1.c", "w");
	file2 = fopen ("linktest2.c", "w");
	if (file1 == NULL || file2 == NULL) {
		fprintf ("can't open files\n");
		exit (1);
	}
	printf (" writing decls,");
	fflush (stdout);
	for (value = 1; value <= count; value++) {
		fprintf (file1, "extern int v%d;\n", value);
	}
	printf (" writing defs and asgns,");
	fflush (stdout);
	fprintf (file1, "main () {\n");
	for (value = 1; value <= count; value++) {
		fprintf (file1, "v%d = %d;\n", value, value);
		fprintf (file2, "int v%d;\n", value);
	}
	fprintf (file1, "}\n");
	fclose (file1);
	fclose (file2);
	printf (" compiling and linking\n");
	fflush (stdout);
	return (system ("cc -o linktest linktest1.c linktest2.c"));
}

======================================================================
-- 
# Fred Fish, 1835 E. Belmont Drive, Tempe, AZ 85284,  USA
# 1-602-491-0048           asuvax!{nud,mcdphx}!estinc!fnf

jbu@sfsup.UUCP (+Urban J.) (05/12/89)

In article <84@estinc.UUCP> fnf@estinc.UUCP (Fred Fish) writes:
>In article <631@jc3b21.UUCP> davis@jc3b21.UUCP (William J Davis) writes:
>>
>>Experimentation further proved that if the number of externs was reduced
>>past some unknown threshold, suddenly the problem would vanish! ;-(
>>
>>I have a call into SCO, but they say they can't
>>help unless they can replicate the problem, short of me sending them
>>5 meg of libraries ( yes, I said 5 meg!)  ;-(
>
>This aroused my curiosity about what the limits were for the SCO 2.3.1
>compiler and linker (actually the 2.2 development system, which is shipped
>with 2.3.1).  This program found that I could have up to 4093 of my own
>external symbols before overflowing the internal linker table (this is 
>suspiciously close to 4096, but then we don't take into account "main"
>and library symbols).
>

I ran this program on UNIX System V/386 Release 3.2 with the C Software Development
Set 4.1.5 installed.  The original program ran fine upto 10000 symbols.  Therefore
I raised the value to 20000.

The CSDS 4.1.5 had NO compiler problems with a file having 15000 symbols in it.
However, when it tried to ld(1) linktest1.o and linktest2.o (a total of 30000
symbols) the ld failed with:

ld internal error: fatal: symbol table overflow.

So this binary search program ran and ran.  Finally it produced:
Maximum of 10181 syms per file.

This meant that the ld(1) successfully load two object files both containing 10181
sysbols for a total of 20362 symbols!

Fred Fish wrote a real nice program here.

Sincerely,

John Ben Urban

crash@jc3b21.UUCP (Frank J. Edwards) (05/12/89)

>>In article <631@jc3b21.UUCP> davis@jc3b21.UUCP (William J Davis) writes:
>>>
>>>Experimentation further proved that if the number of externs was reduced
>>>past some unknown threshold, suddenly the problem would vanish! ;-(
>>>
>>>I have a call into SCO, but they say they can't
>>>help unless they can replicate the problem, short of me sending them
>>>5 meg of libraries ( yes, I said 5 meg!)  ;-(

As someone who has since spoken with Bill Davis, here is SCO's interpretation
of the problem and how to "fix" it:

Apparently the linker does not generate storage allocations in the bss
segment of a task (under certain conditions) unless the variable is
initialized (which means, essentially, that it's no longer BSS but DATA).
For example,
	int xyzzy = 0;	/* works fine since space MUST be allocated */
and
	int xyzzy;	/* may not work, although compiles okay */

The "solution" then is to set all of your external variables to some
initialization value.  While this is tedious (to say the least!) is
does not harm the portability of the code.  I would guess that the
size of binary is increased, though...

Aside to Bill:  last I heard you were supposed to get a call from SCO
about a (possibly) fixed ld(1) -- has this happened?

Frank "Crash" Edwards
...!uunet!pdn!jc3b21!crash