[comp.std.c] detecting invalid pointers

Kevin_P_McCarty@cup.portal.com (03/07/89)

Is there any guaranteed way to detect an out of range pointer,
i.e., one which is supposed to point into an array but might not?

For example, a friend had something like

     int  x[TABLESIZE];
     int  *p;
     ...
     checkrange(p, __FILE__, __LINE__);

where

     checkrange(int *p, char *fname, int lineno)
     {
          if ((p < x) || (p >= x+TABLESIZE)) {
               /* error message: out of range pointer */
          }
     }

My first reaction was that one should not rely on plausible behavior
of an out of range pointer in a comparison, since that is undefined.
For example, the null pointer need not pass either comparison.  That's
easy to remedy; append `|| (p == NULL)' to the condition.

It is conceivable, and I can't find anything that would rule it out,
that a non-null pointer p which did not point into x might fail
(p < &x[0]) and/or fail (p >= &x[TABLESIZE]).  Trichotomy can fail
because pointers need not have a global linear order.

My initial response was to recommend what I thought was a stronger,
more reliable test, namely

     if ((p != NULL) && (p >= x) && (p < x+TABLESIZE)
          /* p is in-range */
     else
          /* p is invalid */

But this is little better.  While it is conceivable that if p and q
are non-null and incomparable (don't point into the same array),
none of (p < q), (p == q), (p > q), (p < q+n) holds,  it is harder to
conceive the possibility that (p > q) *and* (p < q+n) could hold, but
I can't find anything to rule that out either.  While perhaps
almost all implementations would behave reasonably here, what
guarantees that one of these comparisons must fail?

A test like
     if ((p != NULL) && (p-x >= 0) && (p-x < TABLESIZE)
is subject to the same doubts.

What's the best way to test this?

Kevin McCarty

henry@utzoo.uucp (Henry Spencer) (03/09/89)

In article <15495@cup.portal.com> Kevin_P_McCarty@cup.portal.com writes:
>Is there any guaranteed way to detect an out of range pointer,
>i.e., one which is supposed to point into an array but might not?

No.  There simply is no way to do this portably, since the results of
pointer comparisons (ignoring NULL for the moment) are well-defined only
within the array.
-- 
The Earth is our mother;       |     Henry Spencer at U of Toronto Zoology
our nine months are up.        | uunet!attcan!utzoo!henry henry@zoo.toronto.edu

barmar@think.COM (Barry Margolin) (03/09/89)

I don't think it is possible to do what you want.  C only defines the
result of pointer less-than and greater-than comparisons when the
pointers are to the same array.

It's easy to imagine an implementation where A < P && P < A+N, yet P
is not a pointer into the array A[N].  If the system is segmented, and
arrays are required to fit within a single segment, the comparison
operations might only compare the offset portion of a pointer.  If A
were at seg1:100, N were 100, and P were seg2:150, the comparison
would be implemented as 100 < 150 && 150 < 200.

Barry Margolin
Thinking Machines Corp.

barmar@think.com
{uunet,harvard}!think!barmar

karl@haddock.ima.isc.com (Karl Heuer) (03/10/89)

In article <15495@cup.portal.com> Kevin_P_McCarty@cup.portal.com writes:
>Is there any guaranteed way to detect an out of range pointer,
>i.e., one which is supposed to point into an array but might not?

Yes (my distinguished colleagues to the contrary notwithstanding):
	int within(void *ptr, void *a, size_t n) {
	    char *p;
	    for (p = (char *)a; p < (char *)a + n; ++p) {
	        if ((char *)ptr == p) return (1);
	    }
	    return (0);
	}
This works because pointer *equality* is well-defined even on pointers into
different arrays.  If you want to do it in constant time rather than linear,
then the answer is No.  However, on any given implementation there ought to be
an unportable way (e.g., with a type pun followed by one or more integer
compares).  So if you absolutely have to do this, just add a bunch of #ifdef's
(and a big comment, in a blinking font) to the within() routine above.

Karl W. Z. Heuer (ima!haddock!karl or karl@haddock.isc.com), The Walking Lint

jsdy@hadron.UUCP (Joseph S. D. Yao) (03/11/89)

In article <15495@cup.portal.com> Kevin_P_McCarty@cup.portal.com writes:
>Is there any guaranteed way to detect an out of range pointer,
>i.e., one which is supposed to point into an array but might not?
>     int  x[TABLESIZE];
>     int  *p;

How about something on the order of:
	if (q != (int *) NULL &&
	    (i = q - x) >= 0 && i < TABLESIZE &&
	    q == &x[i]) {
		...
	}

I don't really think that the first comparison against NULL is nece-
ssary, but feel free to contradict.  (I know I couldn't stop ya.)

	Joe Yao		jsdy@hadron.COM (not yet domainised??)
	hadron!jsdy@{uunet.UU.NET,dtix.ARPA,decuac.DEC.COM}
	Xarc,arinc,att,avatar,blkcat,cos,decuac,\
	dtix,ecogong,empire,gong,grebyn,inco,    \
	insight,kcwc,lepton,lsw,netex,netxcom,    >!hadron!jsdy
	paul,phw5,research,rlgvax,seismo,sms,    /
	smsdpg,sundc,telenet,uunet              /