rfg@NCD.COM (Ron Guilmette) (12/22/90)
In article <CLINE.90Dec21171912@cheetah.ece.clarkson.edu> cline@sun.soe.clarkson.edu (Marshall Cline) writes: > >So the basic problem is that C++ pointers aren't ``smart'' enough. Ie: >a copy-collect GC routine moves objects, and the pointers are machine >addresses rather than some higher level `logical' pointers, so the >machine addresses become invalid. > >A possible solution is to ban `Cons*' and replace it with ConsPtr, >which is a smart pointer... Great! Now how do I "ban" the use of the type Cons* ??? Marshall's idea of "banning" the use of regular pointers is good one except for one thing. The C++ language (as currently defined) does not provide any form of support for "banning" the use of specific pointer types. This notion of banning pointers to "managed" or "controlled" classes seems to be the general answer for a lot of the questions which arise when `smart pointers' are used. It seems that `smart pointers' work wonderfully, but only so long as nobody on your project team forgets what's going on and then (accidently) codes up a declaration for a T* object rather than a smart_pointer_to_T object. Of course we could even deal with *those* cases if we were allowed to overload such things as operator= for T*'s, but we can't because the language rules say that we can't. (The justification for this is a bit esoteric, and has to do with language mutability. But I digress...) Anyway, given that we can't defend ourselves (e.g. by overloading operator= for pointer types) from the random dummies who may happen to get assigned to our project teams at any given instant in time, wouldn't it be nice if (as an alternative) we could have some way of instructing the compiler to absolutely "ban" people from declaring objects of type T* for cases where we want Ts to be accessed only via values of some `smart_pointer_to_T' type? Rather than inventing yet another keyword (banned?) how about something like this: class T; class smart_pointer_to_T { /* ... magical stuff, possibly including ref counts, etc. */ class T* pointer; public: smart_pointer_to_T (); smart_pointer_to_T& operator= (smart_pointer_to_T); T& operator * (); }; protected class T { /* ... innards of T ... */ }; The idea here is that once we declare `class T' to be `protected', then from that point forward (in the compilation) it becomes impossible to use the type T* in any way, shape, or form. In particular, it would be impossible to declare objects to be of type T* or objects of type T**, or objects of type T*[] or anthing like that. It would also become illegal to cast anything to (T*) or (T**) or (T*[]) or (T*&) or anything like that. Furthermore, it would be impossible to use, or even to generate values of type T*. Assume for the moment that C++ already included the concept of "protected" classes (as described herein). Looking in my crystal ball, I see one very curious side-effect of making a given class "protected". Consider: class T T_array[10]; ... T_array[i] ... // ERROR: value of type T* generated/used The indicated line would contain an error because in C and C++ any reference to an array element gets converted (internally in the compiler) to it's simpler equivalent. For an array reference like `a[i]' the compiler effectively converts this to `*(a+i)' where the evaluation of the sub-expression `a' yields a pointer to the first element of the array `a'. I'm not at all sure whether this curious side-effect of declaring a class to be "protected" would be good or bad in practice. I can see where it might at first seem to be an irritant, but I have a funny feeling that it would end up *aiding* in the enforcement of the programmer's wish to insure that all accesses to type T object go (indirectly) through some object of type `smart_pointer_to_T'. In summary, I have believed for quite some time that the concept of `smart pointers' is a terrific one, but that C++ desperately needs some way to ENFORCE their universal use for certain "controlled" types. I believe that allowing classes to be declared as "protected" could be a good way to provide such enforcement because it should be relatively simple to implement in actual C++ compilers. After all, it involves no new keywords and no new run-time semantics. All it does is to provide the programmer with a convenient way to tell the compiler to treat a slightly larger class of things as compile-time errors. I welcome discussion of the idea of "protected" classes. If people have no objection to the idea, I may just decide to ask x3j16 to consider it. I'd like to get some feedback first however. P.S. When I spoke earlier about "random dummies" getting assigned to project teams, I was definitely *NOT* refering to anybody here at NCD. We don't have any dummies here at all... random or otherwise. Everybody I work with here is incredibly bright. Unfortunately, I cannot say the same thing about *all* of the places that I have worked over the years. :-( -- // Ron Guilmette - C++ Entomologist // Internet: rfg@ncd.com uucp: ...uunet!lupine!rfg // Motto: If it sticks, force it. If it breaks, it needed replacing anyway.
rfg@lupine.ncd.com (Ron Guilmette) (01/13/91)
Well, I'm gratified that there has been so much discussion on this
question of how to deal with the "smart pointer problem". At the very
least this should indicate to all concerned that this is an important
issue and is worthy of careful consideration (in particular by x3j16).
To recap (for those of you who were sleeping) the "smart pointer problem"
is just this:
If you have a class type T and another class type smart_pointer_to_T
there is currently no way in C++ to force all "pointers" to type T
objects (except those within some limited contexts) to be
smart_pointer_to_T type values rather than T* type values.
It may somtimes be important to enforce such a restriction on
T*, either because your garbage collection scheme requires it
or for some other reason.
The basic problem is to prevent values of type T* from "leaking"
out of a certain region of your source code.
Here is a (perhaps biased) summary of the highlights of the discuss so far.
In a separate and subsequent message, I offer my own (definitely biased)
analysis of the state of the discussion and consider yet another solution.
---------------------------------
Marshall Cline really started this thread by mentioning a programming
problem in which it would be useful to "ban" the use of stupid pointers
(perhaps not entirely, but at least over some part of the code in a given
program).
I proposed a combination of new syntax/semantics to allow the programmer
to say (in effect) that the use of values of type T* was restricted to
only some certain limited lexical contexts. The idea rested on what I
called "protected" class types. Subsequently I amended this proposal.
After that, Marshall Cline (cline@cheetah.ece.clarkson.edu) proposed that
the solution to this problem was already available within the current
definitions of the C++ language, via the judicious use of nested classes.
(More on this later).
Then Robert C. (Bob) Martin (rmartin@clear.com) proposed an alternative
to my proposal in which pointer types themselves would be allowed to be
treated as classes, thus allowing class definitions like:
class T* { ... } ;
where the '...' could be replaced with constructors, destructors, type
convertors, and all sorts of other data members, member functions,
operators (including operator=) and what-not.
Later on, Bob amended his proposal and cross-posted it to comp.std.c++
for additional comments.
Henry J. Cobb (hcobb@ccwf.cc.utexas.edu) proposed simply making all of the
constructors for the type T private so that they would be difficult to
create at ramdom (unrestricted) places. (I assume that he also would
suggest making the class smart_pointer_to_T a friend of type T so that
at least smart_pointer_to_T could create T's.)
Jeremy Grodberg (jgro@lia.com) also independently proposed the same
thing that Bob Martin proposed (i.e. allowing T* to be declared as its
own class type and to have operators and what-not defined for it.
Jeremy Grodberg (jgro@lia.com) also noted that there is an important
(and currently uncontrolable) source of "leakage" of regular pointers
built-in to the current definition of the C++ language, i.e. doing
a new() on an array of T's always returns a T* because the language rules
say that new() for an array always uses the global new() rather than
any class-specific new() operator.
Andrew Ginter (gintera@cpsc.ucalgary.ca) suggested that any solution would
have to take into account issues relating to pointers returned from functions,
references (to pointers?), the `this' pointer, and temporaries generated
during expression evaluation (even where their ordering of creation &
destruction may not be known). Furthermore, he suggested that data
members of class objects (which he refered to as "instance variables" in
the Smalltalk tradition of nomenclature) could be a cause for concern
if people were allowed to randomly take their addresses. It is apparent
from this that Andrew G. is not aware of the (relatively new) C++ features
with respect to "pointer-to-members".
Tim Atkins (tma@osc.UUCP) seemed to feel that if you could not control
*all* of the pointers to T everywhere in the program (and make them
all the smart kind) then you would still have a problem. He also
alluded to some (unspecified) problems which he believed might arise
in the generated C code (because it might make use of T* values in
uncontrolled ways behind the back of the C++ programmer). In particular,
he seemed to feel that T* values might end up in registers, and that this
could cause problems when and if a garbage collector needed to relocate
the pointer at object(s).
Piercarlo Grandi (pcg@cs.aber.ac.uk) said that the solution to this problem
required a "capability" language and that C++ was not such a language.
Jim ADCOCK (jimad@microsoft.UUCP) said that the solution to this problem
required a new set of features relating to "metaclasses".rfg@lupine.ncd.com (Ron Guilmette) (01/13/91)
Now for my detailed comments on the "smart pointer problem" discussion
so far. (This is where it really starts to get biased! :-)
--------------------------------------------------------------------------
I believe that the concerns expressed by Andrew Ginter are, for the most
part, non-issues. I don't see where functions which return pointers
(either smart or stupid ones) need to cause us any special concerns.
Likewise for temporaries and expression evaluation ordering. The `this'
pointer is worthy of note only in that it must have type T* and thus,
the type T* should be unrestricted wherever `this' is accessible.
Taking the address of a data member of an object of type T need not cause
us any special concer either because the value yielded by this use of the
unary & operator will be of some pointer-to-member-of-T type, which cannot
be subsequently be used in isolation. Ratherany such pointer-to-member-
of-T may only be used in conjunction with honest-to-goodness pointers to
objects of the type T and if these rae maintained correctly than all will
work out just fine.
Likewise, Tim Atkins concerns are (I believe) misplaced. I don't think
that it is necessary to have *all* pointers to some type T be smart in
order for a program containing objects of type T to be useful. Quite
the contrary, it seems to me that for any pointed-at type (T) you may
want to use smart pointers (to T) in most places and you will absolutely
have to use stupid pointers to T in certain (limited) places. Additionally,
I don't see where low level implementation-specific details (e.g. the
code that cfront generates) needs to enter into this discussion unless
cfront has bugs that become aparent when we are fiddling with smart
(or stupid) pointers. The issue of T*'s in registers also seems
unrelated, unless of course our garbage collector can be triggered into
action asynchronously (e.g. as the result of a signal). In that case,
it may be wise to declare all of our stupid pointers to be volatile
(so that we don't get into memory/register synchronization problems)
but that is all unrelated to the point of this discussion.
I believe that both Peter Grandi and Jim Adcock are saying that we need
to restrict the use of the type T* at run-time via run-time mechanisms.
If so, I disagree with both of them. I feel that we ought to be able
to do something at compile-time where the performance cost is not so high
as it is for things done at run-time.
Henry Cobb's idea to make all constructors for type T private is somewhat
similar to Marshall Cline's suggestion of nesting the declaration (and
definition) of the class T within a smart_pointer_to_T class. In both
cases, the idea seems to be to restrict the ability to create objects
of type T to some particular (limited) set of lexical scopes (all of
which are under complete control of the smart_pointer_to_T type).
To varying degrees, these two proposals solve the "smart pointer
problem" by making the type T unknown to the outside world. (In the case
of Henry's proposal, the whole program could at least say `sizeof(T)'
whereas in Marshall's proposal, even that would be illegal outside of
the encapsulating outer class.)
Anyway, these two proposals succeed by hiding the type T from those who
would attempt to use it directly, and by forcing such potential users
to ask for assistance from the smart_pointer_to_T type in order to
do anything (including creation and destruction) with an object of type T.
These solutions have definite merits, but there is a downside to hiding
the type T. (More on this later.)
The solution proposed by Bob Martin and (independently) also by Jeremy
Grodberg to allow the type T* to be treated like a class (which can
be declared and which can have member functions and operators defined
for it) is clever and I had myself considered it, however I fear that
Bjarne will never like it. The reason? Well, it makes the language
"mutable" (in Stroustrup's terms). One early (and related) idea
which I had some time ago for solving this "smart pointer" problem
was to allow stuff like:
T*& operator= (T*, T*&);
T operator* (T*);
In effect, I wanted to let the user just redefine the meaning of = and
(unary) * for plain old pointer types. If you could do that, then you
could be in complete control of all operations done with stupid pointers.
That idea was almost the same as allowing:
class T* {
public:
T*& operator= (T*&);
T operator* ();
};
But in both cases, you are allowing the user to change the existing
meaning of things whose meaning is already well defined in the language
(e.g. the meaning of unary * when applied to a pointer type value).
Bjarne doesn't want to open that Pandora's box. I tend to feel that
this one important case (of pointer types) might warrant a bit of
"mutability" being allowed to stick its nose into the tent, but it
doesn't much matter what I think. I doubt that Bjarne will have any
part of it.
Of all of these ideas, I think that I like Marshall Cline's the best.
It certainly has good prospects of being implemented widely so that we
can all start to use it soon. After all, it relies only on features of
C++ which are already described in current drafts of the x3j16 working
documents! In effect, nested classes are already "in" the standard.
(I hope nobody in x3j16 kills me for having said that.)
Likewise, Henry Cobb's idea (to make all constructors for T private
and then to just make functions and classes which actually have to
create T's into friends of T) is a good solution which ought to work
even with current implementations.
I do see some problems with these two ideas however. First and foremost,
by using either of these approaches, I have to give up the ability
(which I would otherwise have) to simply declare an object of type T
as a storage-class `static' file-scope variable, or as storage-class
`auto' variable (local to a function) or even as a member.
I don't like that one bit! Just because I want the use of T*'s to be
to be restricted does not mean that I also want to be restricted in
what I can do with a T. Gosh darn it! I want my cake and I want to
eat it too!
Another problem with both Marshall's idea and with Henry's idea is that
they both require me to put the entire *definition* of the (controlled)
class T into header files where I don't even want it to be! That slows
down compilation unnecessarily (which irks me). For example, with Henry's
proposal, I have to put this into my header file:
smart_tp.h:
---------------------------------------------------------------
class T {
/* ... the complete definition of T ...*/
friend class smart_pointer_to_T;
};
class smart_pointer_to_T {
/* ... definition of smart_pointer_to_T ... */
};
---------------------------------------------------------------
Here, both definitions of both classes have to be scanned and compiled
for each .C file which includes "smart_tp.h". Many of these may not even
need to know *any* of the details of the definition of class T.
Likewise, for Marshall's proposal, I need:
smart_tp.h:
---------------------------------------------------------------
class smart_pointer_to_T {
class T {
/* ... the complete definition of T ...*/
};
/* ... definition of smart_pointer_to_T ... */
};
---------------------------------------------------------------
Which is equally wasteful of compile time.
Now somebody else was asking over in comp.std.c++ if it was legal to
incompletely declare a nested class, so that you could have (for example):
smart_tp.h:
---------------------------------------------------------------
class smart_pointer_to_T {
class T; /* incomplete declaration of T */
/* ... definition of smart_pointer_to_T ... */
};
---------------------------------------------------------------
and then later on in a different file:
complete_t.C:
---------------------------------------------------------------
#include "smart_tp.h"
class smart_pointer_to_T::T {
/* completion of type smart_pointer_to_T::T */
};
---------------------------------------------------------------
In my opinion, that would be "way cool" if you could do that, but I don't
think that it is legal. Furthermore, even if it is legal, it only
provides a way of eliminating one of my two objections to Marshall's
proposed solution to the "smart pointer problem". The other (more
important) objection still remains. You still couldn't declare T
objects all over the place. You could only created them where the
smart_pointer_to_T type would let you (probably only in the heap).
My initial proposal was intended solve the "smart pointer problem"
while keeping the language "immutable", allowing declarations of T
objects in most places, and avoiding any need to have a complete
definition of the type T preceed the definition of the type
smart_pointer_to_T.
I believe that my proposal did all that, but I'm now starting to wonder
if it was really such a hot idea after all.
My proposal simply provided a means for telling the compiler that (in
certain contexts) it sould treat uses of type T* values as illegal
(thus forcing the user to use the smart pointer type in those contexts
instead).
Perhaps I grabbed the problem by the wrong end.
I now believe that it might be equally effective to simply make it
impossible to even generate a valid (non-null) stupid pointer-to-T
value in certain contexts. Obviously, if you can prevent valid
values of type T* from leaking out into some area then you don't
even need to worry about whether or not operations on T*'s are
restricted (over that area) or not.
Obviously, for a class type T, you can overload operator& (either as
a member function of T or as a global function taking a T&).
That right there puts you in control of most of the cases where a T*
could potentially be generated.
Unfortunately, there are others that you (currently) can't control.
As Jeremy Grodberg (jgro@lia.com) noted, the language rules currently say
that if you new() an array of objects of some type T, the global operator
new is invoked for this regardless of whether or not the type T has its
own class-specific operator new() defined. As a result, whenever you
new() an array of T, you'll get back a value of type T* even if you
would have preferd getting back a value of type smart_pointer_to_T.
This is a one means by which which unwanted (but valid and non-void)
values of type T* may leak into some context. This leakage is very bad
and it ought to be rectified by x3j16.
Also, there is one more leakage problem. Given some local or global
variable called `ta' of type array of T, the following expression
yields a value of type T* even if the type T has its own class-specific
operator& defined for it:
ta
That's it! The name of an array is generally converted (implicitly) into
a pointer to the zeroth member of that array. This implicit conversion
currently circumvents any class-specific operator& definition (if one
is present) for the class T and allows values of type T* to leak into
contexts where they may not be welcomed.
If both of these unfortunate leaks in the language could be plugged, we
might be able to achieve really water-tight "safe" smart pointer types
just by overloading operator& (and having it yield a smart pointer type)
for some "controlled" type T.
Both leaks could be easily plugged while doing little harm to the existing
language.
For the first leak, it would be easy enough to say that a class-specific
operator new() for a class T is called whenever a single object *or* an
array of objects of type T is new'ed. Such operators could then be defined
by the user to return some smart pointer type.
For the second leak, we could simply redefine the semantics of "array-name"
(where "array-name" names an array of objects of some class type) to be
equivalent to invoking (implicitly) an applicable operator& (either member
or global) on the zeroth element of the array.
There now. That was simple, eh?
Note that by plugging these two leaks, we have not destroyed the user's
ability to declare objects of type T (or even objects of type T*)
but what we have done is to give the user all of the tools he/she needs
in order to insure that no useful values of type T* (other than NULL)
ever leak into a given area (where they might be misused).