[comp.unix.wizards] Smail3, SunOS and shared libraries

tron@veritas.uucp (Ronald S. Karr) (08/27/90)

The Smail3 program does not run well on several recent releases of
SunOS, and I don't understand why.  I do not actually have a Sun
myself, so I can't look into the problem directly, though some of the
sites that have Smail3 sources and Sun machines have tried to find
the problem, and so far nobody has reported a solution.

The problem is this:  Smail3 cannot exec programs that require
shared libraries, if the smail binary is setuid to root, or is run
by root.  One time, over the phone, I talked someone through the
process of getting smail to exec an interactive shell by mailing
a message through smail.  This worked, but only commands without
shared libraries could be executed.

I can only think of a few things in the smail sources that might
cause these problems, and so everyone that I have talked to about
this believes that these should cause a problem.  These are the
things that may (even remotely conceivably) be related:

	1.  the environment is completely reinitialized.

	2.  all file descriptors greater than 2 are closed, both
	    at startup and before execing something in a forked
	    process.

	3.  setgid() and setuid() are used to set the effective
	    (and hopefully the real) user and group IDs before
	    execing another program.

I would very much appreciate a response from anyone who has an idea
on what the problem is and how it can be fixed.

One last note:  Smail3 runs just fine under SVR4.0, which has a very
similar (though not identical) shared library structure.
-- 
	tron |-<=>-|		ARPAnet:  veritas!tron@apple.com
      tron@veritas.com		UUCPnet:  {amdahl,apple,pyramid}!veritas!tron

terry@eece.ksu.edu (Terry Hull) (08/28/90)

tron@veritas.uucp (Ronald S. Karr) writes:

>The Smail3 program does not run well on several recent releases of
>SunOS, and I don't understand why.  I do not actually have a Sun
>myself, so I can't look into the problem directly, though some of the
>sites that have Smail3 sources and Sun machines have tried to find
>the problem, and so far nobody has reported a solution.

>The problem is this:  Smail3 cannot exec programs that require
>shared libraries, if the smail binary is setuid to root, or is run
>by root.  One time, over the phone, I talked someone through the
>process of getting smail to exec an interactive shell by mailing
>a message through smail.  This worked, but only commands without
>shared libraries could be executed.

Hmmm.   If my memory serves me correctly, executables that are suid
root and linked with shared libraries, must have those libraries in
/usr/lib or /lib.  Could it be these executables are trying to find
shared libraries in /usr/local/lib?  
--
Terry Hull 
Department of Electrical and Computer Engineering, Kansas State University
Work:  terry@eece.ksu.edu, rutgers!ksuvax1!eecea!terry

guy@auspex.auspex.com (Guy Harris) (08/29/90)

>Hmmm.   If my memory serves me correctly, executables that are suid
>root and linked with shared libraries, must have those libraries in
>/usr/lib or /lib.  Could it be these executables are trying to find
>shared libraries in /usr/local/lib?  

If they are, this shouldn't be a problem, as "/usr/local/lib" is also
considered an OK directory by SunOS's run-time loader.

Perhaps S5R4 has a more flexible way of specifying the list of OK
directories for shared libraries (they're hardcoded into "ld.so" in
SunOS), or perhaps when the program was built under S5R4 it wasn't
picking up any shared libraries from non-OK directories?