[comp.unix.internals] executing a stream

barmar@think.com (Barry Margolin) (01/17/91)

In article <1991Jan15.204849@IASTATE.EDU> spam@IASTATE.EDU (Begley Michael L) writes:
>   uncompress -c microemacs|execute   /* uncompress microemacs.Z     */
>                                      /* into a stream, and execute  */

>I've been told that it can't be done because of swapping...
>Can anyone help?

It's doable, but not trivial.  Basically, the execute command would have to
duplicate the functionality of the kernel's exec*() system calls.  It could
fork a process, and then use something like ptrace() to initialize the
process from the contents of stdin.  It would have to recognize all the
appropriate magic numbers, parse executable file headers, locate all the
sections of the input stream (text, bss, data), perhaps fill in the user
data structure with something more useful than "execute", fill in argv,
argc, hardware registers, and set the PC so that execution begins at the
appropriate place.  I don't think it can be written portably.

Swapping isn't a problem; the process will simply page out of the swap
area, rather than directly from the executable (since there isn't an
executable).

Sounds like an interesting project, and it would be a fine example to use
whenever the "how do I find the pathname of the current executable"
question comes up.
--
Barry Margolin, Thinking Machines Corp.

barmar@think.com
{uunet,harvard}!think!barmar

moliver@shadow (Mike Oliver) (01/17/91)

In article <1991Jan16.201910.8646@Think.COM> barmar@think.com (Barry Margolin) writes:
>In article <1991Jan15.204849@IASTATE.EDU> spam@IASTATE.EDU (Begley Michael L) writes:
>>   uncompress -c microemacs|execute   /* uncompress microemacs.Z     */
>>                                      /* into a stream, and execute  */
>
>It's doable, but not trivial.  Basically, the execute command would have to
>duplicate the functionality of the kernel's exec*() system calls.  It could
>fork a process, and then use something like ptrace() to initialize the
>process from the contents of stdin.  It would have to recognize all the
>appropriate magic numbers, parse executable file headers, locate all the
>sections of the input stream (text, bss, data), perhaps fill in the user
>data structure with something more useful than "execute", fill in argv,
>argc, hardware registers, and set the PC so that execution begins at the
>appropriate place.  I don't think it can be written portably.

Why not just create a file under /tmp (say, /tmp/executable.$$), copy
from stdin into that file, then on feof(stdin) simply close() the file
and exec() it.

A better alternative for the tidy-minded would be to fork() a child to
exec() the file and have the parent wait() and unlink() it on the death
of the child's.

Arguments can be supplied to the parent and passed through the exec().

The only tricky part is deciding where to pick up standard input in the
exec()'d program.  /dev/tty is my best guess, with an option to
override it on the command line.

Or am I missing something obvious ?

>                                  [...] it would be a fine example to use
>whenever the "how do I find the pathname of the current executable"
>question comes up.

It would indeed.

Cheers, Mike.

moliver@pyramid.com
{allegra,decwrl,hplabs,munnari,sun,utai,uunet}!pyramid!moliver

barmar@think.com (Barry Margolin) (01/17/91)

In article <141325@pyramid.pyramid.com> moliver@shadow.pyramid.com (Mike Oliver) writes:
>Why not just create a file under /tmp (say, /tmp/executable.$$), copy
>from stdin into that file, then on feof(stdin) simply close() the file
>and exec() it.

I guess I trimmed the original article too much, as yours is the second
response I've seen that said this.  The original poster (who,
unfortunately, didn't put a "Subject" line in his message, so readers may
be having trouble finding it) mentioned this, but dismissed it as
distasteful.  Personally, I think it's better than my ptrace-based
solution, as it is really simple and completely portable; in fact, I was
sort of hoping that my description of the complexity of implementing it
without a temp file would cause him to accept the temp-file solution.
--
Barry Margolin, Thinking Machines Corp.

barmar@think.com
{uunet,harvard}!think!barmar

casper@fwi.uva.nl (Casper H.S. Dik) (01/17/91)

barmar@think.com (Barry Margolin) writes:

>In article <1991Jan15.204849@IASTATE.EDU> spam@IASTATE.EDU (Begley Michael L) writes:
>>   uncompress -c microemacs|execute   /* uncompress microemacs.Z     */
>>                                      /* into a stream, and execute  */

>>I've been told that it can't be done because of swapping...
>>Can anyone help?

>It's doable, but not trivial.  Basically, the execute command would have to
>duplicate the functionality of the kernel's exec*() system calls.  It could
>fork a process, and then use something like ptrace() to initialize the
>process from the contents of stdin.  It would have to recognize all the
>appropriate magic numbers, parse executable file headers, locate all the
>sections of the input stream (text, bss, data), perhaps fill in the user
>data structure with something more useful than "execute", fill in argv,
>argc, hardware registers, and set the PC so that execution begins at the
>appropriate place.  I don't think it can be written portably.

In SunOS 4.x, you could make execute position independent and create the
image using mmap, after relocating the code of execute.
Then you jump to the entry point.

I have one problem with this program: what should the new executable have
for stdin?
--
NOTE: Some machine instructions			|	Casper H.S. Dik
      must be executed on the CPU.		|	casper@fwi.uva.nl
(a manual page on the Gould PowerNode)		|	NIC: !cd151

craig@Veritas.COM (Craig Harmer) (01/18/91)

In article <1991Jan17.062934.10863@Think.COM> barmar@think.com (Barry Margolin) writes:
>In article <141325@pyramid.pyramid.com> moliver@shadow.pyramid.com (Mike Oliver) writes:
>>Why not just create a file under /tmp (say, /tmp/executable.$$), copy
>>from stdin into that file, then on feof(stdin) simply close() the file
>>and exec() it.
>
>I guess I trimmed the original article too much, as yours is the second
>response I've seen that said this.  The original poster (who,
>unfortunately, didn't put a "Subject" line in his message, so readers may
>be having trouble finding it) mentioned this, but dismissed it as
>distasteful.  Personally, I think it's better than my ptrace-based
>solution, as it is really simple and completely portable; in fact, I was
>sort of hoping that my description of the complexity of implementing it
>without a temp file would cause him to accept the temp-file solution.

actually, i think the question was basically philospohical.  to rephrase
it, it would be "why isn't an executable a stream of bytes?", or
"why does an executable need to be a file on disk with magic numbers,
structure, and all that stuff?"

the proposed solutions are to copy the executable to a file, and exec it,
or to duplicate the functionality of exec(), with the ability to run off
a memory image.  why, for example, shouldn't we be able to create a named
pipe, send the file to it, and execute that?

i suppose the answer is that a machine executable is an application, and
the kernel is the interpreter, in the same sense that the shell is an
interpreter for shell scripts.  the kernel (as interpreter) won't work
on a stream of bytes.

-- 
{apple,pyramid,amdahl}!veritas!craig			craig@hoser.veritas.com
(415) 626-6827 (h)					(408) 433-5588 x220 (w)
	[views expressed above shouldn't be taken as 
	Veritas' views, or your views or even as my views]

jat@xavax.com (John Tamplin) (01/26/91)

In principle, it seems like fexec* is missing from the system call interface,
perhaps because it was deemed unreasonable.  There is stat() and fstat(), etc,
which operate on either a file or a stream, but no fexecv().  The kernel
implementation would be a few differences at the start, but the main part
would be identical.  Make the synopsis be:

	fexecv(int fd,char *args[])

-- 
John Tamplin						Xavax
jat@xavax.COM						2104 West Ferry Way
...!uunet!xavax!jat					Huntsville, AL 35801