scott@cstvax.UUCP (Scott Larnach) (04/12/86)
Have any of you wizards out there any idea of how much of the time a
shell script spends being interpreted by the shell and how much it
spends actually doing the work? Would a program which turned a shell
script into an equivalent C program (which would handle i/o
redirections, fork/exec the appropriate commands, etc.) usefully
improve the speed of my scripts? Has anybody ever used such a beast?
Is there one in the public domain?
Many Thanks
- Scott
--
** REPLACE THIS NETWORK WITH A BAGFUL OF MAGTAPE **
Scott Larnach Janet: scott@uk.ac.ed.itspna
Edinburgh Unix Support Arpa: scott@itspna.ed.ac.uk
Tel: +44 31 667 1081 x2629 Uucp: scott@cstvax.uucpchris@hwcs.UUCP (04/14/86)
In article <96@cstvax.UUCP> scott@cstvax.UUCP (Scott Larnach) writes: > ... Would a program which turned a shell >script into an equivalent C program (which would handle i/o >redirections, fork/exec the appropriate commands, etc.) usefully >improve the speed of my scripts? I am fairly sure that if your shell has "test" built in then a simple translation of shell control constructs into C will gain you nothing significant on large scripts, since most of the time is spent doing fork/exec. If you have a "clever" translator that knows about a few common "grep", "sed", "awk" and "ls" usages in shell scripts, and puts them directly into C code without exec-ing the corresponding program, you may gain quite a lot more; I don't really think such a translator would be a very good idea, since it would almost inevitably be large, clumsy and as-hoc (and buggy!). As a very rough and ready estimate to the relative load of interpreting control constructs to path-search and fork/exec, I created the following trivial script, "shtest": #!/bin/sh ECHO="$1" for i in 0 1 2 3 4 5 6 7 8 9 ; do for j in 0 1 2 3 4 5 6 7 8 9 ; do for k in 0 1 2 3 4 5 6 7 8 9 ; do $ECHO $i$j$k done done done the following script, "nul.sh": #!/bin/sh exit 0 and the following C program, "nul": main() { exit(0); } and got the following timings (take with the usual pinch of salt, especially "elapsed" since the system was not single-user): user system elapsed shtest : 10.4 2.1 0:18 shtest echo 57.2 306.0 7:17 shtest /bin/echo 52.2 240.9 8:16 shtest nul 57.9 356.4 11:04 shtest nul.sh 100.8 441.9 15:43 Notes: - "." is the LAST directory on my path, so "nul" requires the longest path search. - "echo" is NOT built in to our version of the shell, which is the Bourne shell as distributed with 4.2, without modification. - All commands were run by "time sh shtest COMMAND >/dev/null". - Timings were done on a VAX 11/750 running 4.2bsd; the kernel recognises "#!" at the start of a script as a "magic number" for exec(), which saves a few cycles in starting up a shell script. The conclusions seem clear: - The time spent in shell interpretation is negligible compared to the time taken to exec other programs. - Searching directories to find commands is cheap compared to fork/exec, but expensive compared to interpreting shell control constructs. - For very small scripts, the startup time of the shell compared to a C program is a significant overhead. Hence, if you have a very small script invoked from the inner loop of another script, it might be worth trying to do something; however, it may well be better to embed the inner script directly in its caller than to translate it to C. Of course, there are many other constructs than "for" and simple variable substitution, but I doubt that any of them is an order of magnitude slower than those. I would also expect exactly the same conclusions to hold for ANY other shell used for script-writing. -- Chris Miller, Heriot-Watt University, Edinburgh ...!ukc!hwcs!chris chris@hwcs.uucp chris@cs.hw.ac.uk
gemini@homxb.UUCP (Rick Richardson) (04/16/86)
Scott Larnach writes: > Have any of you wizards out there any idea of how much of the time a > shell script spends being interpreted by the shell and how much it > spends actually doing the work? Would a program which turned a shell > script into an equivalent C program (which would handle i/o > redirections, fork/exec the appropriate commands, etc.) usefully > improve the speed of my scripts? Has anybody ever used such a beast? > Is there one in the public domain? Paul Ruel wrote such a compiler, called 'shacc', and marketed by Concentric Associates (if still in business). Look in an old copy of UNIX World (Review?). If the script does a lot of fork/exec as opposed to control flow, you're not going to see much (any?) of an improvement. Rick Richardson, PC Research, Inc. (201) 922-1134, (201) 834-1378 @ AT&T-CP ..!ihnp4!castor!{rer,pcrat!rer} <--Replies to here, not to homxb!gemini, please. [Disclaimer: I've known Paul for years, and his work is good. I have no connection to Concentric Associates, and neither does Paul, these days]
gwyn@brl-smoke.UUCP (04/21/86)
In article <796@brahma.cs.hw.AC.UK> chris@cs.hw.AC.UK (Chris Miller) writes: >The conclusions seem clear: >- The time spent in shell interpretation is negligible compared to > the time taken to exec other programs. >- Searching directories to find commands is cheap compared to > fork/exec, but expensive compared to interpreting shell control > constructs. >- For very small scripts, the startup time of the shell compared > to a C program is a significant overhead. We have both an Adventure shell and a C shell implemented as Bourne (SVR2) shell scripts, with acceptable performance. The shell is fairly well designed for its application domain (running processes on UNIX), and having just the one program source rather than both source and compiled version is a big win. The shell actually parses the script into an internal form, then acts as an interpreter; I've written other application-specific utilities that work on the same general principle and find that they can run pretty darn fast.
wls@astrovax.UUCP (William L. Sebok) (04/26/86)
In article <157@brl-smoke.ARPA> gwyn@brl.ARPA writes: >We have both an Adventure shell and a C shell implemented as Bourne >(SVR2) shell scripts, with acceptable performance. At least for the 4.2 BSD Bourne shell I don't agree that performance is acceptable for the Adventure shell. I have never gotten very far in the adventure shell, before long I get tired of waiting for a response and quit. There are times it takes many minutes to respond to a command. -- Bill Sebok Princeton University, Astrophysics {allegra,akgua,cbosgd,decvax,ihnp4,noao,philabs,princeton,vax135}!astrovax!wls
gwyn@brl-smoke.ARPA (Doug Gwyn ) (04/28/86)
In article <771@astrovax.UUCP> wls@astrovax.UUCP (William L. Sebok) writes: >In article <157@brl-smoke.ARPA> gwyn@brl.ARPA writes: >>We have both an Adventure shell and a C shell implemented as Bourne >>(SVR2) shell scripts, with acceptable performance. > >At least for the 4.2 BSD Bourne shell I don't agree that performance is >acceptable for the Adventure shell. I have never gotten very far in the >adventure shell, before long I get tired of waiting for a response and quit. >There are times it takes many minutes to respond to a command. As is well known (I thought), the Bourne shell that is shipped with 4.2BSD (a) is rather buggy; (b) is rather slow. That's why I indicated the version of the Bourne shell in my posting. By the way, I recently converted the Adventure shell to use shell functions, which reduces its size somewhat.
henry@utzoo.UUCP (Henry Spencer) (04/28/86)
> >We have both an Adventure shell and a C shell implemented as Bourne > >(SVR2) shell scripts, with acceptable performance. > > At least for the 4.2 BSD Bourne shell I don't agree that performance is > acceptable for the Adventure shell... Doug was referring to modern Bourne shells, e.g. the SVR2 one. The 4.2BSD Bourne shell ought to be in a museum. If you can get licensed for SVR2 at a reasonable price, it's worth it for the shell alone. -- Support the International League For The Derision Henry Spencer @ U of Toronto Zoology Of User-Friendliness! {allegra,ihnp4,decvax,pyramid}!utzoo!henry
wcs@ho95e.UUCP (#Bill_Stewart) (05/08/86)
In article <6637@utzoo.UUCP> henry@utzoo.UUCP (Henry Spencer) writes: >Doug was referring to modern Bourne shells, e.g. the SVR2 one. The 4.2BSD >Bourne shell ought to be in a museum. If you can get licensed for SVR2 at >a reasonable price, it's worth it for the shell alone. Alternatively, if you have a System V (probably System III?) license, get the Korn Shell (ksh) from us. It works on 4.*BSD, System V**, V8, etc. and is the first shell I've seen that did history right. And the V8 addicts don't like it, which may also say something :-) -- # Bill Stewart, AT&T Bell Labs 2G-202, Holmdel NJ 1-201-949-0705 ihnp4!ho95c!wcs