[comp.unix.wizards] #! magic number

chris@mimsy.UUCP (Chris Torek) (06/24/87)

In article <21681@sun.uucp> guy%gorodish@Sun.COM (Guy Harris) writes:
>"#!" was not concocted merely to solve the C shell vs. Bourne shell
>problem.

And indeed, it does more:

	% egrep uucp:: /etc/passwd
	uucp::16:16:UNIX-to-UNIX Copy:/usr/spool/uucppublic:/usr/lib/uucp/recording
	% cat /usr/lib/uucp/recording
	#! /bin/awk NR > 1 { print }

		There is no general-use uucp account on this machine.
		Machines which poll us are assigned individual accounts.
		If you wish to establish a uucp connection, call
		(301) 454-7690 and ask for Fred Blonder or Chris Torek.

	%

`#! /bin/sed 1d' works too.
-- 
In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 454 7690)
Domain:	chris@mimsy.umd.edu	Path:	seismo!mimsy!chris

barnett@vdsvax.steinmetz.UUCP (Bruce G Barnett) (06/25/87)

In article <7180@mimsy.UUCP> chris@mimsy.UUCP (Chris Torek) writes:
>	#! /bin/awk NR > 1 { print }
>
>		[recorded message]

Good example. But I cannot figure out how ELSE to use this feature
with awk. How can I create a t.awk that contains an awk script,
and make it executable? 

That is, in file t.awk, I have

	#!/bin/awk -f t.awk

No matter what I try, I get the "bailing out near line 1" error.

And could someone explain to me why the first line isn't treated as a
comment within the awk script?


-- 
Bruce G. Barnett  (barnett@ge-crd.ARPA) (barnett@steinmetz.UUCP)
-- 
"The difference between a Buddha and an ordinary man is that one knows
the difference and the other does not."

barnett@vdsvax.steinmetz.UUCP (Bruce G Barnett) (06/27/87)

In article <1877@vdsvax.steinmetz.UUCP> I wrote: 
>That is, in file t.awk, I have
>
>	#!/bin/awk -f t.awk
>
>No matter what I try, I get the "bailing out near line 1" error.
 ^^^^^^^^^^^^^^^^^^^^

Yes, I did try #!/bin/awk -f

I STILL get the "bail" error:
Exhibit A, Ultrix 1.2:

csh >cat t.awk
#!/bin/awk -f 
  BEGIN { print "Begin"}
  { 
  print;
  }
  END { print "End"}
csh> awk -f t.awk </dev/null
Begin
End
csh> t.awk
awk: syntax error near line 1
awk: bailing out near line 1  

I also tried "#!/bin/awk -f -"

Does someone have an invocation that WORKS?

Why am I "bailing out"?
-- 
Bruce G. Barnett  (barnett@ge-crd.ARPA) (barnett@steinmetz.UUCP)
-- 
"The difference between a Buddha and an ordinary man is that one knows
the difference and the other does not."

allbery@ncoast.UUCP (Brandon Allbery) (06/27/87)

As quoted from <1877@vdsvax.steinmetz.UUCP> by barnett@vdsvax.steinmetz.UUCP (Bruce G Barnett):
+---------------
| In article <7180@mimsy.UUCP> chris@mimsy.UUCP (Chris Torek) writes:
| Good example. But I cannot figure out how ELSE to use this feature
| with awk. How can I create a t.awk that contains an awk script,
| and make it executable? 
| 
| That is, in file t.awk, I have
| 
| 	#!/bin/awk -f t.awk
| 
| No matter what I try, I get the "bailing out near line 1" error.
+---------------

That isn't quite valid.  Firstly, some systems (not yours, apparently)
require a space between the #! and the command name.

Second, you should not include the program name; the kernel (or bexec if
you got it) adds it automatically.  Both also package everything after the
command name in the #! sequence as a SINGLE argument.  As a result, awk is
seeing the command:

	/bin/awk '-f t.awk' t.awk

which means "for all lines in t.awk which for which the condition "-f t.awk"
is true, print the lines."  Unfortunately, "-f t.awk" is not a valid condition;
it's a syntax error.  Remove the "t.awk" from the top line of your script and
you will be all set.

++Brandon
-- 
     ---- Moderator for comp.sources.misc and comp.binaries.ibm.pc ----
Brandon S. Allbery	<BACKBONE>!cbosgd!ncoast!allbery
aXcess Company		{ames,mit-eddie,harvard,talcott}!necntc!ncoast!allbery
6615 Center St. #A1-105	{well,sun,pyramid,ihnp4}!hoptoad!ncoast!allbery
Mentor, OH 44060-4101	necntc!ncoast!allbery@harvard.HARVARD.EDU (Internet)
+01 216 974 9210	ncoast!allbery@CWRU.EDU (CSnet)

levy@ttrdc.UUCP (Daniel R. Levy) (06/28/87)

In article <7180@mimsy.UUCP>, chris@mimsy.UUCP (Chris Torek) writes:
< In article <21681@sun.uucp> guy%gorodish@Sun.COM (Guy Harris) writes:
< >"#!" was not concocted merely to solve the C shell vs. Bourne shell
< >problem.
< And indeed, it does more:
< 	% egrep uucp:: /etc/passwd
< 	uucp::16:16:UNIX-to-UNIX Copy:/usr/spool/uucppublic:/usr/lib/uucp/recording
< 	% cat /usr/lib/uucp/recording
< 	#! /bin/awk NR > 1 { print }
		[message] 
< 	%
< `#! /bin/sed 1d' works too.

This is like using a cannon to kill a gopher.

Why not

/bin/cat << !
	[message]
!

which will also work on systems other than "Berserkeley" UNIX.
-- 
|------------dan levy------------|  Path: ..!{akgua,homxb,ihnp4,ltuxa,mvuxa,
|         an engihacker @        |		vax135}!ttrdc!ttrda!levy
|    at&t data systems division  |  Disclaimer:  try datclaimer.
|--------skokie, illinois--------|

barnett@vdsvax.steinmetz.UUCP (Bruce G Barnett) (06/29/87)

In article <1895@vdsvax.steinmetz.UUCP> i (Bruce G Barnett) wrote:
>
>	#!/bin/awk -f
>
>No matter what I try, I get the "bailing out near line 1" error.

Thanks for the many cards and letters. 

The problem is another Ultrix 1.2 'feature'. It works properly on Suns,
Pyramids, 4.3bsd, and Ultrix 2.0.

-- 
Bruce G. Barnett  (barnett@ge-crd.ARPA) (barnett@steinmetz.UUCP)

dce@mips.UUCP (David Elliott) (06/29/87)

In article <1788@ttrdc.UUCP> levy@ttrdc.UUCP (Daniel R. Levy) writes:
>In article <7180@mimsy.UUCP>, chris@mimsy.UUCP (Chris Torek) writes:
>< And indeed, it does more:
>< 	% egrep uucp:: /etc/passwd
>< 	uucp::16:16:UNIX-to-UNIX Copy:/usr/spool/uucppublic:/usr/lib/uucp/recording
>< 	% cat /usr/lib/uucp/recording
>< 	#! /bin/awk NR > 1 { print }
><		[message] 
>This is like using a cannon to kill a gopher.
>Why not
>
>/bin/cat << !
>	[message]
>!
>which will also work on systems other than "Berserkeley" UNIX.

What? You mean that your /etc/password will allow you to put multiple
lines in the shell field? What you suggest is that

	uucp::16:..:/bin/cat << !
		[message]

is a valid entry in the password file.

The point was that with #! you can use shell scripts or any other
scripts as login shells or anywhere an executable program can go.
Sure, you could say

	#!/bin/sh
	/bin/cat << !
		[message]
	!

and have it work, but you could not simply say

	/bin/cat << !
		[message]
	!

and have it work unless login(1) executes the script vi sh.

The fact that #!/bin/awk or #!/bin/sed or #!/usr/ucb/tail +2 will
work is due to the generality of the #!, and generality is a big
win.

-- 
David Elliott		{decvax,ucbvax,ihnp4}!decwrl!mips!dce

randy@ncifcrf.UUCP (The Computer Grue) (06/29/87)

In article <1895@vdsvax.steinmetz.UUCP>, barnett@vdsvax.steinmetz.UUCP (Bruce G Barnett) writes:
> Yes, I did try #!/bin/awk -f
> 
> I STILL get the "bail" error:
> Exhibit A, Ultrix 1.2:
> 
> csh >cat t.awk
> #!/bin/awk -f 
>   BEGIN { print "Begin"}
>   { 
>   print;
>   }
>   END { print "End"}
> csh> awk -f t.awk </dev/null
> Begin
> End
> csh> t.awk
> awk: syntax error near line 1
> awk: bailing out near line 1  
> 

    You are not going to f**king believe this.  I got curious enough
  about your question to start fiddling (I use several awk 'shell
  scripts' and they work fine for me) and I think I figured out what
  the problem is.  You have a space (that's right, hex 0x20, char ' ',
  whatever) immediately after your '-f'.  This seems to be a problem
  because the kernel is passing '-f ' (as opposed to '-f', which is what the
  shell would pass it) to awk as it's first argument and it doesn't
  seem to know how to deal with a space immediately after a flag.
  Since the argument parsing is done in the kernel for the 'interpeter
  line' of shell scripts and in the shell for straight command line
  arguments, awk's getting a little hosed when called through the
  'exec' mechanism.

    I confirmed this hypothesis two ways.  First, I deleted the space
  at the end of the first line and Bruce's program runs fine.  That is to
  say, the program:
-------------------------
#!/bin/awk -f
  BEGIN { print "Begin"}
  { 
  print;
  }
  END { print "End"}
---------------------

  works fine for me (check it; no space) where the original program
  does not.  Second, I invoked "awk '-f ' awk.inp" and reproduced the
  syntax error message.

    Weird.  I'm not sure whether I'd call this a bug in the kernel
  processing of the '#!' magic number, or whether awk should be able
  to handle '-f ' as an argument.  Bad synergy.

					-- Randy Smith

-- 
  Randy Smith    @	NCI Supercomputer Facility
  c/o PRI, Inc.		
  PO Box B, Bldng. 430  Phone: (301) 698-5660                  
  Frederick, MD 21701  	Uucp: ...!seismo!elsie!ncifcrf!randy

jpn@teddy.UUCP (John P. Nelson) (06/29/87)

>>No matter what I try, I get the "bailing out near line 1" error.
> ^^^^^^^^^^^^^^^^^^^^
>csh >cat t.awk
>#!/bin/awk -f 

I think the problem is relatively simple.  I used "SUN" cut+paste to grab
the script in question, and I had the same problem.  I then noted that the
#!/bin/awk line had a BLANK at the end of the line.  I have to assume that
when the kernel sees the blank, it inserts a "null" argument.

Anyway, after deleting the extraneous blank, the script worked fine!

levy@ttrdc.UUCP (Daniel R. Levy) (07/01/87)

In article <488@quacky.UUCP>, dce@mips.UUCP (David Elliott) writes:
< In article <1788@ttrdc.UUCP> levy@ttrdc.UUCP (Daniel R. Levy) writes:
< >In article <7180@mimsy.UUCP>, chris@mimsy.UUCP (Chris Torek) writes:
< >< And indeed, it does more:
< >< 	% egrep uucp:: /etc/passwd
< >< 	uucp::16:16:UNIX-to-UNIX Copy:/usr/spool/uucppublic:/usr/lib/uucp/recording
< >< 	% cat /usr/lib/uucp/recording
< >< 	#! /bin/awk NR > 1 { print }
< ><		[message] 
< >This is like using a cannon to kill a gopher.
< >Why not
< >
< >/bin/cat << !
< >	[message]
< >!
< >which will also work on systems other than "Berserkeley" UNIX.
< What? You mean that your /etc/password will allow you to put multiple
< lines in the shell field? What you suggest is that
< 	uucp::16:..:/bin/cat << !
< 		[message]
< is a valid entry in the password file.

No, no, that's not what I meant.  I meant to use a script containing that text.

< The point was that with #! you can use shell scripts or any other
< scripts as login shells or anywhere an executable program can go.

I looked at the source for /bin/login on a SVR2 machine (3B20).  It uses
execlp() to execute the login shell, so it can be a shell script.  (This
is probably a moot point anyhow since System V doesn't interpret "#!".)

< ...but you could not simply say
< 
< 	/bin/cat << !
< 		[message]
< 	!
< 
< and have it work unless login(1) executes the script vi sh.

I also tested a script containing just

/bin/cat << !
	[message]
!

as a "login shell" upon a real live BSD 4.2 machine.  Surprise, surprise,
it worked just dandy.  I also tried putting in a "ps -axww" and it showed the
script being executed by /bin/sh.  I tend to doubt that the machine I performed
this test upon is nonstandard in that respect.

< The fact that #!/bin/awk or #!/bin/sed or #!/usr/ucb/tail +2 will
< work is due to the generality of the #!, and generality is a big
< win.
< -- 
< David Elliott		{decvax,ucbvax,ihnp4}!decwrl!mips!dce

In some cases this IS nice.  But in the example given, it was still like using
a cannon to kill a gopher.
-- 
|------------dan levy------------|  Path: ..!{akgua,homxb,ihnp4,ltuxa,mvuxa,
|         an engihacker @        |		vax135}!ttrdc!ttrda!levy
|    at&t data systems division  |  Disclaimer:  try datclaimer.
|--------skokie, illinois--------|

chris@mimsy.UUCP (Chris Torek) (07/01/87)

In article <4145@teddy.UUCP> jpn@teddy.UUCP (John P. Nelson) writes:
>... I then noted that the #!/bin/awk line had a BLANK at the end of
>the line.  I have to assume that when the kernel sees the blank,
>it inserts a "null" argument.

No, for the kernel code that handles `#!' magic numbers is purposely
extremely simple.  It no longer requires exactly one blank in the
right places (if indeed it ever did so), but it creates at most
one `argv' argument.  The call

	execl("bin/foo", "argv0", "argv1", (char *)0),
	
where file foo begins with `#! /bin/awk -f ', becomes equivalent to

	execl("/bin/awk", "awk", "-f ", "bin/foo", "argv1", (char *)0).

The original argv[0] simply vanishes.  The script I gave earlier
began

	#! /bin/awk { if (NR > 1) print }

This passes the single argument `{ if (NR > 1) print }' to awk.
The implementation is quite economical; it is most certainly *not*
a full shell parser.  If you require a full shell parser, you can
begin your file with `#! /bin/sh'.
-- 
In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 454 7690)
Domain:	chris@mimsy.umd.edu	Path:	seismo!mimsy!chr EP

allbery@ncoast.UUCP (Brandon Allbery) (07/06/87)

As quoted from <4145@teddy.UUCP> by jpn@teddy.UUCP (John P. Nelson):
+---------------
| >>No matter what I try, I get the "bailing out near line 1" error.
| > ^^^^^^^^^^^^^^^^^^^^
| >csh >cat t.awk
| >#!/bin/awk -f 
| 
| I think the problem is relatively simple.  I used "SUN" cut+paste to grab
| the script in question, and I had the same problem.  I then noted that the
| #!/bin/awk line had a BLANK at the end of the line.  I have to assume that
| when the kernel sees the blank, it inserts a "null" argument.
+---------------

Nope.  awk checks for -f with strcmp, but the kernel is bundling the space
into the (single) argument it produces, so the command line ends up being:

/bin/awk "-f " filename

which tells awk to run the program "-f " (which is a syntax error) on the
input file "filename".  Zap the space, the strcmp in awk succeeds, and all
is well.

++Brandon
-- 
       ---- Moderator for comp.sources.misc and comp.binaries.ibm.pc ----
Brandon S. Allbery	<BACKBONE>!cbosgd!ncoast!allbery (NOW!!!!)
aXcess Company		{ames,mit-eddie,harvard,talcott}!necntc!ncoast!allbery
6615 Center St. #A1-105	{well,sun,pyramid,ihnp4}!hoptoad!ncoast!allbery
Mentor, OH 44060-4101	necntc!ncoast!allbery@harvard.HARVARD.EDU (Internet)
+01 216 974 9210	ncoast!allbery@CWRU.EDU (CSnet -- if you dare)
NCOAST ADMIN GROUP	Brandon Allbery on 157/504 (Fidonet/Matrix/whatever)
* ncoast -- Public Access UN*X -- (216) 781-6201, 24 hrs., 300/1200/2400 baud *
 * ncoast is proud to be carrying alt.all -- contact me for more information *

allbery@ncoast.UUCP (Brandon Allbery) (07/06/87)

As quoted from <1792@ttrdc.UUCP> by levy@ttrdc.UUCP (Daniel R. Levy):
+---------------
| In article <488@quacky.UUCP>, dce@mips.UUCP (David Elliott) writes:
| < In article <1788@ttrdc.UUCP> levy@ttrdc.UUCP (Daniel R. Levy) writes:
| < >In article <7180@mimsy.UUCP>, chris@mimsy.UUCP (Chris Torek) writes:
| < >< And indeed, it does more:
| < >< 	% egrep uucp:: /etc/passwd
| < >< 	uucp::16:16:UNIX-to-UNIX Copy:/usr/spool/uucppublic:/usr/lib/uucp/recording
| < >< 	% cat /usr/lib/uucp/recording
| < >< 	#! /bin/awk NR > 1 { print }
| < ><		[message] 
| < >This is like using a cannon to kill a gopher.
| < >Why not
| < >
| < >/bin/cat << !
| < >	[message]
| < >!
| < >which will also work on systems other than "Berserkeley" UNIX.
| < What? You mean that your /etc/password will allow you to put multiple
| < lines in the shell field? What you suggest is that
| < 	uucp::16:..:/bin/cat << !
| < 		[message]
| < is a valid entry in the password file.
| 
| No, no, that's not what I meant.  I meant to use a script containing that text.
+---------------

Unfortunately, while /bin/login uses execlp, /bin/su seems to use execl.  As
a result, I've been forced to make su-able shells by linking /bin/sh to another
name and checking for "-name" in /etc/profile.  (Not that you'd want to do
a "su - uucp".)

Also, using execlp raises the question of how to execute a csh script from
/etc/passwd?  (Which may not mean much to you, but csh is d*mned near the
standard shell for BSD.)

++Brandon
-- 
       ---- Moderator for comp.sources.misc and comp.binaries.ibm.pc ----
Brandon S. Allbery	<BACKBONE>!cbosgd!ncoast!allbery (NOW!!!!)
aXcess Company		{ames,mit-eddie,harvard,talcott}!necntc!ncoast!allbery
6615 Center St. #A1-105	{well,sun,pyramid,ihnp4}!hoptoad!ncoast!allbery
Mentor, OH 44060-4101	necntc!ncoast!allbery@harvard.HARVARD.EDU (Internet)
+01 216 974 9210	ncoast!allbery@CWRU.EDU (CSnet -- if you dare)
NCOAST ADMIN GROUP	Brandon Allbery on 157/504 (Fidonet/Matrix/whatever)
* ncoast -- Public Access UN*X -- (216) 781-6201, 24 hrs., 300/1200/2400 baud *
 * ncoast is proud to be carrying alt.all -- contact me for more information *