[comp.unix.shell] sed 's/foobar/$string/g'.... can't do this?

eddjp@edi386.UUCP ( Dewey Paciaffi ) (01/17/91)

In article <1991Jan17.003856.469@unicorn.cc.wwu.edu> n8743196@unicorn.cc.wwu.edu (Jeff Wandling) writes:
-
-I tried:
-
-set string = newfoobar
-cat file | sed 's/foobar/$string/g' ....
-
-Is there a way for sed to use something like `echo $string` instead 
-of "$string" ??
-

You might try this :

	string=newfoobar
	cat file | sed 's/foobar/'$string'/g' ...

Note that you don't want to use the set command to set the variable,
and by using the additional single quotes in the sed command, the shell
can replace the $string variable.
-- 
Dewey Paciaffi           ...!uunet!edi386!eddjp

]) (01/22/91)

>>In article <1991Jan17.003856.469@unicorn.cc.wwu.edu> n8743196@unicorn.cc.wwu.edu (Jeff Wandling) writes:
>>-
>>-I tried:
>>-
>>-set string = newfoobar
>>-cat file | sed 's/foobar/$string/g' ....
>>-
>
>>You might try this :
>
>>	string=newfoobar
>>	cat file | sed 's/foobar/'$string'/g' ...
>

I usually do it like:

	string=newfoobar
	sed "s/foobar/$string/g" file

which saves a fork/exec and pipe for cat.  If you really don't want to
specify the file as an arg to sed,

	sed "s/foobar/$string/g" < file

If the matched string (foobar in the example) needs to include a shell-
special character, obviously you'd need to escape it.  For instance,
if foobar is double-quote delimited and so must the replacement text be:

	string='"newfoobar"'
	sed "s/\"foobar\"/$string/g" < file

The quotes around the definition of string keep the double-quotes from
being evaluated, and the double-quotes around the sed command keep them
from be evaluated there, too.  I think these are easier to read than the
composite string

	sed 's/foobar/'$string'/g'

and safer, too, because the value stored in string won't be evaluated
by the shell if it contains metacharacters.  For example, if string was
set to 'newfoobar?*' (without the contained double-quotes) and the files
newfoobar12 and newfoobar13 existed, sed would replace 'foobar' with
'newfoobar12 newfoobar13' if called 's/foobar/'$string'/g' , making its
editing PWD-dependent.

...Kris
-- 
Kristopher Stephens, | (408-746-6047) | krs@uts.amdahl.com | KC6DFS
Amdahl Corporation   |                |                    |
     [The opinions expressed above are mine, solely, and do not    ]
     [necessarily reflect the opinions or policies of Amdahl Corp. ]

eddjp@edi386.UUCP ( Dewey Paciaffi ) (01/22/91)

In article <1991Jan21.124531.27867@siesoft.co.uk> stuart@siesoft.co.uk (Stuart Hood) writes:
>eddjp@edi386.UUCP ( Dewey Paciaffi ) writes:
-
->In article <1991Jan17.003856.469@unicorn.cc.wwu.edu> n8743196@unicorn.cc.wwu.edu (Jeff Wandling) writes:
->-
->-I tried:
->-
->-set string = newfoobar
->-cat file | sed 's/foobar/$string/g' ....
->-
-
->You might try this :
-
->	string=newfoobar
->	cat file | sed 's/foobar/'$string'/g' ...
-
-Small point, to be safe you should do:
-
-cat file | sed 's/foobar/'"$string"'/g' ...
-

Point taken.

>which doesn't fall over if "$string" has any whitespace in it.
>
>>Note that you don't want to use the set command to set the variable,
>>and by using the additional single quotes in the sed command, the shell
>>can replace the $string variable.
>
>Why can't you use the set command?

My mistake. There isn't any reason you can't when using the csh, as many have 
pointed out to me since. 



-- 
Dewey Paciaffi           ...!uunet!edi386!eddjp

martin@mwtech.UUCP (Martin Weitzel) (01/23/91)

In article <7ejp01gY0c.900@amdahl.uts.amdahl.com> krs@amdahl.uts.amdahl.com (Kris Stephens [Hail Eris!]) writes:
...
>	sed 's/foobar/'$string'/g'
>
>... value stored in string won't be evaluated
>by the shell if it contains metacharacters.  For example, if string was
>set to 'newfoobar?*' (without the contained double-quotes) and the files
>newfoobar12 and newfoobar13 existed, sed would replace 'foobar' with
>'newfoobar12 newfoobar13' if called 's/foobar/'$string'/g' , making its
>editing PWD-dependent.

For exactly that reason I've fallen into the habbit (and it normally
doesn't hurt) to double-quote every variable used in a shell script.
I'd prefer to write the above as follows:

	sed 's/foobar/'"$string"'/g'

There are few occasions where yo want file name expansion and/or IFS-
interpretation for the contents of some variable. If you are not sure
about the contents (maybe it is read as input or is a user-supplied
argument) double-quoting any usage seems strongly advisable.

BTW: There's no simple way to stop sed from complaining when the variable
`string' contains a slash, except if you know that it does contain a slash
and use some other seperator in the s-commando.
-- 
Martin Weitzel, email: martin@mwtech.UUCP, voice: 49-(0)6151-6 56 83

allbery@NCoast.ORG (Brandon S. Allbery KB8JRR) (01/23/91)

As quoted from <1991Jan21.124531.27867@siesoft.co.uk> by stuart@siesoft.co.uk (Stuart Hood):
+---------------
| eddjp@edi386.UUCP ( Dewey Paciaffi ) writes:
| >In article <1991Jan17.003856.469@unicorn.cc.wwu.edu> n8743196@unicorn.cc.wwu.edu (Jeff Wandling) writes:
| >-set string = newfoobar
| >-cat file | sed 's/foobar/$string/g' ....
| 
| >You might try this :
| >	string=newfoobar
| >	cat file | sed 's/foobar/'$string'/g' ...
| 
| Small point, to be safe you should do:
| cat file | sed 's/foobar/'"$string"'/g' ...
| 
| >Note that you don't want to use the set command to set the variable,
| Why can't you use the set command?
+---------------

"Cultural differences".  In csh, you *must* use "set"; in Bourne shell, you
must *not* use it.

As long as we're nit-picking:  it's faster to say

	string=newfoobar # csh: set string = newfoobar
	sed 's/foobar/'"$string"'/g' < file

because you don't have to start the "cat".

++Brandon
-- 
Me: Brandon S. Allbery			    VHF/UHF: KB8JRR on 220, 2m, 440
Internet: allbery@NCoast.ORG		    Packet: KB8JRR @ WA8BXN
America OnLine: KB8JRR			    AMPR: KB8JRR.AmPR.ORG [44.70.4.88]
uunet!usenet.ins.cwru.edu!ncoast!allbery    Delphi: ALLBERY

gwc@root.co.uk (Geoff Clare) (01/28/91)

In <1064@mwtech.UUCP> martin@mwtech.UUCP (Martin Weitzel) writes:

>	sed 's/foobar/'"$string"'/g'

>BTW: There's no simple way to stop sed from complaining when the variable
>`string' contains a slash, except if you know that it does contain a slash
>and use some other seperator in the s-commando.

Yes there is.  Simply create a copy of $string with '/' changed to '\/'.
While you're at it, you can treat '&' and '\' the same way to prevent
sed from interpreting '&', '\1', etc. if they happen to occur in $string.

Xstring=`sed 's/[\/&\\]/\\&/g' <<!
$string
!
`
sed 's/foobar/'"$Xstring"'/g'


-- 
Geoff Clare <gwc@root.co.uk>  (Dumb American mailers: ...!uunet!root.co.uk!gwc)
UniSoft Limited, London, England.   Tel: +44 71 729 3773   Fax: +44 71 729 3273

maart@cs.vu.nl (Maarten Litmaath) (01/29/91)

In article <2586@root44.co.uk>,
	gwc@root.co.uk (Geoff Clare) writes:
)In <1064@mwtech.UUCP> martin@mwtech.UUCP (Martin Weitzel) writes:
)
)>	sed 's/foobar/'"$string"'/g'
)
)>BTW: There's no simple way to stop sed from complaining when the variable
)>`string' contains a slash, except if you know that it does contain a slash
)>and use some other seperator in the s-commando.
)
)Yes there is.  Simply create a copy of $string with '/' changed to '\/'.
)While you're at it, you can treat '&' and '\' the same way to prevent
)sed from interpreting '&', '\1', etc. if they happen to occur in $string.
)
)Xstring=`sed 's/[\/&\\]/\\&/g' <<!
)$string
)!
)`
)sed 's/foobar/'"$Xstring"'/g'

Did you actually try your solution?  I guess not.
The following works:

Xstring=`sed 's/[\/&\\\\]/\\\\&/g' <<!
$string
!
`
--
kinnersley@kuhub.cc.ukans.edu (Bill Kinnersley):
	"Do phonograph turntables turn the other way in Australia?"
gjh@krebs.acc.Virginia.EDU (Galen J. Hekhuis):
	"How do you think satanic messages were discovered on records?"

martin@mwtech.UUCP (Martin Weitzel) (01/30/91)

In article <2586@root44.co.uk> gwc@root.co.uk (Geoff Clare) writes:
>In <1064@mwtech.UUCP> martin@mwtech.UUCP (I) writes:
>
>>	sed 's/foobar/'"$string"'/g'
>
>>BTW: There's no simple way to stop sed from complaining when the variable
               ^^^^^^^^^
>>`string' contains a slash, except if you know that it does contain a slash
>>and use some other seperator in the s-commando.
>
>Yes there is.  Simply create a copy of $string with '/' changed to '\/'.
                ^^^^^^
>While you're at it, you can treat '&' and '\' the same way to prevent
>sed from interpreting '&', '\1', etc. if they happen to occur in $string.
>
>Xstring=`sed 's/[\/&\\]/\\&/g' <<!
>$string
>!
>`
>sed 's/foobar/'"$Xstring"'/g'

Obviously our view of "simple" differs :-). IMHO no need to argue further.

Geoff's proposual to also protect embedded backslashes is important for
another reason: What would happen if "$string" contains the two characters
backslash and slash in sequence and you would only change the "/" to "\/" ?

One loophole still remains even in Geoff's solution: embedded newlines in
$string. It might not be important in many cases, but if the contents of
"$strings" comes from any kind of user input (procedure arg, interactive
read), the user may be able to create trouble. Again that might be not
important, but trying to break some program in an unforseen way might be
the first step for an intruder to create a security breach - one of the
best practical examples for this was the "Internet Worm".
-- 
Martin Weitzel, email: martin@mwtech.UUCP, voice: 49-(0)6151-6 56 83

gwc@root.co.uk (Geoff Clare) (01/31/91)

In <8872@star.cs.vu.nl> maart@cs.vu.nl (Maarten Litmaath) writes:

}In article <2586@root44.co.uk>,
}	gwc@root.co.uk (Geoff Clare) writes:
})
})Xstring=`sed 's/[\/&\\]/\\&/g' <<!
})$string
})!
})`
})sed 's/foobar/'"$Xstring"'/g'

}Did you actually try your solution?  I guess not.

Yes and no.  I tested the sed command on its own, but then stuck it
inside backquotes without noticing that extra quoting would be required.
Sorry about that - I'll try to be more careful in future.

}The following works:

}Xstring=`sed 's/[\/&\\\\]/\\\\&/g' <<!
}$string
}!
}`

Yes it does.  I would have doubled the backslash in "\/" as well, but
clearly it's not needed.

-- 
Geoff Clare <gwc@root.co.uk>  (Dumb American mailers: ...!uunet!root.co.uk!gwc)
UniSoft Limited, London, England.   Tel: +44 71 729 3773   Fax: +44 71 729 3273

tchrist@convex.COM (Tom Christiansen) (02/01/91)

From the keyboard of gwc@root.co.uk (Geoff Clare):
:})
:})Xstring=`sed 's/[\/&\\]/\\&/g' <<!
:})$string
:})!
:})`
:})sed 's/foobar/'"$Xstring"'/g'
:
:}Did you actually try your solution?  I guess not.
:
:}The following works:
:
:}Xstring=`sed 's/[\/&\\\\]/\\\\&/g' <<!
:}$string
:}!
:}`

Gotta love these languages where the backslashes just keep building
until almost that's all there is.  Reminds me of writing troff macros.
Just keep adding backslashes until it works. :-)

--tom
--
"Hey, did you hear Stallman has replaced /vmunix with /vmunix.el?  Now
 he can finally have the whole O/S built-in to his editor like he
 always wanted!" --me (Tom Christiansen <tchrist@convex.com>)

maart@cs.vu.nl (Maarten Litmaath) (02/02/91)

In article <1069@mwtech.UUCP>,
	martin@mwtech.UUCP (Martin Weitzel) writes:
)[...]
)One loophole still remains even in Geoff's solution: embedded newlines in
)$string. [...]

Here's an example that deals with everything, as far as I know.
Enjoy.

--------------------cut here--------------------
#!/bin/sh
# sed(1) safe substitute example

newline='
'

string='this string contains an embedded newline,
a backslash, a slash, an ampersand and 2 terminating newlines: \ / &

'

nl=

case $string in
*$newline)
	nl=$newline
esac

Xstring=`sed -e 's-[/&\\\\]-\\\\&-g' -e '$!s/$/\\\\/' <<!
$string
!
`$nl

sed "s/.*/$Xstring/" <<!
hello
!
--
Temporary files like /tmp/sh$$ are an abomination.