[comp.unix.wizards] ksh incompatabilities with sh?

gnu@hoptoad.uucp (John Gilmore) (05/23/88)

> > 	(okay - you could rename ksh to sh)

guy@gorodish.Sun.COM (Guy Harris) wrote:
> Well, no, you really can't, not always.  There are supposedly a few real live
> incompatibilities that may break some scripts.

I have heard this too, over the years, and would like real
documentation on what's wrong.  Are there gross bugs in "sh" that ksh
is too polite to perpetuate, but which users depend upon?  It would
seem the best course to fix the ksh to match the sh in all respects,
rename it to /bin/sh, and throw away the old sh.  I thought that was
the original idea; what happened?
-- 
John Gilmore  {sun,pacbell,uunet,pyramid,ihnp4}!hoptoad!gnu        gnu@toad.com
"Use the Source, Luke...."

guy@gorodish.Sun.COM (Guy Harris) (05/24/88)

> > Well, no, you really can't, not always.  There are supposedly a few real
> > live incompatibilities that may break some scripts.
> 
> I have heard this too, over the years, and would like real
> documentation on what's wrong.

All I know is that somebody tried doing exactly that (replacing "/bin/sh" with
the Korn shell) on a system with an S5R2 or later Bourne shell, and found that
a lot of his shell scripts broke.  It happened a while ago, and he doesn't
remember what broke; I didn't see much point in requesting that he go through
the whole exercise again to reconstruct the problems.  He does admit to using
the Bourne shell (and lots of other UNIX tools) in ways that most people
wouldn't think of, so it may be that he hit some corner case where the two
shells behave incompatibly.

The Korn shell (the pre-"ksh-i" version, anyway) does come with a file named
COMPATIBILITY that lists a number of "sh"/"ksh" incompatibilities.

scott@zorch.UU.NET (Scott Hazen Mueller) (05/24/88)

In article <54188@sun.uucp> guy@gorodish.Sun.COM (Guy Harris) writes:
>All I know is that somebody tried doing exactly that (replacing "/bin/sh" with
>the Korn shell) on a system with an S5R2 or later Bourne shell, and found that
>a lot of his shell scripts broke.

When David Korn spoke at the SVNet meeting last month, the question
of compatibility came up briefly (very!).  The one item that I specifically
recall is the venerable ^ when used as a synonym for |.  Supposedly an
early version of /etc/shutdown for System V used ^ as a pipe...

I wouldn't base any purchasing decisions on this recollection...
-- 
Scott Hazen Mueller   scott@zorch.UU.NET
(408) 245-9461        (pyramid|tolerant|uunet)!zorch!scott

kre@munnari.oz (Robert Elz) (05/24/88)

In article <4627@hoptoad.uucp>, gnu@hoptoad.uucp (John Gilmore) writes:
> I have heard this too, over the years, and would like real
> documentation on what's wrong.

Its not a matter of "wrong", its just that there is no sensible way to
extend the sh language (as ksh does) without making it incompatible.

Consider the following trivial (one line) shell script ..

	print $1

when run using the Bourne shell as /bin/sh, and when run using the Korn
shell as /bin/sh.

They're quite likely to do quite different things.

Unfortunately, other that reducing ksh to being a sh clone (with perhaps
some of the keyboard control still there) there's nothing that can be
done about this.

If your site happens not to have any sh scripts that behave differently when
run with ksh, then by all means replace sh.  But don't be too surprised if
you find one or two, or get one or two later.

kre

davidsen@steinmetz.ge.com (William E. Davidsen Jr) (05/25/88)

  There are a number of little places where ksh will break scripts. Some
of the Xenix install scripts break under ksh, and some of the DWB tools
in shell break, also. These last can be easily rewritten to work
correctly under both.
-- 
	bill davidsen		(wedu@ge-crd.arpa)
  {uunet | philabs | seismo}!steinmetz!crdos1!davidsen
"Stupidity, like virtue, is its own reward" -me

dlm@cuuxb.ATT.COM (Dennis L. Mumaugh) (05/25/88)

In article <4627@hoptoad.uucp> gnu@hoptoad.uucp (John Gilmore) writes:
   I have heard this too, over the years, and would like real
   documentation on what's wrong.  Are there gross bugs in "sh" that ksh
   is too polite to perpetuate, but which users depend upon?  It would
   seem the best course to fix the ksh to match the sh in all respects,
   rename it to /bin/sh, and throw away the old sh.  I thought that was
   the original idea; what happened?

I have seen a couple:

1) the test builtin for sh accepts extra arguments:
	test -r /bin/*
while the ksh test builtin complains about syntax error


2) Also, several environment variables are set  readonly  in  ksh
but  not  in  sh and when cron/at is setting up an at job it will
attempt to re-establish  the  user's  enviornment  and  sets  the
"read-only" variables.
-- 
=Dennis L. Mumaugh
 Lisle, IL       ...!{ihnp4,cbosgd,lll-crg}!cuuxb!dlm

lvc@tut.cis.ohio-state.edu (Lawrence V. Cipriani) (05/25/88)

In article <54188@sun.uucp>, guy@gorodish.Sun.COM (Guy Harris) writes:
< > > Well, no, you really can't, not always.  There are supposedly a few real
< > > live incompatibilities that may break some scripts.
< > 
< > I have heard this too, over the years, and would like real
< > documentation on what's wrong.
<
<	anecdote of someones experience replacing sh with ksh removed
< 
< The Korn shell (the pre-"ksh-i" version, anyway) does come with a file named
< COMPATIBILITY that lists a number of "sh"/"ksh" incompatibilities.

The book (still unpublished) by Korn and Bolsky on ksh programming has
an appendix that lists all the incompatibilities between ksh and SVR2 sh.

-- 
Larry Cipriani, AT&T Network Systems and Ohio State University
Domain: lvc@tut.cis.ohio-state.edu
Path: ...!cbosgd!osu-cis!tut.cis.ohio-state.edu!lvc (weird but right)

avr@mtgzz.UUCP (XMRP50000[jcm]-a.v.reed) (05/25/88)

In article <4627@hoptoad.uucp>, gnu@hoptoad.uucp (John Gilmore) writes:
< > > 	(okay - you could rename ksh to sh)
< 
< guy@gorodish.Sun.COM (Guy Harris) wrote:
< > Well, no, you really can't, not always.  There are supposedly a few real live
< > incompatibilities that may break some scripts.
< 
< I have heard this too, over the years, and would like real
< documentation on what's wrong.  Are there gross bugs in "sh" that ksh
< is too polite to perpetuate, but which users depend upon?  It would
< seem the best course to fix the ksh to match the sh in all respects,
< rename it to /bin/sh, and throw away the old sh.  I thought that was
< the original idea; what happened?

All the cases of "ksh breaking scripts" I've ever come across were due
to name conflict between standard utilities expected by the script, such
as grep(1), and user-defined aliases and functions. In one not atypical
case, a user who did not like case sensitivity had

	alias -x grep='/bin/grep -i'

in her ksh $ENV file. Then scripts that depend on the standard
case-sensitive behavior of grep, of which there are many, would break
under ksh, but run fine with sh, which doesn't have aliases. I advised
her to change this alias to "mygrep" instead of just "grep", and presto,
her shell scripts started to work with ksh as with sh, just faster.

				Adam Reed (mtgzz!avr)

john@frog.UUCP (John Woods) (05/25/88)

In article <546@zorch.UU.NET>, scott@zorch.UU.NET (Scott Hazen Mueller) writes:
>In article <54188@sun.uucp> guy@gorodish.Sun.COM (Guy Harris) writes:
>>All I know is that somebody tried [replacing "/bin/sh" with
>>the Korn shell and] a lot of his shell scripts broke.
> When David Korn spoke at the SVNet meeting last month, the question
> of compatibility came up briefly (very!).

I just leafed through the COMPATIBILITY file that came with ksh-i.  Perhaps
David Korn could arrange to post the actual thing, but I'll just give a VERY
brief summary.  It looked to me like the most-commonly-used things that
changed were the handling of IFS (now done later, after parameter and command
substitution); global environment handling (ksh passes modified environment
parameters to children without needing export); and the use of (( to introduce
an arithmetic expression instead of a subshell-in-a-subshell.

I have not yet met an sh script that ksh couldn't handle just fine -- of
course, such exist, but my experience is that they aren't too common.
-- 
John Woods, Charles River Data Systems, Framingham MA, (617) 626-1101
...!decvax!frog!john, john@frog.UUCP, ...!mit-eddie!jfw, jfw@eddie.mit.edu

No amount of "Scotch-Guard" can repel the ugly stains left by REALITY...
		- Griffy

ron@topaz.rutgers.edu (Ron Natalie) (05/28/88)

At one point KSH had different syntax for things like shell functions
and the like as KSH was based on the System V release of the shell, when
those functions were unavailable in the shell.  The shell in VR2 had these
functions, but slightly different.   I believe most of this was resolved
when the legally available version of ksh came out.  If you have a non-
compatible one you are either an AT&T employee or have a illegitimate
copy of an early release.

-Ron

tony@uqcspe.OZ (Tony O'Hagan) (06/01/88)

I haven't checked this recently, but I thought another incompatibility
was that deeply nested loops in Bourne shell were forked as new
processes.  This means that variable assignments in these inner loops
are not exported outside these deep loops.  Korn shell fixed up this mess.

I think I stumbled on this in our old V7 Unix.

Tony O'Hagan	tony@uqcspe.oz

davidsen@steinmetz.ge.com (William E. Davidsen Jr) (06/05/88)

The biggest problem I've had in scripts and ksh is the use of ^ for a
pipe. I hit this in some part of DWB (style? spell?) and fixed the
script. There is something in the install scripts for vpix that breaks,
too, but I just ran them under sh.
-- 
	bill davidsen		(wedu@ge-crd.arpa)
  {uunet | philabs | seismo}!steinmetz!crdos1!davidsen
"Stupidity, like virtue, is its own reward" -me

rbj@icst-cmr.arpa (Root Boy Jim) (06/08/88)

   From: "William E. Davidsen Jr" <davidsen@steinmetz.ge.com>

   The biggest problem I've had in scripts and ksh is the use of ^ for a
   pipe. I hit this in some part of DWB (style? spell?) and fixed the
   script. There is something in the install scripts for vpix that breaks,
   too, but I just ran them under sh.

Do people *actually* still use ^ for pipes? This is the kind of thing
that should be eradicated from existence. 

	   bill davidsen		(wedu@ge-crd.arpa)
     {uunet | philabs | seismo}!steinmetz!crdos1!davidsen
   "Stupidity, like virtue, is its own reward" -me

	(Root Boy) Jim Cottrell	<rbj@icst-cmr.arpa>
	National Bureau of Standards
	Flamer's Hotline: (301) 975-5688
	The opinions expressed are solely my own
	and do not reflect NBS policy or agreement
	My name is in /usr/dict/words. Is yours?

lvc@tut.cis.ohio-state.edu (Lawrence V. Cipriani) (06/09/88)

In article <16094@brl-adm.ARPA> rbj@icst-cmr.arpa (Root Boy Jim) writes:
>Do people *actually* still use ^ for pipes? This is the kind of thing
>that should be eradicated from existence. 
>	(Root Boy) Jim Cottrell	<rbj@icst-cmr.arpa>

Many of our customers still use ^ for pipes, that's what they got used
to.  They have been using various versions of UNIX for at least 12 years.
Old habits die hard as the saying goes.

-- 
Larry Cipriani, AT&T Network Systems and Ohio State University
Domain: lvc@tut.cis.ohio-state.edu
Path: ...!cbosgd!osu-cis!tut.cis.ohio-state.edu!lvc (strange but true)

nick@ccicpg.UUCP (Nick Crossley) (06/10/88)

In article <16094@brl-adm.ARPA> rbj@icst-cmr.arpa (Root Boy Jim) writes:
>Do people *actually* still use ^ for pipes? This is the kind of thing
>that should be eradicated from existence. 
>
>	   bill davidsen		(wedu@ge-crd.arpa)
>     {uunet | philabs | seismo}!steinmetz!crdos1!davidsen

Some terminals have ^ but not |.  An example is the Wang 2236 DW or DE, used
on the Wang 2200 processor.  CCI sells a system which can replace the 2200
processor, but run the same Wang Basic software and use the existing wiring
and terminals.  Since these terminals do not have any of the characters
{}\`|, Escape or Control, using Unix on them is near impossible, so we insist
on supplying at least one normal terminal.  However, there have been times I
have needed to do something on a user's Wang terminal, and I have typed ^ to
use a pipe.  On those few occasions, I have been very thankful for the synonym.

-- 

<<< standard disclaimers >>>
Nick Crossley, CCI, 9801 Muirlands, Irvine, CA 92718-2521, USA
Tel. (714) 458-7282,  uucp: ...!uunet!ccicpg!nick

shankar@hpclscu.HP.COM (Shankar Unni) (06/10/88)

> 
> Many of our customers still use ^ for pipes, that's what they got used
> to.  They have been using various versions of UNIX for at least 12 years.
> Old habits die hard as the saying goes.
>
Maybe a judicious dose of WARNINGS (a' la' the C compiler) might have turned
them off a long time ago...

--
Shankar.

robert@pttesac.UUCP (Robert Rodriguez) (06/10/88)

A colleauge of mine found an incompatibility between ksh and sh.

In the System V "sh" the following test works:

if [ -f /usr/anyusr/* ] ; then
  # There are files there

But "ksh" claims this is a syntax error.

Our $0.02 worth.

While I'm posting, I'm 'gonna up it to $0.04.

A guy posted a message asking for a AWK to C converter.  He was flamed,
told to find /bin/cc.  That stinks in my opinion.  The guy who flamed
him should read the guest article in the January issue of UNIX World.

friedl@vsi.UUCP (Stephen J. Friedl) (06/12/88)

In article <16094@brl-adm.ARPA> rbj@icst-cmr.arpa (Root Boy Jim) writes:
>Do people *actually* still use ^ for pipes?

In article <36911@ccicpg.UUCP>, nick@ccicpg.UUCP (Nick Crossley) writes:
< 
< Some terminals have ^ but not |. [certain] terminals do not have any of
< the characters {}\`|, Escape or Control, using Unix on them is near
< impossible, so we insist > on supplying at least one normal terminal.

Great!  Trigraphs for the shell! :-)
-- 
Steve Friedl    V-Systems, Inc. (714) 545-6442      3B2-kind-of-guy
friedl@vsi.com     {backbones}!vsi.com!friedl    attmail!vsi!friedl

Nancy Reagan on ptr args with a prototype in scope: "Just say NULL"

rbj@ICST-CMR.ARPA (Root Boy Jim) (06/13/88)

? From: "Lawrence V. Cipriani" <lvc@tut.cis.ohio-state.edu>

? In article <16094@brl-adm.ARPA> rbj@icst-cmr.arpa (Root Boy Jim) writes:
? >Do people *actually* still use ^ for pipes? This is the kind of thing
? >that should be eradicated from existence. 
? >	(Root Boy) Jim Cottrell	<rbj@icst-cmr.arpa>

My apologies for the redundant phrase `out of existence' :-)
 
? Many of our customers still use ^ for pipes, that's what they got used
? to.  They have been using various versions of UNIX for at least 12 years.
? Old habits die hard as the saying goes.

What I want to know is how they got used to it. Hasn't `|' *always* been
the symbol for a pipe? When was `^' introduced? A history lesson please!

As for braindamaged terminals, they should be thrown away, period.

? Larry Cipriani, AT&T Network Systems and Ohio State University
? Domain: lvc@tut.cis.ohio-state.edu
? Path: ...!cbosgd!osu-cis!tut.cis.ohio-state.edu!lvc (strange but true)

	(Root Boy) Jim Cottrell	<rbj@icst-cmr.arpa>
	National Bureau of Standards
	Flamer's Hotline: (301) 975-5688
	The opinions expressed are solely my own
	and do not reflect NBS policy or agreement
	My name is in /usr/dict/words. Is yours?

lvc@tut.cis.ohio-state.edu (Lawrence V. Cipriani) (06/14/88)

In article <16147@brl-adm.ARPA> rbj@ICST-CMR.ARPA (Root Boy Jim) writes:
?? From: "Lawrence V. Cipriani" <lvc@tut.cis.ohio-state.edu>
	...
?? Many of our customers still use ^ for pipes, that's what they got used
?? to.  They have been using various versions of UNIX for at least 12 years.
?? Old habits die hard as the saying goes.
?What I want to know is how they got used to it. Hasn't `|' *always* been
?the symbol for a pipe? When was `^' introduced? A history lesson please!

Before the Bourne shell there was the Mashey shell.  It supported ^ for
the pipe symbol not |.  This was a very primitive shell. A lot of its
functionality was in separate programs due to limited memory size.
For example wild card were expanded in a separate process, yuck.
When the Bourne shell was written it inherited the ^, not sure why
the change occurred.  Maybe it wasn't universally available, or perhaps
it was just easier to type on Bourne's terminal :-).

-- 
Larry Cipriani, AT&T Network Systems and Ohio State University
Domain: lvc@tut.cis.ohio-state.edu
Path: ...!cbosgd!osu-cis!tut.cis.ohio-state.edu!lvc (strange but true)

rbj@ICST-CMR.ARPA (Root Boy Jim) (06/16/88)

? From: "Lawrence V. Cipriani" <lvc@tut.cis.ohio-state.edu>
? 
? In article <16147@brl-adm.ARPA> rbj@ICST-CMR.ARPA (Root Boy Jim) writes:
? ?? From: "Lawrence V. Cipriani" <lvc@tut.cis.ohio-state.edu>
? 	...
? ?? Many of our customers still use ^ for pipes, that's what they got used
? ?? to.  They have been using various versions of UNIX for at least 12 years.
? ?? Old habits die hard as the saying goes.
? ?What I want to know is how they got used to it. Hasn't `|' *always* been
? ?the symbol for a pipe? When was `^' introduced? A history lesson please!
? 
? Before the Bourne shell there was the Mashey shell.  It supported ^ for
? the pipe symbol not |.  This was a very primitive shell. A lot of its
? functionality was in separate programs due to limited memory size.
? For example wild card were expanded in a separate process, yuck.
? When the Bourne shell was written it inherited the ^, not sure why
? the change occurred.  Maybe it wasn't universally available, or perhaps
? it was just easier to type on Bourne's terminal :-).

OK. I will grant you that `^' preceded '|'. But why do `many of (y)our
customers still use ^ for pipes'? The original CACM paper, which I
believe appeared in 74, predated most people's exposure to UNIX, and
it used the symbol '|' for piping. So why would anyone have used `^'?

I can see people at TPC using `^' out of habit, but customers?

As far as ease of typing goes, old bit-pairing terminals, such as the
adm-3a (with arrows on the hjkl keys) and the Datamedia 152[01] did
have `^' as an unshifted key, while `|' was shifted. Might explain it.

? Larry Cipriani, AT&T Network Systems and Ohio State University
? Domain: lvc@tut.cis.ohio-state.edu
? Path: ...!cbosgd!osu-cis!tut.cis.ohio-state.edu!lvc (strange but true)

	(Root Boy) Jim Cottrell	<rbj@icst-cmr.arpa>
	National Bureau of Standards
	Flamer's Hotline: (301) 975-5688
	The opinions expressed are solely my own
	and do not reflect NBS policy or agreement
	My name is in /usr/dict/words. Is yours?

lvc@tut.cis.ohio-state.edu (Lawrence V. Cipriani) (06/16/88)

In article <16183@brl-adm.ARPA> rbj@ICST-CMR.ARPA (Root Boy Jim) writes:
	... discussion of the origin and use of ^ vs. | canned
>OK. I will grant you that `^' preceded '|'. But why do `many of (y)our
>customers still use ^ for pipes'? The original CACM paper, which I
>believe appeared in 74, predated most people's exposure to UNIX, and
>it used the symbol '|' for piping. So why would anyone have used `^'?

I suppose the ^ was considered archaic even in '74.

Our customers are phone companies and have been using our product
for over 16 years.  Many of the *users* that were involved with our
product back then are still using it.  Their exposure to UNIX predated
the CACM paper by a few years (see "Advanced UNIX Programming" by
Rochkind pg 156-157 for a brief glimpse of Columbus UNIX, which is
what several of AT&T's (older) internal products ran under).

The documentation for sh that was distributed way back showed ^ and |.
Before divesture of the RBOCS our customers could, at their option,
get source listings of our product for a modest printing fee.
They certainly are capable of understanding it, even code as obtuse as
sh.  Also, the shell scripts that were part of the product ocassionally
used ^.  The training classes might have infected them with ^.
Wherever they picked it up from, they've got and have a hard time kicking
the habit.

The ^ problem"is like the <> redirection operator.  Its not documented
but it is in most versions of the Bourne shell.  It means open for
read *and* write.  I've seen it used only once.  <> will be part of
the next ksh release.

>I can see people at TPC using `^' out of habit, but customers?

What's TPC, "The Phone Company" ?  It doesn't exist any more.

-- 
Larry Cipriani, AT&T Network Systems and Ohio State University
Domain: lvc@tut.cis.ohio-state.edu
Path: ...!cbosgd!osu-cis!tut.cis.ohio-state.edu!lvc (strange but true)

gwyn@brl-smoke.ARPA (Doug Gwyn ) (06/16/88)

In article <15726@tut.cis.ohio-state.edu> lvc@tut.cis.ohio-state.edu (Lawrence V. Cipriani) writes:
>The ^ problem"is like the <> redirection operator.  Its not documented
>but it is in most versions of the Bourne shell.

Actually, <> is incorrectly implemented in most of the Bourne shells I've
seen.  We fixed it in ours.

scs@athena.mit.edu (Steve Summit) (06/17/88)

The most stupefying ksh imcompatibility I ever saw, which I
submit for amusement value only because I don't have many
relevant details any more, was that dbx aliases didn't seem to
work when dbx was invoked under ksh.  The same dbx appeared to
work correctly when invoked from sh or csh.  This is, of course,
prima facie absurd, and the only possible mechanism by which the
choice of shell could make this kind of difference would be via
something in the environment.

It should be noted, however, that the Korn shell (at least the
versions I've used) does _n_o_t keep the environment alphabetized,
as do the other shells; this is a documented inconsistency and is
defended by observing that the environment is not guaranteed to
be sorted, and that programs which depend on its being so
therefore don't deserve to work.

Did anyone else ever have any problems with dbx and ksh?  Or was
I hallucinating?

                                            Steve Summit
                                            scs@adam.pika.mit.edu

nick@ccicpg.UUCP (Nick Crossley) (06/18/88)

In article <16147@brl-adm.ARPA> rbj@ICST-CMR.ARPA (Root Boy Jim) writes:
>As for braindamaged terminals, they should be thrown away, period.

Unfortunately, the Wang terminals I mentioned are *so* brain-damaged that
they cannot easily be replaced.  They have a unique key layout, with a large
number of special keys, including 17 function keys, and send very odd codes.
The character set is non-standard, and the flow control is not Xon-Xoff but
something else invented by Wang.  They have line and box drawing facilities
which cannot easily be emulated by other common terminals, since the lines
do not occupy character positions but pass between them, and a character
written to somewhere on the screen does not overwrite said lines or boxes.
And so on.  They are horrible to fit onto Unix (require a special driver for
the Xon-Xoff flow control and a special line discipline for the function
keys), so I wish they could be thrown away, but the applications need their
peculiar characteristics!

-- 

<<< standard disclaimers >>>
Nick Crossley, CCI, 9801 Muirlands, Irvine, CA 92718-2521, USA
Tel. (714) 458-7282,  uucp: ...!uunet!ccicpg!nick

jpn@teddy.UUCP (John P. Nelson) (06/21/88)

>It should be noted, however, that the Korn shell (at least the
>versions I've used) does _n_o_t keep the environment alphabetized,
>as do the other shells; this is a documented inconsistency ...

Sorry, but csh doesn't seem to keep environment variables alphabetized,
either.  This would appear to be an ideosyncracy unique to "sh".

-- 
     john nelson

UUCP:	{decvax,mit-eddie}!genrad!teddy!jpn
smail:	jpn@genrad.com

koala@ddsw1.UUCP (Karl Meiser) (06/24/88)

In article <4862@teddy.UUCP> jpn@teddy.UUCP (John P. Nelson) writes:
>>It should be noted, however, that the Korn shell (at least the
>>versions I've used) does _n_o_t keep the environment alphabetized,
>>as do the other shells; this is a documented inconsistency ...
>
>Sorry, but csh doesn't seem to keep environment variables alphabetized,
>either.  This would appear to be an ideosyncracy unique to "sh".
>

But is this a plus or a minus?  Is there anything serious wrong with
not alphabetizing?  Suppose you want to find the last changed variable.

It all depends on what you want on how you look at it

--

Karl Meiser    ..!att!spl1!ddsw1!koala

I said that??  I musta been sleeping!