simcha@kurz-ai.UUCP (Simcha Lerner) (06/01/89)
Has anyone had any luck getting atty running under Sun OS 4.0 on a Sun 3? I get an Abort with "set pty modes: Inappropriate ioctl for device" whenever I try to start atty up. simcha lerner talcott!kurz-ai!simcha
schmidt@ics.uci.edu (Doug Schmidt) (06/02/89)
In article <280@kurz-ai.UUCP> simcha@kurz-ai.UUCP (Simcha Lerner) writes:
++
++ Has anyone had any luck getting atty running under Sun OS 4.0
++ on a Sun 3?
++
++ I get an Abort with "set pty modes: Inappropriate ioctl for device"
++ whenever I try to start atty up.
Did you happen to compile with GCC by any chance? If so, did you remember
to supply the -traditional option? If you didn't, this is what you're
probably going to get. Use -traditional, and the problem should go away...
Doug
--
On a clear day, under blue skies, there is no need to seek.
And asking about Buddha +------------------------+
Is like proclaiming innocence, | schmidt@ics.uci.edu |
With loot in your pocket. | office: (714) 856-4043 |roy@sts.sts.UUCP (06/02/89)
/* Written 10:58 am Jun 1, 1989 by schmidt@uci-ics.UUCP in sts:comp.sources.bugs */
In article <280@kurz-ai.UUCP> simcha@kurz-ai.UUCP (Simcha Lerner) writes:
++
++ Has anyone had any luck getting atty running under Sun OS 4.0
++ on a Sun 3?
++
++ I get an Abort with "set pty modes: Inappropriate ioctl for device"
++ whenever I try to start atty up.
Did you happen to compile with GCC by any chance? If so, did you remember
to supply the -traditional option? If you didn't, this is what you're
probably going to get. Use -traditional, and the problem should go away...
Doug
--
On a clear day, under blue skies, there is no need to seek.
And asking about Buddha +------------------------+
Is like proclaiming innocence, | schmidt@ics.uci.edu |
With loot in your pocket. | office: (714) 856-4043 |
/* End of text from sts:comp.sources.bugs */
I had the same problem with a Sun3 running SunOS 4. Here is the exact
error message ...
set pty modes: Inappropriate ioctl for device
IOT trap (core dumped)
I compiled this with gcc -O -traditional -c ... The results were the
same with the Sun C compiler too. The -O might have done it, but it
doesn't sound like it from the error message.
==================================================================
Roy Bixler | UUCP: ...!{claris,sun}!sts!roy |
Semiconductor Test Solutions | Internet: sts!roy@Claris.COM |
4101 Burton Dr. | phones: (408) 727-2885 x132 (work)|
Santa Clara, CA 95054 | (408) 289-1035 (home)|
==================================================================simcha@kurz-ai.UUCP (Simcha Lerner) (06/02/89)
In article <16363@paris.ics.uci.edu> Doug Schmidt <schmidt@zola.ics.uci.edu> writes: >In article <280@kurz-ai.UUCP> simcha@kurz-ai.UUCP (Simcha Lerner) writes: >++ >++ Has anyone had any luck getting atty running under Sun OS 4.0 >++ on a Sun 3? >++ >++ I get an Abort with "set pty modes: Inappropriate ioctl for device" >++ whenever I try to start atty up. > >Did you happen to compile with GCC by any chance? If so, did you remember >to supply the -traditional option? If you didn't, this is what you're >probably going to get. Use -traditional, and the problem should go away... > >Doug Alas, I am using the standard Sun provided cc, so this doesn't fix it. Has anyone out there had success getting atty up on a Sun? simcha lerner talcott!kurz-ai!simcha no return mail via bbn please
glad@daimi.dk (Michael Glad) (06/03/89)
I see no need to discuss the compiler flags used in compiling atty
on Suns.
A few days ago, I compiled atty and tried to run it with the same
disappointing result as reported in this news group.
I recompiled it using -g and used gdb (the nice GNU debugger) to
locate the crash.
Atty core dumps because it calls abort. It calls abort because an pty
ioctl fails. The ioctl fails because it tries to set terminal modes
on the master side of the pty instead of on the slave side.
I don't know if that's possible on other systems but it won't work on
SunOS 4.0 suns, see the enclosed program:
#include <sys/file.h>
#include <sys/types.h>
#include <sys/termio.h>
#include <sys/termios.h>
/* Michael Glad, glad@daimi.dk, 1989-06-03 */
main(argc,argv)
int argc;
char **argv;
{ int s;
struct termios t;
/* Too lazy to search for a pty, let's hope this one is free */
/* Change it if not configured in your kernel */
if((s=open("/dev/ptyr0",O_RDONLY))<0)
{ perror("couldn't open pty");
exit(1);
}
if(ioctl(s,TCGETS,(char*)&t)<0)
{ perror("ioctl failed");
exit(1);
}
close(s);
exit(0);
}
When compiled using gcc or cc it produces the reported error message:
"Inappropriate ioctl for device".
I think atty needs patches for Sun's. I'm not going to do anything
about it as I'm very satisfied with the command editing facilities
in tcsh (a patched csh).
/Michael Glad
glad@daimi.dkguy@auspex.auspex.com (Guy Harris) (06/04/89)
>Atty core dumps because it calls abort. It calls abort because an pty >ioctl fails. The ioctl fails because it tries to set terminal modes >on the master side of the pty instead of on the slave side. > >I don't know if that's possible on other systems but it won't work on >SunOS 4.0 suns, see the enclosed program: OK, here's the scoop. 1) On the vanilla BSD pseudo-tty driver, you can do tty "ioctl"s on either the master or the slave side. This works because the pseudo-tty driver can get at the slave side's line discipline from the master side. 2) The SunOS 4.0 driver does not make this easy, being streams-based. The streams-based slave-side driver doesn't know anything about the modules pushed above it (except that it expects them to understand a couple of specialized messages to turn input processing on and off, so that remote mode works). However, there is a truly slimy kludge in the 4.0 pseudo-tty driver that lets it work *most* of the time for the *BSD* terminal "ioctl"s. It does *not* work for the System V-style terminal "ioctl"s, because, frankly, I didn't give a damn about making them work on the master side; the only reason the kludge is there is so that old binaries, and crufty programs written for BSD that bang on the master side rather than the slave side, work. Furthermore, if the slave side isn't open, there *is* no line discipline there for on which the "ioctl"s can even act, so even the aforementioned kludge wouldn't be sufficient; it might work with even more kludgery, but I figured I'd reached the point of diminishing returns by then.... I've not yet run across any program that couldn't be fixed to do the "ioctl"s on the slave side; I expect the System V Release 4 pseudo-tty driver not to bother making them work on the master side, so I suggest people not do tty "ioctl"s on the master side in new code. (I also expect the S5R4 pseudo-tty driver to have some hooks that will permit programs like "atty" to be cleaned up a fair bit....) The reason your test program fails is that it's doing the S5-style "TCGETS" on the master side. Were you to change it to do a BSD-style "ioctl" such as TIOCSETP, it would fail because you don't have the slave side open. The first problem doesn't apply to "atty", since it doesn't use the S5-style "ioctl"s. The second problem does, because it tries to set the modes on the master side before it even opens the slave side. The following patch to "atty.c" seemed to fix the problem (I was able to start "atty" up, and type at it a bit, at least); it moves the setting of the initial pseudo-tty modes from code in the parent which bangs on the master side to code in the child that bangs on the slave side after it's opened): *** atty.c.ORIG Sat Jun 3 16:25:00 1989 --- atty.c Sat Jun 3 16:34:32 1989 *************** *** 215,222 **** #endif if (ioctl(ptyfd, TIOCPKT, (char *)&one) < 0) fatal("TIOCPKT"); - if (setttymodes(ptyfd, &ptymodes) < 0) - fatal("set pty modes"); fillttymode(); copyttysize(); if ((childpid = fork()) == -1) { --- 215,220 ---- *************** *** 311,316 **** --- 309,316 ---- close(2); dup(0); /* file descriptor 1 = dup of 0 */ dup(0); /* file descriptor 2 = dup of 0 */ + if (setttymodes(2, &ptymodes) < 0) + fatal("set pty slave modes"); if (ioctl(2, TIOCSPGRP, (char *)&mypid) < 0) printf("TIOCSPGRP failed, errno=%d\n", errno); basename = shell;
roy@sts.sts.UUCP (06/04/89)
/* Written 4:49 pm Jun 3, 1989 by guy@auspex.UUCP in sts:comp.sources.bugs */ >Atty core dumps because it calls abort. It calls abort because an pty >ioctl fails. The ioctl fails because it tries to set terminal modes >on the master side of the pty instead of on the slave side. > >I don't know if that's possible on other systems but it won't work on >SunOS 4.0 suns, see the enclosed program: OK, here's the scoop. . . . However, there is a truly slimy kludge in the 4.0 pseudo-tty driver that lets it work *most* of the time for the *BSD* terminal "ioctl"s. It does *not* work for the System V-style . . . The following patch to "atty.c" seemed to fix the problem (I was able to start "atty" up, and type at it a bit, at least); it moves the setting of the initial pseudo-tty modes from code in the parent which bangs on the master side to code in the child that bangs on the slave side after it's opened): *** atty.c.ORIG Sat Jun 3 16:25:00 1989 --- atty.c Sat Jun 3 16:34:32 1989 /* End of text from sts:comp.sources.bugs */ I applied the patch to atty.c and re-compiled with gcc. The result was that I'm able to get the initial prompt and type a few characters, but when I hit Return, I get the same core dump as before and, apparently, for the same reason. Naturally, since atty(1) is not meant to run under System V and SunOS 4.0 is partially based on System V, it doesn't work too well here. My fix is to run ash(1) under the cmushell in GNU Emacs. That way, I get the Emacs editing power as well as command history. ================================================================== Roy Bixler | UUCP: ...!{claris,sun}!sts!roy | Semiconductor Test Solutions | Internet: sts!roy@Claris.COM | 4101 Burton Dr. | phones: (408) 727-2885 x132 (work)| Santa Clara, CA 95054 | (408) 289-1035 (home)| ==================================================================
len@array.UUCP (Leonard Vanek) (06/05/89)
I compiled it on a Sun 3/160 running (I think) SunOS v. 3.5 and it works fine. No changes were required to the make file. Len
guy@auspex.auspex.com (Guy Harris) (06/06/89)
>I applied the patch to atty.c and re-compiled with gcc. The result >was that I'm able to get the initial prompt and type a few characters, >but when I hit Return, I get the same core dump as before and, >apparently, for the same reason. Naturally, since atty(1) is not >meant to run under System V and SunOS 4.0 is partially based on System >V, it doesn't work too well here. "Naturally" that has little, if anything, to do with the problem. The reasons "atty" won't run under S5 are that: 1) it uses BSD-style tty "ioctl"s - the *documented* use of which SunOS 4.0 supports; 2) it uses BSD-style pseudo-ttys, and S5 didn't even *have* pseudo-ttys until recently (I think S5R3.2 might have them, albeit undocumented, for the benefit of e.g. "xterm" - but not BSD-style ones) - but SunOS 4.0 has BSD-style pseudo-ttys; 3) it may depend on other BSDisms - which SunOS 4.0 also has. The SunOS 4.0 tty driver isn't based on the S5 one; parts of its interface are, but the code isn't. The SunOS 4.0 *pseudo*-tty driver is what's at issue here, and it isn't based on the S5 one because S5 hasn't *had* one until recently. The problem is, again, that one particular hack that wasn't ever *documented* as being supported by BSD-style pseudo-ttys, namely the ability to do arbitrary tty "ioctl"s on the master side, is being used; the fix is to do them on the slave side instead. A patch that fixes, as far as I can tell, all of the instances where it's doing tty "ioctl"s on the master side is: *** atty.c.ORIG Sat Jun 3 16:25:00 1989 --- atty.c Mon Jun 5 11:57:49 1989 *************** *** 43,49 **** struct tty realtty; /* tty modes of real tty */ struct tty ptymodes; /* modes of pty */ int realfcntl; /* flags for real tty file descriptor */ ! int ptyfd; /* file descriptor of pty */ char ptydevice[12]; /* pty device */ int childpid; /* pid of child process */ jmp_buf sigjmp; /* where to jump on signal */ --- 43,50 ---- struct tty realtty; /* tty modes of real tty */ struct tty ptymodes; /* modes of pty */ int realfcntl; /* flags for real tty file descriptor */ ! int ptyfd; /* file descriptor of pty controller */ ! int ttyfd; /* file descriptor of pty slave */ char ptydevice[12]; /* pty device */ int childpid; /* pid of child process */ jmp_buf sigjmp; /* where to jump on signal */ *************** *** 215,221 **** #endif if (ioctl(ptyfd, TIOCPKT, (char *)&one) < 0) fatal("TIOCPKT"); ! if (setttymodes(ptyfd, &ptymodes) < 0) fatal("set pty modes"); fillttymode(); copyttysize(); --- 216,225 ---- #endif if (ioctl(ptyfd, TIOCPKT, (char *)&one) < 0) fatal("TIOCPKT"); ! if ((ttyfd = open(ptydevice, O_RDWR)) < 0) { ! fatal("Can't open tty"); ! } ! if (setttymodes(ttyfd, &ptymodes) < 0) fatal("set pty modes"); fillttymode(); copyttysize(); *************** *** 297,302 **** --- 301,307 ---- close(0); close(1); close(ptyfd); + close(ttyfd); fd = open("/dev/tty", O_RDWR); if (fd >= 0) { ioctl(fd, TIOCNOTTY, (char *)0); *************** *** 790,796 **** int pgrp; int zero = 0; ! if (ioctl(ptyfd, TIOCFLUSH, (char *)&zero) < 0) fatal("TIOCFLUSH"); needclearin++; if (ioctl(ptyfd, TIOCGPGRP, (char *)&pgrp) < 0) --- 795,801 ---- int pgrp; int zero = 0; ! if (ioctl(ttyfd, TIOCFLUSH, (char *)&zero) < 0) fatal("TIOCFLUSH"); needclearin++; if (ioctl(ptyfd, TIOCGPGRP, (char *)&pgrp) < 0) *************** *** 851,857 **** int changereal; int flags; ! if (getttymodes(ptyfd, &ptymodes) < 0) fatal("get pty modes"); noedit = 0; #ifdef notdef --- 856,862 ---- int changereal; int flags; ! if (getttymodes(ttyfd, &ptymodes) < 0) fatal("get pty modes"); noedit = 0; #ifdef notdef
guy@auspex.auspex.com (Guy Harris) (06/07/89)
>I compiled it on a Sun 3/160 running (I think) SunOS v. 3.5 and it >works fine. No changes were required to the make file. It's a pre-4.x vs. 4.x difference; the streams-based tty driver in 4.x makes it harder to make the "tty 'ioctl's on the master side'" grot work than does the pretty much vanilla 4.xBSD tty driver in pre-4.x releases.
mathew@jane.Jpl.Nasa.Gov (Mathew Yeates) (06/09/89)
In article <1747@auspex.auspex.com> guy@auspex.auspex.com (Guy Harris) writes: >A patch that fixes, as far as I can tell, all of the instances where >it's doing tty "ioctl"s on the master side is: > >patch posted I applied Guy's patch and everything seems to work fine except when I enter vi. Upon entering I'm immediately placed in insert mode. Also, upon leaving vi it appears that the keys are no longer bound. Anybody have any ideas? -mathew mathew@jane.jpl.nasa.gov
sid@friday.rtech.COM (Sid Shapiro) (06/09/89)
In article <16522@elroy.Jpl.Nasa.Gov> mathew@jane.Jpl.Nasa.Gov (Mathew Yeates) writes: >I applied Guy's patch and everything seems to work fine except when I >enter vi. Upon entering I'm immediately placed in insert mode. Also, >upon leaving vi it appears that the keys are no longer bound. Anybody >have any ideas? > >-mathew >mathew@jane.jpl.nasa.gov Yes - the same with more(1). Looks like going into raw mode screws you. Oh well. / Sid /
guy@auspex.auspex.com (Guy Harris) (06/10/89)
>I applied Guy's patch and everything seems to work fine except when I >enter vi. Upon entering I'm immediately placed in insert mode. Also, >upon leaving vi it appears that the keys are no longer bound. Anybody >have any ideas? The author made an inappropriate assumption about TIOCPKT_IOCTL; apparently, he thought that you get them "for free" if you turn on TIOCPKT or TIOCREMOTE mode. Bad assumption - you *don't* get them for free. You get them by turning on an undocumented mode which I think I already spoke of as a hack (which is probably in part why it's undocumented) *and* which is, as I remember, mutually exclusive with TIOCPKT mode (I don't remember if the information you can get with TIOCPKT mode, such as the TIOCPKT_FLUSH{READ,WRITE} messages, can be gotten in that mode or not). Try removing all code surrounded by #ifdef TIOCPKT_IOCTL ... #endif and removing the surrounding #ifdef/#endif on all code surrounded by #ifndef TIOCPKT_IOCTL ... #endif (or #undeffing TIOCPKT_IOCTL after all the include files have been pulled in); I tried that (the #undeffing, actually) and it seemed to work with "vi". You shouldn't be any worse off than you are on any vanilla 4.3BSD system....