[comp.editors] Can "vi" handle chars above 127 ASCII?

dcarson@unix.cis.pittsburgh.edu (David Carson) (06/16/89)

I work on a Sun 3/50 and have a font which is bilingual in nature.
The low order characters (the normal 7-bit ASCII chars) are English
and the high order chars (top bit set) are Greek.  Therefore, I should be 
able to switch nicely between the two without having to change fonts.

I have figured out how to enter and display the high order chars
on the Sun (see note at end on this, for those interested). 
Now I want to be able to do something useful with the font
like edit a file with it!!!

I will have to admit that I use "vi" and take my chances on the 
flames that will result.  Does anyone know how to set up "vi"
so that it can handle a file that contains chars with the 8th bit 
set as an ASCII file.  (By default, "vi" calls
this a non-ASCII file, and therefore it cannot be edited.)

I don't think I'm quite ready to be converted to another editor
yet, though soon I will be (when I finish my master's).  Do
other editors allow editing of such files?


* What I know about accessing the high order chars:

Open a window specifying the appropriate font.
Run the Bourne shell (Yes that's right.  For some reason, the C shell
	chokes on this.)
Allow 8 bit chars by typing: stty pass8
Type any letter with the "Left" or "Right" key depressed.  The resulting
	character output will be the char in the same position in the
	high order of the font.
If you have a file with such characters in it, these will be displayed
	as such also, when, for example, you do a "cat."
Type: stty -pass8 to terminate 8-bit mode
Kill the Bourne shell (unless you are some kind of masochist and like the
	Bourne shell).

lars@iclswe.UUCP (Lars Tunkrans) (06/21/89)

In article <18473@unix.cis.pittsburgh.edu> dcarson@unix.cis.pittsburgh.edu (David  Carson) writes:
>I work on a Sun 3/50 and have a font which is bilingual in nature.
>The low order characters (the normal 7-bit ASCII chars) are English
>and the high order chars (top bit set) are Greek.  Therefore, I should be 
>able to switch nicely between the two without having to change fonts.
>
>I have figured out how to enter and display the high order chars
>on the Sun (see note at end on this, for those interested). 
>Now I want to be able to do something useful with the font
>like edit a file with it!!!
>
>I will have to admit that I use "vi" and take my chances on the 
>flames that will result.  Does anyone know how to set up "vi"
>so that it can handle a file that contains chars with the 8th bit 
>set as an ASCII file.  (By default, "vi" calls
>this a non-ASCII file, and therefore it cannot be edited.)

It is probably not trivial to find a version of an 8-bit version of vi 
that will run on your system. American computer manufacturers mostly forget
that there are other languages than English in the world. However the 
Europe based ( I think ) standardisation organisation X/OPEN has now 
specified that the text-handling Unix utilities shall be 8-bit and use 
the ISO 8859/1 character set which includes most of the different non-
English characters found in European languages. Unfortunatly Greek is a
compleatly different ballgame since it does not use the Latin alphabet.

The box I am running news on is one of the first systems that is a full 
8-bit implementations of Unix, The ICL DRS300 80386 based machine running
sVr3.1. Can't be demonstrated over the net though since there are a few 
7-bit systems between you and me.

-- Lars

-- 
//     /////  ///     | Lars Tunkrans  Distributed Resource Systems support.
 //  ///      ///     | UUCP: {uunet,mcvax,munnari,cernvax,diku,inria,prlb2,tut
///  ///      ///     | ,ukc,unido} !sunic!iclswe!lars  Phone +46 (0)76096368
///    /////  /////// |            ( Standard Disclaimer )

bengsig@oracle.nl (Bjorn Engsig) (06/21/89)

From article <193@iclswe.UUCP> by lars@iclswe.UUCP (Lars Tunkrans):
|the ISO 8859/1 character set which includes most of the different non-
|English characters found in European languages. Unfortunatly Greek is a
|compleatly different ballgame since it does not use the Latin alphabet.
Yes, but one of the other ISO 8859/N character sets has the greek
alphabet in it, and using a terminal with both latin and greek alphabets
should do the trick, (if such a thing exists)
|
|The box I am running news on is one of the first systems that is a full 
|8-bit implementations of Unix, The ICL DRS300 80386 based machine running
|sVr3.1. 
You should also mention HP-UX and AIX here, and probably others.  As far as
I know, all SysV.3.1 based have the 8-bit support.
-- 
Bjorn Engsig, ORACLE Europe         \ /    "Hofstadter's Law:  It always takes
Path:   mcvax!orcenl!bengsig         X      longer than you expect, even if you
Domain: bengsig@oracle.nl           / \     take into account Hofstadter's Law"

jewett@hpl-opus.HP.COM (Bob Jewett) (06/22/89)

> >Does anyone know how to set up "vi"
> >so that it can handle a file that contains chars with the 8th bit 
> >set as an ASCII file.
> 
> It is probably not trivial to find a version of an 8-bit version of vi 
> that will run on your system. American computer manufacturers mostly forget
> that there are other languages than English in the world.

Hewlett-Packard presently supports 8-bit and 16-bit characters in vi, as
well as various other tools, such as "sort"*, as part of NLS (Natural
Language System).  I use roman8 8-bit often, mostly to access special
characters, like 1/2, and +-.  Other 8-bit character sets include kana8,
arabic8, turkish8, and greek8, but I've never tried them.

(People who speak Indo-european languages mostly forget that there are
languages in the world not based on alphabets.  (;-)

Bob

* (according to the documentation) in Swedish "a-circle" sorts after
"z", while in German, it follows "a", and Canadian French upper-cases
c-cedilla to C-cedilla, while (European) French drops the cedilla when
upper-casing.

[not an official statement of the Hewlett-Packard Company]

prc@erbe.se (Robert Claeson) (06/22/89)

In article <18473@unix.cis.pittsburgh.edu> dcarson@unix.cis.pittsburgh.edu (David  Carson) writes:

>I will have to admit that I use "vi" and take my chances on the 
>flames that will result.  Does anyone know how to set up "vi"
>so that it can handle a file that contains chars with the 8th bit 
>set as an ASCII file.  (By default, "vi" calls
>this a non-ASCII file, and therefore it cannot be edited.)

The machine I'm writing this on, an Encore Multimax running UMAX V (System V
Release 3.1) has a vi that can handle 8-bit characters. Whenever it encounters
an 8-bit character in "type-in" mode, it is printed as an octal number like
in the C programming language (ie, "\0253"). When I hit the Escape key to go
out of insert (or whatever) mode, or press Return while in insert mode, the
line is rewritten to show the true characters (ie, u-umlaut instead of the
octal variety). As far as I know, this is the way that vi works on all SVR3.1
systems.

As for your Sun, I don't think there's much you can do except for switching
to some other editor (or wait for SVR4 to appear on it). I currently use
MicroEmacs 3.9e, with the "strip-the-eight-bit" mask removed from the keyboard
input function. Works like a charm with the ISO 8859/1 8-bit character set.

-- 
          Robert Claeson      E-mail: rclaeson@erbe.se
	  ERBE DATA AB

leech@Apple.COM (Jonathan Patrick Leech) (06/25/89)

In article <193@iclswe.UUCP> lars@iclswe.UUCP (Lars Tunkrans) writes:
>It is probably not trivial to find a version of an 8-bit version of vi
>that will run on your system. American computer manufacturers mostly forget
>that there are other languages than English in the world.

    This is not neccessarily a bad thing, from my point of view.
DEC's incredibly brain-damaged LK201 keyboard may be partially
attributed to internationalization. I have not the least interest in
non-English keyboards and wish they had kept American users in mind in
this case.
--
    Jon Leech (leech@apple.com)
    Apple Integrated Systems
    __@/

bink@aplcen.apl.jhu.edu (Ubben Greg) (06/25/89)

In article <739@ecrcvax.UUCP> Diomidis Spinellis writes:
> In article <18473@unix.cis.pittsburgh.edu> David Carson writes:
>> ...
>> Does anyone know how to set up "vi" so that it can handle a file that
>> contains chars with the 8th bit set as an ASCII file?
> ...
> On a last count vi had a hard coded mask for the lower 7 bits in at least
> 46 different places.  Thus using it or modifying it for an 8 bit character
> set is not trivial.
> ...

Ok, so the SVR2 version of vi can't handle non-ASCII files.  What I really
want is a way to make vi work with a weird terminal type we have where all
the CONTROL CODES (such as "move up one line") are above 0200.  This is the
only thing keeping us from using vi on our terminals.  The terminfo entry
is correct (curses works fine), but vi insists on stripping the 8th bit.
Is the answer to this any easier than for getting vi to work with 8-bit
files?  (We are using SVR2 Unix.)
						-- Greg Ubben
						   bink@aplcen.apl.jhu.edu

ked@garnet.berkeley.edu (Earl H. Kinmonth) (06/25/89)

In article <32661@apple.Apple.COM> leech@Apple.COM (Jonathan Patrick Leech) writes:

>This is not neccessarily a bad thing, from my point of view. I have not
>the least interest in non-English keyboards and wish they had kept
>American users in mind in this case.

As everyone with an IQ of more than 127 knows, GAWD proclaimed that anyone
who did not speak ASCII with an American accent was bound to go to hell,
the opinions of Europeans, Japanese, Chinese, Indians, and ninety-plus
percent of the computer-using bodies on this planet notwithstanding.

leech@Apple.COM (Jonathan Patrick Leech) (06/25/89)

In article <25745@agate.BERKELEY.EDU> ked@garnet.berkeley.edu (Earl H. Kinmonth) writes:
>In article <32661@apple.Apple.COM> leech@Apple.COM (Jonathan Patrick Leech) writes:
>>This is not neccessarily a bad thing, from my point of view. I have not
>>the least interest in non-English keyboards and wish they had kept
>>American users in mind in this case.
>As everyone with an IQ of more than 127 knows, GAWD proclaimed that anyone
>who did not speak ASCII with an American accent was bound to go to hell,
>the opinions of Europeans, Japanese, Chinese, Indians, and ninety-plus
>percent of the computer-using bodies on this planet notwithstanding.

    As anyone who read the complete text of my posting will realize,
your comment is completely irrelevant to mine.	I referred to
keyboards, not character sets.
--
    Jon Leech (leech@apple.com)
    Apple Integrated Systems
    __@/

ked@garnet.berkeley.edu (Earl H. Kinmonth) (06/25/89)

In article <32663@apple.Apple.COM> leech@Apple.COM (Jonathan Patrick Leech) writes:
>In article <25745@agate.BERKELEY.EDU> ked@garnet.berkeley.edu (Earl H. Kinmonth) writes:
>>In article <32661@apple.Apple.COM> leech@Apple.COM (Jonathan Patrick Leech) writes:

>>As everyone with an IQ of more than 127 knows, GAWD proclaimed that anyone
>>who did not speak ASCII with an American accent was bound to go to hell,
>>the opinions of Europeans, Japanese, Chinese, Indians, and ninety-plus
>>percent of the computer-using bodies on this planet notwithstanding.
>
>    As anyone who read the complete text of my posting will realize,
>your comment is completely irrelevant to mine.	I referred to
>keyboards, not character sets.

Wellllll, pardon me for jerking your chain. I should have realized that
anyone affiliated with "Apple Integrated Systems" must have an inside
track to GAWD. Nevertheless, as an American (three generations on the
maternal side, five generations on the paternal side), I still have a
dim awareness that not all of the world ~types~ American acented ascii.
Indeed, true red-blooded, DAR-certified American that I am, there are
times (about every two or three minutes) when I wish that ~keyboard~
manufacturers realized that not everyone was typing the names of
generic wasps (smith, jones, leech, etc.) into an airline reservation
system.

I'd like a nice, neat way of typing some real god damn foreigner names
without twisting my fingers in knots.

Finally, on the principle that a barking dog ought to really have
something to bark about, let me observe, as a specialist in modern
Japanese history, that Apple Fornicated (or otherwise) Systems has an
almost zero presence in the world's second largest computer market.
Even the Intentionally Banal Machines Corporation can do better than
that (about 27 percent better).

leech@Apple.COM (Jonathan Patrick Leech) (06/26/89)

In article <25753@agate.BERKELEY.EDU> ked@garnet.berkeley.edu (Earl H. Kinmonth) writes:
>Wellllll, pardon me for jerking your chain. I should have realized that
>anyone affiliated with "Apple Integrated Systems" must have an inside
>track to GAWD. Nevertheless, as an American (three generations on the
>maternal side, five generations on the paternal side), I still have a
>dim awareness that not all of the world ~types~ American acented ascii.
>Indeed, true red-blooded, DAR-certified American that I am, there are
>times (about every two or three minutes) when I wish that ~keyboard~
>manufacturers realized that not everyone was typing the names of
>generic wasps (smith, jones, leech, etc.) into an airline reservation
>system.
>
>I'd like a nice, neat way of typing some real god damn foreigner names
>without twisting my fingers in knots.

    Fine.  I'd like a nice, neat way of typing control-A and ESC
without twisting my fingers in knots, characters I need to type far
more often than "god damn foreigner names."  Internationalized
keyboards don't tend to provide this capability, which was the point I
tried to make.

>Finally, on the principle that a barking dog ought to really have
>something to bark about, let me observe, as a specialist in modern
>Japanese history, that Apple Fornicated (or otherwise) Systems has an
>almost zero presence in the world's second largest computer market.
>Even the Intentionally Banal Machines Corporation can do better than
>that (about 27 percent better).

    Once again, your gratuitous obscenities are unrelated to my
original posting.  The only person whose "chain was jerked" here is
yourself.

    Try reading the Usenet posting guidelines sometime.

    This is completely irrelevant to comp.editors at this point.
Followups redirected to alt.flame.
--
    Jon Leech (leech@apple.com)
    Apple Integrated Systems
    __@/

guy@auspex.auspex.com (Guy Harris) (06/28/89)

 >    Fine.  I'd like a nice, neat way of typing control-A and ESC
 >without twisting my fingers in knots, characters I need to type far
 >more often than "god damn foreigner names."  Internationalized
 >keyboards don't tend to provide this capability, which was the point I
 >tried to make.

The problem of control and ESC keys, and of internationalized keyboards,
are separate.  Some vendors have either eliminated the ESC key or put it
somewhere other than e.g. to the left of the (! 1) key; they may have
done so to match some international standards.  However, it is *NOT*
necessary to do this for US keyboard layouts; other vendors have
internationalized keyboards that, at least in the US layout, don't stick
ESC off in the hinterlands or put an extra key between Shift and Z, for
example. 

diomidis@ecrcvax.UUCP (Diomidis Spinellis) (07/22/89)

In article <18473@unix.cis.pittsburgh.edu> dcarson@unix.cis.pittsburgh.edu (David  Carson) writes:
>I work on a Sun 3/50 and have a font which is bilingual in nature.
>The low order characters (the normal 7-bit ASCII chars) are English
>and the high order chars (top bit set) are Greek.  Therefore, I should be 
>able to switch nicely between the two without having to change fonts.
>[...]
>Does anyone know how to set up "vi"
>so that it can handle a file that contains chars with the 8th bit 
>set as an ASCII file.  (By default, "vi" calls
>this a non-ASCII file, and therefore it cannot be edited.)
Many versions of vi check the magic number (the first two bytes in the
file) before reading the file.  One of the checks performed is to see
if any of those first two characters has the high bit set.  If this is
the case the error ``Non-ascii file'' is reported.  Even if you
succeeded in making vi read such a file you would still have the
problem of displaying it and entering characters.  On a last count vi
had a hard coded mask for the lower 7 bits in at least 46 different
places.  Thus using it or modifying it for an 8 bit character set is
not trivial.  I think the next (or the following ones) major releases
of SunOS and System V are supposed (among other things) to fully support 
8 bit character sets.

>I don't think I'm quite ready to be converted to another editor
>yet, though soon I will be (when I finish my master's).  Do
>other editors allow editing of such files?
Emacs (as one would expect) does.

Diomidis
-- 
Diomidis Spinellis          European Computer-Industry Research Centre (ECRC)
Arabellastrasse 17, D-8000 Muenchen 81, West Germany        +49 (89) 92699199
USA: diomidis%ecrcvax.uucp@pyramid.pyramid.com   ...!pyramid!ecrcvax!diomidis
Europe: diomidis@ecrcvax.uucp                      ...!unido!ecrcvax!diomidis

ked@garnet.berkeley.edu (Earl H. Kinmonth) (07/22/89)

The version of vi supplied with the MKS Toolkit for MSDOS handles the
extended charcter set.

debra@alice.UUCP (Paul De Bra) (08/02/89)

>...  I think the next (or the following ones) major releases
>of SunOS and System V are supposed (among other things) to fully support 
>8 bit character sets.

As I recall vi in System V r3.2 handles 8 bit characters.
This does *not* imply that it can be used as a binary editor though.
Line length is still limited.

Paul.
-- 
------------------------------------------------------
|debra@research.att.com   | uunet!research!debra     |
------------------------------------------------------

irf@kuling.UUCP (Bo Thide') (08/03/89)

In article <9723@alice.UUCP> debra@alice.UUCP () writes:
>>...  I think the next (or the following ones) major releases
>>of SunOS and System V are supposed (among other things) to fully support 
>>8 bit character sets.
>
>As I recall vi in System V r3.2 handles 8 bit characters.
>This does *not* imply that it can be used as a binary editor though.
>Line length is still limited.
>
>Paul.

The HP-UX version of 'vi' correctly handles 7, 8 and 16 bit characters
and has done so for many HP-UX releases now.  I use it all the time
to produce documents in Swedish (I use 'vi' as a word processor..!)
and get the correct national Swedish glyphs displayed on the screen
as well as the correct ASCII codes (127-255) on the file.  The HP-UX
'vi' even knows about case shifting with '~' for those characters.
Very neat!


   ^   Bo Thide'--------------------------------------------------------------
  | |       Swedish Institute of Space Physics, S-755 91 Uppsala, Sweden
  |I|    [In Swedish: Institutet f|r RymdFysik, Uppsalaavdelningen (IRFU)]
  |R|  Phone: (+46) 18-403000.  Telex: 76036 (IRFUPP S).  Fax: (+46) 18-403100 
 /|F|\        INTERNET: bt@irfu.se       UUCP: ...!uunet!sunic!irfu!bt
 ~~U~~ -----------------------------------------------------------------sm5dfw

guy@auspex.auspex.com (Guy Harris) (08/04/89)

 >>...  I think the next (or the following ones) major releases
 >>of SunOS and System V are supposed (among other things) to fully support 
 >>8 bit character sets.
 >
 >As I recall vi in System V r3.2 handles 8 bit characters.

The one in S5R3.1 does, and the one in the next SunOS minor release
(i.e., 4.1 - the next major release would presumably be called 5.0) is
based on the S5R3.1 one and as such should support 8-bit character sets. 
(It did when I tried it, anyway....)