[misc.handicap] Live Captioning

34AEJ7D@CMUVM.BITNET (Bill Gorman) (10/19/90)

Index Number: 11196

Here's a little tidbit from another list that some of the HI members of
this echo might find of interest.
----------------------------Original message----------------------------

|Today's EDUCOM keynote speech, by former President Jimmy Carter was
|open-captioned for the hearing impaired.  The big-video display in
|the auditorium showed a textual representation of the speech as
|Carter spoke. This was provided by "11-Alive," an Atlanta
|television station.
|
|The system must have used some kind of voice-recognition algorithm,
|because no human typist that I know could have kept up with the
|speaker at times.  The weakness of the voice-recognition system was
|made painfully obvious to attendees, when those with the ability to
|hear the presentation noticed substitutions like:
|
|"man well" noriega, "wak dem iks" for academics, "oath yope yam"
|for Ethiopia, "Jap neens" for Japanese, "My Robe by" N Nairobi for
|"Ken Yeah" for Kenya, "Home Jean yes" for homogeneous.
|
|Carter's speech was thoughtful and moving (he talked about
|academia's moral responsibilities to the third world), but the
|seriousness of the speech was undercut by the occasional giggle
|from the audience.
|
|Later in the speech, human control seemed to be asserted a bit
|more, and "another country" was frequently substituted for the name
|of a third-world country, but Carter must have been most puzzled
|when he explained that the Carter Foundation was nonpartisan, and
|that "Prominent Republicans" worked closely with him on every major
|project.  It was presented to the hearing-impaired (and to the rest
|of us) as "Prominent Rubble Cans."  Of course, the audience broke
|into laughter, and I suspect he still doesn't know why.
|
|I'm a strong supporter for appropriate adaptive technology, but a
|low-tech solution (an ASL interpreter) would have been less
|distracting and reached many (though not all) of the
|hearing-impaired.

SMT@CORNELLC.cit.cornell.edu (Mike Oltz) (10/24/90)

Index Number: 11239

I was present at Jimmy Carter's talk at EDUCOM '90.  I would be
surprised if voice recognition was used for that.  I assumed that
there was someone operating a court recorder machine connected to
a computer.  A court recorder machine is a black box with some
buttons on it similar to a Perkins brailler.  Each combination of
buttons stands for a particular syllable.  Usually, this gadget
prints out in a somewhat encoded form on adding-machine tape.
It is used in court because it is absolutely silent, and because
with an experienced operator it can keep up with a normal rate
of speech.  Next time you or a friend watch a news report of a
court case, occasionally the court recorder will show up.

Some companies have automatic telephone answering equipment that
responds to touch-tone codes.  In some cases, where the topic is
sufficiently specific, you can spell out a word simply by pushing
the touch-tone buttons with the appropriate letters on them, and
the computer will guess what word you mean by looking it up in a
table.  If it is confused, it will ask you which of several choices
you really meant.  This kind of lookup could be done quite easily
with court recorder output.  However, in live captioning, you cannot
back up to fix mistakes.  'Pun' type errors would be expected.

The kinds of errors seen in the open captioning, combined with
the obvious human interpretation at times (neither the human nor
the computer had ever heard of 'Zambia') are just what one would
see with a court recorder and computer lookup combined.  At least,
that is my hypothesis.

The open captioning equipment was borrowed from channel 11, Atlanta.
Therefore, it was not surprising that the computer was familiar with
vocabulary Jimmy Carter would use.  'Rosalynn' was no problem.
---
Mike Oltz, Interactive Multimedia Group, Cornell University
myk@cornella.bitnet