clouds@ccwf.cc.utexas.edu (Kathy Strong) (11/23/90)
The author of this email has okayed me posting it; I think it's fascinating! From: Sherman Wilcox <wilcox@hydra.unm.edu> To: clouds@ccwf.cc.utexas.edu Subject: Re: Alone Thanks for the feedback, Kathy. I'm gonna answer this message private to you then figure out a way to start a thread on the newsgroup. We have started thinking about "disk real-estate management" more in terms of word frequency -- what will be the most likely words that users will want, then put them together. Let me give you a brief idea of what I envision for the ASL dictionary (yeah, I agree -- ain't it a SUPER use of the technology? -- I yearn to answer the folks who say that multimedia is just a bunch of hype -- this technology is the only viable solution to the problem of creating a signed language dictionary, I know, because we have the guy who wrote the *FIRST* ASL dictionar, William Stokoe, on our project!) We want a dictionary with the following features: (1) bilingual (ASL/English). This means users can start with an English word and look for an ASL translation, or start with an ASL word, perhaps one they've seen and don't know the meaning of, and search for an English translation (more on this later); (2) it will incorporate not only ASL (videodisk) and English (text) words, but also bilingual contextual sentences, definitions, usage notes, grammatical info, etc- in other words, it'll be a real dictionary; (3) there will be no direct link between ASL words and English words -- the user will have to go thru meaning (just as when you ask "How do you say 'critical' in language X" and I have to ask "What do you *mean* by the word 'critical' -- 'important' or 'to criticize' or 'life-threatening' (as in critical condition)?" the dictionary will do the same); (4) extensible (in at least two senses: user can enter notes about particular entries, sort of notes in the margin of the dictionary, or, the interface and dictionary engine can be used with videodisks other than those that we will provide, if the user is willing to do the coding herself. A key element of the design which will (god, I hope!) allow us to implement some of these features is that the dictionary will have a relational database underlying it. Codings (formal features of ASL words -- handshapes, movements, locations on the body, facial features, etc; frame numbers of ASL entries; English translation equivalents; etc etc) will be stored in the database. We are doing our first prototype in MacroMind Director. We hope to implement the database in a thing called HyperHIT -- a relational database engine implemented entirely in XCMDs (intended for use with Hypercard, but I think it'll hook to director as well -- here, I'm praying every night for luck!!!). So, I bet you are thinking of some interface goodies, aren't you. It makes me freak out when I think of all the neat interface ideas we could try out. For example, think about leading the user thru an ASL "phonetic" lookup -- the case where he sits down with an ASL word in mind and wants to know what it means (by the way, this is one of the examples of how multimedia enables this type of work: there is only *ONE* print media ASL dictionary in the world that lets you do this [written by Bill Stokoe] and most people don't understand how to use it because it requires them to learn a new, ASL "alphabet"). We will have to lead them to "sound out" (geez, what is the exact parallel for visual languages -- "light out"???) the word: the interface will say, in effect, "show me what the word looked like..." Then we need to give the user feedback on the results of the search given the info provided so far -- "if you give me only the shape of the hand and the location on the body, the dictionary has 30 words which might fit your description -- do you want to provide more info or look at those 30 words?" Etc etc et.... OK, I've taken up enough of your time. Do you think we have a do-able project here? I've got a 6-month NIH feasibility grant, and have to convince them to give me the real bucks, the phase two money, so that we can go into production. I'm convinced -- I hope they will be! ===================== I think there are analogies here to the way Chinese is currently done on computers: you start with a radical, maybe two, and the computer presents you with, say, 12 characters based on that radical: is it one of these? (This is Kathy talking now...) What are the radicals of an ASL word? Gesture? Direction? Hand shape? Distance? Position near body? --Kathy -- ........................................................................... : Kathy Strong : "Try our Hubble-Rita: just one shot, : : (Clouds moving slowly) : and everything's blurry" : : clouds@ccwf.cc.utexas.edu : --El Arroyo : :..........................................................................:
clouds@ccwf.cc.utexas.edu (Kathy Strong) (11/25/90)
More discussion on the ASL project was mistakenly send as email instead of posted... Here it is. ----------------------------- From: Sherman Wilcox <wilcox@hydra.unm.edu> To: clouds@ccwf.cc.utexas.edu Subject: Re: Interactive videodisc project--American Sign Language Newsgroups: comp.ivideodisc In-Reply-To: <40067@ut-emx.uucp> Organization: University of New Mexico, Albuquerque Cc: In article <40067@ut-emx.uucp> you write: >(This is Kathy talking now...) What are the radicals of an ASL word? >Gesture? Direction? Hand shape? Distance? Position near body? > In the work of William Stokoe that I mentioned, he identified what have come to be called the parameters of an ASL word (the equivalent, I think, of what you are calling the radicals): handshape, movement, and location. I fourth parameter was later proposed by other linguists, hand (palm) orientation. Stokoe developed a writing system for ASL based on these parameters. More recently, a writing system called "SignFont" was developed by Emerson & Stern (the same folks that I've seen mentioned lately doing development on speech recognition systems for Mac). The E&S system uses four parameters too: handshape, movement, location, and what they called "action area." We plan to use any & all of these systems in coding ASL words for retriebval in the ASL Hypermedia Dictionary. We hope that one of the advantages of building in hooks to a relational database is that we will be able to have multiple or redundant codings. This will allow us to have only citation forms on the real-estate-poor videodisks, but store codings in the real-estate rich database for variant forms (words signed or "pronounced" slightly differently, for example). And now, a question for anyone out there who might be reading this. I keep hearing that we will soon have the technology to do a project like this on CD-ROM. Does anyone have any idea approximately how much full-motion video we can expect to store on a CD-ROM? I realize this depends on many things (as I understand it, picture size, frames per second, compression factors, etc). Any ballpark figures? Sherman Wilcox Dept. of Linguistics University of New Mexico ----------------------- About CD-ROM: it's my understanding that a videodisc holds about 3.5 gigabytes of information, and a CD-ROM about 0.6 gigs. Can anyone confirm or dispute these figures? My grey matter is notoriously unreliable... --K -- ........................................................................... : Kathy Strong : "Try our Hubble-Rita: just one shot, : : (Clouds moving slowly) : and everything's blurry" : : clouds@ccwf.cc.utexas.edu : --El Arroyo : :..........................................................................:
wilcox@hydra.unm.edu (Sherman Wilcox) (11/25/90)
In article <40113@ut-emx.uucp> clouds@ccwf.cc.utexas.edu (Kathy Strong) writes: >About CD-ROM: it's my understanding that a videodisc holds about 3.5 gigabytes >of information, and a CD-ROM about 0.6 gigs. Can anyone confirm or dispute >these figures? My grey matter is notoriously unreliable... > How does video compression figure into this? I've seen articles that say video compression can yield compression ratios of over 100-to-1. Does this mean that the .6 gig CD-ROM now holds 6 gig? Speaking with someone at New Video last week (they are working on a DVI board for the Mac), he told me that he expected that I could get about 100 minutes of video on a CD-ROM (using their compression). -- Sherman Wilcox
clouds@ccwf.cc.utexas.edu (Kathy Strong) (11/26/90)
Got some more mail--yelled at the sender for not posting to the group, but as it happens he is unable to post from his site in Japan, so here goes.. -------------------- From trc!jnoc.go.jp!olling@ccut.cc.u-tokyo.ac.jp Sun Nov 25 18:52:50 1990 Date: Mon, 26 Nov 90 09:48:28 JST From: olling@jnoc.go.jp (Cliff Olling) To: clouds@ccwf.cc.utexas.edu Subject: Re: Interactive videodisc project--American Sign Language Well, it was pretty interesting. You could also advise your friend that there already exist computer-driven ASL output _and_ input systems! Output means that a user types a word on the keyboard and a mechanical hand moves to form the ASL "characters", which a ASL-competent person "reads" with by holding it with _their_ hand. Input means that the ASL-competent person wears a "data glove". This conveys the position of the hand and fingers to a computer which is running a program that recognizes the ASL word or char- acter, then produces the corresponding ascii text and synthesized speech. I saw this on PBS about 6 months ago. Clifford Olling Japan National Oil Corporation $@@PL}8xCDJ Technology Research Center $@@PL}3+H/5;=QJ Chiba City, Japan olling@trc.jnoc.go.jp $@KkD%K\6?1XJ 24hrs/day=>81+472-73-5831 -------------------- P.S. Apparently Cliff does get to READ the group, so if you have replies or comments, share 'em ... :-) -- ........................................................................... : Kathy Strong : "Try our Hubble-Rita: just one shot, : : (Clouds moving slowly) : and everything's blurry" : : clouds@ccwf.cc.utexas.edu : --El Arroyo : :..........................................................................:
wilcox@hydra.unm.edu (Sherman Wilcox) (11/26/90)
>Subject: Re: Interactive videodisc project--American Sign Language > >Well, it was pretty interesting. You could also advise your friend that >there already exist computer-driven ASL output _and_ input systems! Output >means that a user types a word on the keyboard and a mechanical hand moves >to form the ASL "characters", which a ASL-competent person "reads" with by >holding it with _their_ hand. Input means that the ASL-competent person >wears a "data glove". This conveys the position of the hand and fingers to >a computer which is running a program that recognizes the ASL word or char- >acter, then produces the corresponding ascii text and synthesized speech. > >I saw this on PBS about 6 months ago. > > Actually, what you saw was a robot hand that fingerspells -- this is not ASL. Fingerspelling is a manual representation of English graphemes (letters of the alphabet) -- one handshape for A, one for B, etc. Of course, when you string fingerspelled letters together, you get --- English. ASL is not English. As a matter of fact, though, the device you saw has some parsing problems. By that, I mean it has problems knowing where fingerspelled words begin and end. To solve this problem, the user (the one wearing the DataGlove) adds a special nonce handshape at the ends of words. One other problem -- the device has problems reading letters in context. Just as speech sounds are modified in context (which is why speech recognition is such a difficult nut to crack), fingerspelled letters are contextually modified (the fancy word for this os "coarticulation", and it happens in speech, fingerspelling, and even in signing ASL). Still, though, the device is very interesting. BTW, I have tried the DataGlove. It's a very intriguing piece of equipment. Very lightweight, doesn't interfere with range of motion, etc. We have considered (down the road!) trying to use the DataGlove as an input device for the ASL Hypermedia Dictionary. If the user wanted to look up an ASL word, she would just put on the DataGlove and sign it! Of course, I expect we would have just as many problems attempting to implement this as folks in speech recognition have. So for now, we are just putting our efforts into developing a useful dictionary. Sherman Wilcox Dept. of Linguistics University of New Mexico