g-chapma@GUMBY.WISC.EDU (Ralph Chapman) (04/20/87)
I was asked to forward this message in response to the article by Linda G. Means: From sklein@rsch.wisc.edu Thu Apr 16 14:20:13 1987 Date: Thu, 16 Apr 87 15:18:42 CDT From: sklein@rsch.wisc.edu (Sheldon Klein) Message-Id: <8704162018.AA06807@rsch.wisc.edu> Subject: Re: grammar checkers I accept the note as one more piece of evidence that the field of Comp Sci, Comp Ling & AI are providing the prosthetic devices to allow otherwise unemployable segments of the World population to function for pay in occupations for which they would have been congenitally unqualified in an earlier era. Those capable of constructing complex sentences which, to some pundits of an earlier era reflected the ability to think complex thoughts, will have to abandon their elitist modes of cognition for the greater benefit of the larger segment of humankind.
MEANS@gmr.COM ("Linda G. Means") (05/04/87)
mom: toaster oven, kimono Todd Ogasawara writes in AILIST Digest v.5 #108: >I think that if these style checking tools are used in conjunction >with the efforts of a good teacher of writing, then these style >checkers are of great benefit. It is better that children learn a >few rules of writing to start with than no rules at all. Of course, >reading lots of good examples of writing and a good teacher are still >necessary. Sure, but the problem is the bogus rules that the child is likely to infer from the output of the style-checking program, like never write a sentence longer than x words, or don't use passive voice, or try not to write sentences with multiple clauses. >On another level... I happened to discuss my response above with one >of my dissertation committee members. His reaction? He pulled out >a recent thesis proposal filled with red pencil marks (mostly >grammatical remarks) and said, "So what if the style checkers are >superficial? Most mistakes are superficial. Better that the style >checker should find these things than me." Sounds like a rather irresponsible attitude to me, given the state of the art of automatic style checkers. Your prof needs a graduate student slave if he dislikes having to correct student grammar errors. Let's consider separately the issues of grammar correction and stylistic advice (the two worlds partially overlap, but remain distinct some areas). 1. Grammar. As your prof points out, lots of grammar errors are superficial, but your commercial grammar checker will fail to find all of them, correct perceived mistakes which really aren't, and give plenty of bad advice. Those programs "know" less about grammar than the students who use them. Any bonafide grammatical errors which can be found by the commercially available software could also be found by the writer if he were to proof his paper carefully. It grieves me to think of students failing to proof their own papers because the computer can do it for them. 2. Style. The analysis of writing style is not a superficial task; it is, in fact, a kind of expertise not found in many "literate" individuals. In my experience, the best way to learn to write well is to scrutinize your work in the company of a good writer who will think aloud with you while helping you to rewrite sentences. I've successfully taught various people to write that way. The second best method is a patient teacher's red pen. In both cases, your prose is being evaluated by someone who is trying to understand what you are trying to communicate in your writing. You must understand that this is not the case with the computer. It probably has no way of representing the discourse as a whole; all analysis is performed at the sentence level with a heavy emphasis on syntax and with no semantic theory of style. The result? Stylistic advice which is so superficial as to be useless. Many years of research in the area of computational stylistics have provided evidence that although some (few) stylistic discriminators can be found through syntactic analysis, the features which contribute to textual cohesion and to a given writer's "stylistic fingerprint" cannot. Researchers are still stymied by the problem of identifying stylistically significant features of a text. Yet the program advocated by Carl Kadie feigns an understanding of the effect that the prose will have on its reader; it generalizes from syntactic structure to stylistic impact. Look at the summary generated at the end of the text. The program equates active voice and short sentences with "directness". I won't take the time here to argue against the use of fuzzy adjectives like 'direct', 'expressive', 'fresh', and so on to describe prose, since the use of such imprecise language is a longstanding tradition in the arena of literary criticism. I can't tell you exactly how to make your writing "direct", but I know that directness cannot always be computed empirically, which is how your machine computes it. A paragraph of non sequiturs probably shouldn't be characterized as direct, even if all sentences are short and contain only active verbs. An aside to Ken Laws: You questioned whether the topic of automatic style checkers is appropriate to AILIST: is it AI? I believe it is. The study of computational stylistics is a difficult natural language problem with a long history. Topics range from authorship studies of anonymous works to trying to identify stylistic idiosyncrasies to automatic style advisors. In general, many theoretical issues carry over from other areas of natural language processing, like discourse analysis and understanding human reasoning processes. Think of a favorite author. You may sometimes recognize a sample of his writing without even knowing who wrote it, or you may say of another writer, "Gee, his style reminds me of X". You may put down a book which you started reading because the style is too "obtuse". How specifically does a writer use the language to produce that effect? What characteristics of a text must we identify to enable a computer to make judgments about style? Of course, any advances made in tackling these issues may also be of use in the area of text generation. - Linda Means GM Research Laboratories means%gmr.com@relay.cs.net
miller@ACORN.CS.ROCHESTER.EDU.UUCP (05/08/87)
Date: Mon, 4 May 87 12:18 EST
From: "Linda G. Means" <MEANS%gmr.com@RELAY.CS.NET>
An aside to Ken Laws:
You questioned whether the topic of automatic style checkers is
appropriate to AILIST: is it AI? I believe it is. The study of
computational stylistics is a difficult natural language problem
with a long history. [...]
- Linda Means
GM Research Laboratories
means%gmr.com@relay.cs.net
Personally, I suspect the question is should the discussion be carried in
AIList or moved to NL-KR. NL-KR is, indeed, already picking it up; further
such things are directly in NL-KR's scope, and the idea of the list was to be
somewhat subtractive from AIList, keeping traffic on Ken's list a little
lower.
Brad Miller
nl-kr-request@cs.rochester.edu
miller@cs.rochester.edu
miller@acorn.cs.rochester.edutodd@uhccux.UUCP (05/08/87)
I would rather see mainstream AI-related topics given space in AIList rather than take up more space with yet another "grammar checker" related messsage. And while I accept the criticism of my comments in the spirit of academic give and take in the exchange of ideas, I will make, I hope, the final comment in this discussion and then consider it closed for the moment. I wish the two following commentators "Linda G. Means" <MEANS%gmr.com@RELAY.CS.NET> gilbert@aimmi.UUCP (Gilbert Cockton) had *read* what I said before they reacted. I wrote: >I think that if these style checking tools are used in conjunction ^^^^^^^^^^^^^^^^^^^ *********** >with the efforts of a good teacher of writing, then these style ^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^ >checkers are of great benefit. It is better that children learn a >few rules of writing to start with than no rules at all. Of course, >reading lots of good examples of writing and a good teacher are still ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ *** ^^^^^^^^^^^^^^^^^^^^^^^^ *** >necessary ^^^^^^^^^ I don't think that anyone would seriously suggest that these borderline "AI" programs be used *exclusively* to teach children (or people of any other age group) to write. My thanks to Ken Laws for allowing this interesting little discussion to take place here instead of forcing us to move it to AI-ED (where it probably belongs, I admit). Now, let's get back to mainstream AI :-) Todd Ogasawara, U. of Hawaii Computing Center UUCP: {ihnp4,seismo,ucbvax,dcdwest}!sdcsvax!nosc!uhccux!todd ARPA: uhccux!todd@nosc.MIL INTERNET: todd@uhccux.UHCC.HAWAII.EDU [NL-KR@CS.ROCHESTER.EDU has also been reprinting these messages. -- KIL]
HAYES@SPAR-20.ARPA.UUCP (05/08/87)
On 'Style checkers'. Of course one shouldnt criticise to extremes, and no doubt a competent adult would find these things useful sometimes. That wasnt what I was complaining about: it was using them to INFLUENCE children. The word was chosen carefully. Marvin isnt going to think that the thing should be taken as an authority on how to write, or that in order to write well he should simply arrange that the style checker doesnt find any problems. But if they are used to grade or influence the way children write in a school setting, that is exactly what almost all kids will rapidly decide. ( Unless an extraordinarily good teacher is in charge, and maybe even then. Just think of the pressures on a teacher to come to rely on the programs judgement, and on a pupil to take the machine as authoritative. The machine finds no fault with Joes essay and complains about Bettys, but the teacher gives Betty a higher grade..... ) Pat Hayes -------
gilbert@aimmi.UUCP.UUCP (05/09/87)
In article <MINSKY.12299573623.BABYL@MIT-OZ> MINSKY@OZ.AI.MIT.EDU writes: >I agree with Todd, Ogasawara: one should not criticise to extremes. What does this mean? I thought accuracy was the only goal in criticism, not avoiding the ends of some quaint invented continuum. Can we have a style checker which rates our extremity with marks out of 10 (0 for credulous and 10 for rampant scepticism perhaps :-)) > I also used it to establish a "gradient". The early >chapters are written at a "grade level" of about 8.6 and the book ends >up with grade levels more like 13.2 - using RightWriter's quaint >scale. How about MIT turning some of its resources towards VALIDATING this quaint gradient? Do you seriously think there is any real computable ordering, partial or otherwise, which can be applied to your chapters and actually square up with any of our everyday evaluations of text complexity? If so, where's the beef? How would US data square up with European data. English teachers in the UK, for example, do not apply unimaginative inflexible rules to students' writing, so it could be that many educated English students will be turned off by an 8.6 introduction. Luckily we have not yet been carried away with the belief that all complex ideas can have banal presentations without bowdlerisation creeping in. Doubtless your style checker would ask me to drop 'bowdlerise'? What should I have used instead, given that I want an EXACT synonym with all its connotations? When I taught, I would have advised my students to find a dictionary (many of them carried them anyway - and I taught children from a wide range of cultural and economic backgrounds). God knows what the French would say to a mechanical style checker (a Franglais remover would go down well though). Finally, how on earth do these style checkers know which words will be commonly understood? Surely they don't use word frequency in newspapers or something like that? Does the overuse of a word in the media imply universal understanding of/consensus on its meaning - eg. 'moral', 'freedom', 'extreme', 'quaint', 'seriously', 'inflexible' etc? Does the limited use of a word in the media imply universal ignorance - eg. 'ok', 'alright', 'balls', 'claptrap', 'space cadet', 'avid', 'stroppy', 'automaton'? I would not regard any of the criticisms of style checkers I have read as 'extreme' at all. The difference seems to be one of gross credulity versus informed criticism. People who know nothing about good style will believe all the things which the style checker hackers have MADE UP - I defy any style checker implementor to point to a sound experimental/statistical basis for the style rules they have palmed off onto their gullible customers. Perhaps they did at least read some books by self-proclaimed authorities, but this would only shift the charge from invention to uncritical acceptance. I'd still be unimpressed. This may sound extreme - that however is irrelevant. The point is, am I accurate?. Note that my substantial assertions are few: i) Style don't compute. Verify by Chinese characters test between a style checker and the editors of the New Yorker (US) or the Listener (UK). Other quality magazine editors will do. Can you spot the editors' critiques? ii) The current 'reading age' metrics have no validity. They are bogus psychometric tools. Operationally I am saying that their will be no strong correlation (say r > 0.9, p < 0.001) between the reading age of text and a reader's performance on a comprehension test. Allow the author to add a glossary and the correlation will weaken. People can learn new words you know. iii) Current measures of popular understanding of words are equally bogus and there is NO decent research to back it up. There has been some good work on correlating vocabulary with educational achievement, but this tells us nothing about the typical adult's vocabulary. Every assertion above is falsifiable, so let's all forget about emotive subjective concepts like extremity (= I disagree a lot and wish you hadn't said that) and get back to an objective, informed debate. The motion is: "All computer based style checkers can stunt the literary growth of their users" A second order effect is that, although 1,000 chimpanzees could between them type out the works of Shakespeare given enough time, they would fail miserably if their output had to be passed by a computer style checker. To be, or not to be, that is the question. >> Sentence starts with infinitive Sentence has no subject. Whether it is .... >> "Whether" may not be understood by people who just read comics. (? spelling mistake = weather ?). -- Gilbert Cockton, Scottish HCI Centre, Ben Line Building, Edinburgh, EH1 1TN JANET: gilbert@uk.ac.hw.aimmi ARPA: gilbert%aimmi.hw.ac.uk@cs.ucl.ac.uk UUCP: ..!{backbone}!aimmi.hw.ac.uk!gilbert
preece%mycroft@GSWD-VMS.ARPA.UUCP (05/12/87)
Out of curiosity, would any of the automated checkers people have been talking about have caught the "their" for "there" error in the following: > ii) The current 'reading age' metrics have no validity. > They are bogus psychometric tools. Operationally I am > saying that their will be no strong correlation (say r > > 0.9, p < 0.001) between the reading age of text and a > reader's performance on a comprehension test. Allow the > author to add a glossary and the correlation will weaken. > People can learn new words you know. -- scott preece gould/csd - urbana uucp: ihnp4!uiucdcs!ccvaxa!preece arpa: preece@gswd-vms
john@viper.lynx.mn.ORG (John Stanley) (05/13/87)
In article <8705121527.AA01698@gswd-vms.ARPA> preece%mycroft@GSWD-VMS.ARPA (Scott E. Preece) writes: >Out of curiosity, would any of the automated checkers people have >been talking about have caught the "their" for "there" error in.... I don't know about the ones people have been talking about, but I do know there is a program under development that can handle "there" vs "their" or, for that matter, the "two" vs "too" vs "to". It's a new program, not yet released, but should be out by the end of the year. The company working on it is a small Minnesota based company working on AI related software products for mini/micro/word-processor applications. --- John Stanley (john@viper.UUCP) Software Consultant - DynaSoft Systems UUCP: ...{amdahl,ihnp4,rutgers}!{meccts,dayton}!viper!john
gilbert@aimmi.UUCP (Gilbert Cockton) (05/21/87)
In article <974@viper.UUCP> viper!john (John Stanley) writes: > > I don't know about the ones people have been talking about, but I >do know there is a program under development that can handle "there" >vs "their" or, for that matter, the "two" vs "too" vs "to". Anyone got one for "which" versus "that"? -- Gilbert Cockton, Scottish HCI Centre, Ben Line Building, Edinburgh, EH1 1TN JANET: gilbert@uk.ac.hw.aimmi ARPA: gilbert%aimmi.hw.ac.uk@cs.ucl.ac.uk UUCP: ..!{backbone}!aimmi.hw.ac.uk!gilbert