[soc.college] Silly survey season

andy@rocky.STANFORD.EDU (Andy Freeman) (10/29/87)

*U.S. News and World Report* surveyed 241 "engineering deans at national
universities" but only 158 bothered to reply.  I'll bet that the quality
of the school correlates strongly with not replying.  If 12 of the 158
people who replied don't think that MIT is one of the 10 best engineering
schools in the US, that's pretty good evidence that the survey is garbage.
12 out of the 96 deans who replied to the law school survey didn't think
that Harvard or Yale were top 10 law schools - should we take them seriously?

"Little" known fact - 94 out of 204 of the college presidents surveyed
didn't respond to the survey on undergraduate schools.  Stanford's
president said "It's a beauty contest, not a serious analysis of
quality.  [Over 35% of the people who reply didn't rank Stanford or Harvard
in the top 10.  I don't even reply to them.]"  "His" school did very well
on both the undergrad and grad versions, so a sour grapes claim is fairly
weak.

Regarding the 85 version of the business school dean survey, the dean
of Stanford's, Professor Robert Jaedicke, said:

"[these surveys] are a disservice to prospective students.

"Business schools approach management education and research in
different ways.  These differences get glossed over in the rankings,
which tend to weed out a lot of heterogeneity.

"When surveys are done on schools, they tend to become the lead
indicator.  People who really need information about educational
programs to make informed decisions may not dig deep enough.

The Deans Council of the Association of American Medical Colleges
urged their colleagues NOT to participate; only 56 out of 144
surveyed did.  Dr. David Korn, dean of Stanford's med school said:

"We Americans are much given to lists of the `best,' and, perhaps
because of our 10 fingers, it is invariably the 10 best.  Only
a pedant would insist on the 10 better, and so it goes, the 10
best-dressed women, the 10 best movies, etc.

"And now the turn of medical schools has come.  The deans who
responded to the questionaire from the 10 best editors of the
*U.S. News and World Report* named the 10 top schools, but there
is no mystery; the particular positions seem to have depended on
how many deans answered the questionaire.

"We are not surprised to see Stanford Medical School listed among
the 10 best, but it's difficult to see anything useful as a
consequence of these rankings.
-- 
Andy Freeman
UUCP:  {arpa gateways, decwrl, sun, hplabs, rutgers}!sushi.stanford.edu!andy
ARPA:  andy@sushi.stanford.edu
(415) 329-1718/723-3088 home/cubicle

jtkung@mit-caf.UUCP (Joseph Kung) (10/30/87)

In article <706@rocky.STANFORD.EDU> andy@rocky.stanford.edu (Andy
Freeman) writes: 
>
>"We are not surprised to see Stanford Medical School listed among
>the 10 best, but it's difficult to see anything useful as a
>consequence of these rankings.
>-- 
>Andy Freeman

How does a person chose where to go to school if that person is an R.
P. Feynman or a future Dirac? He choses based on quality of faculty
and facilities. And this usually gets reflected in a good survey
(Gorman Report). It is no mystery why so many past Nobel Laureates
have had advisors or friends that were themselves Nobel Laureates. As
for the question of surveys (specifically the one in US News and World
Report), I would compare that with the Gorman Report, which is not
based on Deans' opinions, but on tangible things such as faculty,
facilities, research money, etc. The two reports parallel each other,
though not perfectly, but close. There is a definite usefulness in a good
factual suvey.

- Joe



-- 

Joseph Kung
Arpa Internet : jtkung@caf.mit.edu

andy@rocky.STANFORD.EDU (Andy Freeman) (11/04/87)

In article <492@mit-caf.UUCP> jtkung@mit-caf.UUCP (Joseph Kung) writes:
>How does a person chose where to go to school if that person is an R.
>P. Feynman or a future Dirac? He choses based on quality of faculty
>and facilities.

This answer is very incomplete.  First the faculty and facilities
have to be relevant to the student's interests.  A strong OS faculty
is of little use to a CS theory student.  Almost as important is
the school's "style".  Both CMU and Stanford are very good schools,
but some students will thrive at one and not the other.

>And this usually gets reflected in a good survey (Gorman Report). ... As
>for the question of surveys (specifically the one in US News and World
>Report), I would compare that with the Gorman Report, which is not
>based on Deans' opinions, but on tangible things such as faculty,
>facilities, research money, etc. The two reports parallel each other,
>though not perfectly, but close. There is a definite usefulness in a good
>factual suvey.

Ah, the famous Gorman report.  Yes, Gorman labels his criteria "faculty
quality", "facilities", and the like, but how does he measure them and
combine these measurements?  Once again, let's use MIT's CS department
as an example.*  The most recent Gorman report said that it was one of
the top 5, but in ones before that, it didn't even make the top 10.
Since CS departments don't change that fast, Gorman was blatantly
wrong either originally or now.  I'd say the former, but that doesn't
mean the current results are worth anything.  I think the numbers are
being cooked to match preconceptions and therefore are useless.

BTW - There are lots of problems with the Deans survey (in U.S. News
and World Report) that haven't been mentioned.  The most critical is
that we don't know what definition of "best" was used.  (I doubt the
surveyers thought of this either.)  Was it "does best research"?  How
about "trains best researchers" or even "trains best engineers"?  There
are lots of possibilities and the rankings depend on which "best" was
used.

-andy

ps - I'm using MIT as an example because most of the postings praising
the surveys seem to come from there.  Even so, MIT is one of the five
best engineering schools in the world by any reasonable standard.
-- 
Andy Freeman
UUCP:  {arpa gateways, decwrl, sun, hplabs, rutgers}!sushi.stanford.edu!andy
ARPA:  andy@sushi.stanford.edu
(415) 329-1718/723-3088 home/cubicle

maslak@sri-unix.ARPA (Valerie Maslak) (11/05/87)

Be careful of the Gorman report: my understanding is that he is
listing things by MAJOR, not by department, at least in the
undergraduate book, so if a school calls its
major by an unorthodox name, it may not be listed where you would
think it would be.

Valerie Maslak

hooner@athena.mit.edu (Hoon D Ko) (11/10/87)

In article <9247@sri-unix.ARPA> maslak@sri-unix.UUCP (Valerie Maslak) writes:
>Be careful of the Gorman report: my understanding is that he is
>listing things by MAJOR, not by department, at least in the
>undergraduate book, so if a school calls its
>major by an unorthodox name, it may not be listed where you would
>think it would be.
>
>Valerie Maslak

Well actually, the Gourman report lists several different "names"
in a department with footnotes- for example, MIT's version of
aerospace engineering is actually called "aeronautical and astronomical
engineering" (something like that, we just call it "aero/astro").
If a school chooses to call its CS program Information Systems, for
example, then it would still be listed in the CS rankings with a 
footnote giving the actual name for the program at the institution.