eugene@wilbur.nas.nasa.gov (Eugene N. Miya) (08/03/90)
In article <9932@hubcap.clemson.edu> aglew@oberon.crhc.uiuc.edu (Andy Glew) writes: >Finally, several people contacted me saying that they have lent their >parallel applications to other researchers in the past, but refuse to >do it any more because they have been "burnt" by the experience - the >other researchers have "trashed" the contributed codes and attacked >the authors of the contributed benchmarks. > This is a sad commentary on the lack of basic etiquette in the >research community. Yes, this is very true. It's part of the black art of benchmarking. It's made harder because 1) parallel programs are more difficult to write or modify [a controversial statement, itself], 2) lack of consistency in parallelism paradigm. The technical problems. Ideally, a science of computing should have a freer exchange of information. But this conflicts with marketing concerns [companies with policies which forbid release of benchmark data], "security" concerns [remember many of those with vested interests in performance are also concern about security, remember who paid for the first computers], embarassment of "not quite good enough results," tenure, and "embarassingly parallel problems." The social problems. --e. nobuo miya, NASA Ames Research Center, eugene@orville.nas.nasa.gov {uunet,mailrus,other gateways}!ames!eugene
tak@eecg.toronto.edu (Mike Takefman) (08/04/90)
In article <9990@hubcap.clemson.edu> eugene@wilbur.nas.nasa.gov (Eugene N. Miya) writes: >In article <9932@hubcap.clemson.edu> aglew@oberon.crhc.uiuc.edu >(Andy Glew) writes: >>Finally, several people contacted me saying that they have lent their >>parallel applications to other researchers in the past, but refuse to >>do it any more because they have been "burnt" by the experience - the >>other researchers have "trashed" the contributed codes and attacked >>the authors of the contributed benchmarks. >> This is a sad commentary on the lack of basic etiquette in the >>research community. > >Yes, this is very true. >It's part of the black art of benchmarking. > >Ideally, a science of computing should have a freer exchange of >information. But this conflicts with marketing concerns [companies with >policies which forbid release of benchmark data], "security" >concerns [remember many of those with vested interests in performance >are also concern about security, remember who paid for the first >computers], embarassment of "not quite good enough results," >tenure, and "embarassingly parallel problems." The social problems. Just to add my $0.02 and personal frustration. My thesis is about improvements to DSP microprocessor architecture and I needed to analyse code for the 56000 DSP microprocessor. Recognizing that some companies would be worried about the security of their source code I offered to go to their site with my tools to generate data. I also offered to sign any non-disclosure they cared to create (with the proviso I get to publish the data)! My requests were often met with responses like... "The research sounds interesting, and we'd like to see the results, but our code is proprietary" "We can't give you code, but could be have a copy of your tools?" I certainly recognize the need for companies to protect their investments and technology, but there must be some middle ground where everyone can benefit. -- Michael Takefman So I was watching the world cup final between University of Toronto West Germany and Argentina, I was wondering E.E. Computer Group who the fugitive Nazi war criminals would be tak@eecg.toronto.edu rooting for? (Late Night with David Letterman)
dfk@grad13.cs.duke.edu (David F. Kotz) (08/04/90)
In article <9990@hubcap.clemson.edu>, eugene@wilbur.nas.nasa.gov (Eugene N. Miya) writes: > Ideally, a science of computing should have a freer exchange of > information. But this conflicts with marketing concerns [companies with > policies which forbid release of benchmark data], "security" > concerns [remember many of those with vested interests in performance > are also concern about security, remember who paid for the first > computers], embarassment of "not quite good enough results," > tenure, and "embarassingly parallel problems." The social problems. > > --e. nobuo miya, NASA Ames Research Center, eugene@orville.nas.nasa.gov > {uunet,mailrus,other gateways}!ames!eugene This exact topic is covered in this month's CACM (8/90), "Sharing Scientific Data", by Theodor D. Sterling and James J. Weinkam. David Kotz Department of Computer Science, Duke University, Durham, NC 27706 USA ARPA: dfk@cs.duke.edu CSNET: dfk@duke UUCP: decvax!duke!dfk