Experimental Evaluation in Computer Science: A Quantitative Study
-
Tagung:
Zeitschriftenartikel
-
Autoren:
Lutz Prechelt
Ernst Heinz
Paul Lukowicz
Walter F. Tichy -
Zusammenfassung
A survey of over 400 recent research articles suggests that computer scientists publish relatively few papers with experimentally validated results. The survey includes complete volumes of several refereed computer science journals, a conference, and 50 titles drawn at random from all articles published by ACM in 1993. The journals Optical Engineering (OE) and Neural Computation (NC) were used for comparison. Of the papers in the random sample that would require experimental validation, 40% have none at all. In journals related to software engineering, this fraction is over 50%. In comparison, the fraction of papers lacking quantitative evaluation in OE and NC is only 15% and 12%, respectively. Conversely, the fraction of papers that devote one fifth or more of their space to experimental validation is almost 70% for OE and NC, while it is a mere 30% for the CS random sample and 20% for software engineering. The low ratio of validated results appears to be a serious weakness in computer science research. This weakness should be rectified for the long-term health of the field.
Bibtex
@article{,
author={Lutz Prechelt, Ernst Heinz, Paul Lukowicz, Walter F. Tichy},
title={Experimental Evaluation in Computer Science: A Quantitative Study},
year=1995,
month=Jan,
volume={28},
url={http://ps.ipd.kit.edu/downloads/zeitschriftenartikel_1994-17.pdf},
abstract={A survey of over 400 recent research articles suggests that computer scientists publish relatively few papers with experimentally validated results. The survey includes complete volumes of several refereed computer science journals, a conference, and 50 titles drawn at random from all articles published by ACM in 1993. The journals Optical Engineering (OE) and Neural Computation (NC) were used for comparison. Of the papers in the random sample that would require experimental validation, 40% have none at all. In journals related to software engineering, this fraction is over 50%. In comparison, the fraction of papers lacking quantitative evaluation in OE and NC is only 15% and 12%, respectively. Conversely, the fraction of papers that devote one fifth or more of their space to experimental validation is almost 70% for OE and NC, while it is a mere 30% for the CS random sample and 20% for software engineering. The low ratio of validated results appears to be a serious weakness in computer science research. This weakness should be rectified for the long-term health of the field.},
number={1},
pages={9-18},
journal={Journal of Systems and Software},
}
-
Jahr:
1995
- Links:
| Titel Vorname Nachname |
|---|
| Prof. em. Dr. Walter F. Tichy |
| Prof. Dr. Lutz Prechelt |