Some Problems and Solutions in the Experimental Science of Technology: The Proper Use and Reporting of Statistics in Computational Intelligence, with an Experimental Design from Computational Ethnomusicology
Download (854 KB)
Statistics is the meta-science that lends validity and credibility to The Scientific Method. However, as a complex and advanced Science in itself, Statistics is often misunderstood and misused by scientists, engineers, medical and legal professionals and others. In the area of Computational Intelligence (CI), there have been numerous misuses of statistical techniques leading to the publishing of insupportable results, which, in addition to being a problem in itself, has also contributed to a degree of rift between the Statistics/Statistical Learning community and the Machine Learning/Computational Intelligence community. This talk surveys a number of misuses of statistical inference in CI settings, including well-known and more rarely discussed examples. These are followed by an overview of concepts and techniques that are central to model evaluation. Finally, an experimental design is presented for a statistically valid comparison of multiple hypotheses for a particular real-world problem combining Information Theory, Neural Networks, Statistics, and Computational Ethnomusicology.
Mehmet Vurkaç is a Ph.D. candidate in Electrical & Computer Engineering. He completed his B.A. in Math-Physics at Whitman College in 1993, and his M.S. in ECE (DSP emphasis) at Portland State University in 1999. He started his doctoral studies in April 2002 after working in the music industry (Roland Corp.) for four years as a hardware engineer. His dissertation research is in Computational Intelligence (Neural Networks), with an Information Theory component and applications to the rapidly growing Computational Ethnomusicology subfield of Music Information Research (MIR). He served as an adjunct instructor at PSU’s ECE department 2003–2009, and at Whitman College in the Music Department (Sound Synthesis) in 1994. He is currently an assistant professor in the Department of Electrical Engineering & Renewable Energy at the Oregon Institute of Technology, and a Ph.D. candidate in Electrical & Computer Engineering at PSU. His research and teaching interests are Neural Networks, Fuzzy Logic, Evolutionary Computation, Computational Ethnomusicology, Information Theory; Statistical Learning, Music Information Research, Psychoacoustics, Music Perception, the history and theory of Afro-Latin musics, Signals & Systems, DSP, Cognitive Science, and general education for critical thinking and social responsibility.
Computational intelligence -- Statistical aspects, Mathematical statistics, Probabilities, Experimental design
Design of Experiments and Sample Surveys | Statistical Methodology | Statistical Models
Vurkaç, Mehmet, "Some Problems and Solutions in the Experimental Science of Technology: The Proper Use and Reporting of Statistics in Computational Intelligence, with an Experimental Design from Computational Ethnomusicology" (2011). Systems Science Friday Noon Seminar Series. 59.