AES E-Library

C. Mendonça, and S. Delikaris-Manias, "Statistical Tests with MUSHRA Data," Paper 10006, (2018 May.). doi: C. Mendonça, and S. Delikaris-Manias, "Statistical Tests with MUSHRA Data," Paper 10006, (2018 May.). doi:
Abstract: This work raises concerns regarding the statistical analysis of data obtained with the MUSHRA method. There is a widespread tendency to prefer the ANOVA test, which is supported by the recommendation. This work analyzes four assumptions underlying the ANOVA tests: interval scale, normality, equal variances, and independence. Data were collected from one experiment and one questionnaire. It is found that MUSHRA data tend to violate all of the above assumptions. The consequences of each violation are debated. The violation of multiple assumptions is of concern. The violation of independence of observations leads to the most serious concern. In light of these findings, it is concluded that ANOVA tests have a high likelihood of resulting in type 1 error (false positives) with MUSHRA data and should therefore never be used with this type of data. The paper finishes with a section devoted to statistical recommendations. It is recommended that when using the MUSHRA method, the Wilcoxon or Friedman tests be used. Alternatively, statistical tests based on resampling methods are also appropriate.

@article{mendonça2018statistical,
author={mendonça, catarina and delikaris-manias, symeon},
journal={journal of the audio engineering society},
title={statistical tests with mushra data},
year={2018},
volume={},
number={},
pages={},
doi={},
month={may},} @article{mendonça2018statistical,
author={mendonça, catarina and delikaris-manias, symeon},
journal={journal of the audio engineering society},
title={statistical tests with mushra data},
year={2018},
volume={},
number={},
pages={},
doi={},
month={may},
abstract={this work raises concerns regarding the statistical analysis of data obtained with the mushra method. there is a widespread tendency to prefer the anova test, which is supported by the recommendation. this work analyzes four assumptions underlying the anova tests: interval scale, normality, equal variances, and independence. data were collected from one experiment and one questionnaire. it is found that mushra data tend to violate all of the above assumptions. the consequences of each violation are debated. the violation of multiple assumptions is of concern. the violation of independence of observations leads to the most serious concern. in light of these findings, it is concluded that anova tests have a high likelihood of resulting in type 1 error (false positives) with mushra data and should therefore never be used with this type of data. the paper finishes with a section devoted to statistical recommendations. it is recommended that when using the mushra method, the wilcoxon or friedman tests be used. alternatively, statistical tests based on resampling methods are also appropriate.},}

TY - paper
TI - Statistical Tests with MUSHRA Data
SP - EP -
AU - Mendonça, Catarina
AU - Delikaris-Manias, Symeon
PY - 2018
JO - Journal of the Audio Engineering Society
IS -
VO -
VL -
Y1 - May 2018 TY - paper
TI - Statistical Tests with MUSHRA Data
SP - EP -
AU - Mendonça, Catarina
AU - Delikaris-Manias, Symeon
PY - 2018
JO - Journal of the Audio Engineering Society
IS -
VO -
VL -
Y1 - May 2018
AB - This work raises concerns regarding the statistical analysis of data obtained with the MUSHRA method. There is a widespread tendency to prefer the ANOVA test, which is supported by the recommendation. This work analyzes four assumptions underlying the ANOVA tests: interval scale, normality, equal variances, and independence. Data were collected from one experiment and one questionnaire. It is found that MUSHRA data tend to violate all of the above assumptions. The consequences of each violation are debated. The violation of multiple assumptions is of concern. The violation of independence of observations leads to the most serious concern. In light of these findings, it is concluded that ANOVA tests have a high likelihood of resulting in type 1 error (false positives) with MUSHRA data and should therefore never be used with this type of data. The paper finishes with a section devoted to statistical recommendations. It is recommended that when using the MUSHRA method, the Wilcoxon or Friedman tests be used. Alternatively, statistical tests based on resampling methods are also appropriate.