AES E-Library

RA. FR. Fela, N. Zacharov, and S. Forchhammer, "Assessor Selection Process for Perceptual Quality Evaluation of 360 Audiovisual Content," J. Audio Eng. Soc., vol. 70, no. 10, pp. 824-842, (2022 October.). doi: https://doi.org/10.17743/jaes.2022.0037 RA. FR. Fela, N. Zacharov, and S. Forchhammer, "Assessor Selection Process for Perceptual Quality Evaluation of 360 Audiovisual Content," J. Audio Eng. Soc., vol. 70 Issue 10 pp. 824-842, (2022 October.). doi: https://doi.org/10.17743/jaes.2022.0037
Abstract: For accurate and detailed perceptual evaluation of compressed omnidirectional multimedia content, it is imperative for assessor panels to be qualified to obtain consistent and high-quality data. This work extends existing procedures for assessor selection in terms of scope (360? videos with high-order ambisonic), time efficiency, and analytical approach, as described in detail. The main selection procedures consisted of a basic audiovisual screening and three successive discrimination experiments for audio (listening), video (viewing), and audiovisual using a triangle test. Additionally, four factors influencing quality of experience, including the simulator sickness questionnaire, were evaluated and are discussed. After the selection process, a confirmatory study was conducted using three experiments (audio, video, and audiovisual) and based on a rating scale methodology to compare performance between rejected and selected assessors. The studies showed that (i) perceptual discriminations are influenced by the samples, the encoding parameters, and some quality of experience factors; (ii) the probability of symptom occurrence is considerably low, indicating that the proposed procedure is feasible; and (iii) the selected assessors performed better in discrimination than the rejected assessors, indicating the effectiveness of the proposed procedure.

@article{fela2022assessor,
author={fela, randy frans and zacharov, nick and forchhammer, søren},
journal={journal of the audio engineering society},
title={assessor selection process for perceptual quality evaluation of 360 audiovisual content},
year={2022},
volume={70},
number={10},
pages={824-842},
doi={https://doi.org/10.17743/jaes.2022.0037},
month={october},} @article{fela2022assessor,
author={fela, randy frans and zacharov, nick and forchhammer, søren},
journal={journal of the audio engineering society},
title={assessor selection process for perceptual quality evaluation of 360 audiovisual content},
year={2022},
volume={70},
number={10},
pages={824-842},
doi={https://doi.org/10.17743/jaes.2022.0037},
month={october},
abstract={for accurate and detailed perceptual evaluation of compressed omnidirectional multimedia content, it is imperative for assessor panels to be qualified to obtain consistent and high-quality data. this work extends existing procedures for assessor selection in terms of scope (360? videos with high-order ambisonic), time efficiency, and analytical approach, as described in detail. the main selection procedures consisted of a basic audiovisual screening and three successive discrimination experiments for audio (listening), video (viewing), and audiovisual using a triangle test. additionally, four factors influencing quality of experience, including the simulator sickness questionnaire, were evaluated and are discussed. after the selection process, a confirmatory study was conducted using three experiments (audio, video, and audiovisual) and based on a rating scale methodology to compare performance between rejected and selected assessors. the studies showed that (i) perceptual discriminations are influenced by the samples, the encoding parameters, and some quality of experience factors; (ii) the probability of symptom occurrence is considerably low, indicating that the proposed procedure is feasible; and (iii) the selected assessors performed better in discrimination than the rejected assessors, indicating the effectiveness of the proposed procedure.},}

TY - paper
TI - Assessor Selection Process for Perceptual Quality Evaluation of 360 Audiovisual Content
SP - 824 EP - 842
AU - Fela, Randy Frans
AU - Zacharov, Nick
AU - Forchhammer, Søren
PY - 2022
JO - Journal of the Audio Engineering Society
IS - 10
VO - 70
VL - 70
Y1 - October 2022 TY - paper
TI - Assessor Selection Process for Perceptual Quality Evaluation of 360 Audiovisual Content
SP - 824 EP - 842
AU - Fela, Randy Frans
AU - Zacharov, Nick
AU - Forchhammer, Søren
PY - 2022
JO - Journal of the Audio Engineering Society
IS - 10
VO - 70
VL - 70
Y1 - October 2022
AB - For accurate and detailed perceptual evaluation of compressed omnidirectional multimedia content, it is imperative for assessor panels to be qualified to obtain consistent and high-quality data. This work extends existing procedures for assessor selection in terms of scope (360? videos with high-order ambisonic), time efficiency, and analytical approach, as described in detail. The main selection procedures consisted of a basic audiovisual screening and three successive discrimination experiments for audio (listening), video (viewing), and audiovisual using a triangle test. Additionally, four factors influencing quality of experience, including the simulator sickness questionnaire, were evaluated and are discussed. After the selection process, a confirmatory study was conducted using three experiments (audio, video, and audiovisual) and based on a rating scale methodology to compare performance between rejected and selected assessors. The studies showed that (i) perceptual discriminations are influenced by the samples, the encoding parameters, and some quality of experience factors; (ii) the probability of symptom occurrence is considerably low, indicating that the proposed procedure is feasible; and (iii) the selected assessors performed better in discrimination than the rejected assessors, indicating the effectiveness of the proposed procedure.