AES Store

Journal Forum

Reflecting on Reflections - June 2014
1 comment

Quiet Thoughts on a Deafening Problem - May 2014
1 comment

Perceptual Effects of Dynamic Range Compression in Popular Music Recordings - January 2014
5 comments

Access Journal Forum

AES E-Library

The Impact of Test Duration on User Fatigue and Reliability of Subjective Quality Ratings

This article investigates the influence of test duration on user fatigue and the reliability of user ratings in the context of subjective Quality-of-Experience (QoE) assessment. The goal is to provide empirically grounded guidance for the design of lab-based quality experiments, particularly as concerns the overall duration of test sessions. Since subjective user tests tend to be time-consuming and costly, aspects of user workload and fatigue are relevant as they relate to a fundamental challenge: how to maximize test duration without compromising results quality by overly exhausting test participants? In order to address this challenge, we investigate the relationships between test duration, user fatigue, and rating behavior. Our analysis is grounded on measurements and observations made during three typical QoE lab studies with mixed audio, video, and web task profiles that assessed the impact of different network conditions on perceived quality. We measured participant workload and fatigue in two complementary ways: subjectively by means of a questionnaire and objectively by performing physiological measurements in terms of eye blink rates (EBR) as well as Electrocardiographs (ECG). Our results show that even after 90 minutes of active testing, participants’ quality gradings were still reliable despite the presence of measurable signs of fatigue. Thus, for comparable QoE lab user experiments, we recommend to stay within this limit in order to achieve a good balance between results quantity and results quality.

Authors:
Affiliation:
JAES Volume 60 Issue 1/2 pp. 63-73; January 2012
Publication Date:

Click to purchase paper or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member and would like to subscribe to the E-Library then Join the AES!

This paper costs $20 for non-members, $5 for AES members and is free for E-Library subscribers.

Learn more about the AES E-Library

E-Library Location:

Start a discussion about this paper!


 
Facebook   Twitter   LinkedIn   Google+   YouTube   RSS News Feeds  
AES - Audio Engineering Society