Mobile phones include noise suppression to facilitate use in noisy environments; therefore listening tests in accordance with ITU-T P.835 are appropriate for comparing handset performance. Objective speech quality measures are an often used cheaper alternative; however the results can be misleading, as rank order compared to listening tests is not always preserved. We compare the outputs of PESQ, POLQA, and 3Quest with the results of P.835 listening tests. As expected, measures intended for use with noise suppression perform that task better than tools that were not initially designed to do so. However, improved measures, that aim to preserve rank order while minimizing both maximum error and RMSE, would improve the reliability of comparative evaluations in background noise.
Click to purchase paper as a non-member or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member and would like to subscribe to the E-Library then Join the AES!
This paper costs $33 for non-members and is free for AES members and E-Library subscribers.