Non-linear quantization of the type INT(x^(M/N) + constant) is commonly used in audio compression techniques, particularly MPEG-1 and MPEG-2 layer III (MP3) and MPEG Advanced Audio Coding (AAC). Finding a suitable DSP implementation is a problem since lookup table methods are prohibitive due to excessive storage requirements, conventional series approximation methods do not give sufficient precision, and not all processors have log/exp assist functions. This paper describes a method which utilizes the property of geometric periodicity of the x^(M/N) function to first normalize the problem to a small range of input x. Subsequently one can choose to perform the x^(M/N) in this limited range based on lookup, interpolation, or series expansion, and finally re-normalize the output to obtain the overall answer. Using a hybrid scheme based on lookup and interpolation very good overall precision is achieved. Compared to direct application of any of the above techniques, there is very little additional computational burden, and the improvement in precision is very significant. Mathematically, this method is shown to be a special case of log-exp based computation where the log is quantized.
https://www.aes.org/e-lib/browse.cfm?elib=9951
Click to purchase paper as a non-member or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member and would like to subscribe to the E-Library then Join the AES!
This paper costs $33 for non-members and is free for AES members and E-Library subscribers.
Learn more about the AES E-Library
Start a discussion about this paper!
