9th April 2002 - Retouch: a unique audio airbrush to remove those squeaks, bleeps & creaks

CEDAR Audio is well known for its noise removal systems that clean up audio artefacts that can be found on various types of media - whether it be clicks from a scratched LP (anyone remember those?) or tape hiss. At April's AES lecture though, Christopher Hicks and Tony Webster from CEDAR Audio presented 'Retouch', a new system that allows the removal of squeaking chairs and unwanted car horns from recordings. Previous audio restoration systems have concentrated on the removal of three types of 'noise'. Impulsive noises like LP scratches tend to be short (100µs - 1ms) and have a defined beginning and end, but due to their transient nature, contaminate all frequencies. To remove these types of noises, the offending click is removed from, and the signal is then reconstituted from the 'good' audio either side of the section removed. Tones like tape line-up, or PAL line frequency bleed-through are normally removed in the frequency domain by using a very high Q notch filter. Care should be taken though, because these 'stationary' tones can often drift, requiring a tracking filter.

The third type of problem is that of Hiss. This occurs at all frequencies and all the time. Hiss removal is performed by a number of processes, ranging from noise gates to 'spectral subtraction'. Spectral subtraction works by taking a snapshot of the spectrum of a 'quiet' piece of a recording and subtracting that from the rest of the recording. However, noise by its very nature is random so the results are imperfect. Unfortunately Clicks, Tones and Hiss are not the only artefacts that can occur on recordings. Many recordings have been ruined by squeaking chairs, watch alarms and more recently mobile phones.

Christopher explained that the problem with these types of sounds is that they are difficult to distinguish unwanted noises from music. For example, how can you tell a computer to keep an orchestral triangle, but remove a watch alarm. Mathematically, the two are very similar. If a computer can't be told how to identify unwanted noises in a recording, then the recording engineer needs to be provided with appropriate tools to do the job. Christopher described how CEDAR adopted the spectrogram to allow an engineer to 'see' what is happening in a recording. A spectrogram shows how the spectral content of a signal changes with time. The horizontal axis of the graph represents time, the vertical axis represents frequency, and colour is used to represent the amplitude. The graph below shows a spectrogram of a sweeping sine wave.

The dark line shows the frequency of the sinusoid increasing (vertically) with increasing time (horizontally). The smearing which can be seen is due to the sinusoid changing frequency during each slice of the FFT.

Christopher played several examples of music with various different problems and showed how easy it was to spot these in the spectrogram. To remove these problems, the engineer highlights the part that they want to remove and hits the 'go' button. Miraculously the offending section has gone. CEDAR used an algorithm based on their de-clicking technology. However, whereas a click contains all frequencies, Retouch only works on frequencies selected for removal by the engineer. Christopher described this as a 'band pass interpolator'. The maths behind this is complicated and the subject of various patent applications but, in essence, Retouch splits the audio into two parts, the part to be removed and the part to be kept. The processing removes the unwanted section of the spectrum and then uses the audio before and after it to reconstitute what was obscured. Christopher and Tony demonstrated this with a section of piano music where a half second 'clunk' obscured part of the piece. In one of the best demonstrations seen at an AES lecture in a long time, Tony quickly loaded the offending piece into Retouch and used the spectrogram to identify the offending clunk. The clunk was highlighted with the mouse and two seconds later it was gone. Playing back the 'de-clunked' piece was immediately followed by applause from the audience - praise indeed.

However, not all pieces are as easy to remove. The same process was used on an excerpt from 'Locus Iste' where car horns outside the church used destroyed an otherwise good recording. One of the car horns occurred moments before the choir started to sing. Because Retouch uses information before and after the offending piece to reconstruct the obscured audio, a little user intervention was required to prevent Retouch using the starting choir to reconstruct what was covered by the car horns. This 'mathematically more challenging' piece took all of one minute to remove. Again playing the corrected piece was immediately followed by applause from the audience.

During questions and answers, almost every other question related to another possible use for Retouch. Christopher likened it to an audio tool which can be put to multiple uses, many of which will be unknown to the inventor.

Howard Farrar