Media.Player

Example-based audio inpainting (Technion)

Summary

VoIP systems, a standard in modern communication, are troubled by unreliability in the communication channel such as the internet. Data losses are frequent and can render audio signals incoherent. Current methods for filling in missing parts rely on plain interpolation and insertion of noise. More advanced methods that use models like sparse representations have been shown to be highly effective for short periods of missing audio, a few dozen missing samples at most, but not for long periods of hundreds of samples and beyond.We developed an algorithm that fills audio data gaps created owing to dropped packets. First, a streaming digital audio signal is captured continuously. All the while, this audio signal is divided into overlapping segments of constant length. Each segment undergoes a feature extraction process, yielding a feature vector representative of this segment. These segments are saved to be used as examples at a later stage. Once a packet is dropped, an audio data gap is formed, and the segments containing it are then used as queries in a task to fill in this hole. The unharmed portions inside these queries undergo a feature extraction process, similar to the one applied to example segments. This process yields query feature vectors which are comparable to example feature vectors. For each optional query we then find several examples that have the most resembling feature vectors out of the prior examples. This collection of examples constitutes a list of potential matches for this hole. Relying on prior statistics, we pick the most suitable example out of the list to fill the hole. Finally, various techniques are used to inlay the selected example into the hole. By this we aim to reduce artifacts and increase intelligibility.

Technology Benefits

Handles relatively long temporal audio gaps
Takes advantage of assumptions in the nature of audio signals
Builds a database of user audio samples

Technology Application

Telephone and cellphone communication
Teleconferencing and video conferencing

ID No.

COM-1502

Country/Region

Israel