Executive summary

The KAMoulox project (ANR-15-CE38-0003) runs from January 2016 to September 2019. It has been funded by ANR, the French national research agency, as a Junior Researcher project with principal investigator Antoine Liutkus. Its name stands for Kernel additive models for online unmixing of large historical arxives. KAMoulox has two main objectives:

• Developing cutting-edge audio restoration techniques through signal processing research.
• Making them available for research in real world sound archives, in an open-source environment.

The main assumption of the project is that audio separation and restoration can help shape new practices regarding our immaterial audio heritage. It can notably benefit to the scientific and educational exploitation of this heritage through the concept of crowd processing.

Assets

The archives of the CNRS - Musée de l’Homme

Description

• Since late 19th century, French ethnologists were sent over the world with the mission to record the world audio heritage
• To this day, this extraordinary database comprises 42k documents, among which 32k are digitized
• The diversity of this archive is tentalizing, with items from 199 countries and 1200 ethnies

Value

• The archives of the CNRS - Musée de l'Homme are one working material for a whole community of scientists in digital humanities.
• As of December 2017, there are 2600 scientists/month regularly using the archive.
• With a content of an extraordinary cultural value, the archives offer rich repurposing possibilities: musical creation, licensing, etc.

The mixed research unit CREM: Centre de Recherche en Ethno-Musicologie is responsible for these archives.

Challenges

Although they are an invaluable source for information and wealth for our immaterial cultural heritage, the archives also come with some specific challenges, which are the focus of the project.

• Most recordings are old, degraded and noisy, often preventing repurposing in cultural creation.
• Users are not sound engineers and may not easily proceed to restoration themselves.
• Due to the sheer volume of these archives and the time required, it is impossible to automatically restore all entries. The need to proceed on demand emerges.

Objectives for the project

• Audio restoration for the non expert
• Embed online audio restoration tools in the archives

Audio signal separation

Description

• Antoine Liutkus and collaborators are specialists in audio source separation, which consists in recovering the differnet sounds in a recording. Denoising and demixing are examples.
• This topic is heavily based on probabilistic models for time series, that comprise two core elements:
• The distribution considered and the corresponding filtering procedure. Our recent research focuses on α-stable models, that appear especially interesting for denoising.
• Models for the spectrograms, that describe how a sound is varying over time and frequency. This is the topic of our recent research on kernel and DNN models.

Challenges

Our current research on probabilistic models for audio processing comprise several important challenges, that need to be addressed befor the methods may be used in real applications.

• Tractability is an issue: applications require fast computations, and models need to be pre-trained. Scaling up training to the archives level remains a challenge, especially considering that archives only feature real data, so that the true underlying noise or sources are unknown.
• Alpha-stable modeling are effective and explain many heuristics used in the field. However, learning their parameters and generalizing them to multichannel cases is a real challenge, due to the lack of an analytical expression for their likelihood function.

Objectives for the project

• Robustness: filtering in noise, robust inference
• Explaining heuristics with the α-stable paradigm
• Flexible and exemplar-based trainable audio models

Work packages

KAMoulox is decomposed into 4 work-packages (WPs) of equal importance, that closely reflect its numerous scientific and technical objectives. From a transversal application perspective, all WPs revolve around incrementally improving a web-based source separation software architecture.

WP1 Robust separation methods

This WP builds on recent advances I proposed in probabilistic signal processing, that are based on the alpha-stable formalism and which notably extend the classical Gaussian methodology to heavy-tailed signals. This part of the project feature two main challenges. The first one is the estimation of parameters in models that do not come with likelihood functions. Ideas include moment fitting and Monte Carlo methods. Second, the case of multichannel signals brings interesting research directions for probabilistic signal processing.

WP2 Extensions of KAM

This WP introduces new nonparametric ways of estimating the parameters of sources, notably their time-frequency energy profiles. This also comes with new ways to format the feedback displayed to the user for interacting with him. This work-package is divided into two main parts: first, the design of new non-parametric spectrogram models, such as deep-nets or nonnegative alpha-stable. Second, the iterative processing achieved through an interaction between the filtering machine and a user.

WP3 Separation web-service

This WP provides computationally efficient implementations to separation and parameter estimation procedures. It will also be critical in embedding the whole architecture on the existing platform of the CNRS - Musée de l'Homme.

WP4 Design and science outreach

This WP will first focus on the design of the client-side application using a resolutely user-centered methodology. It will then lead to the actual realization of this multimedia client, but also to the promotion of this technology through the realization of multimedia content and through the participation of strategical social events.