Luigi Gresele Profile picture
DDSA postdoc (@DataScienceDK) at @UCPH_Research. Interested in the cocktail-party problem, causal inference and identifiability. Previously @MPI_IS.

Nov 19, 2021, 9 tweets

Can we make progress on nonlinear blind source separation by drawing inspiration from the field of causal inference?

Introducing our #NeurIPS2021 paper "Independent mechanism analysis, a new concept?", joint work with @JKugelgen, @VStimper, @bschoelkopf and @MichelBesserve

1/9

We consider the "cocktail party problem", i.e. blind source separation (BSS):

In a room with many speakers, recording devices pick up (nonlinear) mixtures of what each person says. Given the recorded mixtures, we would like to reconstruct (separate) the original sources.

2/9

This can be formalised through nonlinear ICA, assuming the sources are statistically independent.

Problem: Unsupervised nonlinear ICA is not identifiable ➡️ Estimating independent components does not solve BSS! This can be shown by constructing "spurious" solutions.

3/9

To recover identifiability, we take inspiration from causal inference and the principle of independent causal mechanisms.

We adapt it to the BSS problem, with the following intuition: the mechanisms by which each source influences the observations should be "independent".

4/9

In our cocktail party metaphor: the speakers’ positions are not fine-tuned to the room acoustics and microphone placement, or to each other.

Technically, our principle is an orthogonality condition on the columns of the mixing's Jacobian.

5/9

We show that, under our assumptions, the most commonly used counterexamples to identifiability can be ruled out.

We also introduce a contrast to quantify violations of our principle, and show that it provides a useful learning signal to solve purely unsupervised BSS.

6/9

Summarising, we showed how the idea of independent causal mechanisms (previously mostly used for causal discovery) can be meaningfully adapted to unsupervised representation learning: a first effort to use ideas from causality (specifically ICM) for BSS.

7/9

Feel free to reach out if you have any questions!

Paper: arxiv.org/abs/2106.05200
Code: github.com/lgresele/indep…

8/9

P.s. The paper’s title is a reference/reverence to the seminal 1994 paper by Pierre Comon, "Independent component analysis, A new concept?"

9/9

Share this Scrolly Tale with your friends.

A Scrolly Tale is a new way to read Twitter threads with a more visually immersive experience.
Discover more beautiful Scrolly Tales like this.

Keep scrolling