Can we make progress on nonlinear blind source separation by drawing inspiration from the field of causal inference?
Introducing our #NeurIPS2021 paper "Independent mechanism analysis, a new concept?", joint work with @JKugelgen, @VStimper, @bschoelkopf and @MichelBesserve
1/9
Introducing our #NeurIPS2021 paper "Independent mechanism analysis, a new concept?", joint work with @JKugelgen, @VStimper, @bschoelkopf and @MichelBesserve
1/9
We consider the "cocktail party problem", i.e. blind source separation (BSS):
In a room with many speakers, recording devices pick up (nonlinear) mixtures of what each person says. Given the recorded mixtures, we would like to reconstruct (separate) the original sources.
2/9
In a room with many speakers, recording devices pick up (nonlinear) mixtures of what each person says. Given the recorded mixtures, we would like to reconstruct (separate) the original sources.
2/9
This can be formalised through nonlinear ICA, assuming the sources are statistically independent.
Problem: Unsupervised nonlinear ICA is not identifiable ➡️ Estimating independent components does not solve BSS! This can be shown by constructing "spurious" solutions.
3/9
Problem: Unsupervised nonlinear ICA is not identifiable ➡️ Estimating independent components does not solve BSS! This can be shown by constructing "spurious" solutions.
3/9
To recover identifiability, we take inspiration from causal inference and the principle of independent causal mechanisms.
We adapt it to the BSS problem, with the following intuition: the mechanisms by which each source influences the observations should be "independent".
4/9
We adapt it to the BSS problem, with the following intuition: the mechanisms by which each source influences the observations should be "independent".
4/9
In our cocktail party metaphor: the speakers’ positions are not fine-tuned to the room acoustics and microphone placement, or to each other.
Technically, our principle is an orthogonality condition on the columns of the mixing's Jacobian.
5/9
Technically, our principle is an orthogonality condition on the columns of the mixing's Jacobian.
5/9
We show that, under our assumptions, the most commonly used counterexamples to identifiability can be ruled out.
We also introduce a contrast to quantify violations of our principle, and show that it provides a useful learning signal to solve purely unsupervised BSS.
6/9
We also introduce a contrast to quantify violations of our principle, and show that it provides a useful learning signal to solve purely unsupervised BSS.
6/9
Summarising, we showed how the idea of independent causal mechanisms (previously mostly used for causal discovery) can be meaningfully adapted to unsupervised representation learning: a first effort to use ideas from causality (specifically ICM) for BSS.
7/9
7/9
Feel free to reach out if you have any questions!
Paper: arxiv.org/abs/2106.05200
Code: github.com/lgresele/indep…
8/9
Paper: arxiv.org/abs/2106.05200
Code: github.com/lgresele/indep…
8/9
P.s. The paper’s title is a reference/reverence to the seminal 1994 paper by Pierre Comon, "Independent component analysis, A new concept?"
9/9
9/9
• • •
Missing some Tweet in this thread? You can try to
force a refresh
