We present RemixIT, a simple yet effective self-supervised method for training speech enhancement without the need of a single isolated in-domain speech nor a noise waveform. Our approach overcomes limitations of previous methods which make them dependent on clean in-domain target signals and thus, sensitive to any domain mismatch between train and test samples. RemixIT is based on a continuous self-training scheme in which a pre-trained teacher model on out-of-domain data infers estimated pseudo-target signals for in-domain mixtures. Then, by permuting the estimated clean and noise signals and remixing them together, we generate a new set of bootstrapped mixtures and corresponding pseudo-targets which are used to train the student network. Vice-versa, the teacher periodically refines its estimates using the updated parameters of the latest student models. Experimental results on multiple speech enhancement datasets and tasks not only show the superiority of our method over prior approaches but also showcase that RemixIT can be combined with any separation model as well as be applied towards any semi-supervised and unsupervised domain adaptation task. Our analysis, paired with empirical evidence, sheds light on the inside functioning of our self-training scheme wherein the student model keeps obtaining better performance while observing severely degraded pseudo-targets.

RemixIT: Continual Self-Training of Speech Enhancement Models via Bootstrapped Remixing

We study the limiting spectral distribution of sample covariance matrices $XX^T$, where $X$ are $p\times n$ random matrices with correlated entries, for the cases $p/n\to y\in [0,\infty)$. If $y>0$, we obtain the Mar\v{c}enko-Pastur distribution and in the case $y=0$ the semicircle distribution (after appropriate rescaling). The entries we consider are Curie-Weiss spins, which are correlated random signs, where the degree of the correlation is governed by an inverse temperature $\beta>0$. The model exhibits a phase transition at $\beta=1$. The correlation between any two entries decays at a rate of $O(np)$ for $\beta \in (0,1)$, $O(\sqrt{np}$) for $\beta=1$, and for $\beta>1$ the correlation does not vanish in the limit. In our proofs we use Stieltjes transforms and concentration of random quadratic forms.

High-dimensional sample covariance matrices with Curie-Weiss entries

We derive and compare various forms of local semicircle laws for random matrices with exchangeable entries which exhibit correlations that decay at a very slow rate. In fact, any $l$-point correlation will decay at a rate of $N^{-l/2}$. We call our ensembles \emph{of Curie-Weiss type}, and Curie-Weiss($\beta$)-distributed entries are admissible as long as $\beta\leq 1$.

Local Semicircle Law for Curie-Weiss Type Ensembles

In this paper, we propose a new interpretation of local limit theorems for univariate and multivariate distributions on lattices. We show that - given a local limit theorem in the standard sense - the distributions are approximated well by the limit distribution, uniformly on intervals of possibly decaying length. We identify the maximally allowable decay speed of the interval lengths. Further, we show that for continuous distributions, the interval type local law holds without any decay speed restrictions on the interval lengths. We show that various examples fit within this framework, such as standardized sums of i.i.d. random vectors or correlated random vectors induced by multidimensional spin models from statistical mechanics.

Interval Type Local Limit Theorems for Lattice Type Random Variables and Distributions.

The almost sure semicircle law for random band matrices with dependent entries

We analyze the empirical spectral distribution of random periodic band matrices with correlated entries. The correlation structure we study was first introduced in 2015 by Hochstattler, Kirsch and Warzel, who named their setup "almost uncorrelated" and showed convergence to the semicircle distribution in probability. We strengthen their results which turn out to be also valid almost surely. Moreover, we extend them to band matrices. Sufficient conditions for convergence to the semicircle distribution both in probability and almost surely are provided. In contrast to convergence in probability, almost sure convergence seems to require a minimal growth rate for the bandwidth. Examples that fit our general setup include Curie-Weiss distributed, correlated Gaussian, and as a special case, independent entries.

The Almost Sure Semicircle Law for Random Band Matrices with Dependent Entries

We study a multi-group version of the mean-field Ising model, also called Curie–Weiss model. It is known that, in the high-temperature regime of this model, a central limit theorem holds for the vector of suitably scaled group magnetisations, that is, for the sum of spins belonging to each group. In this article, we prove a local central limit theorem for the group magnetisations in the high-temperature regime.

/pdf/local-central-limit-theorem-for-multi-group-curie-weiss-1h29ubnpln.pdf

Local Central Limit Theorem for Multi-group Curie–Weiss Models

In this paper, we derive limit laws for the empirical spectral distributions of random band and block matrices with correlated entries. In the first part of the paper, we study band matrices with approximately uncorrelated entries. We strengthen previously obtained results while requiring weaker assumptions, which is made possible by a refined application of the method of moments. In the second part of the paper, we introduce a new two-layered correlation structure we call SSB-HKW correlated, which enables the study of structured random matrices with correlated entries. Our results include semicircle laws in probability and almost surely, but we also obtain other limiting spectral distributions depending on the conditions. Simple necessary and sufficient conditions for the limit law to be the semicircle are provided. Our findings strengthen and extend many results already known.

Michael Fleermann

Papers

The almost sure semicircle law for random band matrices with dependent entries

The Almost Sure Semicircle Law for Random Band Matrices with Dependent Entries

Interval Type Local Limit Theorems for Lattice Type Random Variables and Distributions.

Local Central Limit Theorem for Multi-group Curie–Weiss Models

Random Band and Block Matrices with Correlated Entries