speech embedding

Contrastive Predictive Coding

Contrastive Predictive Coding (CPC) learns self-supervised representations by predicting the future in latent space by using powerful autoregressive models. The model uses a probabilistic contrastive loss which induces the latent space to capture information that is maximally useful to predict future samples. It describes a form of unidirectional modeling in the feature space,
where the model learns to predict the near future frames in
an acoustic sequence while contrasting with frames from other
sequences or frames from a more distant time.

Autoregressive Predictive Coding

The APC approach uses an autoregressive model to encode
temporal information of past acoustic sequence; the model then
predicts future frames like a recurrent-based LM while
conditioning on past frames.

TERA

TERA, which stands for Transformer Encoder Representations from Alteration, is a self-supervised speech pre-training
method.

experiment design

Amount of labeled data needed to perform well.
with pre-trained and without pre-trained.