Timezone: »

 
Spotlight
Principal Bit Analysis: Autoencoding with Schur-Concave Loss
Sourbh Bhadane · Aaron Wagner · Jayadev Acharya

Thu Jul 22 06:45 AM -- 06:50 AM (PDT) @
in Semisupervised Learning 2 »

We consider a linear autoencoder in which the latent variables are quantized, or corrupted by noise, and the constraint is Schur-concave in the set of latent variances. Although finding the optimal encoder/decoder pair for this setup is a nonconvex optimization problem, we show that decomposing the source into its principal components is optimal. If the constraint is strictly Schur-concave and the empirical covariance matrix has only simple eigenvalues, then any optimal encoder/decoder must decompose the source in this way. As one application, we consider a strictly Schur-concave constraint that estimates the number of bits needed to represent the latent variables under fixed-rate encoding, a setup that we call \emph{Principal Bit Analysis (PBA)}. This yields a practical, general-purpose, fixed-rate compressor that outperforms existing algorithms. As a second application, we show that a prototypical autoencoder-based variable-rate compressor is guaranteed to decompose the source into its principal components.

Keywords: Algorithms · Components Analysis (e.g., CCA, ICA, LDA, PCA)

Author Information

Sourbh Bhadane (Cornell University)
Aaron Wagner (Cornell University)
Jayadev Acharya (Cornell University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors