Timezone: »

 
Poster
Simple Disentanglement of Style and Content in Visual Representations
Lilian Ngweta · Subha Maity · Alex Gittens · Yuekai Sun · Mikhail Yurochkin

Thu Jul 27 04:30 PM -- 06:00 PM (PDT) @ Exhibit Hall 1 #109
in Poster Session 6 »

Learning visual representations with interpretable features, i.e., disentangled representations, remains a challenging problem. Existing methods demonstrate some success but are hard to apply to large-scale vision datasets like ImageNet. In this work, we propose a simple post-processing framework to disentangle content and style in learned representations from pre-trained vision models. We model the pre-trained features probabilistically as linearly entangled combinations of the latent content and style factors and develop a simple disentanglement algorithm based on the probabilistic model. We show that the method provably disentangles content and style features and verify its efficacy empirically. Our post-processed features yield significant domain generalization performance improvements when the distribution shift occurs due to style changes or style-related spurious correlations.

[ PDF]  [ Poster] 

Author Information

Lilian Ngweta (Rensselaer Polytechnic Institute (RPI))

I am a computer science PhD student at Rensselaer Polytechnic Institute (RPI) in Troy, New York. My research interests are in the areas of out-of-distribution (OOD) generalization in machine learning and the applications of Machine Learning/AI methods to solve real world problems. Before joining RPI, I worked as a research intern at Microsoft Research in Redmond, Washington in Summer 2018, where I implemented a robust and scalable video processing engine that performed real-time image classification and labeling using a state-of-the-art deep neural network, and measured its performance. I received my Bachelor of Science in Software Engineering from Barrett, The Honors College at Arizona State University. Outside of work and school, I like to hike, dance and listen to music. I also like to travel and to interact with people from different cultures!

Subha Maity (University of Michigan, Ann Arbor)
Alex Gittens (Rensselaer Polytechnic Institute)

Alex Gittens's research focuses on using randomization to reduce the computational costs of extracting information from large datasets. His work lies at the intersection of randomized algorithms, numerical linear algebra, high-dimensional probability, and machine learning.

Yuekai Sun (University of Michigan)
Mikhail Yurochkin (IBM Research, MIT-IBM Watson AI Lab)

More from the Same Authors