Back To Schedule
Tuesday, October 26 • 1:30pm - 1:45pm
Generalized Zero-Shot Learning via Normalizing Flows

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
Technical Presentations Group 3: Algorithms, Foundations, Visualizations, and Engineering Applications

Generalized Zero-shot Learning (GZSL) in Computer Vision refers to the task of recognizing images for which classes are not available during training, but other data such as textual descriptions for all classes are available. The idea is to leverage the information from these language descriptions to recognize both seen and unseen classes by transferring knowledge from each modality. This setup poses a more realistic scenario in image classification problems, where it is not possible to manually collect and annotate all the images for a specific class, but it is more viable to use natural language descriptions. In this work, we explore Normalizing Flows to generate features from a shared latent space that aligns the image and textual representations. These new features synthetically generated by our model are then used to enlarge the training set, so that the aligned representations for all seen and unseen classes can be used to train a classifier in a supervised manner. For this purpose, we simultaneously train two Invertible Neural Networks, one for the image representation, and the other for the textual description. Our aim is that the features encoded in the forward pass would work as data embeddings which we align so that they share the same feature space. In the reverse pass, both networks are enforced to reconstruct their corresponding input as a supervised signal for each modality. In this way, our approach outperforms previous generative models that use Variational Autoencoders and Generative Adversarial Networks in the CUB dataset by significant margins.

Authors: Paola Cascante-Bonilla (University of Virginia), Yanjun Qi (University of Virginia) and Vicente Ordonez (Rice University)


Paola Cascante-Bonilla

University of Virginia

Tuesday October 26, 2021 1:30pm - 1:45pm CDT

Attendees (2)