Review — DrLIM: Dimensionality Reduction by Learning an Invariant Mapping

DrLIM: Contrastive Learning for Dimensionality Reduction

Dimensionality Reduction by Learning an Invariant Mapping (DrLIM)

Outline

1. The Contrastive Loss Function in General Form

2. The Contrastive Loss Function in Exact Form

Graph of the loss function L against the energy DW. The dashed (red) line is the loss function for the similar pairs and the solid (blue) line is for the dissimilar pairs.

2.1. Exact Loss Function

2.2. Training

3. Spring Model Analogy

3.1. Attracting Force

Attractive Force

3.2. Repelling Force

Repelling Force

3.3. Equilibrium

The situation where a point is pulled by other points in different directions, creating equilibrium.

4. Network Architecture for GW

4.1. Network Architecture for MNIST

Architecture of the function GW

4.2. Network Architecture for Airplane Images in NORB Dataset

5. Experimental Results

5.1. MNIST

DrLIM in a trivial situation with MNIST digits

5.2. MNIST Distorted by Adding Samples that have been Horizontally Translated

5.2.1. DrLIM

DrLIM in MNIST data with horizontal translations added (-6, -3, +3, and +6 pixels)

5.2.2. LLE

LLE’s embedding of the distorted MNIST set

5.2.3. DrLIM Considering Translation

Better DrLIM in MNIST data with horizontal translations added (-6, -3, +3, and +6 pixels)

5.3. Airplane Images in DORB Dataset

3d embedding of NORB images by LLE algorithm
DrLIM learned a mapping to 3d space for images of a single airplane (The output manifold is shown under five different viewing angles)

Reference

Data Visualization

My Other Previous Paper Readings

PhD, Researcher. I share what I've learnt and done. :) My LinkedIn: https://www.linkedin.com/in/sh-tsang/, My Paper Reading List: https://bit.ly/33TDhxG