Word2Vec: Using CBOW or Skip-Gram to Convert Words to Meaningful Vectors, i.e. Word Representation Learning
In this story, Efficient Estimation of Word Representations in Vector Space, (Word2Vec), by Google, is reviewed. In this paper:
- Word2Vec is proposed to convert words into vectors that have semantic and syntactic meanings, i.e. word representations, e.g.: vector(”King”)-vector(”Man”)+vector(”Woman”) results , we can obtain vector(”Queen”).
- Two approaches are proposed for Word2Vec…
Using Triplet Loss for Contrastive Learning
In this story, FaceNet: A Unified Embedding for Face Recognition and Clustering, by Google, is reviewed. In this paper:
- A mapping from face images to a compact Euclidean space is learned where distances directly correspond to a measure of face similarity.
Feedforward Neural Network for Word Prediction
In this story, Adaptive Importance Sampling to Accelerate Training of a Neural Probabilistic Language Model, by Université de Montréal, is reviewed. This is a paper by Prof. Yoshua Bengio. In this paper:
- A feedforward neural network is trained to approximate probabilities over sequences of words.
- Adaptive importance sampling is designed…
DrLIM: Contrastive Learning for Dimensionality Reduction
In this story, Dimensionality Reduction by Learning an Invariant Mapping, (DrLIM), by New York University, is reviewed. This is a paper by Prof. LeCun. Originally, this method is proposed for Face Recognition in 2005 CVPR. In this paper:
- A method called Dimensionality Reduction by Learning an Invariant Mapping (DrLIM) is…
Contrastive Loss + LeNet-Like CNN Siamese Network for Face Recognition
In this story, Learning a Similarity Metric Discriminatively, with Application to Face Verification, by New York University, is briefly reviewed. This is a paper from Prof. LeCun. In this paper:
- Contrastive loss function is used for training the Siamese network for face verification/recognition.
- Specifically, a function is learnt to map…
Using Attention Decoder, Automatically Search for Part of Source Sentence at Encoder for Machine Translation
In this story, Neural Machine Translation by Jointly Learning to Align and Translate, (Attention Decoder/RNNSearch), by Jacobs University Bremen, and Universit´e de Montr´eal, is reviewed. This is a paper by the group of Prof. Bengio. In previous RNN Encoder-Decoder and Seq2Seq, a fixed-length vector is used in between encoder and…
Neural Image Caption (NIC) for Caption Generation
In this story, Show and Tell: A Neural Image Caption Generator, by Google, is reviewed. In this paper:
- Neural Image Caption (NIC) is designed for image captioning.
- BN-Inception / Inception-v2 generates the image representation.
- LSTM generates natural sentences describing the image.
CNN for Image, Bidirectional RNN for Senstences, Generate Descriptions over Images Regions
In this story, Deep Visual-Semantic Alignments for Generating Image Descriptions, by Stanford University, is reviewed. This is a paper by Prof. Li Fei-Fei, In this paper:
- A Convolutional Neural Network (CNN) is used over image regions, and a bidirectional Recurrent Neural Network is used over sentences, and a structured objective…
Using CNN+RNN for Captioning, Generate Sentence from Image
In this story, Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN), by University of California, and Baidu Research, is reviewed. In this paper:
- A multimodal Recurrent Neural Network (m-RNN) model is designed for generating novel image captions.
- The model consists of two sub-networks: a deep recurrent neural network for sentences…
Designed a New Hidden Unit for Statistical Machine Translation (SMT) Using RNN Encoder-Decoder
In this paper, Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation, (RNN Encoder-Decoder), by Universit´e de Montr´eal, Jacobs University, and Universit´e du Maine, is reviewed. This is also a paper by Prof. Bengio. In this paper:
- RNN Encoder-Decoder is proposed that consists of two recurrent neural networks (RNN)…
Sik-Ho Tsang
PhD, Researcher. I share what I've learnt and done. :) My LinkedIn: https://www.linkedin.com/in/sh-tsang/, My Paper Reading List: https://bit.ly/33TDhxG
