PP-YOLO, Outperforms EfficientDet & YOLOv4 — PP-YOLO: An Effective and Efficient Implementation of Object Detector PP-YOLO, by Baidu Inc., 2020 arXiv v3, Over 90 Citations (Sik-Ho Tsang @ Medium) Object Detection, YOLO series PP-YOLO is proposed, which is a new object detector based on YOLOv3. It combines various existing tricks that almost not increase the number…

LRCN, Using Both CNN & LSTM, Outperforms m-RNN — Long-term Recurrent Convolutional Networks for Visual Recognition and Description, LRCN, by UT Austin, UMass Lowell, and UC Berkeley, 2015 CVPR, Over 6000 Citations (Sik-Ho Tsang @ Medium) Video Classification, Image Captioning, Video Captioning Long-term Recurrent Convolutional Networks (LRCNs) is proposed, which uses CNN and LSTM jointly for image description and…

Self-Supervised Learning by Playing Rubik’s Cube — Rubik’s Cube+: A self-supervised feature learning framework for 3D medical image analysis, Rubik’s Cube+, by Chinese Academy of Sciences, and Tencent Jarvis Lab, 2020 JMIA, Over 50 Citations (Sik-Ho Tsang @ Medium) Self-Supervised Learning, Medical Image Segmentation, Medical Image Classification, Image Segmentation, Image Classification A pretext task, Rubik’s Cube+, is…

IRNN, RNN Initialized Using Identity Matrix — A Simple Way to Initialize Recurrent Networks of Rectified Linear Units IRNN, by Google 2015 arXiv v2, Over 700 Citations (Sik-Ho Tsang @ Medium) Recurrent Neural Network, RNN To overcome vanishing and exploding gradients, instead of using LSTM, identity matrix or its scaled version is proposed to initialize the recurrent…

RegNet, Simple & Regular Networks are Designed, By Analyzing the Network Design Space — Designing Network Design Spaces RegNet, by Facebook AI Research (FAIR) 2020 CVPR, Over 500 Citations (Sik-Ho Tsang @ Medium) Image Classification, Convolutional Neural Network, CNN, Neural Architecture Search, NAS Authors design network design spaces that parametrize populations of networks. By exploring the structure aspect of network design at design space…

Design Deep RNN Without Using Memory-Based LSTM/GRU — How to Construct Deep Recurrent Neural Networks DT-RNN, DOT-RNN, sRNN, by Université de Montréal 2014 ICLR, Over 1000 Citations (Sik-Ho Tsang @ Medium) Recurrent Neural Network, RNN, Sequence Model, Language Model At that moment, there was a bottleneck of how to construct a efficient deep RNN when LSTM and GRU…

SGD+CR, Solving the Gradient Explosion and Vanishing Problems in RNN — On the Difficulty of Training Recurrent Neural Networks SGD+CR, by Universite de Montreal 2013 ICML, Over 5000 Citations (Sik-Ho Tsang @ Medium) Recurrent Neural Network, Character Prediction, Language Model A gradient norm clipping strategy is proposed to deal with exploding gradients. A soft constraint is proposed for the vanishing gradients…

YOLOv7, Train From Scratch, Outperforms All Object Detectors — YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors, YOLOv7, by Institute of Information Science 2022 arXiv v1 (Sik-Ho Tsang @ Medium) Object Detection, YOLO Series Unlike the previous YOLOv5 and YOLOv6, YOLOv7 comes from the author of YOLOv4 Alexey Bochkovskiy. Extended efficient layer aggregation networks (E-ELAN), model scaling…

Rectified Linear Unit (ReLU) Introduced — Rectified Linear Units Improve Restricted Boltzmann Machines ReLU, by University of Toronto 2010 ICML, Over 17000 Citations (Sik-Ho Tsang @ Medium) Activation Function, Restricted Boltzmann Machine, Image Classification, Face Recognition Rectified Linear Unit (ReLU) is introduced, which outperforms Sigmoid. This is a paper from Hinton’s research group. Outline