PersonLab, for Human Pose Estimation & Person Instance Segmentation — PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model, PersonLab, by Google Research, 2018 ECCV, Over 500 Citations (Sik-Ho Tsang @ Medium) Human Pose Estimation, Keypoint Detection, Instance Segmentation First, PersonLab learns to detect individual keypoints and group keypoints into person pose instances.

DeLighT, Parameter Reduction for Transformers — DeLighT: Deep and Light-weight Transformer, DeLighT, by University of Washington, Facebook AI Research, and Allen Institute for AI, 2021 ICLR, Over 60 Citations (Sik-Ho Tsang @ Medium) NLP, LM, NMT, Transformer Within each Transformer block, a deep and lightweight transformation is used using DeLighT block. Across blocks, block-wise scaling is…

ACMix, a MIXed model that enjoys the benefit of both self-Attention and Convolution (ACmix), while — On the Integration of Self-Attention and Convolution, ACMix, by Tsinghua University, Huawei Technologies Ltd., and Beijing Academy of Artificial Intelligence, 2022 CVPR, Over 40 Citations (Sik-Ho Tsang @ Medium) Image Classification, Convolution, Self-Attention ACmix, A mixed model that enjoys the benefit of both self-Attention and Convolution, is proposed.

DALS, Using Fully Convolutional Network (FCN) and Active Contour Model (ACM) — Deep Active Lesion Segmentation, DALS, by University of California, and Stanford University, 2019 MLMI, Over 40 Citations (Sik-Ho Tsang @ Medium) Medical Imaging, Medical Image Analysis, Image Segmentation Deep Active Lesion Segmentation (DALS), a fully automated segmentation framework is introduced. DALS leverages the powerful nonlinear feature extraction abilities of fully…

SAM, Simultaneously Minimizing Training Loss and Sharpness Loss, Using Loss Landscape — Sharpness-Aware Minimization for Efficiently Improving Generalization, Sharpness-Aware Minimization (SAM), by Google Research, and Blueshift, Alphabet, 2021 ICLR, Over 400 Citations (Sik-Ho Tsang @ Medium) Image Classification A novel, effective procedure for instead simultaneously minimizing training loss value and loss for sharpness.

RA-UNet, first Work To Use Attention Residual Mechanism for Tumor Segmentation from 3D Medical Volumetric Images. — RA-UNet: A Hybrid Deep Attention-Aware Network to Extract Liver and Tumor in CT Scans, RA-UNet, by Tianjin University, CSIRO Data61, Tianjin University of Traditional Chinese Medicine, and La Trobe University 2020 J. Front. Bioeng. Biotechnol., Over 220 Citations (Sik-Ho Tsang @ Medium) Medical Imaging, Medical Image Analysis, Image Segmentation, U-Net

VAN, Large Kernel Convolution by Decomposition — Visual Attention Network, VAN, by Tsinghua University, Nankai University University, and Fitten Tech, 2022 arXiv v5, Over 80 Citations (Sik-Ho Tsang @ Medium) Image Classification Applying self-attention in computer vision is challenging: (1) Treating images as 1D sequences neglects their 2D structures. (2) The quadratic complexity is too expensive for…

cGAN+AC+CAW: Conditional GAN (cGAN), Atrous Convolution (AC), Channel Attention with Weighting (CAW), are Used — Breast Tumor Segmentation in Ultrasound Images Using Contextual-Information-Aware Deep Adversarial Learning Framework, cGAN+AC+CAW, by Universitat Rovira i Virgili, Khalifa University of Science and Technology, Aswan University 2020 J. ESWA, Over 30 Citations (Sik-Ho Tsang @ Medium) Medical Imaging, Medical Image Analysis, Image Segmentation

Loss Landscape Visualization, Looking for Sharpness — Visualizing the Loss Landscape of Neural Nets, Loss Landscape, by University of Maryland, United States Naval Academy, and Cornell University, 2018 NIPS, Over 1200 Citations (Sik-Ho Tsang @ Medium) Sharpness, Visualization Filter-Wise Normalization is proposed to calculate the loss landscape for visualization. Based on the visualization, we can know the…