Neural Architecture Search (NAS) for Depth & Width in U-Net — AdwU-Net: Adaptive Depth and Width U-Net for Medical Image Segmentation by Differentiable Neural Architecture Search, AdwU-Net, by Shanghai Jiao Tong University, and National Medical Products Administration, 2022 MIDL (Sik-Ho Tsang @ Medium) Biomedical Image Segmentation 2015 … 2022 [UNETR] [Half-UNet] [BUSIS] [RCA-IUNet] [Swin-Unet] [DS-TransUNet] [UNeXt] 2023 [DCSAU-Net] [RMMLP] ==== My Other Paper…

LIMA, Quality is More Important Than Quantity — LIMA: Less Is More for Alignment, LIMA, by Meta AI, Carnegie Mellon University, University of Southern California, and Tel Aviv University, 2023 arXiv v1 (Sik-Ho Tsang @ Medium) Language Model 1991 … 2022 [GPT-NeoX-20B] [GPT-3.5, InstructGPT] [GLM] [MT-NLG 530B] [Chinchilla] [PaLM] [AlexaTM] [BLOOM] [AlexaTM 20B] [OPT] [Switch Transformers] [LaMDA] [LoRA] [Galactica]…

Galactica (GAL) 120B, Outperforms PaLM 540B — Galactica: A Large Language Model for Science, Galactica, by Meta AI, 2022 arXiv v1, Over 50 Citations (Sik-Ho Tsang @ Medium) Language Model 1991 … 2022 [GPT-NeoX-20B] [GPT-3.5, …

VLMo, VLM or Foundation Model, Using Multiway Transformer, Inspired by MoE and Switch Transformer — VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts, VLMo, by Harbin Institute of Technology, and Microsoft Corporation, 2022 NeurIPS, Over 90 Citations (Sik-Ho Tsang @ Medium) Vision Language Model (VLM) / Foundation Model 2017 … 2021 [CLIP] [VinVL] [ALIGN] [VirTex] [ALBEF] [Conceptual 12M (CC12M)] 2022 [FILIP] [Wukong] [LiT] [Flamingo] [FLAVA] [SimVLM] 2023 [GPT-4] ====…

CrossFormer, Outperforms DeiT, PVT, Swin, Twins, etc. — CrossFormer: A Versatile Vision Transformer Hinging on Cross-scale Attention, CrossFormer, by Zhejiang University, Tencent, and Columbia University, 2022 ICLR, Over 110 Citations (Sik-Ho Tsang @ Medium) Image Classification 1989 … 2023 [Vision Permutator (ViP)] [ConvMixer] ==== My Other Paper Readings Are Also Over Here ==== CrossFormer are designed, with Cross-scale…

DHVT, Solve Two Weaknesses of ViTs When Training from Scratch — Bridging the Gap Between Vision Transformers and Convolutional Neural Networks on Small Datasets, DHVT, by University of Science and Technology of China, 2022 NeurIPS (Sik-Ho Tsang @ Medium) Image Classification 1989 … 2023 [Vision Permutator (ViP)] [ConvMixer] ==== My Other Paper Readings Are Also Over Here ==== There are two…

LDBM, Self-Supervised Learning (SSL) + Self-Distillation (SD) — Fair Visual Recognition in Limited Data Regime using Self-Supervision and Self-Distillation, LDBM, by IIT Kanpur, IIT Roorkee, and University of Bath, 2022 WACV (Sik-Ho Tsang @ Medium) Self-Supervised Learning 1993 … 2022 [BEiT] [BEiT V2] [Masked Autoencoders (MAE)] [DiT] [SimMIM] Bias mitigation techniques assume that a sufficiently large number of…

By Analyzing ViT, AlterNet is Proposed, Mixing Convs & MSAs — How Do Vision Transformers Work?, AlterNet, by Yonsei University, and NAVER AI Lab, 2022 ICLR, Over 170 Citations (Sik-Ho Tsang @ Medium) Image Classification: 1989 … 2023 [Vision Permutator (ViP)] [ConvMixer] ==== My Other Paper Readings Are Also Over Here ==== Multi-head Self Attentions (MSAs) improve not only accuracy but…

RMMLP: MEEMLP+DMMLP+DSFF — RMMLP: RollingMLP and Matrix Decomposition for Skin Lesion Segmentation, RMMLP, by Jiangnan University, and Wuxi People’s Hospital, 2023 J. BSPC (Sik-Ho Tsang @ Medium) Biomedical Image Segmentation 2015 … 2022 [UNETR] [Half-UNet] [BUSIS] [RCA-IUNet] [Swin-Unet] [DS-TransUNet] [UNeXt] 2023 [DCSAU-Net] An MLP-based model, RMMLP, is proposed using matrix decomposition and rolling…