Brief Review — CC-3D-FCN: 3-D Fully Convolutional Networks for Multimodal Isointense Infant Brain Image Segmentation

CC-3D-FCN, Adding Convolutions to Skip Connection for Downsampling Path to Upsampling Path

Multimodality MRI data of an infant subject scanned at six months old (isointense phase). From left to right: T1 MRI, T2 MRI, and FA image.

3-D Fully Convolutional Networks for Multimodal Isointense Infant Brain Image Segmentation,
CC-3D-FCN, by University of North Carolina at Chapel Hill, and Korea University, 2018 TCYB, Over 140 Citations (Sik-Ho Tsang @ Medium)
Medical Imaging, Medical Image Analysis, Image Segmentation, U-Net

• Conventional 2-D FCN architectures is extend to 3-D for segmentation of isointense phase brain MR images.
• Coarse (naturally high-resolution) and dense (highly semantic) feature maps are integrated to better model tiny tissue regions. In addition, a transformation module is further proposed to better connect the aggregating layers. Also, A fusion module to better serve the fusion of feature maps.
• Moreover, multimodal information is used, which further boosts the segmentation performance.

Outline

1. PT-3D-CNN (PT: Pass-Through)
2. CC-3D-CNN (CC: Convolution and Concatenate)
3. Experimental Results

1. PT-3D-CNN (PT: Pass-Through)

PT-3D-CNN Architecture

• FCN actually consists of two major operations: down-sampling and up-sampling. (Similar to U-Net)
• Pass-Through (PT) operations are included in our architecture, in which the coarse feature maps of the down-sampling layers are concatenated to the whole learned feature maps to the up-sampling layers. This is similar to the operations used in U-Net.
• Fusion modules (extra convolutional layers) after the concatenation layers.
• This is named as PT-3D-CNN, as baseline.

2. CC-3D-CNN (CC: Convolution and Concatenate)

CC-3D-CNN Architecture

• A transformation module is introduced to boost the low-level features to be complementary for the high-level features.
• The whole PT operation is replaced by the Convolution and Concatenate (CC) subprocedures.
• Also, batch normalization is used after convolution.
• The weighted cross entropy loss is used.
• Patch size 32×32×32 is used.

Comparison of CC-3-D-FCN and PT-3-D-FCN

• Dice ratio is higher for CC-3D-FCN.

3. Experimental Results

3.1. Multimodality Information

Average Dice ratios of our proposed method with respect to different combinations of three imaging modalities.

Using more imaging modalities, i,e, T1+T2+FA as inputs, generally results in more accurate segmentations.

3.2. Performance

Dice Ratio and MHD on First Dataset

Dice Ratio and MHD on Second Dataset

For both datasets, CC-3-D-FCN outperforms other methods such as 3D U-Net.

3.4. Time

Average Time Cost in Minutes

CC-3D-FCN method is significantly faster than any of the other methods.

3.5. Visualization

Comparison of segmentation results by different methods

CC-3D-FCN obtains better visual segmentation.

Reference

[2018 TCYB] [CC-3D-FCN]
3-D Fully Convolutional Networks for Multimodal Isointense Infant Brain Image Segmentation

4.2. Biomedical Image Segmentation

2015 … 2018 [CC-3D-FCN] … 2020 [MultiResUNet] [UNet 3+] [Dense-Gated U-Net (DGNet)]

My Other Previous Readings