Brief Review — Fusion of Deep Convolutional Neural Networks for No-Reference Magnetic Resonance Image Quality Assessment
Fusion of Multiple Models for Medical IQA
Fusion of Deep Convolutional Neural Networks for No-Reference Magnetic Resonance Image Quality Assessment
Model Fusion + SVR, by Rzeszow University of Technology, Jagiellonian University Medical College, AGH University of Science and Technology
2021 MDPI Sensors, over 1500 Citations ( @ Medium)Quality Assessment
2020 [Mean Teacher + ROI Consistency] 2022 [Swin-MIQA]
==== My Healthcare and Medical Related Paper Readings ====
==== My Other Paper Readings Are Also Over Here ====
- The quality of magnetic resonance (MR) images may influence the diagnosis and subsequent treatment.
- Model fusion with SVR is used to predict the MR image quality so that image with low quality can be retaken.
Outline
- Model Fusion + SVR
- Results
1. Model Fusion + SVR
- The performance of a single network turned out to be insufficient to provide superior performance in the MRIQA task.
- Thus, The model fusion is applied.
- An exemplary fusion of ResNet-18, ResNet-50, and GoogLeNet is presented in Figure 1.
- The last three network layers of each used network, configured for 1000 classes, are replaced with a fully connected layer and the regression layer.
- Among the last elements in each network are the global average pooling layer.
- And there is an addition layer that fuses their outputs, fully connected layer, and regression layer.
- The weights of fused networks were modified using a small learning rate instead of frozen.
- Mean squared error (MSE) is used to train the model:
- Once the training of the network is finished, a second step of the approach is executed in which concatenated feature vectors, which are then used as an input to the SVR module to obtain a quality prediction model.
- The SVR technique is commonly used to map perceptual features to MOS.
- The radial base function (RBF) is used as the kernel function.
2. Results
2.1. Datasets
- There are 2 datasets: One is DB1. One is proposed DB2.
2.2. Performance Evaluation
- There are approaches tried: ResNet-50_GoogLeNet_MobileNet-V2 (R18GR50M), GoogLeNet_DenseNet-201 (GD201), GoogLeNet_MobileNet-V2 (GM), GoogLeNet_ResNet-101 (GR101), GoogLeNet_ResNet-18 (GR18), GoogLeNet_ResNet-50 (GR50), MobileNet-V2_ResNet-101 (MR101), MobileNet-V2_ResNet-18 (MR18), MobileNet-V2_ResNet-50 (MR50), ResNet-50_GoogLeNet_ResNet-18 (R50GR18), ResNet-50_Inception-V3 (R50Iv3), ResNet-50_ResNet-101 (R50R101), and ResNet-50_ResNet-18 (R50R18).
Specifically, the R50GR18 (composed of three network architectures) and MR50 (fusing two networks) obtained the best SRCC and KRCC performances for the first database, outperforming the recently introduced NOMRIQA.
It is worth noticing that NR MRIQA methods that do not require training, ENMIQA and SNRTOI, produce poorly correlated quality opinions.
For the database introduced in this paper, DB2, the R18GR50M, and R50GR18 present superior performance, followed by the SEER and MR50.
- All methods were run in Matlab. (As authors mentioned)
The proposed models are of moderate complexity, being on par with the fastest and more reliable approaches. Further reduction of the computation time can be achieved by a parallel feature extraction process or providing a native implementation (e.g., C++).
- In this experiment, learning-based methods are trained on one database and tested on another.
The introduced fusion models outperform other techniques and exhibit stable prediction accuracy.
