No-Reference Image Quality Assessment for Multiple Distortions Using Saliency Map Based on Dual-Convolutional Neural Networks

10.6138/JIT.2017.18.7.20170801

Jian-Jun Li；Lan-Lan Xu；Zhi-Hui Wang；Chin-Chen Chang

Convolutional neural networks ； NR-IQA ； Multiply distorted image quality assessment ； HSV ； Saliency map

網際網路技術學刊

18卷7期（2017 / 12 / 01）

1701 - 1710

英文

No-reference image quality assessment (NR-IQA) is a challenging topic in image processing and image assessment, especially for multiple distortions. We propose a novel NR-IQA method based on a dual-convolutional neural network structure. Both gray and color features in Hue Saturation Value (HSV) space are extracted to train a model in order to improve the accuracy of logistic regression. In addition, our method combines with saliency detection which is in line with human visual system (HVS). The saliency map contains more information of an image and is considered to be the most attractive region. Thus, a well-trained model is used to detect the saliency map of an input image. The saliency map is employed as a weighting function to reflect the important distortion brought by the local region. The first CNN structure includes two convolutional layers with maximum pooling, inception module and two fully connected layers. The second CNN structure only contains convolutional and pooling layers. Our proposed algorithm achieves a Pearson (Linear) Correlation Coefficient (LCC) of 0.964 and a Spearman's Rank Ordered Correlation Coefficient (SROCC) of 0.962 on the LIVE database, which demonstrates the excellent generalization ability in cross database experiment. Furthermore, we fine-tune a CNN model trained on LIVE database to predict multiply distorted image quality. Experiment result shows that our method outperforms the state-of-the-art methods on LIVE multiply distorted image database.