Review — Dong’s ICDSP’16: Morphological Analysis and Deep Learning (Camera Anomaly/Tampering Detection)
Morphological Analysis and Deep Learning to Detect Simple and Complicated Camera Anomalies Respectively
In this story, Camera Anomaly Detection based on Morphological Analysis and Deep Learning, by Zhejiang University of Technology, and Hebei University of Technology, is reviewed. In this paper:
- A morphological analysis and deep learning based camera anomaly detection method are proposed to detect comprehensive types of anomaly.
- Morphological analysis (Non deep learning approach) is used to detect simple camera anomalies to accelerate the processing speed.
- Deep learning is utilized to detect complicated camera anomalies to improve the accuracy.
This is a paper in 2016 ICDSP. (Sik-Ho Tsang @ Medium) The above figure shows some examples of anomaly to be detected.
Outline
- Anomaly Detection based on Morphological Analysis
- Anomaly Detection based on Deep Learning
- Experimental Results
1. Anomaly Detection based on Morphological Analysis
- Morphological analysis based anomaly detection has sufficiently good performance to detect five camera anomalies.
1.1. Brightness Fault (Too Bright or Too Dark)
- A brightness threshold L and a darkness threshold D are set.
- Then, all grey level of pixels in a video image is compared with L and D.
If the number of pixels with grey level great than L is greater than threshold NL, the video image is identified as too-bright-anomaly.
Similarly, if the number of pixels with grey levels less than D is greater than threshold ND, the video image is identified as too-dark-anomaly.
1.2. Brightness Fault (Gain Disorder)
- YUV color space is utilized to detect gain disorder.
- As in the figure, the greater the distance from the origin, the deeper the color is. Video image is segmented by setting deep color pixel threshold dcp.
When the number of deep pixels is greater than 1/2 of the number of all pixels, and each partition of coordinate system as shown in the figure and each image partition have uniform number of deep pixels, the current video image is an image with gain disorder.
1.3. Freeze Abnormal
When a certain number of consecutive video images are identical, freeze abnormal appears.
1.4. Lose Abnormal
- When a screen picture of a monitoring video is missing, blue or black screen presents.
Therefore, lose abnormal detection is implemented by counting the number of blue screen frames or black screen frames.
1.5. Scrolling Abnormal
- In scrolling abnormal, the content change of adjacent image frames is little, but the locations of their corresponding contents are shaking.
- If the adjacent image frames are aligned, ghosting phenomenon will occur and their contents will increase.
- First, a gradient map of current image is calculated:
- where I(x,y) is the pixel value of a pixel (x,y).
If ||fk(x,y)|-|fk+2(x,y)||>Tk, pixel (x,y) is identified as a contour point of the scrolling image region.
- For current image, the number of contour points N1 and the number of the edge points N2 are calculated.
If |N2-N1|is greater than 1/10 of the area of current image, the current image frame is identified as scrolling image.
1.6 Color Cast (too green, too blue, too red, too yellow)
- The RGB values first are transformed to the XYZ color space, and then the XYZ values are transformed to the Lab color models.
- Here, the values of Xn, Yn and Zn are 95.047, 100, and 108.883 respectively.
- After Lab values are calculated, image average chrominance D, central moment of image chrominance M.
- Color cast factor K=D/M are computed.
Then, color cast is determined according to color cast factor calculation criteria:
2. Anomaly Detection based on Deep Learning
- There are some complicated anomalies, such as strip interference, camera occlusion and image blurring, which are difficult to be detected by morphological analysis based detection methods.
- The CNN is used to detect such complicated anomalies.
- It consists of 4 convolutional layers, 3 pooling layers, 1 fully-connected layer. Softmax loss is used to train a classifier.
- (It is not sure but I believe the single current frame is used as input for anomaly detection of the current frame. The number of classes to be predicted or the number of output neurons is also not sure. Not much information about the network is provided in the paper.)
3. Experimental Results
- A certain amount of surveillance videos from two real application scenarios is collected.
- One scenario is a residence community where the VSS includes 10 16-channel Hikvision DVR surveillance devices and 160 cameras.
- Another is a school where the VSS includes 14 16-channel Hikvision DVR surveillance devices and 194 cameras.
- The accuracy of the proposed method in two application scenarios exceeds 95%.
- The false alarm rates in various different times of a day are shown above.
- False alarms occur when light condition changes relatively greatly.
- “detection-train-detection” tactics is applied in which video images leading to false alarms are used as training samples to train a new classifier for the following camera anomaly detection.
- As shown in the above tables, the tactics enhances the robustness of the anomaly detection method and prevents the recurrence of the same false alarm.
- As shown above, deep learning based detection method has more excellent accuracy for complicated anomalies.
- As time goes by, when enough samples leading to false alarm are added as training samples, the accuracy of our deep learning based detection method can be further improved.
Reference
[2016 ICDSP] [Dong’s ICDSP’16]
Camera Anomaly Detection based on Morphological Analysis and Deep Learning
Camera Tampering Detection
2016 [Dong’s ICDSP’16] 2019 [VFI-ConvLSTM] [UHCTD]
