Projects:2019s2-22501 Maritime Ship Detection for Harbour and Coastal Monitoring
1 ABSTRACT The proposed project aims to present a system for maritime ship detection for harbour surveillance using optical imagery. This process involves extraction of ship candidates from a complex background and confirmation of the candidates from false positives. This system involves detecting and where possible recognising ship types from images captured by a camera placed on a fixed/moving platform in the sea. The detected ships are then compared to the image with the database to recognise the ship. In the proposed project the three main steps are: horizon detection, ship detection and ship recognition. By exploiting the concepts such as edge detection, Gaussian mixture models, Saliency maps and template matching we <gallery> <gallery> Example.jpg|Caption1 simply extract the required ship data from the image for detection and recognition.
INTRODUCTION Oceans and seas are home to numerous significant tasks, such as trade and tourism. International tourism and cargo transport, fishing, and different military shipments are brought out through these waters. Maritime surveillance tasks are critical to guarantee the tasks on the oceans or seas are completed securely and that national and global security is kept up with no issues. Ship targets are a key goal of maritime surveillance and wartime battle, and the automation in detection and recognition ships is of critical advantage with numerous applications in both civil and defence spaces. There have been numerous past examinations on ship recognition in Synthetic aperture radar (SAR) imagery, High Frequency Surface Wave Radar (HFSWR) imagery, regular ship-based radars imagery, and Infrared and optical sensors imagery and video. These sorts of strategies have points of interest due minimal influence by climate and time, but face challenges due to low resolution SAR images and a long return period [1]. With the fast advancement of optical remote imaging innovation, a few scientists have given more consideration to the recognition of ships with optical imagery due of their higher resolution and detailed spatial data, in contrast to SAR images [1]. In contrast with SAR detection and recognition algorithms, ship detection and recognition using optical remote-detecting imagery) is a recent development. At present, the techniques for ship location can be summed up in three stages: sea, sky and land separation, ship feature extraction and false alarm expulsion.[1]
Constraints- This is a passive optical based detection The camera(s) direction limits the accurate look of ship Platform is in continuous motion, therefore accurate registration or accurate detection of minute movements may to be practical. The weather conditions also pose as a constraint due to their effect on the imagery. Example- Fog, mist or cloudy day. Recognition is challenging due to wide range of ship types and unknown aspect ratio and range.
Project Proposal
Horizon Detection Horizon is the line which separates the sky from the rest of the image Colour based horizon detection uses the intensity of colour difference between sea , sky and horizon [2].
Edge based horizon detection aims at determining the principle edge in the image, this is achieved by computing the binary edge map and subsequently identifying the most prominent edge in the image[2].
DCT based horizon detection divides the original image is into 8x8 blocks for DCT. The blocks are then divided into sea/sky regions. The horizon line is the points joining the bottommost part of the sky region[1]
Temporal information (MOVING CAMERA) Gaussian Mixture Model (GMM)-Background Modelling and subtraction. GMM a distribution by a linear combination of Gaussian densities. [6] GMM can be used to cluster N pixels into L class labels.[6] GMM does not consider spatial information. [6] Morphological erosion- Removes irrelevant size details from binary image Shrinks the image. Gives accurate estimation on location of edge.
SALIENCY MAPS (FIXED CAMERA) [6] Saliency- Series of techniques for highlighting sections of the image that differ from surrounding. Differing features- Cluster of Texture, Shape, color [6] Spectral residual- Redundancies are removed. Only unexpected signals are passed on to the next stages. [3]
PHASE QUATERNION TRANFORM SALIENCY- Utilizing the exponential of spectral residual rather than the amplitude spectrum, furthermore, keeping the phase spectrum, the image can be reconstructed to obtain the saliency.
GRAPH BASED VISULA SALIENCY - The standard approached depend on the feature selection governed biologically, the local gradients are then highlighted by processing only that required segment of the image resulting in ‘master map’.
IMAGE SIGNATURE - The model of obtaining image signature is developed based on discrete cosine transform (DCT) algorithm.
CONSTANT FALSE ALARM RATE BASED DETECTION The principle of operation is the 2P-CFAR detector. Sliding window is used in the detector consisting of guard region, cell under test region (target window) and the background. Considering the cell under test is denoted by xt, mean is denoted by μb and standard deviation is given by σb and the threshold is given T.
DATA COLLECTION This project is based on maritime surveillance for harbour and coastal monitoring, majority of the applications of this system are in the military and defence activities. Therefore, the datasets of the maritime imagery are difficult to obtain. The required imagery for experiments is 1) horizon images, 2) ship on horizon images and 3) different ship models and silhouettes information if possible. However, to test the algorithms images downloaded from the internet are used. We are also using the Singapore Maritime Dataset from which we have obtained 97 videos for horizon and ship detection, using both on-shore and off-shore camera. This dataset includes scenarios such as haze, mist, fog, rain etc. The dataset also includes ground truth labelling data, which can be used to map bounding boxes for training detection dataset. We have collected 200 images for horizon detection and 300 images for ship detection. Real-time pictures are also included in the dataset that are captured by our own cameras. Data collection is aimed to be an ongoing task more images will be added to the dataset from time to time.
GROUND TRUTH LABELLING Training dataset is an important task of the project. This enables us to train the system for efficient detection and provides a reference for the detection results using proposed algorithm. The ground truth labelling is required for horizon detection, ship detection and ship recognition. Once the dataset is trained, the test data can be compared to the trained data for evaluation of system accuracy. Detections encased in bounding boxes are matched against the ground truth data and decided to be true or false positives by estimating bounding box measures. Using ground truth labeller application in MATLAB makes it easy to mark the bound boxes around the region of interest. Rectangular shape boxes are used to mark the detection area of the ship. The bounding boxes can be labelled as ship to train the dataset for ship detection. The ground truth table is then generated which includes the coordinates of the bounding boxes. This coordinates when obtained from detection results can be compared to the trained object detection dataset so decide if the coordinates are the same and the bounding boxes overlap. There has been no further development in this area, as the evaluation technique used in the final analysis is different method.
EVALUATION The evaluation process involves matching the training dataset with the test results obtained. Initially the plan was to perform ground truth labelling for the entire dataset and once the detector is trained, the resultant bounding boxes would be compared to the detection bounding boxes of the ground truth dataset as shown in figure 22. The resultant detections at this stage are saliency maps with blobs at the detection location, these blobs are not compared to the bounding boxes but instead an informal visual assessment is carried to quantify the analysis. The method used is basically rating the system methods on 5-point basis. The tabular column below describes each case from 1 to 5 points.
Results Horizon detection was done for 300 images with the same approach but by tuning the filter, the images were derived from different data set and varied from simple to harsh cases. The first method used the canny edge detector provided by the matlab tool and the second one using the canny edge detector with ‘6’ as its window size. In the initial project proposal model, we proposed a system where we had performed the ground truth labelling for the 300 images and the final results would be compared with those and results would be tabulated. But as we are not using the horizon information the model was not designed. A substitute model where the results are rated from 1 to 3 based on the accuracy of the result is designed, where 1 represents false detection and 3 proper horizon detection.
The initial project proposal had a system which compares the ground truth images of marked ships with the detected ships and would quantify the difference in detection. As the research progressed, we came across a lot of interesting ship detection techniques and tried to develop a working system on ship detection using Gaussian mixture modelling, Discreate Cosine Transform and RCNN, due to which we are analysing the result with a rating system of 1 to 5 with one representing no detection and 5 perfect detection.
The output obtained in saliency maps are blobs at the location of ship. And to compare the validity of the method the ground-truth labelling uses bounding boxes. The blobs cannot be compared to the bounding boxes, a subjective evaluation is carried out for all the methods on 200 imagery datasets. Using the subjective evaluation, the saliency techniques are rated based on 5 stars, this rating is purely observation based on this particular model and dataset and will be irrelevant for any other application. The image signature technique is observed to be 5star rank, that considers the discrete fourier transform of the image as it gives the best output as compared to other methods. This method provides correct detection of ships for 53% of the images. The algorithm helps to detect ships in complex backgrounds such as in the case of multiple ships and variations in sea and sky texture. The computation time is also low for the algorithm. The algorithm does fail in the conditions of haze and unclear ship pixels but when compared to other techniques, it detects ship blobs for various images.
The quaternion transform does strongly compete with GBVS method. The quaternion transform 4-star ranked, does provide clear detection with not much of the unnecessary background clutter in the maps. 45% of the images have proper ship blobs. GBVS method 3-star ranked is observed to have higher tendency to spread the detection in along the background clutter in the presence of multiple targets. When ship targets are closely packed, or the sea texture is rough the GBVS technique fails to detect only ship targets. In this technique multiple maps are examined, 9 feature maps and multiscale images, whereas in Quaternion and Image signature only 3 channels are considered to provide more accurate output detection. GBVS holds good for 42% of the detections.
conclusion In this project both the segments: Horizon detection and ship detection were carried out using multiple approaches present by tuning the parameters for accurate detections. The obtained results for ship detection using saliency gives us 66% detection blobs using various techniques mentioned and RCNN detection is 86% true results from the above analysis.
FUTURE SCOPE The future scope of this project is to compute the ship outline from the saliency maps and use for recognition. Recognition of ship targets are a huge section of research and is not yet begun. The techniques mentioned in literature for recognition are to be explored on the existing models of saliency, the parameters such filter tuning, window size, thresholding can be modified to obtain better version. The dataset of 200 was accumulated and 110 images were analyzed for algorithm comparison. Therefore, another scope includes assessing a large data more than 200 images to improve the performance rate of the algorithm. Ground truth labelling was also done in order to evaluate the functioning by comparing the bounding boxes obtained. Hence, ground truth labelling can be more developed to create a formal trained dataset.
