Projects:2018s1-108 Machine Learning Multi-Spectral Simulation
Contents
Introduction
This project aims to develop a machine learning system for generating simulated real-world environments from satellite imagery. The environments generated will be incorporated into VIRSuite, a multi-spectral real-time scene generator. The system will need to generate material properties to allow VIRSuite to render the environment realistically from visible to long-wave infrared spectra. The system will need to be able to regenerate existing environments as new satellite images become available, but doesn't need to be real-time or closed loop. Secondary phases of the project are to build a validation process to check that generated environments match real-world likeness across all spectra. The satellite imagery available is up to 0.3m per pixel and the environments generated should reflect this. This project requires students to: gain knowledge in deep and convolutional machine learning technologies, develop and adapt machine learning techniques for satellite imagery (GIS Data), and conduct validation experiments using multi-spectral knowledge and real-life data.
Image Segmentation
In order to identify objects in the imagery, the objects have to first be separated. This is done by partitioning the imagery into regions. Region merging approach is applied to perform image segmentation, based on techniques described in [1]. The image is first oversegmented using low-level method. Possible low-level segmentation algorithms are watershed transformation, Felzenszwalb’s graph based segmentation, and simple linear iterative clustering (SLIC). Region adjacency graph (RAG) and nearest neighbour graph (NNG) are created from the segmented regions.
RAG and NNG
RAG is an undirected graph which consists of nodes and edges. Nodes represents regions in the image, two nodes are connected by an edge if the two nodes are neighbouring regions. The weight of the edge is the similarity distance between the two regions, which is the combination of spectral, texture and shape feature distances. NNG is a directed graph that can be constructed from RAG by only keeping the edge with the lowest weight for each node. Each node has an edge that points to its most similar neighbour. If two nodes point to each other, an NNG circle is formed.
Merging Process
- Create RAG and NNG and calculate the weights of edges base on similarity
distance as described below.
Spectral Feature
Spectral feature distance can be calculated by considering standard deviations of pixels in the regions. The following equation is used to calculate spectral feature distance.
Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle d_{spe} = \sum_{i=1}^{n}\left(\text{std}_i - \frac{a \times \text{std}_{ai} + b \times \text{std}_{bi}}{a+b}\right)} \begin{equation} d_{spe} = \sum_{i=1}^{n}\left(\text{std}_i - \frac{a \times \text{std}_{ai} + b \times \text{std}_{bi}}{a+b}\right) \end{equation}
Project Members
Supervisors
Students
References
- ↑ D. Liu, L. Han, X. Ning, and Y. Zhu, “A segmentation method for high spatial resolution remote sensing images based on the fusion of multifeatures”, IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 8, pp. 1274–1278, 2018, ISSN: 1545-598X. DOI: 10.1109/LGRS.2018.2829807
