Projects:2018s1-108 Machine Learning Multi-Spectral Simulation

Introduction

This project aims to develop a machine learning system for generating simulated real-world environments from satellite imagery. The environments generated will be incorporated into VIRSuite, a multi-spectral real-time scene generator. The system will need to generate material properties to allow VIRSuite to render the environment realistically from visible to long-wave infrared spectra. The system will need to be able to regenerate existing environments as new satellite images become available, but doesn't need to be real-time or closed loop. Secondary phases of the project are to build a validation process to check that generated environments match real-world likeness across all spectra. The satellite imagery available is up to 0.3m per pixel and the environments generated should reflect this. This project requires students to: gain knowledge in deep and convolutional machine learning technologies, develop and adapt machine learning techniques for satellite imagery (GIS Data), and conduct validation experiments using multi-spectral knowledge and real-life data.

Image Segmentation

In order to identify objects in the imagery, the objects have to first be separated. This is done by partitioning the imagery into regions. Region merging approach is applied to perform image segmentation, based on techniques described in . The image is first oversegmented using low-level method. Possible low-level segmentation algorithms are watershed transformation, Felzenszwalb’s graph based segmentation, and simple linear iterative clustering (SLIC). Region adjacency graph (RAG) and nearest neighbour graph (NNG) are created from the segmented regions.

RAG and NNG

RAG is an undirected graph which consists of nodes and edges. Nodes represents regions in the image, two nodes are connected by an edge if the two nodes are neighbouring regions. The weight of the edge is the similarity distance between the two regions, which is the combination of spectral, texture and shape feature distances. NNG is a directed graph that can be constructed from RAG by only keeping the edge with the lowest weight for each node. Each node has an edge that points to its most similar neighbour. If two nodes point to each other, an NNG circle is formed.

Merging Process

Create RAG and NNG and calculate the weights of edges base on similarity distance as described below.

Spectral Feature

Spectral feature distance can be calculated by considering standard deviations of pixels in the regions. The following equation is used to calculate spectral feature distance.

a and b are area of region a and area of region b respectively, std_ai and std_bi are standard deviation of region a and standard deviation of region b respectively in band i, std_i is the standard deviation of two region a and region b combined.

Texture Feature

Nonsubsampled contourlet transform (NSCT) is used to obtain the texture feature. The texture feature distance is obtained by following the steps below:

1. Perform principal component analysis (PCA) on the multispectral image. The first principal component is taken as the input to NSCT.
2. Perform NSCT on the input image with scale number of k. The directional decomposition number for each scale is d_m = 2^m for scale 1 ≤ m ≤ k. k' bandpass images for each scale are generated, denoted by I₁, I₂, ..., I_k' respectively. For each bandpass image I_m (where m = 1,2,...,k), d_m directional subimages are generated, expressed as I_m¹, I_m²,...,I_m^dm
3. From the directional subimages, form texture feature vector that corresponds to every pixel.
4. Perform fuzzy C-means clustering analysis on all texture feature vectors. Suppose g clusters are created, the cluster centres are denoted by CC₁,CC₂,...,CC_g.

Project Members

Supervisors

• Danny Gibbins
• Sebastien Wong

Students

• Alwin Kai En Ng
• Thomas Focareta