Spatiotemporal Clustering of Driving Situations Using Unsupervised Learning for Sensor Data Encoding

W. Siddiqui


All drivers have their own habitual choice of driving style, but these driving styles also vary under specific environmental scenarios. To improve the functionality of advanced driver systems, information regarding the surrounding is of utmost importance and can help to predict driving behaviours under different circumstances. To better understand different environmental scenarios, this thesis aims to develop a system that can differentiate environmental scenes based on weather, traffic or road conditions. Images and point clouds of different scenes are gathered from a driving vehicle driven by different drivers on the same route. Using unsupervised manifold learning, the dimensionality of the raw sensor data is reduced to a 2D mapping and then clustered into representative driving situations. Not-Too-Deep (N2D) has been picked as a baseline system. An autoencoder from N2D is used for extracting features from images. Apart from that, beta-VAE is also used for image feature extraction. For point cloud feature extraction FoldingNet, LRGM and CenterPoint autoencoders have been used. Features from images and point clouds are also fused to obtain feature points with better environmental information. Principal component analysis (PCA) is first used to reduce the dimensions of these features after PCA uniform manifold approximation, and projection (UMAP) or pairwise controlled manifold approximation projection (PaCMAP) is used to reduce the dimensions to two. Finally, hierarchical density-based spatial clustering of applications with noise (HDBSCAN) is used for clustering. For evaluation, internal and external validation criteria are used, and a comparison is made between clustering using image features, clustering using point cloud features and clustering using fused features.