Visual Scene Analysis

In Order to detect victims and determine their vital functions the robot continuously scans its environment. A visual scene analysis is performed in order to detect victims, dangerous materials marked with hazmat signs and QR codes. The information an victims will then be added to the generated map and enrich it with semantic information.

In order to search victims, calibration and registration of thermal, video and depth cameras is required. Victim hypotheses from thermal camera images are validated using images from a video and a depth camera.


Thermal image		Color image

In order to detect hazmat signs, all contours in the continuous stream of images are analyzed. Afterwards, a classification bounded by the detected regions is performed using histograms of color images and support vector machines (SVM).


Input image		Contour detection


Detected Hazmat sign: Flameable gas		Detected QR code: Y_01 chair-002% -yolks