Modelling the fusion and evaluation of networked multimodal sensors

Dr. Szirányi Tamás


Progress towards dense sensor networks in the cities and on the roads is a main challenge now. Investigating their behavior and developing new algorithms for high level fusion of the distributed information is an existing demand. With the dense sensor networks around us, the future is about the definition of virtual cooperation and the formation of sensor clusters inside the infinite network driven by specific scenarios/tasks.

In the fields of sensor networks and data fusion, one objective beyond the state of the art that we address is the development of the fundamental theoretical background and modelling framework for understanding the measured environment in a network of freely configured sensors.

•     Selecting virtual cooperative groups in the agent network focusing on real or virtual regions of interests; finding cores of attention and synchronized 3D camera groups in arbitrary sensor nets and exploring for synchronized views;

•     Finding the focus of attention evolving as a coherent action of different sensors and cross-modal interactions;

•     Geometrical and information inference constraints for building the topology of the sensor network: developing new graph structures for describing geometrical (and/or geographical) directional connectedness of sensors, guided by region of interest areas.


•     Registration of moving mobile and fixed (surveillance) sensor networks;

•     Object detection and tracking shared among the networked mobile and fix sensors;

•     Connection graph modelling in the sensor network;

•     Connection dynamics for multicamera network systems;

•     Analysis on stability and connectedness;

•     Simultaneous localization and Mapping (SLAM) models;

•   3D geometry modelling: 3D model of the scene, sensor fusion, image based calibration

Planned scientific results: Publications in the leading image processing and autonomous driving journals and conferences.

Key-words: autonomous driving, sensor network, 3D recognition, simultaneous localization and mapping (SLAM), graph theory.

Expected background:

·         The applicant should have good knowledge in image processing a machine learning;

·         Good math, programming in MATLAB and C/C++.