Description

Quadrupedal robots face challenges navigating complex or dynamic environments. This lab focuses on reproducing DiPPeST, a diffusion-based trajectory planner originally trained on synthetic mazes. Students will first implement and evaluate the planner in simulation, then adapt it to egocentric RGB camera input, enabling zero-shot, real-time path planning. The lab emphasizes handling variable camera views, lighting conditions, and dynamic obstacles, and deploying the system on a real quadruped robot.

Objectives

Adapt DiPPeST from simulation to egocentric RGB camera input.
Evaluate generalization under varying image resolutions, pixel intensities, and viewpoints.
Implement real-time navigation on a quadruped robot and compare with standard local planners.

Expected Outcomes

Hands-on experience with diffusion models and robotics.
Implementation of a zero-shot vision-based path planner.
Benchmark results for navigation success in static and dynamic environments.

Skills Required

Python and PyTorch programming.
Basic knowledge of robotics, computer vision, or motion planning.
Interest in learning robot deployment and real-time perception integration.

The applicant will use publicly available datasets and receive technical support from HUN-REN SZTAKI.

External Supervisor from SZTAKI:  András László Majdik, Ph.D., Senior Research Fellow; Email: majdik.andras@sztaki.hun-ren.hu