projects | Fan Yang

A growing collection of my personal projects.

3D Lidar Inertial Odometry

ESKF Lidar Inertial Odometry

This project implements 3D lidar inertial odometry based on error state kalman filter. Given 3D lidar scan and IMU sensor data, the goal is to create 3D point cloud map of the environment. The key elements include ESKF for state estimation, point cloud registration estimating transformation between two point clouds, and lidar-inertial odometry creating keyframes which form the 3D point cloud map. Two types of LIO is implemented, namely loosely coupled LIO which only takes point cloud alignment as the observation in ESKF and tightly coupled LIO which takes residuals of point cloud alignment as part of the ESKF observation model.

IMU Preintegration based Lidar Inertial Odometry

This project implements a tightly coupled 3D lidar inertial odometry based on IMU preintegration. Given 3D lidar scan and IMU sensor data, the goal is to create 3D point cloud map of the environment. The state estimation is achieved by pose graph optimization with both imu preintegration measurement and point cloud alignment involved as residuals. The optimization problem is built for estimating the state of two consecutive frames, though it can be further adapted to integrate more frames and observations. Moreover the marginalization is implemented and executed for smoothing the state estimation.

3D Mapping

3D Point Cloud Mapping

This project designs and implements a complete pipline of the 3D point cloud mapping system. The system is comprised of two parts: online frontend module and offline backend module. The frontend is responsible for collecting sensor data (3d lidar scan, imu data, gnss data), creating keyframes and pose graph structure, and saving point cloud data. The backend then perform pose graph optimization to reduce accumulated error and improve the consistency of the map. The main job of the backend is to detect and evaluate the loop closure, eliminate sensor data outliers and optimize the keyframe poses and partition the map.

Real-time Localization

This project implements a real-time lidar localization system based on error state kalman filter. Given the 3D point cloud map, the pose of vehicle is estimated by matching laser scan to the map. Then the states are fused with imu data using ESKF. The initial pose is determined by synchronization of lidar scan with GNSS data which provides physical coordinates, and a grid search for estimating orientation. A map management component is implemented to accelerate scan matching and save memory usage. Running in a seperate thread, the map managment loads active submap and releases inactive submap based on the state of vehicle.

2D Mapping

2D SLAM System for AMR

This project implements a 2D Lidar SLAM system for AMR (autonomous mobile robots). The system includes lidar odometry, map management and loop closure. Given 2D lidar scan, robot estimate the pose by scan matching with a map or scan. The lidar scan is then integrated to the map. To reduce the impact of the accumulated error, instead of one single map, a sequence of submaps are created. Each submap is formatted as an occupancy grid map. The real-time loop closure module is implemented to find the spatial relationship between current frame and previous submaps, and improve the global consistency by performing global optimization with loop closure edges.

GINS

GNSS/Odometer/IMU integrated navigation system based on error state Kalman Filter

This project implements a simplified GNSS/Odometer/IMU integrated navigation system with error state Kalman Filter. Three types of data: GNSS, Odometry and IMU are used for the sensor fusion. The error state Kalman Filter is used to estimate the state of the system which includes position, rotation, velocity, bias of accerometer, bias of gyroscope and gravity.

GNSS/Odometer/IMU integrated navigation system with IMU preintegration

This project implements a simplified GNSS/Odometer/IMU integrated navigation system using pose graph optimization. Similar to ESKF, GNSS, Odometry and IMU are the input sensor data used for the sensor fusion. The residuals/factors in the pose optimization includes preintegrated imu factors, GNSS, odom, bias of imu measurement and prior factors.

SLAM

Online SLAM with Extended Kalman Filter

The first SLAM solution in history is based on the Extended Kalman Filter. This project implements an EKF SLAM system. Given the sensor readings and odometry information, the robot computes a feature-based map while simultaneously localizing itself.

Online SLAM with Unscented Kalman Filter

The idea of Unscented Kalman Filter is similar to EKF by linearizing the transformation of a Gaussian. UKF performs a stochastic linearization through the use of a weighted statistical linear regression process. This project impelements an UKF SLAM system.

Full SLAM with FastSLAM Algorithm

This project implements FastSLAM for full SLAM problem. Unlike the online SLAM algorithms such as EKF that computes the posterior in the current time step, FastSLAM estimates the full path posterior and the locations of features given the range-bearing sensor reading and odometry information.

Graph-based SLAM

This project implements GraphSLAM algorithm which solves the full SLAM problem. Given a graph in which each node refers to a robot pose or landmark position and the edge represents a spatial constraint between two poses, the algorithm update the node configuration such that the error introduced by the constraints is minimized.

Localization

Particle Filter for Localization

Particle filter is a nonparametric filter which does not rely on a fixed functional form of the posterior, such as Gaussian. This project implement a complete particle filter using Python.

Extended Kalman Filter for Localization

Extended Kalman Filter is an extension of the Kalman Filter which deals with the nonlinearities in the motion and observation model.This project implement a complete extended Kalman Filter using Python.

Mapping & Planning

Occupancy Grid Map

Buiding Map is a fundamental problem in mobile robotics. Many applications like localization, path planning, navigation depend on the map of the environment. This project implements the occupancy grid mapping algorithm with the assumption that the robot poses are known.

Path planning algorithms

This project implements path planning algorithms that compute the path in graphs from a start to a goal using Python. Given a occupancy grid map, the algorithms construct a graph in which each free cell is consider a node and any two free cells are considered connected if they are adjacent either horizontally, vertically or diagonally. Then Dijstra and A* algorithms are implemented to find the shortest path between the given two points.

ROS

ROS Simulation

Simulate the SLAM and navigation in Gazebo using ROS navigation stack. In the SLAM simulation, user specify the pose for robot to visit in the unknown environment. During the exploration robot computes the occupancy grid map. Once the map is obtained, we can perform the Navigation simulation. Given the goal pose specified by user in rviz, the robot moves to the target and avoids the obstacle without teleoperation during the navigation. I also simulate the SLAM with multiple robots in Gazebo. Each robot moves around and computes its own map with seperate gmapping node. Then the maps are merged to a single map by multirobot-map-merge package. Further the project includes multi-robot navigation simulation. The robots have to avoid the collision with not only the static obstacle but not other moving robots.

Multi-robot System

Simulation of Multiple Robots in Warehouse

Control and simulate multiple robots working in the open warehouse environment where other uncontrolled agents are working around. The robots are expected to visit the assigned tasks and avoid not only the obstacles that are represented by the occupancy map (e.g. shelf, wall) but also the uncontrolled static and moving agents that are not represented by the occupancy map.

Computer Vision

Bag of Visual Word for Finding Similar Images

This project creates a C++ application that finds similar looking images using the idea of the bag of visual words. Given a database of images and a query image, the program will find N most similar database images to the query image. In robotics applications, finding similar image can provide additional information for determining the correspondence of the observation especially when the map features cannot be uniquely identified by range sensors like laser.

Object Detection using YOLO model

In this project, a real-time object detection application is created for the self-driving car using YOLO model. Given images taken from the car-mounted camera, the program outputs a list of bounding boxes indicating not only the position and size of objects but also the class of objects. In particular, a Deep CNN is used to convert the preprocessed image to an encoding, from which the bounding boxes with high probability is computed by non-max suppression.

Machine Learning

Mobile Robot Collision Avoidance using Reinforcement Learning

Use DDPG to train mobile robots to avoid obstacle. Given the range sensor reading the robots learning the policy to produce the velocity command to avoid colliding into obstacles. The project is implemented with multiple robots under the environment.

Mobile Robot Navigation using Reinforcement Learning

This is a follow-up project of collision-avoidance using DDPG. In this project, instead of moving randomly without collision, robot is expected to move to a given pose in the map without collision. In addition to the sensor readings, robot also receives its current pose to determine the linear and angular velocity.

Cpp

A* Route Planning Algorithm

C++ implementation of A* route planning algorithm. Given the inital and goal position on a map downloaded from the Open Street Map, the algorithm will compute an optimal path in term of distance. The map information of the region is contained in the OSM file which is a XML-formatted data in the form of "nodes" (points), "ways" (connections), and "relations" (street and object properties, such as tags).

System Monitor

The system monitor for linux operation system created using C++. It provides information such as cpu usage, memory usage, processes information. The program reads system information from various files in directories like /proc/ and /etc/, computes parameters and shows them in a graphic interface in terminal.

Memory Management Chatbot

The chatbot will talk with you about basic concepts in C++ memory management. You can ask related topic of memory management in C++, like unique pointer and get answer from chatbot. In this program, the smart pointers and move semantics are used to manage resources and optimize the code.