Object Detection Developed an advanced object detection system, Locus Learning, utilizing transfer learning with to detect LocusBots, persons, carts and other objects in real-time within indoor warehouse environments.

• Optimized model inference by porting it from Python to C++, achieving a 15% reduction in inference time and a 35% decrease in CPU load.
• Single-handedly integrated the object detector into the existing Locus framework, converted PyTorch weights into ONNX format for faster Intel iGPU inference, and introduced a lightweight inference visualizer for enhanced detection performance.
• Integrating a state-of-the-art Kalman-based Multi-Object Tracker, ByteTracker, with the Object detector. This advanced tracking system is employed to track and avoid forklifts in warehouses.

Cart Filter for Vectors Designed and implemented a per-sensor filtering system to exclude carts from sensor data once a robot attaches itself to a cart, significantly improving navigation performance and reducing mission time by 14% (Video).This system prevents the robot from misinterpreting the attached cart as an external obstacle, allowing for smoother trajectory planning and avoiding unnecessary detours.

• LiDAR Filtering: Each LiDAR sensor operates its own dedicated filtering module to dynamically remove cart leg reflections from the scan data. The filtering range is fully customizable, allowing users to adjust parameters on-the-fly based on specific cart dimensions or deployment environments.
• Depth Camera Filtering: For every depth sensor, a real-time mask is computed based on the camera’s field of view and the known geometry of the cart. These masks filter out cart-related depth points before they reach the obstacle costmap. The entire processing pipeline is offloaded to the onboard iGPU, ensuring minimal latency and maintaining real-time performance under compute constraints.

Forklift Detection via Apriltags Used a new Apriltag3 family with Forklifts to avoid robot-forklift collisions. Detected fiducial markers are used to define dynamic ‘danger zones’ in the robot’s costmap, treated as obstacles during navigation. This approach significantly reduced collision incidents, lowering repair costs and boosting operational efficiency.

Fiducial Marker Detection Upgraded the fiducial marker detection system to AprilTag3, resulting in a 22% increase in frame processing speed and a 28% improvement in recall. Replaced image undistortion with Region of Interest (RoI) rectification for tag detectors, reducing NUC load by approximately 5%. Additionally, integrated Locus’s fiducial markers with the state-of-the-art deep-learning tag detector, DeepTag, to enhance overall detection accuracy and efficiency.

Camera Calibration Replaced individual camera calibrations with a standard calibration matrix for all cameras mounted on the robot, ensuring calibration errors remained within 1% of use-case-specific tolerance limits. This approach streamlined the calibration process, reducing robot deployment time by 6% and eliminating the need for per-camera calibrations for each robot.

We propose to augment smart wheelchair perception with the capability to identify potential docking locations in indoor scenes. ApproachFinder-CV is a computer vision pipeline that detects safe docking poses and estimates their desirability weight based on hand-selected geometric relationships and visibility. Although robust, this pipeline is computationally intensive. We leverage this vision pipeline to generate ground truth labels used to train an end-to-end differentiable neural network that is 15 times faster.

ApproachFinder-NN is a point-based method that draws motivation from Hough voting and uses deep point cloud features to vote for potential docking locations. Both approaches rely on just geometric information, making them invariant to image distortions. A large-scale indoor object detection dataset, SUN RGB-D, is used to design, train, and evaluate the two pipelines.

Potential docking locations are encoded as a 3D temporal desirability cost map that can be integrated into any real-time path planner. As a proof of concept, we use a model predictive controller that consumes this 3D costmap with efficiently designed task-driven cost functions to share human intent. This wheelchair navigation controller outputs a nominal path that is safe, goal-oriented, and jerk-free for wheelchair navigation.

Designed and graded homework assignments, quizzes, and examinations for the following courses:

• CPSC 404 - Advanced Relational Databases x 4 (Winter 2019, Winter 2020, Winter 2021)
• CPSC 304 - Introduction to Relation Databases x 1 (Summer 2020)

I participated in several research projects focused on warehouse automation using industrial manipulators. My work included 3D pose estimation of heterogeneous-sized boxes using point clouds and motion planning for Universal Robots with ROS.
Here are some selected projects I worked on:

• Long Distance Container (LDC) Packing Video
• Chitrakar: Robotic System for Drawing Jordan Curve of Facial Portrait Video
• Amazon Robotic Challenge Video

For a detailed description of these projects, please refer to my Curriculum Vitae.