SDV Autonomous QCar

This repository contains the source code for a full-stack embedded autonomous driving project built on the Quanser QCar2 platform.

---

Project Context

This project is a Self-Driving Vehicle (SDV) system built on:

• Platform: Quanser QCar2
• Compute: NVIDIA Jetson Orin
• OS: Ubuntu 20.04
• Middleware: ROS2 Humble
• Language: Python 3.8

This is a real hardware-based embedded autonomy system involving perception, sensor fusion, planning, control, and safety logic.

---

Core Objective

The goal is to build a controlled, explainable autonomous system that can:

• Follow lanes or cone-defined corridors
• Detect and react to obstacles safely
• Integrate traffic sign understanding
• Operate reliably on the Quanser map

This is structured autonomy with safety and reasoning.

---

System Architecture

The system follows a modular autonomy pipeline:

1. Perception

   • Lane detection (CSI camera)
   • Traffic sign detection (RT-DETR)
   • Depth-based obstacle reasoning (RealSense)
   • LiDAR-based obstacle detection

2. Sensor Fusion

   • fusion_confidence = vision_score × depth_score
   • Sync gating (reject if timestamp mismatch > 100ms)
   • Depth veto over vision

3. Planning

   • Lane center tracking
   • Corridor following (lane OR cones)
   • Obstacle avoidance (Bug algorithm style)

4. Control

   • Steering based on lane deviation: steer = (lane_center - image_center) / image_center
   • Speed based on confidence levels

5. Safety Logic (priority-based)

   • LiDAR hard stop (highest priority)
   • Depth obstacle veto
   • Sync failure → STOP
   • Lane loss → STOP

---

Sensor Setup

• CSI Front Camera → Primary lane perception
• RealSense Depth → Forward clearance
• LiDAR → Emergency stop + obstacle direction
• RealSense RGB → Debug/fallback

Important constraints:

• Resolution mismatch (CSI vs Depth)
• Timestamp mismatch
• Invalid depth pixels (~38%)

---

Key Implementations

1. Traffic Sign Detection

   • Model: RT-DETR
   • ~92.6% mAP
   • ~24 FPS
   • Fixed issues:
     • Left/right confusion (augmentation)
     • Class imbalance
     • Far-distance detection

2. Lane Detection Pipeline (8-step)

   • Grayscale → Blur → Canny → ROI → Hough → Lane grouping
   • Lane center estimation
   • Steering from deviation
   • Exponential smoothing

3. Sensor Fusion

   • Vision + Depth combined
   • Depth acts as a safety veto
   • Sync validation required

4. Obstacle Avoidance

   • LiDAR sector-based (Left / Center / Right)
   • Bug algorithm:
     • Choose clearer side
     • Avoid obstacle
     • Return to lane
   • Backing logic if too close

5. Stability Improvements

   • Frame-based state confirmation
   • Steering smoothing
   • LiDAR anti-flicker logic

---

Current Status

Working:

• All sensors operational (CSI, RealSense, LiDAR)
• Lane detection + steering
• Traffic sign detection model
• Sensor fusion prototype
• Obstacle avoidance logic
• Integrated multi-sensor loop

Partial:

• Depth tuning (false positives)
• Steering stability on curves
• Lane offset filtering

Not complete:

• Full stable autonomous driving
• Traffic sign integration into control loop
• Path planning around stopped vehicles
• Calibration between sensors

---

Next Development Goals

• Stable lane/corridor following
• Reduce false depth stops
• Improve steering smoothness
• Visualize ego path and lane center
• Integrate traffic sign decisions
• Handle stopped vehicles intelligently

---

Important Engineering Insight

This project demonstrates:

• Real-world sensor debugging
• Embedded AI deployment
• Multi-sensor fusion design
• Transition from reactive → autonomous control
• Full-stack autonomy pipeline