AeroSight - Hardware-in-the-Loop Infrastructure Inspection
An end-to-end drone inspection platform that detects structural defects on the edge with a quantized ONNX model, commands a Pixhawk flight controller to hover and capture, and streams telemetry to a web command center with a live 3D digital twin.
- Python
- ONNX YOLO
- MAVLink/Pixhawk
- OpenCV
- React Three Fiber
SYSTEM BUILD
What went into it
- 01
Pipelines RTSP, USB, and simulated video through an int8-quantized ONNX YOLO detector with NMS and tracking, gated by severity thresholds and cooldown debounce.
- 02
Bridges detections to MAVLink flight control — LOITER hover, still capture, auto-resume — with a heartbeat watchdog and a mock flight controller for CI testing.
- 03
Multiplexes binary JPEG frames and JSON telemetry over a single WebSocket endpoint into a Next.js ops console with HUD gauges and an R3F quadcopter twin mirroring live attitude.
- 04
Gave every hardware boundary — video source, model, flight controller — a first-class simulation mode so the full loop develops and CI-tests without hardware.
MEASURED OUTPUT
What changed
Proved the full severe-defect → hover → capture → resume chain in a real edge-to-browser integration run: 156 frames, 124 telemetry envelopes, and the complete event chain with zero drops.
Ships 14 green pytest cases including a full-loop trigger E2E and 8 Playwright specs against the ops console.