Edge AI & Computer Vision
Measured Under Field LoadRaising Edge AI Inference from 31 fps to 127 fps
+310% inference speed · −41% power · NVIDIA Jetson Orin NX 16GB
NVIDIA Jetson Orin NX 16GB · TensorRT 8.6 · JetPack 6
01Problem
Problem: 31 fps on a SWaP-C Constrained Platform
On an autonomous unmanned aerial system, standard FP32 ONNX runtime inference produced only 31 fps. On a size, weight, power, and cost (SWaP-C) constrained platform this was insufficient for real-time object detection and tracking, and the power budget directly limited battery life.
- 0131 fps caused frame skipping and track loss on fast-moving objects
- 02FP32 inference saturated the GPU, leaving no headroom for concurrent image pre-processing
- 03~14 W power draw kept the platform below its target flight time
- 04The model was recompiled from ONNX on every startup — initialization delayed operations
- 05CPU-GPU copies and kernel launch overhead in the pipeline had never been measured
02System Context
03Root Cause Analysis
- 01The model ran in FP32 — the Orin NX's INT8 Tensor Cores and DLA capacity sat completely idle
- 02ONNX Runtime used generic execution; TensorRT's layer fusion and kernel auto-tuning were not in play
- 03A separate kernel launch chain per frame generated CPU overhead — GPU stalls were visible in profiling
- 04The engine was rebuilt on every startup; serialization was not used
- 05The entire load sat on the GPU — the DLA cores went unused, leaving power efficiency on the table
04What We Changed
Applied INT8 quantization to YOLOv8-m, calibrated with a representative dataset
→ Inference throughput rose ~4× over FP32; detection accuracy validated within mission requirements
Enabled TensorRT engine serialization
→ Recompilation at startup eliminated — model load time optimized
Collapsed the per-frame kernel launch chain into a single CUDA graph
→ CPU launch overhead minimized; GPU feed continuity maintained
DLA co-execution: eligible layers dispatched to DLA, the rest to GPU
→ Power draw fell to 8.4 W (−41%) — battery life target met
Moved pre-processing to the GPU (CUDA resize/normalize), reduced CPU-GPU copies
→ Pipeline bottleneck removed — 127 fps sustained end-to-end
05Benchmark Results
06Why It Mattered
fps alone is meaningless; on a SWaP-C constrained platform the meaningful metric is fps/W. 15.1 fps/W meant both longer flight time and more concurrent perception tasks on the same battery.
- 01Real-time multi-object detection and tracking became possible on a single device
- 02−41% power draw directly extended battery life and mission duration
- 03Freed GPU capacity became available for additional perception workloads (sensor fusion)
- 04The INT8 calibration and validation flow was delivered as a reusable pipeline — model updates follow the same path
To evaluate your Edge AI & Computer Vision needs on your own platform, schedule an embedded architecture audit or scope your platform class with the system requirements calculator.
Methodology Note
- fps figures are reported over the end-to-end pipeline (capture → detection → output), not an isolated model benchmark
- Power was measured externally, under sustained load, after thermal equilibrium
- INT8 accuracy loss was measured against the FP32 reference using a mission-representative validation set
- Comparisons used identical hardware, identical input streams, and identical power modes
- Project details remain confidential under NDA — client identity is anonymized
Is your Edge AI pipeline missing its fps or power budget?
Free technical assessment — let's review the latency and determinism challenges in your system together.
All performance data was obtained in our own lab under repeatable conditions. Project-specific details are confidential under NDA. The methodology document is available on request.