Real Time Has a Cost: When Not to Compute and How to Choose N

Running an object detector thirty times a second means recomputing, in most frames, what the system already knows: between two frames the object is still there, moving where physics says it should. So running the heavy model on one frame in N and carrying the frames in between with a cheap motion estimate is one of the most direct compute savings on an embedded system. It is not a speed trick but a budget decision: the control loop's deadline, the object's acceleration, the allowed prediction error and the risk of a new object entering the scene set N together. This article shows how to choose it.

In short

• Running the heavy model on every frame pays full price for what you already know. Detect on one frame in N and let a cheap estimate carry the rest.
• The gain is never a full factor of N, because fixed costs are paid on every frame. The error of a constant-velocity estimate grows with ½·a·t².
• Two constraints set N: prediction error for known objects and time to notice new ones. Uncertainty, a manoeuvre or a change signal should pull detection forward.
• Source: Çalışkan and Koruk, ICHORA 2025. The GPU saving reported there belongs to that hardware and task.

The film and the work behind it

Our 60-second concept film Real Time Has a Cost shows the idea: a camera head on a mast watches a piece of sky and an aircraft crosses the frame. Of four glass plates in a compute room, only the first lights up (DETECT); the other three read CARRY. The facility, aircraft, sensors and overlays are our own 3D animation; the boxes, tracks and trajectories are illustration, not detector output.

The idea comes from ICHORA 2025 work by H. H. Çalışkan and T. Koruk of Bursa Technical University; the second author, Talha Koruk, is Spikedge's founder. The paper combines a modified YOLOv8 (YOLO-Prime) for aerial vehicle detection on embedded systems such as Jetson Xavier NX and Rockchip RK3588, a morphological CNN that classifies the detected regions, and a dynamic linear regression model that predicts future coordinates. Detection and classification run on one frame in four; the other three are carried by prediction. The abstract reports that adding the regression model cut the GPU utilisation of the deep learning models by approximately 66%. That was measured on that hardware for that task; our film does not repeat the measurement, and we have not measured the method on our own hardware.

A cost model: why you never get a full factor of N

Split the per-frame cost into a fixed part C_fixed paid on every frame (decode, pre-processing, association), C_detect paid only on detection frames and C_predict paid on the frames in between. With detection on one frame in N:

average cost per frame = C_fixed + (C_detect + (N - 1) * C_predict) / N
speed-up = (C_fixed + C_detect) / average cost per frame
ceiling  = (C_fixed + C_detect) / (C_fixed + C_predict)     # as N grows large

When C_predict ≪ C_detect the fraction approaches C_detect/N, but C_fixed never shrinks: the fixed share, not N, caps the gain (Amdahl's law). With C_fixed at zero the speed-up is N / (1 + (N − 1)·r), with r = C_predict / C_detect, which is Equation 7 of the Deep Feature Flow paper. C_predict is usually small: the SORT paper reports its tracking component, detector excluded, running at 260 Hz on one core of a 2.5 GHz Intel i7.

Less compute also means less heat. NVIDIA's Jetson Xavier documentation describes clocks being lowered through DVFS once a temperature passes a throttling trip point, so a detection frame that fits the frame period at room temperature may not fit in a closed enclosure in the sun. And "GPU utilisation" is usually a time ratio (for example GR3D_FREQ in tegrastats), not watts; for watts, measure the power rails of the NVIDIA Jetson module or the enclosure input.

The deadline: how far does the object travel between detections?

Worked example: the values below are assumptions, not measurements. 30 fps; the object crosses the line of sight at 20 m/s; the acceleration the constant-velocity model does not see is 5 m/s²; the target is 0.5 m wide; detections arrive without delay; C_fixed = 4 ms, C_detect = 20 ms, C_predict = 0.5 ms.

The object moves 20/30 ≈ 0.67 m per frame and 2.67 m (about 2.7 m) over a detection cycle of N = 4 (4/30 ≈ 133 ms). A constant-velocity estimate carries that motion; its error comes from the unmodelled acceleration: e = ½·a·t².

The ceiling is 4.5 ms per frame, or 5.33×. Cost saturates while error grows with the square: going from N = 8 to N = 16 saves 1.2 ms per frame but quadruples the error, past the 0.5 m target width. There the predicted box and the new detection no longer overlap and IoU drops to zero; a centre-distance gate scaled by the covariance (Mahalanobis) holds up better. The table does not reproduce the paper's result.

What carries the frames in between?

The cheaper the carrier, the less it knows:

• Constant velocity. The cheapest; the error grows with ½·a·t².
• Sliding-window linear regression. Fit a line to the last k observations and extend it. A short window follows a manoeuvre quickly but is noisy; a long one is smooth but late. The paper calls its model "dynamic linear regression"; the abstract gives no further detail.
• Kalman filter (Kalman, 1960). The same motion model, recursive, with noise modelled explicitly. Its real benefit is the covariance: it grows at every prediction step and tells you when to pull detection forward. SORT pairs a constant-velocity Kalman filter with IoU-based Hungarian assignment.
• Constant turn rate and other manoeuvre models. Better for a turning target, at the price of a non-linear filter and extra states (Li and Jilkov's survey compares them).

When should detection run early?

A fixed N is the simplest scheduler, and the literature says it is not enough. Deep Feature Flow notes that a new key frame should ideally be allocated when the image content changes drastically; Detect or Track ties detection frequency to tracking quality with a small scheduler network that decides "detect or track" on every frame. Without a learned scheduler, four triggers do most of the work: the period has run out; the Kalman covariance has passed a limit; at the last detection a track went unmatched or the innovation was larger than expected, so the object is probably manoeuvring; a cheap motion trigger saw movement outside the known tracks.

# Adaptive detection scheduler (pseudocode; thresholds are system-specific)
def on_frame(frame, t, s):
    s.tracks.predict_to(t)                              # cheap, every frame
    due     = t - s.last_submit >= N_MAX / FPS
    unsure  = any(tr.pos_sigma() > SIGMA_MAX for tr in s.tracks)
    trouble = s.tracks.lost_or_maneuvering()
    novel   = s.watch.score(frame, mask=s.tracks.boxes()) > CHANGE_MAX
    if (due or unsure or trouble or novel) and s.detector.idle():
        s.detector.submit(frame, t)                     # asynchronous
        s.last_submit = t
    for det in s.detector.results():                    # may arrive late
        s.tracks.update_at(det.t_capture, det)          # apply at capture time, predict forward
        s.tracks.count_misses(det.t_capture)            # misses count on detection frames only
    return s.tracks.state_at(t)

What if a new object enters between two detections?

Prediction only carries known tracks. An object that enters between detections stays invisible for up to about N/fps plus the detection delay. Cheap remedies:

• A motion trigger. Frame differencing or background subtraction (such as Zivkovic's method, the basis of OpenCV's MOG2) runs on every frame and only triggers. When a pan-tilt head moves the whole image changes; without ego-motion compensation the trigger fires every frame and N falls to 1.
• A low-resolution full-frame scan. Cheap, but it erases small targets; our edge drone detection article (in Turkish) explains why.
• Rotating tiles. Known tracks are confirmed in small windows around their predicted positions, and one tile of the frame is scanned at full resolution each frame. Total compute stays similar, but the peak flattens and jitter drops.

In the sky, an approaching aircraft does not enter from the edge; it appears mid-frame as it becomes resolvable. A trigger that only watches the edges misses it.

How to choose N: two constraints and a cost curve

1. Write down the allowed position error ε for known objects and an upper bound a on unmodelled acceleration: t_max = √(2ε/a).
2. The worst-case prediction horizon is about N/fps + L, where L is the detection delay. N/fps + L ≤ t_max gives N₁.
3. Write down the time T_new within which a new object must be noticed. N/fps + L ≤ T_new gives N₂.
4. N = min(N₁, N₂). If the gain has saturated at that N, a larger N buys nothing; if the detection frame does not fit the frame period, run detection asynchronously.

Worked example: in addition, L = 50 ms, ε = 0.25 m (half the target width) and T_new = 220 ms. t_max = √(2 · 0.25 / 5) ≈ 0.316 s. N/30 + 0.05 ≤ 0.316 gives N₁ = 7; N/30 + 0.05 ≤ 0.22 gives N₂ = 5. The new-object constraint binds: N = 5, 8.4 ms per frame (2.86×), and a constant-velocity error of about 11.7 cm at the worst-case horizon of 217 ms, inside the 25 cm budget. Another speed, acceleration or alarm time gives another N.

Doing this sum with the real costs of your own pipeline is the first step of our edge AI pipeline work.

Where it goes wrong

1. Counting a skipped frame as a missed detection. In the SORT paper's experiments a track is deleted after one frame without a detection (T_Lost = 1), partly because the constant-velocity model is a poor predictor of the true dynamics. Feed such a tracker empty detection lists on skipped frames and every track dies between detections; the miss counter should only advance where the detector actually ran.
2. Watching the average and forgetting the worst frame. Worked example: with C_detect = 40 ms the detection frame takes 44 ms and overruns the 33.3 ms frame period once every four frames, although the average at N = 4 is 14.4 ms. In a synchronous design that is a periodic jitter spike.
3. Writing a late detection into "now". An asynchronous detection belongs to its capture time. Worked example: ignoring a 50 ms delay on an object at 20 m/s is a systematic lag of 1.0 m, two body widths on a 0.5 m target.
4. Treating carried points as measurements. Predicted points follow the motion model exactly. Fed to a classifier that works on track shape, a bird's track can look like a drone's; see our bird or drone article. The sampling rate of a periodic signature also drops to fps/N. Worked example: 7.5 Hz at 30 fps and N = 4, with a Nyquist limit of 3.75 Hz.

What this article cannot tell you

We have not measured this method on our own hardware; every millisecond, metre and centimetre above is a labelled worked example. The paper's result of approximately 66% was measured on that task, on that hardware and with a GPU utilisation metric; it does not transfer to another task, device or metric. The film's scene is an illustration. We also do not know your object's acceleration limits or how often new objects enter your scene, and those set N. To answer those questions, we would measure:

• Prediction error against N: run detection on every frame of recorded video, then replay it at different N; report p95 and worst case, not the mean.
• Time from observation to actuation: from capture to output (for example a GPIO edge), with hardware timestamps.
• Control loop jitter: with synchronous and asynchronous detection, under full load.
• Watts and junction temperature at the enclosure: after thermal settling, at real ambient temperature, detection on every frame next to the chosen N.

The measurement conditions (same input stream, same power mode, thermal settling) are in our inference pipeline method note.

Frequently asked questions

Why is running detection on every frame wasteful?

Between two consecutive frames the scene changes very little, so an object's position and velocity can largely be estimated from the previous frame. Running the heavy model on every frame pays full price for information you already have.

How do I choose N, the number of frames per detection?

Write two constraints: prediction error for known objects (½·a·t² ≤ ε, with t roughly N/fps plus the detection delay) and the time you have to notice a new object. The smaller N wins; then check whether fixed costs have already flattened the gain at that N.

What happens if a new object enters between two detections?

Prediction only carries known tracks, so a new object stays invisible until the next detection frame. A cheap motion trigger on every frame, or rotating tile scans, bound that delay; when the trigger fires, detection runs without waiting for N.

Does this reduce accuracy?

On carried frames the position is an estimate, not a measurement, and its error grows with the square of the prediction horizon and with unmodelled acceleration. Deep Feature Flow, for example, reports up to a 10× speed-up on ImageNet VID detection for a drop in mAP from 73.9 to 69.5; the loss in your system depends on the task, the hardware and N.

Sources

• Çalışkan, Koruk: Optimized YOLOv8 (YOLO-Prime), M-CNN, and Dynamic Linear Regression for Real-Time Aerial Vehicle Detection in Embedded Systems (ICHORA 2025): the work behind the film; the abstract's approximately 66% lower GPU utilisation.
• Zhu et al.: Deep Feature Flow for Video Recognition (CVPR 2017): key frames, Equation 7, ImageNet VID results.
• Luo et al.: Detect or Track: Towards Cost-Effective Video Object Detection/Tracking (AAAI 2019): detection frequency tied to tracking quality; scheduler network.
• Bewley et al.: Simple Online and Realtime Tracking (ICIP 2016): constant-velocity Kalman, IoU association, T_Lost = 1, 260 Hz.
• Kalman: A New Approach to Linear Filtering and Prediction Problems (1960): the original Kalman filter paper.
• Li, Jilkov: Survey of Maneuvering Target Tracking. Part I: Dynamic Models (IEEE TAES 2003): survey of manoeuvre models, turn models included.
• Zivkovic: Improved Adaptive Gaussian Mixture Model for Background Subtraction (ICPR 2004): the method behind OpenCV's MOG2.
• NVIDIA Jetson Linux Developer Guide: Tegrastats Utility: GR3D_FREQ (share of time the GPU is active in a period), power rails.
• NVIDIA Jetson Linux Developer Guide: Jetson Xavier NX Series and Jetson AGX Xavier Series: throttling trip points and clock throttling through DVFS.

Sources accessed on 26 September 2026.