AM67A Vision Pipeline: How a Frame Travels from Sensor to Decision

Seeing an ISP, an optical-flow engine and a stereo-disparity engine on a SoC feature list is reassuring. But the presence of accelerators does not make end-to-end latency low. What determines latency is not which blocks exist — it is how many times a frame is written to and read from memory, which resource is shared across how many cameras, and where in the chain a copy appears. This article follows one frame through the TI AM67A from sensor to decision and shows, at each stop, where the time goes. Sources are TI's technical reference manual SPRUJB3D (August 2026) and the AM67x datasheet SPRSPA3B. Spikedge has not benchmarked this platform for this article; every figure quoted is a published peak value.

The route

sensor → CSI-2 RX (×4) → video stream MUX → VISS → [SL2] → LDC / MSC
       → DDR ↔ DMPAC (DOF, SDE) → C7x + MMA → Cortex-A53 → R5F

Four accelerators and at least two memory tiers. Each stop below covers what it does and what it charges you.

This article is about the vision subsystem, not the SoC as a whole; for how these blocks sit inside the device and which variants carry them, see our AM67A architecture deep dive.

1. Sensor to CSI-2

The AM67A has four independent CSI-2 receivers, each a 4-lane D-PHY at up to 2.5 Gb/s per lane, MIPI CSI-2 v1.3 and D-PHY 1.2 compliant, with up to 16 virtual channels per input.

Two design decisions follow:

• Lane budget. Four lanes at 2.5 Gb/s gives 10 Gb/s of raw bandwidth. A 12-bit RAW 1920×1080 stream at 60 fps runs at roughly 1.5 Gb/s, so trimming lane count to the sensor is not pedantry — it is power saved.
• Virtual channels or separate ports? A deserializer can aggregate several sensors onto one port, at the cost of sharing that port's bandwidth. Four separate ports partition bandwidth but cost pins and board area.

One errata item lands directly here: i2190 reports that the CSI receive interface can enter an unknown state after an incomplete or CRC-failed frame, requiring a module reset. On a multi-camera system under vibration this is a driver-level recovery path you have to design — it shows up when a cable connection degrades in the field, not on the bench.

2. The video stream MUX: four cameras, one VPAC

This is the most important section of the article.

In the TRM's VPAC block diagram, four CSI_RX_IF blocks feed a video stream MUX, and a single VISS comes out of it. The TRM says so plainly:

"The VPAC is designed to support multiple cameras by working in time-multiplexing mode."

So the AM67A has four camera inputs and one image processing pipeline, shared by those cameras in time. The device tables list one VPAC instance and one DMPAC instance.

That lands straight in the latency budget:

• In a four-camera system, ISP capacity is divided among the cameras, not multiplied. The 600 MP/s the datasheet quotes is the pipeline's total, not a per-camera figure.
• Time multiplexing means queueing. The fourth camera's frame waits behind the other three. That wait looks small on average and approaches a frame period in the worst case.
• Camera synchronisation is a separate job. Four frames captured simultaneously are not four frames processed simultaneously.

"Supports four cameras" is true. "Processes four cameras in parallel" is not. If a stereo pair and a surround camera share this SoC, build the latency budget on that time-sharing.

3. VISS: from raw pixels to an image

VISS is the part of the VPAC that processes raw sensor data. The sub-blocks the TRM lists:

The datasheet refers to this block by the reference name VPAC3L and gives 600 MP/s ISP throughput, 12-bit RGB-IR support, up to 16-bit RAW input and line support up to 4096. The TRM simply calls it "VPAC"; the sub-block list only exists in the TRM. A block you cannot find in the datasheet is not necessarily absent — NSF4V, GLBCE and CAC do not appear in the datasheet feature list but do appear in the TRM.

Two capabilities matter more than the rest here.

RGB-IR. Taking both a visible and an infrared channel from one sensor and separating them in the ISP removes the need for a second IR camera — and with it the board cost and the alignment problem.

Simultaneous HV + MV output. The TRM states the VPAC produces "simultaneous visual (HV) and analytics (MV) output to system memory", which it describes as "saving need for additional read from system memory for HV+MV processing". In other words, the image for human viewing and the image for machine processing are produced in one pass. In a multi-camera system the memory bandwidth this saves is substantial.

That single feature accounts for much of the gap between a naively built pipeline and a well-built one.

4. SL2 and HTS: the real story inside the VPAC

If you want to know where latency goes in an accelerator chain, this is the section to read.

The VPAC contains two pieces of infrastructure:

• SL2 (shared level 2) — memory local to the VPAC. VISS, LDC, MSC and the other sub-modules exchange data through it, and transfers to and from DDR/MSMC use the K3 DMA architecture.
• HTS (hardware thread scheduler) — runs the messaging between sub-modules and the local DMA engine (UTC) in hardware.

The consequence: transitions between blocks inside the VPAC do not have to go via DDR. Data leaving VISS can reach LDC through SL2, and the TRM separately lists a "VISS to LDC direct OTF (on-the-fly) for multi-camera" path — streaming, without a memory write at all.

By contrast, LDC and MSC are described as reading "data from memory (DDR or on-chip) to internal SL2 memory". Invoke those blocks independently and the input comes from memory.

That is where the design decision lives:

• Build the pipeline chained (VISS → LDC on the fly) and the DDR round trip, along with its bandwidth cost, disappears.
• Build it block by block — write to memory at each step, read it back at the next — and every transition adds a DDR write plus a read. On a four-camera system that becomes visible quickly on a 32-bit LPDDR4 bus.

The latency difference between two pipelines built from the same accelerators usually comes from here, not from block speed.

5. LDC and MSC

LDC (lens distortion correction) applies perspective transforms and lens distortion correction, fisheye included. If you use a wide-angle sensor and your model expects rectified geometry, doing that correction in hardware takes a serious load off the A53s.

MSC (multi-scaler) reads memory into SL2 and produces up to 10 scaled outputs from one or two inputs. That number matters: in a multi-scale inference pipeline — several models at different resolutions, or a pyramid-based approach — taking every scale in one pass is far cheaper than a separate resize per scale.

Practical consequence: give the downscale to the model's input resolution to the MSC. Doing it on the A53 or the GPU costs both core time and a memory round trip. Where pre-processing happens is the single most frequently overlooked component of measured latency.

6. DMPAC: depth and motion

DMPAC is the block that separates the AM67A from most of its family, and it holds two engines. The limits the TRM publishes actually define what the block can do:

Dense optical flow (DOF) uses TI's own algorithm: coarse-to-fine block search over image pyramids with proprietary binary pixel descriptors. Its limit: flow vectors reach up to ±191 pixels horizontally and ±62 pixels vertically. If something in your scene moves further than that between frames, DOF cannot track it. The remedies are a higher frame rate or a lower resolution — both of which land in the budget.

Stereo disparity engine (SDE) supports search ranges of 64, 128 or 192, operating over "0 … SR−1" or "−3 … SR−4". Sub-pixel accuracy is 1/16 pixel; the output passes through a 2D median filter and carries an 8-level confidence score for every disparity value.

That confidence score may be the block's most valuable output from a system design point of view: knowing which regions of the depth map not to trust stops stereo from silently returning wrong answers on low-texture surfaces.

DOF and SDE share their own SL2 memory, a DMA engine called UTC, and an HTS. So the same "pass without touching memory" architecture exists inside DMPAC too.

What the limits tell you: a 2 MPix input ceiling and a 1280×720 simultaneous-operation limit say that 4K stereo depth is not what this block delivers. DMPAC is an assistant, not an unbounded resource.

7. C7x + MMA: inference

By this point the frame has become a tensor. The AM67A carries two deep learning accelerators, each a C7x256V vector DSP paired with an MMA, together rated up to 4 TOPS peak.

Three things matter for latency:

Operator coverage. Layers the compiler cannot place on the accelerator fall back to the Cortex-A53. That is a domain change mid-pipeline, and a domain change means a memory round trip. A model "running" and a model "being accelerated" are different things.

Where pre-processing happens. Letterboxing, normalisation and colour conversion are part of the pipeline. The difference between doing them in the VPAC and doing them on the A53 exceeds the inference time itself in many measurements.

What two accelerators buy. TI's default inference mode runs a model on a single core regardless of how many exist; using both is an explicit compile-time option. The clearest return on two blocks is two independent streams in parallel, not one model at double speed.

The workflow that gets a model onto these accelerators — compilation, quantisation, artifacts, fallback — is covered in our TIDL deployment guide.

8. Cortex-A53: application and post-processing

Inference output is raw tensors. Turning it into boxes, classes and tracks takes decode, thresholding, NMS, tracker update and business logic.

This step is the pipeline's invisible cost. After an inference that finishes in 20 ms on the accelerator, a poorly chosen tracker can add hundreds of milliseconds by itself. Post-processing complexity is independent of model complexity and has to be measured separately.

9. Cortex-R5F: the real-time reaction

If there is an actuator at the end of the chain — a stop command, a servo, a relay — that reaction should not depend on Linux scheduling. The AM67A's R5F domain exists precisely for that, and the decision moves out of the A53 into a different timing contract there.

How to build that partition, the bridge between the domains and the usual mistakes: running Linux alongside Cortex-R5F on the AM67A.

Latency budget: accelerators ≠ low latency

The article's thesis in one line: a pipeline's latency is not the sum of its blocks' speeds.

What actually enters the total:

1. Sensor exposure and readout (no accelerator shortens this).
2. CSI-2 transfer.
3. VPAC queueing — if four cameras share it in time.
4. VISS processing.
5. DDR round trips at block transitions, if the pipeline is not chained.
6. LDC / MSC.
7. DMPAC, if used.
8. Pre-processing, wherever it runs.
9. Inference.
10. The A53 cost of layers that did not reach the accelerator.
11. Post-processing and tracking.
12. Decision and output to the actuator.

The most common reason a system is "slow" is not a heavy model at line 9. It is the sum of lines 5, 8 and 11 — none of which appear in a datasheet.

Pipeline bottleneck matrix

Symptom → likely cause → where to look. A starting map for finding out why a pipeline is slower than expected.

Vision pipeline design checklist

Questions to answer before writing code.

Camera and input

1. How many sensors, across how many CSI-2 ports, using how many lanes?
2. Do the sensors need to be synchronised? How does the time-multiplexed VPAC affect that?
3. What is the ISP budget per camera — how many ways are you dividing the 600 MP/s total?
4. Is there a recovery path designed for a CSI interface fault?

VPAC 5. Is the pipeline chained (VISS → LDC on the fly), or does every step pass through memory? 6. Is scaling done in the MSC, or on the A53/GPU? 7. How many different resolutions does the model need? Is the MSC's multi-output capability being used? 8. If both a display and an analytics stream are needed, is the simultaneous HV+MV output being used? 9. Is RGB-IR required? If so, has the separate IR camera been dropped from the plan?

DMPAC 10. Are depth or flow genuinely required, or on the list out of habit? 11. Is the resolution under the 2 MPix ceiling? If running SDE and DOF together, is the 1 MPix limit understood? 12. Does the maximum motion in the scene fit within DOF's ±191 / ±62 vector range? 13. Is the SDE confidence score used in the system decision, or discarded?

Inference 14. Which layers of the model land on the accelerator? Has the compilation report been read? 15. Are pre-processing and post-processing measured separately from inference time? 16. Are there genuinely two independent streams for the two accelerator blocks?

System 17. Has an end-to-end latency budget been written, or is only inference time being discussed? 18. Does the decision reaching the actuator depend on Linux scheduling? 19. Has thermal behaviour under sustained load been measured? 20. Has memory bandwidth been measured with AI and storage traffic running together?

What to measure

Every figure in this article is a published peak. Only measurement tells you what a pipeline actually does. The minimum set:

• Time from sensor exposure to VISS output, per camera.
• Effective per-camera ISP throughput with four cameras active.
• The difference between a chained pipeline and a block-by-block one, on the same task.
• Pre-processing, inference and post-processing distributions separately — p95/p99, not averages.
• Memory bandwidth pressure with DMPAC on and off.
• End to end: sensor exposure to decision.
• Where all of the above settle after the first thirty seconds under sustained load.

Every published number carries its conditions: which setup, which method, measured by whom.

Summary

The AM67A's vision subsystem is rich: an ISP with RGB-IR, a VPAC with its own shared memory and hardware scheduler, a scaler with up to ten outputs, a stereo engine that emits confidence scores, and TI's own optical flow algorithm.

But all of those blocks share one VPAC instance and one DMPAC; four cameras run time-multiplexed; and whether data touches memory between blocks is left to you. Those three decisions set the pipeline's speed — not the list of blocks.

To work out the partitioning and latency budget on your own pipeline, our Edge AI and vision work is the starting point; to scope it, we run an embedded architecture audit.

Sources

1. Texas Instruments — J722S/TDA4VEN/TDA4AEN/AM67 Processor Silicon Revision 1.0 Technical Reference Manual, SPRUJB3D, March 2024, revised August 2026 — https://www.ti.com/lit/zip/sprujb3 (TI distributes this document as a ZIP, not a PDF)
2. Texas Instruments — AM67x Processors datasheet, SPRSPA3B, March 2024, revised June 2026 — https://www.ti.com/lit/ds/symlink/am67a.pdf
3. Texas Instruments — J722S/TDA4VEN/TDA4AEN/AM67 Processor Silicon Revision 1.0 Errata, SPRZ575 Rev. A — https://www.ti.com/lit/pdf/sprz575
4. Texas Instruments — AM6xA ISP Tuning Guide, SPRAD86 Rev. A — https://www.ti.com/lit/pdf/sprad86
5. Texas Instruments — Processor SDK Linux (AM67A) documentation — https://software-dl.ti.com/jacinto7/esd/processor-sdk-linux-am67a/latest/exports/docs/index.html

Documents accessed 5 September 2026. The TRM and datasheet carry different revision dates; use the TRM for sub-block detail and the datasheet for electrical and variant information.