Synthetic Data Generation & Computer Vision
R&D · Synthetic Data PipelinePhysically-Correct Synthetic Training Data for Railway Anomaly Detection
Physically-correct compositing · 8 anomaly classes · automatic YOLO labeling
Python · Google Colab GPU · SDXL · BiRefNet · YOLOv8-seg
01Problem
Problem: Black-Swan Obstacle Data Can't Be Collected in the Field
Training a forward-camera model to detect obstacles on the track requires images of objects — rocks, tree branches, sacks, animals — lying on the rails. But these "black-swan" events are rare, dangerous, and expensive to capture in reality; you cannot deliberately place an obstacle in front of a train to collect data. The result: a scarcity of positive samples and severe class imbalance.
- 01Real on-track obstacle imagery barely exists — the model is trained without ever seeing the rare but critical event
- 02Manual labeling is done frame by frame; for thousands of frames the cost and error rate are high
- 03Naive copy-paste composites are physically wrong (incorrect scale, floating objects, lighting mismatch) — the model learns spurious cues
- 04Class distribution is naturally imbalanced; some anomaly types are not represented at all
- 05Single-frame composites don't generalize to video — identity/position jump across consecutive frames
02Pipeline Context
03Why Naive Approaches Fall Short
- 01Field collection is impossible: deliberately creating an obstacle is dangerous and illegal — the positive class can't grow naturally
- 02Scale inconsistency: without knowing the rail's position in perspective, a placed object appears at a physically wrong size
- 03Geometry violation: without ground-contact and occlusion, the object floats in mid-air — the domain gap widens
- 04Lighting/color mismatch: when the source asset doesn't match the scene tone, the model learns to separate the object by color (shortcut learning)
- 05Temporal inconsistency: if frames are generated independently, object identity and position jump across the video, breaking tracker training
04How the Pipeline Works
Selected diverse representative frames from the train video via CLIP embedding + clustering
→ A base frame pool with high scene diversity, without inflation from near-duplicate frames
Segmented the rail and ballast corridor with YOLOv8-seg (ray_alan.pt); vanishing-point + Hough fallback on failure
→ Where the object goes is geometrically known — locked to the rail corridor, not random
Generated anomaly assets with SDXL and matted with BiRefNet; automatic QC enforced single-object, edge-touch and pHash/CLIP dedup
→ A clean, transparent, duplicate-free asset library; multi-object and copy artifacts are filtered out
Derived placement scale from the standard rail gauge (1.435 m); applied class-conditional zones + pose priors
→ The object sits at a perspective-correct, physically-plausible size and a reasonable position for its class
Added ground-contact shadow, Reinhard color matching, distance-based blur and sensor noise in harmonization
→ The object grounds into the scene; color/light shortcuts are reduced, lowering the domain gap
Generated the label automatically from the post-transform visible alpha; filtered via on-rail QC and a visibility threshold
→ Zero-touch YOLO labels; samples that fall off-rail or become invisible are auto-rejected
At the video level, added a fixed track-id, KLT optical-flow anchor and EMA temporal smoothing
→ A single identity across the clip, locked to the rail and jitter-free — temporally consistent output
05Capability Comparison (Naive Baseline → Pipeline)
06Why It Matters
This pipeline makes critical obstacle scenarios — impossible to collect in the field — available as synthetic, automatically labeled data. For autonomous railway safety (ODS), it aims to eliminate the scarcity of positive samples and class imbalance in training obstacle-detection models.
- 01Unlimited, balanced data in the desired anomaly classes — without dangerous/costly field collection
- 02Manual labeling cost approaches zero — labels are derived automatically from the transform
- 03Physically-correct placement and harmonization deliberately shrink the sim-to-real domain gap
- 04Image and frame-consistent video output: usable for both detector and tracker training
- 05Modular config flags (depth, diffusion refine, z-buffer occlusion) for staged future strengthening
To evaluate your Synthetic Data Generation & Computer Vision needs on your own platform, schedule an embedded architecture audit or scope your platform class with the system requirements calculator.
Methodology & Scope Note
- This is an R&D / capability effort; it is not a delivered client win or a field-verified metric.
- The figures above are pipeline capability descriptors (class count, gauge scale, automation) — they contain no fabricated accuracy/fps metric.
- The international standard rail gauge of 1.435 m is used as the scale reference.
- Sim-to-real performance requires separate validation on a real anomaly test set and is in progress.
- Heavy inference (SDXL, segmentation, video) runs on GPU (Google Colab).
Need a synthetic, automatically-labeled dataset for your own domain?
Free technical assessment — let's review the latency and determinism challenges in your system together.
* R&D / capability case. Black-swan obstacle data can't be safely collected in the field, so it is generated synthetically. The values here are pipeline capability descriptors, not field-verified accuracy metrics. Sim-to-real validation is in progress.