By Adam Mills
AI Editor
0
Share
Newsweek is a Trust Project member
See more of our trusted coverage when you search.Prefer Newsweek on Googleto see more of our trusted coverage when you search.
For years, the debate around autonomous systems has centered on scale: how many miles have been driven, how much data has been collected and how quickly models are improving. By that measure, progress has been substantial. Autonomous vehicles alone have logged millions of real-world miles.
Yet the hardest problems persist.
“Because miles aren’t evenly informative,” Bálint Pasztor, co-founder and CEO of DiffuseDrive, told Newsweek, referring to the challenge of training autonomous systems on rare scenarios. DiffuseDrive is a generative AI company that helps AI teams analyze real-world datasets and generate high-quality synthetic data for training and validation. “Most miles are ‘normal driving,’ while many of the hardest cases are rare, long-tail and combinatorial.”
Read More on Newsweek.AI
Those edge cases—unusual interactions, adverse weather layered with lighting conditions, unexpected objects or ambiguous construction zones—are statistically unlikely to appear in real-world data.
...
“The probability of seeing the exact dangerous configuration you care about can be extremely low,” Pasztor said, “yet those are exactly the cases that define safety and reliability.”
The issue, he argues, is not just frequency but coverage. “A system can operate for a long time and still under-sample entire regions of conditions, especially when you consider how quickly the scenario space explodes with additional variables,” he said, citing factors such as timing, intent, visibility and sensor artifacts.
When Real-World Data Stops Scaling
This limitation becomes even more pronounced beyond passenger vehicles. While autonomous cars benefit from large fleets and continuous data collection, many other AI-driven systems operate in environments where real-world scale is expensive, risky or impractical.
“Many other autonomy domains—defense, drones, industrial robotics, medical and inspection systems—don’t get that same real-world scale,” Pasztor said. In those settings, waiting for rare scenarios to occur naturally is not only slow but sometimes unethical or unsafe.
“That’s where ‘virtual scale’ matters most,” he said, particularly in what he describes as “'the 4D environments' autonomy often targets—Dull, Dirty, Dangerous and Dear.”
Relying solely on real-world data introduces additional constraints. Pasztor points to three structural limits: “Coverage and bias,” where common conditions are overrepresented; “Cost, time and risk,” which make capturing rare hazards difficult; and “Ground truth and consistency,” referring to the challenge of reliably observing and labeling what an AI system perceived or inferred at a given moment in real-world environments.
“Real-world data is essential,” he said, “but it doesn’t scale to ‘proof’ across the full scenario space on its own.”
Why Synthetic Data Looks Different Now
Synthetic data is not new to autonomy. What has changed is how it is generated and why it is used.
“Generative AI reduces friction,” Pasztor said. “You can create diverse, targeted variations of scenarios faster—without hand-authoring everything like classic simulation pipelines.”
At the same time, the bottleneck has shifted. “As autonomy stacks become more capable, progress is less about ‘more data’ and more about systematically covering the long-tail and validating performance across scenario catalogs,” he said.
In that context, synthetic data has moved “from ‘nice-to-have augmentation’ to practical necessity for coverage and confidence.”
Crucially, the goal is not visual realism for its own sake. “You don’t just generate ‘pretty scenes,’” Pasztor said. “You generate distributions and then stress them.”
That means intentionally modeling the messy parts of reality—occlusions, sensor dropouts, ambiguous geometry—and rejecting outputs that violate physics or operational constraints. “Synthetic is only as good as the validation harness wrapped around it,” he said.
From Training Aid to Validation Tool
Inside enterprise development pipelines, synthetic data now shows up at multiple stages, from early prototyping to deployment readiness.
In verification and validation, Pasztor said, teams use “scenario-based regression suites and virtual testing to complement physical testing—especially for ‘hard-to-reproduce’ situations.” Before deployment, synthetic data supports “targeted ‘go/no-go’ evidence” tied to defined operational boundaries.
Trust remains a hurdle. Common concerns include whether synthetic scenarios transfer to the real world, whether they introduce hidden bias and whether they can be explained and audited. “The fastest way to earn trust,” Pasztor said, “is to treat synthetic as measured engineering, not content generation.”
That distinction becomes critical as AI-generated data plays a larger role in safety-critical systems, where synthetic testing could otherwise be mistaken for a substitute for real-world responsibility. High-performing teams, he said, “treat synthetic as part of a safety case, not a replacement for accountability.”
Responsibility still depends on traceability and evidence. “Synthetic changes the tooling, not the obligation,” Pasztor said. “Companies still need a defensible chain from requirements to test methods to evidence.”
...
A Shift in What Comes Next
Regulators, too, are beginning to reflect that shift. Pasztor said standards bodies are “more open to virtual testing and assessment, as long as credibility is demonstrated,” particularly in high-stakes domains where real-world testing is constrained—such as autonomous systems used in aerospace, defense or industrial environments, where failures can be costly, dangerous or difficult to reproduce safely.
Looking ahead, the competitive edge will not come from generating more data. “In a world where raw data is no longer the bottleneck, the hardest problems shift to evaluation, governance, and iteration speed without sacrificing safety,” Pasztor said.
The companies that succeed, he argues, will be “AI-first in process, not just in models.” They will rebuild development workflows around continuous testing and defensible evidence, while legacy organizations remain slowed by tooling inertia and long validation cycles.
In that sense, autonomy’s challenge is no longer about seeing more of the world. It is about proving—systematically and credibly—that AI systems can handle the parts they have never seen at all.
View 0 Comments
Submit Correction
View Editorial & AI Guidelines