Flaky-test noise: capture sampled run snapshots and auto-compare reports

Flaky tests are a persistent source of wasted engineering time and noisy CI pipelines: industry surveys often report 10–30% of test failures are non-deterministic, creating repeated triage work for QA leads, SREs, and feature teams. The consequence is slowed release cadence and diverted senior engineers investigating transient failures instead of shipping features. A practical product would sample test runs, capture lightweight run snapshots (logs, selected traces, screenshots, environment metadata) and generate automated run-to-run comparisons with diffs, similarity grouping, and actionable triage reports surfaced in the CI and PR flow. With a $15.0B addressable market (3,000,000 engineering teams x $5,000 ACV) and market/revenue scores of 88/100 and 84/100 respectively, trends like CI/CD acceleration, shift-left testing, and AI-assisted observability make the ROI for triage automation clearer now than five years ago. This idea can differentiate by minimizing overhead (sampled rather than full capture), providing deterministic hashing and source-level diffs, offering explainable ML grouping and suspected root-cause hints, and shipping open SDKs and first-class CI/IDE integrations. Strengths include measurable reclaimed engineering hours and predictable ACV potential; key challenges are instrumenting diverse test frameworks with low performance impact, avoiding new sources of noise from imperfect ML, proving security/compliance for captured artifacts, and competing in a medium-competition field—so an early focus on a few high-value verticals and a tight pilot with 1–3 teams is the most pragmatic next step to validate product-market fit.