Three SBOM tools disagree on the same 3,000 projects

Three widely used SBOM generators produce different results on the same codebase, according to a study on arXiv that ran the tools against more than 3,000 JavaScript and Rust projects and checked their output against dependency lockfiles.

The disagreement shows up in two places: which dependencies a tool lists at all, and how complete each listed entry is. The paper traces this to differing assumptions about dependency scope, naming conventions, provenance tracking, and how a dependency graph gets represented in the SBOM format itself. None of it is a bug in the ordinary sense. Each tool is doing what it was built to do, and the three builds don't agree.

That matters because the EU Cyber Resilience Act makes SBOMs mandatory in December 2027. A software bill of materials is supposed to be a complete, checkable inventory of what's inside a product, the thing a regulator or a customer points to and asks "is this accurate." If three respected generators produce three different inventories from the identical source tree, "accurate" stops being a property of the practice and becomes a property of which vendor's tool you happened to run.

The paper doesn't name which of the three tools comes closest to the lockfile ground truth on either axis, coverage or completeness, so there's no ranking to hand a team picking a generator right now. What it does establish is that the choice isn't neutral. A team that locks in a generator 14 months out is also locking in that generator's specific blind spots on dependency naming and provenance, and won't find out which ones until an SBOM gets audited against the regulation.