US federal government documents concerning software supply chain security have been confusing and may even impose unrealistic compliance time frames, industry experts have warned.
Software supply chain security has quickly risen to high-profile stature among enterprise IT teams and vendors following major security incidents such as the SolarWinds breach and Log4j vulnerability, as well as a 2021 presidential executive order on cybersecurity.
In response, some government agencies have begun to draw up general guidelines for secure software development, such as this month’s Recommended Practices Guide on securing the software supply chain published by the National Security Agency, Cybersecurity Infrastructure Security Agency and the Office of the Director of National Intelligence.
It has taken some time for IT experts to digest the 64-page document, but one IT pro posted a pointed DevOps-based critique of the guidance this week. He pointed out ways in which the document flies in the face of Agile and DevOps principles such as small incremental changes, high delivery velocity and cross-functional roles within development teams.
Bryan FinsterDistinguished engineer, DevOps Unicorns
“It’s a maddening collection of good ideas mixed with really bad ideas,” wrote Bryan Finster, distinguished engineer and value stream architect at defense contractor Defense Unicorns, in a LinkedIn post Sept 20. “It’s absolutely clear that the people who wrote it do not continuous delivery (they think it’s automation), nor do they understand how defects get passed downstream.”
Other cybersecurity experts concurred with this criticism, and pointed out the Recommended Practices Guide also contradicts the government’s own published technical definitions in at least one area — it refers to “open source and commercial software products” as if they are separate categories, when the Department of Defense designates open source to be commercial software, said David A. Wheeler, director of open source supply chain security at The Linux Foundation.
“I have to admit I’m a little conflicted,” said Wheeler, who emphasized that his statements were his personal opinion and made on behalf of the foundation officially.
The document contains some statements Wheeler said he was pleased to see, such as guidance that software should meet cryptographic standards. The document acknowledges such standards may not necessarily be the same as those laid out by the National Institute of Standards and Technology for federal government use.
Other technical specifics left him scratching his head, Wheeler said, such as the recommendation that developer systems be restricted to development operations only, without other activity being conducted on them such as email or internet access.
“In high assurance situations, you might want to have a specialized computer on a virtual machine, where that virtual machine doesn’t have internet access,” Wheeler said. “But the developers, certainly not.”
Federal SBOM statements deemed contradictory, confusing
Last year’s presidential executive order kicked the development of software bills of material (SBOM) into high gear. These machine-readable lists of the software components and dependencies that make up an application were mentioned in the executive order to ensure transparency from federal software suppliers and shore up software supply chains. The order was followed last week by a memo from the White House Office of Management and Budget reiterating that SBOM may be required by federal agencies from software suppliers based on the criticality of the application.
These mandates were as expected, but multiple industry organizations raised the alarm about legislation currently working its way through Congress under the 2023 National Defense Authorization Act (NDAA). Earlier this month, a group of organizations comprised of the Software Alliance for Digital Innovation, Cybersecurity Coalition and Information Technology Industry Association published an open letter to Congressional leaders raising objections to SBOM requirements in the bill.
As drafted, the legislation “provides conflicting requirements with respect to certifications and notifications,” the open letter from the groups states. “In one instance, the provision requires certification that the items in the BOM are free of vulnerabilities or defects, and in another it requests a plan to mitigate all identified vulnerabilities.”
Any such vagueness and contradiction could provide a path to working around the intent of requirements, which is to shore up software security, said Daniel Kennedy, an analyst at S&P Global.
“The idea that continually updated software can be free of vulnerabilities is not a realistic goal,” Kennedy said. “There are gaps between identification and fix, and patch availability and application, and while those gaps can be a result of inattention, they can also be a function of a proper approach to testing the effects of a software patch before deployment in an environment. “
Wheeler said he expects many of the semantic kinks within the NDAA bill to be worked out or addressed by the Department of Homeland Security (DHS) follow-on guidance regarding relevant contracts for which the bill also calls.
But as written, the NDAA would also impose a 180-day deadline to comply with SBOM requirements following this DHS publication, which Wheeler said is far too short a time frame given the state of SBOM technology.
“Generating SBOM is not something the software industry has done yet, in general, other than in special cases,” he said. “This is a huge change in the software industry, and I do think that we’re going to get there, [but not] in any reasonable way in the time frames that they’re envisioning here.”
Some problems in SBOM, such as support for rapidly changing cloud-native applications, remain largely unsolved, although projects are afoot to address these issues. But even the most mature SBOM tech struggles with the fundamental problems of transitive dependencies, where nested layers of software components make transparency difficult, as well as false positives and negatives in detecting vulnerabilities.
It may be that 180-day requirement is eliminated before the bill passes the Senate, or that the DHS interpretation accepts “best effort” SBOM at the 180-day deadline, Wheeler said. In the worst-case scenario, Wheeler said he worries that too short a time frame for compliance could hurt the public perception of SBOM effectiveness.
“I fear that it will actually hurt, because then people will say, ‘Oh, SBOM are crappy and not reliable,'” he said.
Software supply chain security confusion adds to risks, for now
SBOM adoption will likely charge ahead, but in the meantime, confusion about how to implement software supply chain security represents its own kind of risk.
“Nobody knows exactly what to do,” said Dan Lorenc, co-creator of the Sigstore project and co-founder and CEO of software supply chain security vendor Chainguard. “There’s a whole bunch of things that claim to solve parts of the problem and don’t really do that.”
Chainguard this week launched a free training resource for developers on all aspects of software supply chain security called Chainguard Academy in a bid to cut down on this confusion. But, it will likely take about a year before third-party auditors are trained on concepts such as SBOM and formal certifications are available in this field, Lorenc estimated.
“We want to build up to current certifications over time, but the compliance frameworks are still shifting around,” he said.
This week also saw the first generally available release of Chainguard’s Enforce supply chain security policy orchestration product, which can be used to ensure that only trusted container images are run in Kubernetes clusters.
This type of “day two” operation is a broad area of SBOM tech that remains nascent, according to Katie Norton, an analyst at IDC.
“The biggest issue with SBOM is not generating them, but … how are SBOM going to be stored and shared?” she said. “To be actually useful in the case of a vulnerability like Log4Shell, a company is going to need to be able to aggregate all their SBOMs and they need to be able to be queried.”
Beth Pariseau, senior news writer at TechTarget, is an award-winning veteran of IT journalism. She can be reached at [email protected] or on Twitter @PariseauTT.