Visible to the public Continuous Compliance

TitleContinuous Compliance
Publication TypeConference Paper
Year of Publication2020
AuthorsKellogg, M., Schäf, M., Tasiran, S., Ernst, M. D.
Conference Name2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE)
Keywordsauditing source code, cloud computing, cloud data stores, compliance, compliance certificate, compositionality, continuous compliance process, cryptographically unsafe algorithms, cryptography, data privacy, Data security, Encryption, encryption audits, FedRAMP, Hard-coded credentials, Industries, key length, large cloud-services company, lightweight verification tools, Manuals, Open Source Software, open-source software, PCI DSS, pluggable type systems, Predictive Metrics, Production, program diagnostics, program verification, pubcrawl, public domain software, Resiliency, SoC, software engineering, source-code compliance requirement, Standards, Tools
AbstractVendors who wish to provide software or services to large corporations and governments must often obtain numerous certificates of compliance. Each certificate asserts that the software satisfies a compliance regime, like SOC or the PCI DSS, to protect the privacy and security of sensitive data. The industry standard for obtaining a compliance certificate is an auditor manually auditing source code. This approach is expensive, error-prone, partial, and prone to regressions. We propose continuous compliance to guarantee that the codebase stays compliant on each code change using lightweight verification tools. Continuous compliance increases assurance and reduces costs. Continuous compliance is applicable to any source-code compliance requirement. To illustrate our approach, we built verification tools for five common audit controls related to data security: cryptographically unsafe algorithms must not be used, keys must be at least 256 bits long, credentials must not be hard-coded into program text, HTTPS must always be used instead of HTTP, and cloud data stores must not be world-readable. We evaluated our approach in three ways. (1) We applied our tools to over 5 million lines of open-source software. (2) We compared our tools to other publicly-available tools for detecting misuses of encryption on a previously-published benchmark, finding that only ours are suitable for continuous compliance. (3) We deployed a continuous compliance process at AWS, a large cloud-services company: we integrated verification tools into the compliance process (including auditors accepting their output as evidence) and ran them on over 68 million lines of code. Our tools and the data for the former two evaluations are publicly available.
Citation Keykellogg_continuous_2020