DevSecOps is a set of software development practices on a mission that combines software development (Dev), security (Sec), and information technology operations (Ops) to secure the outcome and shorten the development lifecycle. The DevSecOps Mission is to develop a Continuous Monitoring (CM) approach for all Department of Defense (DoD) mission partners that monitors and provides compliance enforcement of containerized applications which cover all the DevSecOps pillars (Develop, Build, Test, Release & Deploy, and Runtime) for a secure posture with the focus being on automation and integration going forward. Here's how this breaks down.
What is DEVSECOPS?
DevSecOps is a set of repeatable software development practices that combine software development (Dev), security (Sec), and operations (Ops) to shorten the development life cycle and deliver a secure product. Security practices are implemented within DevOps at both the Dev and the Ops phases and is not a separate phase in and of itself. By implementing a DevSecOps process, developers can now deliver software features, patches, and fixes more quickly to users because the process can be automated and repeatable.
Development categories in devsecops
Development is the phase in which development of the application is performed. This phase is typically broken into five categories: Coding, Building, Testing, Securing, and Publishing. These categories may have other names or be merged to become one category, but the same general tasks will exist within this phase. During development, any category can cause a return to a previous category.'
- Coding is the process of application and infrastructure code development, code design and security review, configuration management of code through the use of source code management tools, and the use of static code analysis tools to locate coding flaws and security risks.
- Building includes the use of continuous integration tools, reporting build statuses, pushing container images to repos, image signing, and the creation of image checksums
- Testing involves using testing tools to perform continuous testing that provides quick and timely feedback on business risks
- Securing is the process of using security tools to perform continuous image scanning for vulnerabilities and security findings, and the creation of whitelist
- Publishing is the process of putting together the artifacts from previous steps and posting those to the artifact repository, performing application pre-deployment staging, manifest creation, and image signing
Coding and Testing
Within the Iron Bank flow, the coding and testing tasks are done by the contributor and the Iron Bank Container Hardener. The contributor is responsible for developing the container and running tests against the container to validate that the container passes a set of test criteria set forth by the organization. Once the coding and testing are performed by the contributor, the Iron Bank Container Hardener works with the contributor to secure the container before the container is introduced to the Iron Bank pipeline. To enable the Iron Bank pipeline to ingest the pull request (PR) from the contributor, standards are set for the form of the artifacts and what those artifacts are permitted to contain. A contributor will meet the requirements set forth by the Iron Bank pipeline for inclusion by following the security requirements to follow.
Building
During the build stage of the Iron Bank pipeline, it is necessary to control the inclusion of dependencies. Iron Bank can control the dependencies through the type of commands in the build files and the use of trusted sources. Though the dependencies come from trusted sources, they are run through Clam AntiVirus (AV) to check for viruses and malware. Once the dependencies are determined to be secure, they are moved to the artifact repository and there is no need to reach outside the Iron Bank infrastructure for the build of the container image. After the build completes, the new image is pushed to the artifact repository. UNCLASSIFIED Container Image and Deployment Guide, V2 R0.6 DISA 02 November 2020 Developed
The Iron Bank pipeline is tailored for building hardened containers without having any awareness or concern for the type (i.e., java, python) of build being performed, so many of the steps in the build process reflect its general purpose.
Securing
Scanning for security vulnerabilities of a newly built container image is kicked off after the successful build of the container image within the pipeline. The new container image is scanned by security tools such as Prisma, Anchore, and OpenScap to determine what vulnerabilities are within the container image. It is necessary to utilize more than one tool to scan since results can differ. These tools generate reports showing the vulnerability, the fix, and severity. From these reports, the Iron Bank Common Vulnerabilities and Exposures (CVE) Approver and the contributor can determine which vulnerabilities need to be fixed before release during the publishing phase. By following the security requirements in Section 2, the vulnerabilities found by the security tools will be minimized allowing for a container image to require a smaller whitelist of findings.
Publishing
Before releasing the container image to the public, any vulnerabilities found during the security scans must be addressed. Throughout this phase, the findings are reviewed by the contributor and the Iron Bank CVE Approver. Each vulnerability is either signed off by the Iron Bank CVE Approver, creating a whitelist of findings, or is resolved by the contributor. In cases where the contributor does not understand the vulnerability, the Iron Bank CVE Approver will work with the contributor to understand the vulnerability and resolve the vulnerability if necessary. If vulnerabilities within the container image are too severe to be released, the contributor must make the recommended fixes and create a new pull request to begin the process once again. The Iron Bank CVE Approver is the primary defense of the Department of Defense (DoD) networks.
Releasing
The first step in releasing the container image to the public is getting approval from the Authorizing Official (AO). The AO is responsible for ensuring no container image that has been brought into the pipeline is made available to the public without proper security scanning and review. The AO is familiar with approving the risk behind container images. Once approval is given, the pipeline will begin the process of moving the container image to the Iron Bank repository. The process will make available the container image and those artifacts created during the pipeline (e.g., scan reports, whitelists, and manifests), the README file and licenses from the contributor, the image will be signed, and a checksum generated.
Configuring
The Iron Bank pipeline addresses configuration of Kubernetes and the platform through usage of Helm Charts & Operators as well as Kubernetes manifests. Configuration is the process of creating an infrastructure for the container. Because Iron Bank is not instantiating the container images released through the pipeline, there is no infrastructure to create; however, the pipeline could create the code to instantiate the infrastructure required for the container. By moving the code for the infrastructure through the pipeline with the container image, the infrastructure is also validated for security and function. The infrastructure will follow the same configuration management practices used by the application code.
Monitoring
Iron Bank continuously monitors the Iron Bank repository using a Sidecar Container Security Stack (SCSS). Monitoring can perform continuous scans of the Iron Bank repository and in a production environment where container images are instantiated; the executing services and infrastructure can also be scanned. Performing continuous scans of the repository and running services allows new vulnerabilities or security risks to be quickly detected and addressed. Once addressed, new builds can be initiated with the goal of quickly releasing new secure container images.
Operations in devsecops
The operations phase consists of categories that take place in production to keep the application available and secure. Operations is usually broken into three categories: Releasing, Configuring, and Monitoring. If during any of these phases a security issue is found or a threshold is met during monitoring, a fix can be requested. Because configuration management in the coding category during development is the single source of truth, changes implemented trigger a new pipeline to execute pushing upgrades automatically into production.
- Releasing involves executing change management procedures, getting release approvals, and then implementing the release automation processes
- Configuring is the process of implementing any infrastructure configurations and the use of infrastructure as code tools
- Monitoring is the continuous process of monitoring the application security posture, the applications performance, end-user experience, and infrastructure drift
How Do You Make a DevSecOps Pipeline?
To implement a sustainable and repeatable DevSecOps process, organizations will implement a pipeline in which DevSecOps steps are applied through code in tools such as Jenkins, Bamboo, or GitLab etc. with continuous integration and continuous delivery or continuous deployment (CI/CD). To fulfill tasks the pipeline cannot perform itself, external tools are called and executed from within the pipeline process. The process can better be understood by breaking a production pipeline into its pieces.
An example of a CI/CD pipeline takes a developed container from a contributor, rebuilds the container while controlling the inclusion of dependencies, scans the container for vulnerabilities, verifies the container is secure, and then releases the hardened container for public consumption. This pipeline is called the Iron Bank pipeline and is implemented by the Air Force.
Iron Bank – DoD Centralized Artifacts Repository (DCAR)
- Iron Bank is the DoD repository of digitally signed, binary container images including both Free and Open-Source software (FOSS) and Commercial off-the-shelf (COTS)
- All artifacts are hardened according to the Container Hardening Guide. Containers accredited in Iron Bank have DoD-wide reciprocity across classifications
- Over 300 containers available today and growing
While this pipeline is used to verify and release hardened container images, the practices implemented would also apply to pipelines building software for release.
DevOps and Cybersecurity
While there are 7 layers of cybersecurity threats in the ISO-OSI model, there are 8 areas of specialization for cybersecurity within a robust mission. Secure DevOps is one of the components of cybersecurity within an organization.
The eight areas of specialization for cybersecurity are:
- Architecture and Policy
- Data Loss Prevention
- Governance, Risk and Compliance
- Identity and Access Management
- Incident Response and Forensic Analysis
- Penetration Testing
- Secure DevOps
- Secure Software Development
Secure DevOps is the hands-on team that actually manages systems in the data center (or cloud). They securely install, configure, and operate systems and software—especially dedicated security products such as firewalls, intrusion detection, and even dedicated HSMs (Hardware Security Modules) to hold sensitive keys and certificates. Often, the team is called DevSecOps to signify that “security is in the middle.” Even in a cloud environment, they still need to manage security processes and functions securely.
DevSecOps Training & Culture
Just as DevOps is a cultural shift, so too is DevSecOps, one that starts with championing the inclusion of cybersecurity team members, processes, and controls in the agile planning and project management process. Understanding that security can also be automated via CI/CD should address any concerns that introducing security upstream will impede agility. There is a military training course available for DevSecOps to educate soldiers and government contractors on gaining a solid understanding of the leadership requirements for successful DevOps adoption at both the tactical and strategic levels for those who help design, influence, implement and motivate the cultural transformation. Click here to view DevSecOps Foundation outline.
The configurations and practices within this guide are meant to show what is needed to have a secure container through the complete lifecycle. Some of the guidance is measurable and can be checked by tools before a container image is instantiated within a production environment or while the container is executing, and other guidance is non-measurable but should be followed to reduce accidental security flaws from being introduced into the container lifecycle. This document does not address the host operating systems, pipeline, and security tools, the container platform, or any other services offered by the container platform. These services need to be secured through the appropriate STIG, SRG, or best practice guidance to have a fully secure environment.
