By Tim Palko Senior Member of the Technical Staff CERT Cyber Security Solutions Directorate
In the realm of DevOps, automation often takes the spotlight, but nothing is more ubiquitous than the monitoring. There is value to increased awareness during each stage of the delivery pipeline. However, perhaps more than any other aspect of DevOps, the act of monitoring raises the question, "Yes, but what do we monitor?" There are numerous aspects of a project you may want to keep an eye on and dozens of tools from which to choose. This blog post explores what DevOps monitoring means and how it can be applied effectively.
By Aaron Volkmann Senior Research Engineer CERT Division
The DevOps philosophy prescribes an increase in communication and collaboration between software development and operations teams to realize better outcomes in software development and delivery endeavors. In addition to bringing development and operations closer together, information security teams should be similarly integrated into DevOps-practicing teams. An automated way of performing complete software security assessments during continuous integration (CI) and continuous delivery (CD) does not exist yet, but this blog post describes how we can apply DevOps principles to application security assessments and integrate the results of those activities with CI/CD.
By Aaron Volkmann Senior Research Engineer CERT Division
You will be hard pressed to find a DevOps software development shop that doesn't employ Vagrant to provision their local software development environments during their development phase. In this blog post, I introduce a tool called Otto, by Hashicorp, the makers of Vagrant.
DevOps principles focus on helping teams and organizations deliver business value as quickly and consistently as possible. While the principles advocate for improving the coordination between development and operational teams, they can be adapted for any number of domains. The key components of DevOps we want to emulate across other domains are:
collaboration between project team roles
infrastructure as code
automation of tasks, processes, and workflows
monitoring of applications and infrastructure
In this blog post, I explore how to apply DevOps to the incident response domain. In the same way that advances in methodologies surrounding software development were gleaned from Toyota's manufacturing processes, we can apply lessons learned from DevOps across domains.
In response to several corporate scandals, such as Enron, Worldcom, and Tyco, in the early 2000s congress enacted the Sarbanes-Oxley (SOX) act. The SOX act requires publicly traded companies to maintain a series of internal controls to assure their financial information is being reported properly to investors. In an IT organization, one of the main tenets of SOX compliance is making sure no single employee can unilaterally deploy a software code change into production. DevOps automation techniques and technologies, such as continuous integration (CI), continuous delivery (CD), and infrastructure as code (IaC), can appear on the surface to throw a shop out of SOX compliance. This blog post examines how DevOps automation can help organizations not only stay compliant, but actually increase their compliance.
The challenges of DevOps--a cultural change, learning new technologies, and making a big-picture impact for a software project team--are possibly even more challenging in contract work. In this blog post, I'll expand on some of my past experiences as a contract software developer and discuss, in retrospect, how DevOps could have worked in different scenarios.
Formal documentation (such as source code documentation, system requirements and design documentation, or documentation for various user types) is often completely ignored by development teams; applying DevOps processes and philosophies to documentation can help alleviate this problem. Software documentation tends to fall into several categories: code, requirement, design, system, and user documentation. One reason documentation is often ignored is that standard documentation tools and processes create an obstacle for development teams since the tools and processes do not fit in well with the suite of tools development teams rely on, such as version control, issue trackers, wikis, and source code. As a consequence of this mismatch, slow the velocity of development teams. This blog post explores three primary challenges to documentation--process, documenting source code, and system documentation--and explains how DevOps-based documentation allows all stakeholders to access a common, trusted source of information for project details.
Since beginning our DevOps blog in November, and participating in webinars and conferences, we have received many questions that span the various facets of DevOps, including change management, security, and methodologies. This post will address some of the most frequently asked questions.
In late 2014, the SEI blog introduced a biweekly series of blog posts offering guidelines, practical advice, and tutorials for organizations seeking to adopt DevOps. These posts are aimed at the ever-increasing number of organizations adopting DevOps (up 26 percent since 2011). According to recent research, those organizations ship code 30 times faster. Despite the obvious benefits of DevOps, many organizations hesitate to embrace DevOps, which requires a shifting mindset and cultural and technical requirements that prove challenging in siloed organizations. Given these barriers, posts by CERT researchers have focused on case studies of successful DevOps implementations at Amazon and Netflix, as well as tutorials on popular DevOps technologies such as Fabric, Ansible, and Docker. This post presents the 10 most popular DevOps posts (based on number of visits) over the last six months.
Container-based virtualization platforms provide a means to run multiple applications in separate instances. Container technologies can provide significant benefits to DevOps, including increased scalability, resource efficiency, and resiliency. Unless containers are decoupled from the host system, however, there will be the potential for security problems. Until that decoupling happens, this blog posting describes why administrators should keep a close eye on the privilege levels given to applications running within the containers and to users accessing the host system.
At a recent workshop we hosted, a participant asked why the release frequency was so high in a DevOps environment. When working with significant legacy applications, release may be a once-in-a-year type event, and the prospect of releasing more frequently sends the engineering teams running for the hills. More frequent releases are made possible by properly implementing risk mitigation processes, including automated testing and deployment. With these processes in place, all stakeholders can be confident that frequent releases will be successful.
This post is the latest installment in a series aimed at helping organizations adopt DevOps. Some say that DevOps is a method; others say it is a movement, a philosophy, or even a strategy. There are many ways to define DevOps, but everybody agrees on its basic goal: to bring together development and operations to reduce risk, liability, and time-to-market, while increasing operational awareness. Long before DevOps was a word, though, its growth could be tracked in the automation tooling, culture shifts, and iterative development models (such as Agile) that have been emerging since the early 1970s.
The federal government continues to search for better ways to leverage the latest technology trends and increase efficiency of developing and acquiring new products or obtaining services under constrained budgets. DevOps is gaining more traction in many federal organizations, such as U.S. Citizenship and Immigration Services (USCIS), the Environmental Protection Agency (EPA), and the General Services Administration (GSA). These and other government agencies face challenges, however, when implementing DevOps with Agile methods and employing DevOps practices in every phase of the project lifecycle, including acquisition, development, testing, and deployment. A common mistake when implementing DevOps is trying to buy a finished product or an automated toolset, rather than considering its methods and the critical elements required for successful adoption within the organization. As described in previous posts on this blog, DevOps is an extension of Agile methods that requires all the knowledge and skills necessary to take a project from inception through sustainment and also contain project stakeholders within a dedicated team.
DevOps can be succinctly defined as a mindset of molding your process and organizational structures to promote
software quality attributes most important to your organization
As I have discussed in previous posts on DevOps at Amazon and software quality in DevOps, while DevOps is often approached through practices such as Agile development, automation, and continuous delivery, the spirit of DevOps can be applied in many ways. In this blog post, I am going to look at another seminal case study of DevOps thinking applied in a somewhat out-of-the-box way: Netflix.
This post is the latest installment in a series aimed at helping organizations adopt DevOps.
Tools used in DevOps environments such as continuous integration and continuous deployment speed up the process of pushing code to production. Often this means continuous deployment cycles that could result in multiple deployments per day. Traditional security testing, which often requires manually running multiple tests in different tools, does not keep pace with this rapid schedule. This blog post introduces a tool called Gauntlt, which attempts to remedy this issue.
When Agile software development models were first envisioned, a core tenet was to iterate more quickly on software changes and determine the correct path via exploration--essentially, striving to "fail fast" and iterate to correctness as a fundamental project goal. The reason for this process was a belief that developers lacked the necessary information to correctly define long-term project requirements at the onset of a project, due to an inadequate understanding of the customer and an inability to anticipate a customer's evolving needs. Recent research supports this reasoning by continuing to highlight disconnects between planning, design, and implementation in the software development lifecycle. This blog post highlights continuous integration to avoid disconnects and mitigate risk in software development projects.
"Software security" often evokes negative feelings among software developers since this term is associated with additional programming effort and uncertainty. To secure software, developers must follow a lot of guidelines that, while intended to satisfy some regulation or other, can be very restricting and hard to understand. As a result a lot of fear, uncertainty, and doubt can surround software security. This blog posting describes how the Rugged Software movement attempts to combat the toxic environment surrounding software security by shifting the paradigm from following rules and guidelines to creatively determining solutions for tough security problems.
The workflow of deploying code is almost as old as code itself. There are many use cases associated with the deployment process, including evaluating resource requirements, designing a production system, provisioning and configuring production servers, and pushing code to name a few. In this blog post I focus on a use case for configuring a remote server with the packages and software necessary to execute your code.
In a computing system, a context switch occurs when an operating system stores the state of an application thread before stopping the thread and restoring the state of a different (previously stopped) thread so its execution can resume. The overhead incurred by a context switch managing the process of storing and restoring state negatively impacts operating system and application performance. This blog post describes how DevOps ameliorates the negative impacts that "context switching" between projects can have on a software engineering team's performance.
The DevOps movement is clearly taking the IT world by storm. Technical feats, such as continuous integration (CI), comprehensive automated testing, and continuous delivery (CD) that at one time could only be mastered by hip, trendy startups incapable of failure, are now being successfully performed by traditional enterprises who have a long history of IT operations and are still relying on legacy technologies (the former type of enterprises are known in the DevOps community as "unicorns," the latter as "horses"). In this post, I explore the experience of a fictional horse, Derrick and Anderson (D&A) Lumber, Inc., a company that hit some bumps in the road on its way to DevOps. As D&A finds out, a DevOps transformation is not a product that can be purchased from the outside, but rather a competency that must be grown from within.
When building and delivering software, DevOps practices, such as automated testing, continuous integration, and continuous delivery, allow organizations to move more quickly by speeding the delivery of quality software features, that increase business value. Infrastructure automation tools, such as Chef, Puppet, and Ansible, allow the application of these practices to compute nodes through server provisioning using software scripts. These scripts are first-class software artifacts that benefit from source code version control, automated testing, continuous integration, and continuous delivery.
Regular readers of this blog will recognize a recurring theme in this series: DevOps is fundamentally about reinforcing desired quality attributes through carefully constructed organizational process, communication, and workflow. When teaching software engineering to graduate students in Carnegie Mellon University's Heinz College, I often spend time discussing well known tech companies and their techniques for managing software engineering and sustainment. These discussions serve as valuable real-world examples for software engineering approaches and associated outcomes, and can serve as excellent case studies for DevOps practitioners. This posting will discuss one of my favorite real-world DevOps case studies: Amazon.
In the post What is DevOps?, we define one of the benefits of DevOps as "collaboration between project team roles." Conversations between team members and the platform on which communication occurs can have a profound impact on that collaboration. Poor or unused communication tools lead to miscommunication, redundant efforts, or faulty implementations. On the other hand, communication tools integrated with the development and operational infrastructures can speed up the delivery of business value to the organization. How a team structures the very infrastructure on which they communicate will directly impact their effectiveness as a team. ChatOps is a branch of DevOps focusing on the communications within the DevOps team. The ChatOps space encompasses the communication and collaboration tools within the team: notifications, chat servers, bots, issue tracking systems, etc.
When Agile software development models were first envisioned, a core tenet was to iterate more quickly on software changes and determine the correct path via exploration--essentially, striving to "fail fast" and iterate to correctness as a fundamental project goal. The reason for this process was a belief that developers lacked the necessary information to correctly define long-term project requirements at the onset of a project, due to an inadequate understanding of the customer and an inability to anticipate a customer's evolving needs.
In our last post, DevOps and Docker, I introduced Docker as a tool to develop and deploy software applications in a controlled, isolated, flexible, and highly portable infrastructure. In this post, I am going to show you how easy it is to get started with Docker. I will dive in and demonstrate how to use Docker containers in a common software development environment by launching a database container (MongoDB), a web service container (a Python Bottle app), and configuring them to communicate forming a functional multi-container application. If you haven't learned the basics of Docker yet, you should go ahead and try out their official tutorial here before continuing.
Docker is quite the buzz in the DevOps community these days, and for good reason. Docker containers provide the tools to develop and deploy software applications in a controlled, isolated, flexible, highly portable infrastructure. Docker offers substantial benefits to scalability, resource efficiency, and resiliency, as we'll demonstrate in this posting and upcoming postings in the DevOps blog.
The federal government is facing a shortage of cybersecurity professionals that puts our national security at risk, according to recent research. "As cyber attacks have increased and there is increased awareness of vulnerabilities, there is more demand for the professionals...