Posted on by DevOpsin
To maintain a competitive edge, software organizations should be early adopters of innovation. To achieve this edge, organizations from Flickr and IBM to small tech startups are increasingly adopting an environment of deep collaboration between development and operations (DevOps) teams and technologies, which historically have been two disjointed groups responsible for information technology development. "The value of DevOps can be illustrated as an innovation and delivery lifecycle, with a continuous feedback loop to learn and respond to customer needs," Ashok Reddy writes in the technical white paper, DevOps: The IBM Approach.
Beyond innovation and delivery, DevOps provides a means for automating repetitive tasks within the software development lifecycle (SDLC), such as software builds, testing, and deployments, allowing them to occur more naturally and frequently throughout the SDLC. This blog post, the second in our series, presents a generalized model for automated DevOps and describes the significant potential advantages for a modern software development team.
I oversee a software engineering team that works within CERT and focuses on research and development of solutions to cybersecurity challenges. Our engineers design and implement software solutions that solve challenging problems for federal agencies, law enforcement, defense intelligence organizations, and industry by leveraging cutting-edge academic research and emerging technologies.
The environment and manner in which software engineering teams (not just my own) write code are constantly evolving. Until recently, most software engineers operated in siloed environments, with developers in one silo and an operations staff of IT professionals who maintained systems and software deployments in another. Members of the operations staff were often uninvolved in development processes, sometimes even unfamiliar with technologies used in development, yet expected to handle deployment, release testing, maintenance, and support of software applications produced by development teams.
Likewise, development teams were unable to design or develop software to be effectively released, tested, and maintained by operations teams. The problem, of course, is that they had no understanding of the needs or processes inherent to their operations teams. This divergence led to wasted time and effort, as well as heavy amounts of risk for software products and deployments.
The concept of DevOps initially evolved in 2009 as an effort to remove barriers between the development and operations teams in software development. At the SEI, we are taking that concept and pushing it forward, alongside many others working in the software industry today.
As a federally-funded research and development center (FFRDC), the SEI must maintain high standards of efficiency, security, and functionality. Our team in particular develops tools and technologies to help federal agencies assess cybersecurity risks, manage secure systems, and investigate increasingly complex cyber attacks and crimes. Cybersecurity is often misunderstood or even ignored as new systems are designed and developed, falling out of view to more high profile quality requirements, such as availability or correctness of software systems. This prioritization necessarily requires addressing security after the primary design phase, or adding security controls after large portions of development have already occurred. Due to CERT's responsibility to our sponsors and the community, security is consistently a first-tier concern, addressed as an early and fundamental requirement for any system developed by our team. In addition to being a defining factor of our software development methodology, this posture heavily influences our approach to DevOps, weaving security considerations into every facet of our software development operation.
As mentioned in my introduction, forward-thinking approaches to process--including heavily automated DevOps techniques--allow us to systematically implement, maintain, and monitor status and quality standards for each of our projects. One large component of the current DevOps movement is Release Automation, which automates the build and deployment cycles of a software project. In Agile software-development scenarios, this process has two benefits:
This process ensures that the entire development and management team is aware of the up-to-the-minute state of the software, including its test status and ability to be deployed to the expected runtime environment, continually throughout development. This process also enables highly confident development, and a great deal of certainty that when the time comes, the software can be successfully transitioned to an operations team for deployment and maintenance. This certainty is based on the fact that the entire test and deployment process has been automated and performed countless times already throughout the project lifecycle.
Continuous deployment means no surprises for operations staff, which translates into predictable, low-risk releases of provably high-quality software. Many players in the modern software industry share this vision. "When it comes to ensuring quality, release automation is another major asset for development teams. It automates the complex workflows required for agile software movement between development, test and production environments by removing manual interaction so humans don't introduce avoidable errors," Ruston Vickers wrote in a March 5 blog post on Wired's Innovation Insights blog. That same post cites a study in which more than half of respondents (52 percent) "identified the ability to simplify, standardize and execute application releases with [fewer] errors as the main benefit of release automation." In my view, release automation allows teams to perform build, deployment, and testing activities thoroughly and continually, dramatically increasing the confidence in the state of the software and all processes surrounding release and transition.
In a traditional, siloed environment, teams of developers deploy software manually, on a periodic basis (such as once per quarter, anum, or project cycle) or as necessitated by a new product release. In contrast, for a team leveraging automated DevOps, continuous integration (the process of building and testing a software project continuously, each time new changes are made) is in place and, as part of the process, the project is continuously deployed to an integration environment for testing and review. Some highly mature DevOps operations practice Continuous Deployment, an uninterrupted process that actually deploys live software to production environment.
Automating a task requires a substantial level of understanding by all involved of the processes, technologies, and complexities of deployments of the software in question. This requisite increase in understanding makes it likely that teams moving towards automated DevOps and continuous deployment capabilities also see a positive impact in the reliability, testability, security, and other quality attributes of their software. To achieve automation at this level, the team must study and deeply consider the needs of the entire project, from inception to deployment, which will result in a superior product due to increased focus on implementation details and operational realities.
To achieve the level of automation described, a number of autonomous systems must be in place, such as source control, build and deployment systems, an integration environment, and other systems to channel data and communications throughout DevOps processes. As the image below illustrates, DevOps involves a number of systems operating in concert and communicating seamlessly with other systems and with humans, to detect changes, perform the necessary autonomous functions, and notify team members of status and results. These structured interactions ensure that developers, quality-assurance staff, managers, and even external stakeholders receive continuous, real-time information on the status of the project, which is hugely beneficial, especially in Agile environments. Consistent with the Agile Manifesto: more information, more often, will lead to better project outcomes.
Systems of an Automated DevOps Environment
The remainder of this blog post explores our generalized model for automated DevOps, identifying the systems required to support an automated DevOps process (as shown in the illustration above):
As I see it, all the processes described above--from source control to documentation--require some system to perform them in every software project. Of course, before automated DevOps, many of these processes were performed manually, especially manual testing and deployments. Even now, generation and formatting of system documentation or management of code reviews are still manual processes in most organizations. My goal with this post was to define all tasks necessary throughout the SDLC, and to promote the automation of as many of them as possible, thus freeing up developers to do what they do best: design and write amazing code.
While this post presented a generalized model for DevOps practices, future posts in this series will present the following topics:
We welcome your feedback on this series, and what DevOps topics would be of interest to you. Please leave feedback in the comments section below.
To listen to the podcast, DevOps--Transform Development and Operations for Fast, Secure Deployments, featuring Gene Kim and Julia Allen, please visit https://resources.sei.cmu.edu/library/asset-view.cfm?assetid=58525.
To view the August 2011 edition of the Cutter IT Journal, which was dedicated to DevOps, please visit http://www.cutter.com/promotions/itj1108/itj1108.pdf.
Additional resources include the following sites: