Untangling the Knot: Enabling Rapid Software Evolution
Created February 2022 • Updated October 2024
To quickly deliver new capabilities or take advantage of new technologies, organizations often need to first improve the modularity of existing software by isolating existing capabilities from surrounding code. Our automated refactoring solution recommends ways to refactor existing software, significantly increasing the efficiency of software evolution.
Software Is Never Done
Software-reliant systems need to evolve over time to meet new requirements, take advantage of new technology, or accommodate changes in everything from the deployment environment to business priorities. Many evolution projects start with a common problem—isolating software capability from its tangle of dependencies. Isolating software is key to working more efficiently with software and deploying it in new ways or to new environments. Common examples include
- decomposing monolithic applications into independently evolvable, deployable units (e.g., microservices)
- reusing capability in a different system or on a different platform
- improving migration of capability to the cloud (e.g., going cloud native instead of relying on lift and shift)
- isolating custom capabilities for easier replacement with third-party or open source alternatives
But all too often the structure of software becomes too complicated to allow rapid and cost-effective improvements. This is common in long-lived systems and even occurs frequently in newer systems, hampering the ability to independently evolve, migrate, or replace business and mission capabilities. Software refactoring is a known technique for improving the structure of software, but it typically relies heavily on slow, manual efforts when applied at scale.
To understand how large-scale refactoring is performed today, we surveyed industry practitioners. We found that large-scale refactoring is common and efforts often take hundreds of staff days to complete. Organizations also commonly forego desired refactoring efforts because they prioritize new features more highly or have concerns about the cost involved. However, foregoing refactoring often leads to a slower pace of capability delivery.
One of the contributing factors to expensive large-scale refactoring efforts is the lack of tools designed to solve that problem. To address this problem, the SEI built an automated refactoring assistant for developers that helps development teams improve the software structure for several common forms of change that involve software isolation. It solves project-specific problems with a semi-automated approach. Our goal is to enable refactoring to be completed in less than one-third of the time required by manual approaches.
Automating Software Isolation
The SEI's refactoring assistant analyzes code written in Java, C#, and C/C++ and recommends specific refactorings that isolate its functionality from its tangle of dependencies. In one proprietary example, a team estimated 14,000 hours of software development work alone—excluding integration and testing—to isolate a mission capability from the underlying hardware platform. The refactoring assistant could reduce the development time required to less than 5,000 hours.
Our prototype combines advances in search-based software engineering with static code analysis and refactoring knowledge. It is unique in its focus on project-specific goals as opposed to improving general software metrics. This goal is incorporated in genetic algorithms through fitness functions that guide the search to solutions for the project-specific isolation goal. In practice, our prototype recommends solutions that solve more than 85% of the problems on typical projects, suggesting that our effort-reduction goal is obtainable. The search algorithm relies on a representation derived from static code analysis and uses formalizations of refactorings as operators to explore many different combinations of code changes during search.
This work addresses a widespread, recurring need in software organizations. Because requirements and technology are never frozen in time, the need to adapt working software to new contexts will remain a common need across many software systems. Our solution has broad implications for moving existing software to modern architectures and infrastructures, such as service-based architectures, microservices, serverless architectures, cloud environments, and containers. It also addresses a pervasive research challenge in improving automated support for developers to perform architecture refactoring tasks.
Looking Ahead
Our prototype currently works on codebases as large as 2 million source lines of code with no problem. We are also actively working on research to integrate additional preferences from users and bias recommendations toward solutions favored by developers.
Learn More
Role of Automation in Reducing Software Refactoring Costs
•Presentation
This talk describes a case study on the use of automation for large-scale software refactoring using an open source example.
Learn MoreDependent or Not: Detecting and Understanding Collections of Refactorings
•Article
This paper describes a theory for reasoning about refactorings by defining an ordering dependency relation and organizing them as a set of refactoring graphs.
ReadSoftware Isolation: Why It Matters to Software Evolution and Why Everybody Puts It Off
•Blog Page
READ
Refactoring for Software Isolation
•Presentation
This project extends a refactoring assistant that automates the majority of the work required for software isolation.
Learn MoreIndustry's Cry for Tools That Support Large-Scale Refactoring
•Conference Paper
This paper introduces an industry survey that assessed which tools developers use in large-scale refactoring efforts and how well those tools support refactoring.
ReadUntangling the Knot: Enabling Architecture Evolution with Search-Based Refactoring
•Conference Paper
This paper describes a search-based algorithm that recommends a series of refactorings that isolate specified software from its architectural dependencies.
ReadUntangling the Knot: Automating Software Isolation
•Presentation
Our prototype tool combines advances in search-based software engineering with static code analysis and refactoring knowledge for cost-effective software improvement.
Learn More