icon-carat-right menu search cmu-wordmark

Untangling the Knot: Enabling Rapid Software Evolution

Created February 2022 • Updated October 2024

To quickly deliver new capabilities or take advantage of new technologies, organizations often need to first improve the modularity of existing software by isolating existing capabilities from surrounding code. Our automated refactoring solution recommends ways to refactor existing software, significantly increasing the efficiency of software evolution.

Software Is Never Done

Software-reliant systems need to evolve over time to meet new requirements, take advantage of new technology, or accommodate changes in everything from the deployment environment to business priorities. Many evolution projects start with a common problem—isolating software capability from its tangle of dependencies. Isolating software is key to working more efficiently with software and deploying it in new ways or to new environments. Common examples include

  • decomposing monolithic applications into independently evolvable, deployable units (e.g., microservices)
  • reusing capability in a different system or on a different platform
  • improving migration of capability to the cloud (e.g., going cloud native instead of relying on lift and shift)
  • isolating custom capabilities for easier replacement with third-party or open source alternatives

But all too often the structure of software becomes too complicated to allow rapid and cost-effective improvements. This is common in long-lived systems and even occurs frequently in newer systems, hampering the ability to independently evolve, migrate, or replace business and mission capabilities. Software refactoring is a known technique for improving the structure of software, but it typically relies heavily on slow, manual efforts when applied at scale.

To understand how large-scale refactoring is performed today, we surveyed industry practitioners. We found that large-scale refactoring is common and efforts often take hundreds of staff days to complete. Organizations also commonly forego desired refactoring efforts because they prioritize new features more highly or have concerns about the cost involved. However, foregoing refactoring often leads to a slower pace of capability delivery.

One of the contributing factors to expensive large-scale refactoring efforts is the lack of tools designed to solve that problem. To address this problem, the SEI built an automated refactoring assistant for developers that helps development teams improve the software structure for several common forms of change that involve software isolation. It solves project-specific problems with a semi-automated approach. Our goal is to enable refactoring to be completed in less than one-third of the time required by manual approaches.

Untangling the Knot: Enabling Rapid Software Evolution

Automating Software Isolation

The SEI's refactoring assistant analyzes code written in Java, C#, and C/C++ and recommends specific refactorings that isolate its functionality from its tangle of dependencies. In one proprietary example, a team estimated 14,000 hours of software development work alone—excluding integration and testing—to isolate a mission capability from the underlying hardware platform. The refactoring assistant could reduce the development time required to less than 5,000 hours.

Our prototype combines advances in search-based software engineering with static code analysis and refactoring knowledge. It is unique in its focus on project-specific goals as opposed to improving general software metrics. This goal is incorporated in genetic algorithms through fitness functions that guide the search to solutions for the project-specific isolation goal. In practice, our prototype recommends solutions that solve more than 85% of the problems on typical projects, suggesting that our effort-reduction goal is obtainable. The search algorithm relies on a representation derived from static code analysis and uses formalizations of refactorings as operators to explore many different combinations of code changes during search.

This work addresses a widespread, recurring need in software organizations. Because requirements and technology are never frozen in time, the need to adapt working software to new contexts will remain a common need across many software systems. Our solution has broad implications for moving existing software to modern architectures and infrastructures, such as service-based architectures, microservices, serverless architectures, cloud environments, and containers. It also addresses a pervasive research challenge in improving automated support for developers to perform architecture refactoring tasks.

Looking Ahead

Our prototype currently works on codebases as large as 2 million source lines of code with no problem. We are also actively working on research to integrate additional preferences from users and bias recommendations toward solutions favored by developers.

Learn More

Role of Automation in Reducing Software Refactoring Costs

Presentation

This talk describes a case study on the use of automation for large-scale software refactoring using an open source example.

Learn More

Dependent or Not: Detecting and Understanding Collections of Refactorings

Article

This paper describes a theory for reasoning about refactorings by defining an ordering dependency relation and organizing them as a set of refactoring graphs.

Read

Refactoring for Software Isolation

Presentation

This project extends a refactoring assistant that automates the majority of the work required for software isolation.

Learn More

Industry's Cry for Tools That Support Large-Scale Refactoring

Conference Paper

This paper introduces an industry survey that assessed which tools developers use in large-scale refactoring efforts and how well those tools support refactoring.

Read

Untangling the Knot: Enabling Architecture Evolution with Search-Based Refactoring

Conference Paper

This paper describes a search-based algorithm that recommends a series of refactorings that isolate specified software from its architectural dependencies.

Read

Untangling the Knot: Automating Software Isolation

Presentation

Our prototype tool combines advances in search-based software engineering with static code analysis and refactoring knowledge for cost-effective software improvement.

Learn More