Aristagoras led the Ionian revolt against the Persian Empire in 499 BCE. The revolt was coordinated with other Greek city-states by means of an early example of steganography, a way to exchange hidden messages or information. His compatriot, Histiaeus of Susa, sent a plea for an uprising by shaving a servant's head, tattooing the message on his skin, waiting for the hair to grow back, and then sending the servant to deliver the message. When the servant successfully completed the trek to Miletus without arousing suspicion, he revealed the hidden message by shaving his head.

The Grecian tactic of hiding messages, information, and codes remains relevant as a core part of today’s computer malware development and deployment. In this blog post, we describe how attackers obscure their activity via alternate data streams (ADSs), which are essentially alternate subfiles inside of a file. We also discuss how to defend against malware attacks that employ ADSs for evasion and subterfuge.

While ADSs are not new, they continue to elude the attention of many professional defenders. Therefore, the security concerns they’ve posed since inception have remained largely unaddressed. This oversight could be a result of the unique niche of ADSs in the filesystem framework: by design, they are subtle, innocuous, and almost entirely invisible.

We often think of a file as a single-compartment container, holding data that might compose a text file, picture, or video. But in an ADS-enabled file system, files can contain many compartments, each independently packaged in parallel, and each filled with its own text or binary data—its own alternate data stream. The crux of this design is that only the initial primary compartment is visible when using common navigational tools like File Explorer and the terminal to search a filesystem. The rest are essentially secret compartments, their existence and contents unknown to the user without a specific targeted search. This innate obfuscation is what makes ADSs streamlined and convenient— both for the standard user and for the malicious attacker.

Today’s most advanced adversary teams—advanced persistent threats or APTs—aim to create and deploy malware that can covertly infiltrate targeted systems and maintain access over time. This strategy relies on remaining hidden from all manner of detection tools and incident-response measures. APTs seek not only to persist on a target system, but also to discover and capitalize on opportunities to silently expand their purview. A successful APT is one that remains undetected for as long as possible. One of the largest concerns with ADS exploitation is the ease with which an adversary can silently store or execute malicious payloads directly from a hidden stream. Later in this post, we discuss more technically how attackers can use ADSs to hide data and activity, as well as provide details from our own ADS use case executed during a training event.

APT malware authors are tirelessly developing new techniques to evade an ever-growing number of security controls. Some of the basic methods for obfuscation include packers, crypters, polymorphism, staged loading, anti-disassembly, rootkits, and injection. In the end, however, malware must exist in some digital space or container—in memory or on disk—and as security researchers, we at the SEI are always interested in those dark corners of a computer where malware might hide.

Alternate data streams have existed since the introduction of Microsoft Windows’ New Technology File System (NTFS). These streams provide compatibility with Apple’s Hierarchical File System (HFS), which implements a similar multi-stream strategy mimicked by ADSs called resource forking (HFS was replaced with the Apple File System [APFS] beginning with MacOS High Sierra in 2017). They are part of the underlying master file table (MFT) structure detailed by Ryan L. Means, who defines an ADS as “The stream in any data attribute on a file or directory other than the default, unnamed stream.”

Alternate data streams on NTFS are essentially alternate subfiles inside of a file. Typically, when a file on an NTFS drive is accessed, it automatically opens its default unnamed data stream, which is referred to as $DATA on the Windows operating system. If, however, the filename handle of an alternate data stream is specified while opening the file, the content of the corresponding ADS will instead be read or executed.

An ADS can be as small or large as any other file on disk, but with the notable limitation that it will be lost if the host file is moved to a non-NTFS filesystem. These streams can therefore disappear or become suddenly inaccessible when copied to a network share, transferred to external media, placed on a USB drive, etc.

There are many legitimate uses for ADSs, such as to

● increase dynamic-link library (DLL) loading speed in Citrix’s virtual memory

● increase performance of scanning files in antivirus applications

● identify high-risk files in Windows System Resource Manager

● encrypt files in the Windows operating system

● scan files in Windows Attachment Manager

● maintain database integrity in Microsoft SQL Server

Perhaps we have come to expect that any file on our system is visible with the tools provided by our operating systems. However, ADSs are not displayed in this way. Their existence is thus at high risk of falling out of sight and out of mind since we must specifically look for them and what they contain. Moreover, there are many lesser known subtleties of ADSs that add to their complexity:

● An ADS can be added to any file of any type—text files, binary files, executable files—that is accessible in the file system. An ADS can be added even to core system files, such as rundll32.exe or similar.

● ADSs can be used to create folders by any user with create folder permissions on the parent directory (these streams must start with an alpha character).

● When we add, modify, or delete an ADS, the base containing file size remains unchanged, even though the ADS can be any size that a normal file on disk might be. We can thus have a 1kb-size file and write an ADS onto that file that is a gigabyte or more in size, but the file size will still show as 1kb in applications, such as File Explorer.

● Similarly, adding, modifying, or deleting ADSs does not modify the file hashes.

● Related, although the file size does not change when we add a new or modify an existing ADS, the containing file timestamp does update.

● ADSs can circumvent locked files using stream-aware applications, such as Windows Notepad. When adding data to an ADS associated with an already locked file, anti-virus (AV) systems cannot find the ADS on a locked file (where essentially the primary unnamed stream is locked), resulting in being able to read and write data from the alternate stream, even while the primary stream remains locked, evading detection of AV systems.

● ADSs provide a handle to their contents like any other file, so an adversary can store and execute malicious payloads directly from a stream and use execution to cause corruption or damage to a system. A command, such as, C:>start c:\fox.txt:rundll32.exe, will execute a binary executable file directly from the stream.

● Lastly, NTFS and the MFT provide no mechanism for disabling ADSs.

While there is no exact analog to ADSs on Linux distributions, there are similar technologies, such as other file systems that support what are called extended attributes (EAs, or in Unix-style systems, xattr). While EAs are near in concept, they are not the same as ADSs and are rather limited in comparison. EAs have a combined size limit (for all EAs on a file) of 64k. Moreover, EAs are not streams—you cannot open a file handle to an EA and read it as a stream.

As we’ve discussed, ADSs are hidden during default filesystem navigation. Luckily, the manual workaround for revealing them is quite simple: the standard DIR command can be used with its /r switch to list all directory files, including streams.

Less fortunately, however, there are a few techniques for circumventing DIR /r :

● If an ADS is added to a file with a Windows reserved device name, it will not appear in a DIR /r listing. Creating files with reserved names such as CON, AUX, or NUL is only possible by prepending \\?\ to the name during creation, ex: C:\temp>echo texthere > \\?\C:\temp\NUL . The resulting file will appear in directory listings as C:\temp\NUL, without the prefix, but any contained ADSs will be visible only if \\?\ is explicitly appended its searchpath name when calling DIR /r.

● An adversary could create confusing or nondescript stream paths, such as C:\Windows\:System32, which could escape casual notice in large directory listings with many files and streams.

● ADSs can be compressed through common programs like WinRAR, which allows their transfer to and preservation in non-NTFS systems. These non-compatible environments have no native methods for alternate stream detection and thus require specialized forensic techniques and tools to facilitate the manipulation and identification of compressed ADSs.

In addition to manual user methods, there are many available NTFS scanning applications that provide an intuitive interface for the detection and inspection of hidden ADS content. Even the free versions of these programs can usually identify most adjunct data streams present in an everyday filesystem. Unfortunately, many of these tools rely on the same Windows API functionality as the DIR /r command, and can thus be thwarted by tricky subversions of Windows naming conventions, such as the \\?\ obfuscation method outlined above.

Microsoft has its own scanning tool available for download called Streams, which had originally been developed as a part of Sysinternals back in 2016. Microsoft’s documentation reports that Streams “makes use of an undocumented native function for retrieving file stream information.” Our testing, however, hasn’t discovered Streams as any less susceptible to the known strategies for evading detection with ADS-naming subterfuge.

Malware that takes advantage of ADSs is not new. MITRE lists over a dozen named malware examples that use ADSs to hide artifacts and evade detection. Attack tools, such as Astaroth, Bitpaymer, and PowerDuke, have been extensively detailed by various parties, providing insight into how these threats take advantage of ADS evasion on a host system. Authors, such as Berghel and Brajkovska, downplay the risks of ADSs. Our opinion, however, is that ADSs introduced the host of concealment and obfuscation techniques outlined above, but little has been done to mitigate these worries since their publication in 2004.

In a recent APT emulation exercise, we explored the use of ADSs to collect and exfiltrate data while obfuscating it from our participants’ casual view. We first designed and developed an exercise event that collected various data logs from the network. We then exfiltrated them from the protected network to an adversarial command-and-control server across a large-scale simulated internet (using our own greybox internet simulator). This exfiltration was done only periodically to reduce network activity that would have it shown on a top-talker or beaconing report. We suspect that our experience sheds light on the difficulties that ADSs introduce into security operations and monitoring, including detecting them and what they contain.

For this adversarial ADS activity, we began with a Microsoft PowerShell script that would be hidden within legitimate system-operation scripts. We copied naming conventions, script headers, and the coding style into our own malicious script. The first action for the script was to locate files of interest and to silently copy them to a hidden location that we could later access for exfiltration.

It turns out that in terms of ease of use, the script itself became a convenient database of data read for later exfiltration. For each file that the script found and wanted to exfiltrate, the script would write that data into an ADS on itself, so the command C:\some\path\> .\script.ps1 would create the ADS, C:\some\path\script.ps1:file_read_path (note that the ADS names are subject to the same Windows OS pathnaming guidelines, and so characters such as “\” need to be properly escaped). Using this method, by the end of the first 24 hours, we had hundreds of ADSs written onto our script, and we were ready to move on to the next step of getting that data out of the network. An initial concern of where to hide megabytes of copied data to exfiltrate was made simple by hiding it in ADS streams out of the casual view of network administrators.

In alignment with the known tactics, techniques, and procedures (TTPs) of our APT emulation target, our primary vehicle for data exfil was the Background Intelligent Transfer Service (BITS), which is a native Windows component for asynchronous, prioritized uploads and downloads using idle network bandwidth. For additional versatility, we designed and monitored our BITS transfers using the BITSAdmin command-line tool. By consolidating some of the convenient BITSAdmin functionalities for job creation and transfer management into our existing ADS Powershell script, we created a single all-encompassing program that would accomplish each of our simulated infiltrator’s target objectives:

1. Copy and stow the contents of the target file into hidden ADS.
2. Slice the file into <1MB chunks for discreet daily exfiltration.
3. Create a BITSAdmin job that uploads single slices of data to our remote server.
4. When the upload completes, delete the job and any associated temporary files.

With automated execution and clean-up already incorporated into the script, the only remaining task was to run a single scheduled-tasks (schtasks) command that queued up daily execution of the script’s exfil function. At the time of writing, this attack has been successfully implemented in the exercise space and daily exfil monitoring remains all green.

Of the many security challenges posed by ADSs, perhaps the greatest threat lies in their inherent invisibility within NTFS Windows systems—and consequently, the increased effort and nuance required to detect their presence. Although the most common malicious usages of ADSs are fairly well documented and understood, their existence remains a security vulnerability by simple nature of the blindspot they occupy in contemporary cybersecurity frameworks.

ADSs are most dangerous when left unmonitored and unaccounted for since they essentially enable all the issues and risks associated with the standard file system. As shown by our case study, unless your security operations center deliberately transcends conventional procedures for the file-review process, it is nearly impossible to know what ADSs may be silently hiding on your system and harboring malicious scripts. To mitigate the potential for rampant ADS abuse across the network, it is critical to begin updating our standard security-operations repertoires with specific diagnostic accommodations and countermeasures against rogue unwanted ADSs.