A zero-day vulnerability refers to a software security vulnerability that has been exploited before any patch is published. In the past, vulnerabilities were widely exploited even when a patch was available, which means they were not zero-day. Today, zero-day vulnerabilities are common. Notorious examples include the recent Stuxnet and Operation Aurora exploits. Vulnerabilities may arise from a variety of sources, but most vulnerabilities are the result of simple coding errors. Consequently, developers need to understand common traps and pitfalls in the programming language, libraries, and platform to produce code that is free of vulnerabilities.

To address this problem, CERT published The CERT Oracle Coding Standard for Java in 2011. This book is version 1 of this standard and was written primarily for Java SE 6, but also covers features introduced in Java SE 7. This coding standard provides secure coding rules that help programmers recognize and avoid vulnerabilities in their products. Each rule provides simple instructions regarding what a programmer must and must not do. Each rule description is accompanied by noncompliant code examples, as well as compliant solutions that can be used instead. In this blog post, I examine a Java zero-day vulnerability, CVE 2012-0507, which infected half a million Macintosh computers, and consider how this exploit could have been prevented through adherence to two secure coding rules.

Java Security Background

Java was designed in the early 1990s with security in mind. Java's creators wanted it to have the ability to run untrusted code that could be immediately delivered and run over the network. With the growth of the World Wide Web, this requirement translated into the ability to run Java applets over the web. Because an applet need not be trusted, Java required that each applet run in a security sandbox maintained by an object called the SecurityManager. This object acts as a chaperone, overseeing what an applet does and generating a SecurityException if the applet violates a security policy. A SecurityException typically causes the applet to terminate without disturbing the rest of the user's browsing experience. Actions forbidden by the SecurityManager include

• accessing the file system
• accessing the network (except the host it came from)
• running external programs
• disabling the SecurityManager
• However, a signed applet may request from the user the ability to do some or all of these privileged actions.

Java's huge (and open-source) core library provides a large attack surface that can conceal many vulnerabilities. As Java evolves, the library grows, and hackers know that a large codebase means many new vulnerabilities can be found and exploited. Each vulnerability is a golden opportunity for hackers to gain recognition among their peers or to sell their exploit for big bucks on the black market. Roger A. Grimes reported in InfoWorld that an increasing number of governments are buying hackers' exploits--or hiring the hackers.

CVE 2012-0507 and the Flashback Trojan

Jeroen Frijters, technical director of Sumatra Software, first discovered the vulnerability later classified as CVE 2012-0507 in 2011 while developing IKVM, a Java Virtual Machine (JVM) for .NET. Frijters practiced responsible disclosure by notifying Oracle when he discovered the vulnerability, coordinating his publication of the vulnerability details with Oracle's release of an update to Java (1.7.0_03), which disabled the vulnerability in February 2012.

Meanwhile, the Mac Flashback Trojan had languished in obscurity for several months until it was altered to exploit the Java vulnerability in March 2012.

Apple had supported Java ever since Mac OS X was released in 2001. Moreover, it had distributed and updated Java as part of OS X itself. Unfortunately, Apple had not applied Oracle's patch until the exploit started attacking Macs in the wild. After its makeover, Flashback managed to infect over 500,000 Apple computers, and 22,000 Apple computers remained infected as of January 2014.

Apple has unbundled Java from OS X and no longer distributes Java. Mac users who wish to install Java must now download it from Oracle.

Preventing Flashback

Earlier this year, Milton Smith, who leads the strategic security program for Java products at Oracle, asked Robert Seacord and me to participate as reviewers on the JavaOne 2014 Security Track review team. He also encouraged us to propose a Java exploit presentation for the track.

Our presentation focused on how the Flashback vulnerability could have been prevented. To prevent future exploits, it is important to understand how these mistakes occurred. Two issues were at play in Flashback, which the developers could have prevented had they followed our Java coding rules:

• One common principle of object-oriented design is that every class has data that is private; only that class can manipulate that data. The AtomicReferenceArray class contains a private array but does not ensure that this array remains private. This class is also serializable, which allows an object of this class to be written to a file that can later be deserialized, that is, read back into memory.
  
• The diagram below illustrates the data structure they produced. This data structure could not have been produced by running Java because the AtomicReferenceArray would not have allowed its private array to be accessed by any other data structure.

The attackers serialized an AtomicReferenceArrayobject to a file. They modified this file so that the deserialized object contained a public handle to the AtomicReferenceArray's private array.

Deserializing this data structure was possible because the AtomicReferenceArray failed to override the default deserialization method, which allowed outside access to the internal array.

Failure to override the default deserialization method violates the following CERT rule:

SER07-J Do not use the default serialized form for classes with implementation-defined invariants

Oracle mitigated the vulnerability by adding the following method to

AtomicReferenceArray:

private void readObject(java.io.ObjectInputStream s) throws java.io.IOException, ClassNotFoundException

Object a = s.readFields().get("array", null);
if (a == null || !a.getClass().isArray())
throw new java.io.InvalidObjectException
"Not array type");
if (a.getClass() != Object[].class)
a = Arrays.copyOf((Object[])a, Array.getLength(a), Object[].class); unsafe.putObjectVolatile(this, arrayFieldOffset, a);}

This method is invoked whenever an AtomicReferenceArrayis deserialized. By making a private copy of its internal array if the array is not of type Object, this class guarantees the privacy of its internal array, disabling the exploit.

• The second issue allowed an attacker to insert an object into the AtomicReferenceArray's private array and then extract it as an object of a different class. The JVM was consequently tricked into thinking that the object is of a different type than it actually is.

In the Flashback exploit, MalClassLoader is a malicious class derived from the benign ClassLoader class, which contains a defineClass() method that allows the creation of new code. This method is accessible only to subclasses, including the malicious MalClassLoader class. The Java security manager prevents an unprivileged applet from creating custom class loaders. Unfortunately, the exploit tricks the JVM into believing that a preexisting class loader object is actually a MalClassLoader object. Consequently, this object could use defineClass() to build a new Java class from an array of bytes and execute its default constructor. This new class is created with full privileges; the security manager was instructed not to prevent the new class from doing anything. The new class then disabled the security manager, which effectively enabled the rest of the exploit code to do anything normally forbidden by the SecurityManager.

Java defines heap pollution as a condition in which one class is expected to contain elements of one type but can inadvertently contain an element of another type. The JVM has mechanisms to prevent heap pollution or detect if it has happened, but heap pollution can still be successfully exploited by an attacker. In confusing a ClassLoader object with a MalClassLoaderobject, this exploit polluted the AtomicReferenceArray object, which violates the following CERT rule:

OBJ03-J Prevent heap pollution

In Version 1.0 of the standard, OBJ03-J was titled "Do not mix generic with nongeneric raw types in new code," but was updated in response to community feedback.

Future Work

Java is a victim of its success: its widespread use and popularity have made it a lucrative target for hackers, similar to Microsoft Windows. Java includes a huge library with many features, some of which are obsolete or deprecated. To prevent future Java vulnerabilities, Oracle is working to comply with Java Secure Coding rules during ongoing development and maintenance of the Java codebase. Likewise, compliance with these rules is critical for Java developers outside of Oracle even if their software is not used as widely as Java itself.

We are currently updating the CERT Oracle Coding Standard for Java for Java SE 8 on our community wiki. We encourage the community to participate in the continuing evolution of the standard. Anyone is free to browse the standard and submit comments, and experts are frequently invited to become editors. Community involvement is essential to producing a high-quality coding standard. There are both commercial and open-source tools such as FindBugs that identify potential violations of the Java Secure Coding rules.

Additional Resources

To sign up for a free account on the CERT Secure Coding wiki, please visit
http://www.securecoding.cert.org.

To subscribe to our Secure Coding eNewsletter, please click here.

For more information about the CERT Oracle Secure Coding Standard for Java, please visit
https://wiki.sei.cmu.edu/confluence/display/java/SEI+CERT+Oracle+Coding+Standard+for+Java.