On behalf of the real author, my colleague David Svoboda (and a couple others who work on the CERT Secure Coding Initiative), here's a post analyzing recent Java exploits.

Java was exploited recently and last August. The August exploit was patched by Oracle on August 30; this most recent exploit now also has a patch available. Strictly speaking, the vulnerabilities that permitted both exploits are independent; the current exploit attacked code that was unused by the August exploit. Nevertheless, these vulnerabilities were quite similar. This blog post examines the vulnerabilities that permitted Java to be exploited in each case, using the proof-of-concept code exploits that have been published for them in January 2013 and August 2012.

Background

Unlike most languages, Java was specifically designed for safe execution of untrusted code. Although the security API involves many classes, the SecurityManager class is the most critical. The System class contains a single static instance of the SecurityManager class. The SecurityManager monitors certain potentially sensitive actions, and causes a SecurityException to be thrown when the program attempts to perform an action for which it lacks permissions. The System.SecurityManager can also be set to null, in which case all actions are permitted. Typical sensitive actions checked by the SecurityManager include the execution of other applications, opening of local files, opening network sockets, and loading additional Java code. The SecurityManager also prevents untrusted code from changing certain system settings, including disabling the SecurityManager.

Most Java desktop applications run with the null SecurityManager; consequently Java applications can perform the same functions permitted by a program written in any other language in the same context. A non-null security manager enforces a security policy put in place for the Java code that is running. For Java applets, there is a restrictive security policy. This security policy prevents applets from opening local files or network sockets, with the exception that applets may open a connection to the web address from which they originated. Colloquially, they are permitted to "phone home." In theory, it is possible to run Java applets from an untrusted source, while preventing the untrusted code from contacting other untrusted websites or tampering with local files. This level of security helped to propel Java applets into widespread use when they were first introduced in 1995.

The recent exploits demonstrate that the Java environment fails to achieve its theoretical security potential. Both exploits can be run as applets. They each work in the context of a SecurityManager and they each manage to disable the security manager to run arbitrary code.

Writing secure Java code can be nontrivial. This is especially true if the code interacts with untrusted code, or otherwise depends on Java's security architecture. To assist developers in writing secure code, CERT has published The CERT Oracle Secure Coding Standard for Java (the rules are also available). Sun also published its own secure coding rules for Java; Sun, and later Oracle, have maintained this document.

Java provides two APIs that support reflection: the Java Beans API and the Reflection API. A Java Bean is a piece of code that specifically knows information about itself; beans are often used to build self-aware code for applications such as IDEs or GUI builders. The Reflection API consists of a series of classes that enable a Java program to answer questions about its internal structure.

Using the reflection API, a Java program can discover what classes are available in a package, including the specific details of the methods and fields a given class supports. A program can also read the value of a field using reflection, or even invoke a method using reflection. Not only can a program examine the classes and other items it uses, it can also discover what specific classes are provided by the Java Virtual Machine (JVM) that runs the program.

Naturally, certain sensitive method calls will be restricted by a security manager; consequently the security manager must be aware of calls to those methods made indirectly via reflection. In addition to a security manager, Java provides access control on the fields and methods of classes, as do many object-oriented languages. For example, a class field that is marked "private" is inaccessible to any code except for the class's own methods. To preserve this property, the reflection API permits indirect access to a private class field only from code that could have accessed the field directly. It is always possible to construct a class that fails to adequately protect its private fields. This could be done by creating a method that returns a Field object containing a reference to one of the class's private fields. However, doing so violates both rule SEC05-J in CERT's rules and rule 6-5 of Oracle's secure coding guidelines.

Current Exploit

The current exploit takes advantage of vulnerabilities that exist in Java's APIs for reflection, class loading, and the security manager. It first creates handles to two classes:
sun.org.mozilla.javascript.internal.Context and sun.org.mozilla.javascript.internal.GeneratedClassLoader. Both classes are in the sun.* package, which indicates that they are part of the internal architecture and are not intended for use by external classes. They are not declared as "private." Instead, the applet SecurityManager prevents access to these classes as part of its security policy. If the exploit code tried to access them directly, the applet would terminate with a SecurityException. So how did the exploit gain access to these classes?

The answer is the MBeanInstantiator class. This class, which is also part of the Java distribution, provides a findClass() method that can be used to access the sun.* classes, even when running with a security manager that is intended to forbid access to such classes. The exploit code must merely create a MBeanInstantiator object; then it can obtain handles to any classes it wants.

The second goal of the exploit is to construct code that runs outside of the security sandbox. To accomplish this goal, it needs a class loader. These objects construct Java code from various sources, such as a local Java .class file, a web stream, or even a string of bytes. This is what the sun.org.mozilla.javascript.internal.GeneratedClassLoader is for. However, the Security Manager would terminate a process that attempts to construct a new class loader. Consequently, the exploit code cannot construct these objects directly. It could use reflection, but the reflection classes also contain security manager checks to prevent the exploit code from using them to do what it can't do itself.

Unfortunately, the java.lang.invoke.MethodHandles.Lookup class fails to perform adequate security manager checks. This class is part of the Invoke API, which is a new addition to the Reflection API in Java 7, and has yet to appear in Oracle's secure coding standards. It can be used to invoke constructors and methods. It does perform a security manager check before proceeding, but, because of a bug involving the new java.lang.invoke.* API, the exploit code passes the security manager check. This bug involves the reflection API trying to determine which method called it.

Because the reflection methods delegate work to each other, they typically ignore each other when trying to determine what method invoked a reflection method. However, they fail to ignore the new methods in java.lang.invoke.*. Instead they incorrectly identify one of the methods in java.lang.invoke.* as the caller, and so they decide the caller method is trusted. As a result, the exploit is able to load and execute untrusted code that is included as a byte string.

The untrusted code simply sets the System.securityManager to null, disabling security, and letting the exploit proceed to do anything it desires.

August Exploit

The August exploit is similar but more direct. It uses several reflection classes in the Beans API to build a single statement outside of the security sandbox and execute it. This statement also sets System.securityManager to null, giving the exploit code full permissions. The security manager should prevent the exploit code from creating new code with no security restrictions but it doesn't. The reason again is that a private-to-Oracle class incorrectly grants full privileges to the statement. In this case, the sun.awt.SunToolkit class returns a Field object pointing to the private Statement.acc field. The Field object can then be set, permitting the Statement to execute with no security check. In this case, the sun.awt.SunToolkit.getField() class violates CERT rule SEC05-J, as it uses reflection to give untrusted code access to a field that the untrusted code could not access directly.

Both exploits access code in the sun.* package, which should normally be inaccessible to applets. The current exploit accomplished this using the MBeanInstantiator class. The August exploit used a different approach. The August exploit used Class.forName() directly rather than using MBeanInstantiator. (In fact, the MBeanInstantiator class also relies on the Class.forName() method to access classes.) The August exploit worked because of a second vulnerability permitting the exploit code to access sensitive classes via Class.forName(). While calling Class.forName() directly would have been caught by the security manager, the exploit uses reflection instead, creating an Expression object that runs Class.forName() when invoked. This is because the java.beans.Statement class (which is invoked by the Expression class) used a special case to allow Class.forName() to be executed, even if the security manager disallowed it.

Conclusions

The exploits examined in this blog post each relied on two vulnerabilities. In each case, the first vulnerability served to grant the exploits access to sensitive private classes that are used internally by Java. Both of these initial vulnerabilities appeared in the Beans API, which provides similar features to the Reflection API, and bypassed normal security manager checks. Once the exploit code had access to the sensitive classes, it was then able to use them to execute a statement that disabled the security manager. The current exploit code used reflection to hide this from the security manager, while the August exploit code used the Beans API, which lacked adequate security manager checks.

While many previous Java vulnerabilities were actually vulnerabilities in the C code of a particular Java implementation, these exploits ran with pure Java--no underlying C/C++ vulnerability was involved. The current proof-of-concept exploit relies on classes that are specific to Oracle's JVM and are unavailable on other implementations, such as OpenJDK. However reports, while unclear, indicate that OpenJDK contains similar vulnerabilities. While OpenJDK lacks the same sun.* classes found in the Oracle JDK, it does contain private class loaders, so this exploit could plausibly work on OpenJDK with minor modifications.

A comparison of Java vulnerabilities with vulnerabilities in C/C++ would be a daunting task. Like many organizations, CERT maintains a database of vulnerabilities. However, although simple statistical searches on the data reveal raw numbers of database entries, these numbers are insufficient for assessing the collective severities of vulnerabilities in one language compared to another. Furthermore, as noted above, many Java vulnerabilities are really C vulnerabilities that occur in an implementation of Java. We can note that many C vulnerabilities, such as buffer overflows, tend to result from writing outside the bounds of an object. This is not possible in a secure implementation of Java; its design mandates memory safety. We can also note that injection attacks, such as SQL injection, cross-site scripting (XSS), and command injection, occur in all languages that permit string manipulation.

Finally, the vulnerabilities analyzed in this blog post exploit Java's security mechanism, which was designed to allow safe execution of untrusted code in a secured environment. The consequence is that the current implementation of the security mechanism is insufficient; the remainder of the language is unaffected. That is, this vulnerability does not prevent Java from running desktop applications, or in fact, any programs that don't use Java's SecurityManager. Java is unusual in providing this feature; most language implementations lack any mechanism for safe execution of untrusted code. These exploits serve as a cautionary warning to other language implementations that wish to permit the execution of untrusted code inside some form of sandbox.