security – Preston Lee's Blog

For all you web application developers, I’d like to briefly note the following stack trace I received from the American Express website while clicking around:

And here is the same trace in plain-text format:


JSPG0036E: Failed to find resource /cardcenter/common/interstitial.jsp

Root Cause:

java.io.FileNotFoundException: JSPG0036E: Failed to find resource /cardcenter/common/interstitial.jsp
	at com.ibm.ws.jsp.webcontainerext.AbstractJSPExtensionProcessor.findWrapper(AbstractJSPExtensionProcessor.java:322)
	at com.ibm.ws.jsp.webcontainerext.AbstractJSPExtensionProcessor.handleRequest(AbstractJSPExtensionProcessor.java:284)
	at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:3548)
	at com.ibm.ws.webcontainer.webapp.WebGroup.handleRequest(WebGroup.java:269)
	at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:818)
	at com.ibm.ws.wswebcontainer.WebContainer.handleRequest(WebContainer.java:1478)
	at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:126)
	at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:458)
	at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewInformation(HttpInboundLink.java:387)
	at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.complete(HttpICLReadCallback.java:102)
	at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165)
	at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
	at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
	at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:136)
	at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:196)
	at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:751)
	at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:881)
	at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1497)

Most developers might say, “So what? You just clicked on a broken link somewhere. Go back and go about your business.”

And that’s exactly what I did, since I just wanted to pay a credit card bill. But what if “my business” had been to gather forensic information about potential American Express website vulnerabilities? A stack trace may look innocuous, but we can be almost certain that this HTTP request was handled by a Java servlet. It uses Java Server Pages (JSPs) for view templating, and the application appears to be running on IBM WebSphere. Not only this, but we have line numbers for each stack frame, and the CSS appears to be some form of default styling.

Large institutions such as AmEx tend to not upgrade to the latest versions of development libraries until a future interal release cycle where the new version(s) can be internally vetted, tested, QA’d and put through a due diligence process. (Or for older apps, dependencies might not be upgraded at all.) This makes stack traces extremely valuable to an attacker. Why? Because I can obtain all historical versions of technologies such as WebSphere, and map every specific call in the stack trace back to the code until I find the software versions where the line numbers match up. The attacker should at least be able to narrow down the software version numbers to a range, if not the specific release version. After gathering this information, the attacker may choose to fixate on the older packages, and consult databases of vulnerabilities in legacy software until a vulnerability is discovered.

Long story short: turn off the rendering of stack traces in public-facing production applications. They’re convenient for validation of production errors, yes, but you don’t know what they may reveal.