Virtuallinks vs character encoding vs ESAPI

One has to wonder what is EMC thinking when they take some decisions: In Webtop 6.8 EMC introduced a “new” library in order to improve protection against XSS vulnerabilities. They decided to use OWASP’s ESAPI library, a project that hasn’t been updated in over 2 years (check ESAPI/esapi-java-legacy · GitHub and ESAPI/esapi-java · GitHub) and doesn’t show any sign of being currently developed (quite the contrary: Off-the-Wall Security)

Anyway, with the introduction of this library they have broken the virtuallinks component, at least for everyone that uses non ASCII characters. To sum up what has been discussed in Webtop 6.8 Download Servlet Escaped Umlaut Problem:

In a “standard” environment:

  • Tomcat running with -Dfile.encoding=UTF-8
  • Default ESAPI.properties
  • Webtop version: 6.8 SP6

Any “weird” character (in spanish: áéíóúñ) will break the virtuallinks functionality in two cases:

1. The user is logged in webtop:
VirtualLink404Handler.getServletErrorRequestUri method has an url conversion

virtualLinkPath = URLDecoder.decode(virtualLinkPath, "UTF-8");

that has the following behaviour (in Chrome and Firefox, IE seems to work just fine):

BEFORE URLDecoder: /folder%20name/%C3%A1%C3%A9%C3%AD%C3%B3%C3%BA
AFTER URLDecoder: /folder name/áéíóú
BEFORE URLDecoder: /folder%20name/%C3%A1%C3%A9%C3%AD%C3%B3%C3%BA
AFTER URLDecoder: /folder name/áéíóú
BEFORE URLDecoder: /folder%20name/%E1%E9%ED%F3%FA
AFTER URLDecoder: /folder name/?????

that “????” string will return “?????” or a sequence of square icons or simply a blank depending of the character encoding. In order to fix it, you need to rewrite the class changing that method to something like:

    String vLink=URLDecoder.decode(virtualLinkPath, "UTF-8");
    if (vLink.indexOf("?")!=-1){//broken UTF-8 encoding
        virtualLinkPath = URLDecoder.decode(virtualLinkPath, "ISO-8859-1");
    }else{
        virtualLinkPath = URLDecoder.decode(virtualLinkPath, "UTF-8");
    }

This will make virtuallinks that have special characters to be correctly opened from any browser.

2. The user hasn’t logged in webtop:
Virtuallink component will redirect to the login component (check url-addressable-components & virtuallinkconnect first) and then you’ll get a popup saying

“There’s been an error. Contact with your administrator”

without any additional info or stacktrace in the logs.

What is happening is that ESAPI is forbidding the redirection to the login compoenent due to the presence of “unsafe” characters as you can see in a the following phantom exception:

HTTP header value: Header Manipulation: FormProcessor: Invalid input. Please conform to regex ^[a-zA-Z0-9()\-=\*\.\?;,+\/:&_ ]*$ with a maximum length of 1024

Obviously, your next step would be adding the special characters to the Validator.HTTPHeaderValue regular expression in ESAPI.properties:

Validator.HTTPHeaderValue=^[%a-zA-Z0-9áéíóúÁÉÍÓÚñÑ()\\-=\\*\\.\\?;,+\\/:&_ ]*$

Restart the server, retry the virtuallink url and:

HTTP header value: Header Manipulation: FormProcessor: Invalid input. Please conform to regex ^[%a-zA-Z0-9áéÃóúÁÉÍÓÚñÑ()\-=\*\.\?;,+\/:&_ ]*$ with a maximum length of 1024

WTF!!?? ESAPI reads the properties file as a stream and doesn’t care about the encoding

You can fix this by overwriting WDKESAPIValidator.getValidHeader (check Re: Webtop 6.8 Download Servlet Escaped Umlaut Problem) or you can do it in a nicer way (considering it is unlikely that the ESAPI jar will be ever updated):

1. Download ESAPI 2.1.0 source
2. Modify org\owasp\esapi\reference\DefaultSecurityConfiguration.java:

protected String getESAPIProperty( String key, String def ) {
  String value = properties.getProperty(key);

with

    String value=null;
    try {
      value = new String(((String)properties.get(key)).getBytes("ISO-8859-1"), "UTF-8");
    } catch (UnsupportedEncodingException e) {
      e.printStackTrace();
    }

And

protected boolean getESAPIProperty( String key, boolean def ) {
  String property = properties.getProperty(key);

with

    String property=null;
    try {
      property = new String(((String)properties.get(key)).getBytes("ISO-8859-1"), "UTF-8");
    } catch (UnsupportedEncodingException e) {
      e.printStackTrace();
    }

3. Replace the class in webtop/WEB-INF/lib/esapi-2.1.0.jar

And now your special characters in ESAPI.properties will be read correctly and you can use again the virtuallink component.

2 thoughts on “Virtuallinks vs character encoding vs ESAPI

  1. Admitted, particularly in todays day and age, the ESAPI library is questionable, if not simply outdated.

    However on loading the ESAPI.properties file, that is done by the library by resorting to the java runtime library which defined the *.properties file format to be only ISO 8859-1 encoded:
    https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html?noframes=true

    “The load(InputStream) / store(OutputStream, String)methods work the same way as the load(Reader)/store(Writer, String) pair, except the input/output stream is encoded in ISO 8859-1 character encoding.
    Characters that cannot be directly represented in this encoding can be written using Unicode escapes as defined in section 3.3 of The Java™ Language Specification; only a single ‘u’ character is allowed in an escape sequence.”

    Thus i should work if in the ESAPI.properties the pattern was defined this way:
    Validator.HTTPHeaderValue=^[%a-zA-Z0-9u\000Au\00E1u\00E9u\00EDu\00F3u\00FAu\00F1()\\-=\\*\\.\\?;,+\\/:&_ ]*$

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.