With the announcements of the Dell-EMC merge and OpenText buying Documentum, it looks like this slipped through. EMC/DellEMC ECD/OpenText released the WebTop with the new content transfer mechanism.

Release notes: https://support.emc.com/docu78653_Documentum-Webtop-6.8.2-Release-Notes.pdf

Webtop Patch notes state that problems with esapi have been fixed:

WEBTOP-32460
Opening a document with accent in name using http mode with Webtop 6.8 using IE11 results in a security exception and the contents of the file are not displayed.

Well, if you check HttpTransportManager.class you can check the “fix” by yourself:

Prepatch:

    try{
       contentDisposition.append(SECURITY.validator().getValidHeader("HttpTransportManager", makeSafeHeaderValue(strCleanFileName), "Header Manipulation"));
    }catch (UnsupportedEncodingException e){
        throw new WrapperRuntimeException(e);
    }

Patch “fix”:

    try{
         ClientInfo localClientInfo = ClientInfoService.getInfo();
         if ((localClientInfo.isPlatform(ClientInfo.WIN)) && (localClientInfo.isBrowser(ClientInfo.MSIE))) {
            contentDisposition.append(makeSafeHeaderValue(str));
          } else {
            contentDisposition.append(SECURITY.validator().getValidHeader("HttpTransportManager", makeSafeHeaderValue(str), "Header Manipulation"));
          }
    }catch (UnsupportedEncodingException localUnsupportedEncodingException){
          throw new WrapperRuntimeException(localUnsupportedEncodingException);
    }

This is obviously a lazy way to “fix” it, as the problem is in the ESAPI library and skipping the security check is not really a way to fix it.

There are basically 3 problems with the character enconding in wdk:

1. The way WDK handles characters: This is a minor problem, as even if WDK does weird things such as converting back and forth the charecters, it “works”
2. ESAPI is not character encoding aware, this makes useless adding your “special” characters to the ESAPI.properties validators, as this would be read incorrectly by ESAPI (more info here: Virtuallinks vs character encoding vs ESAPI)
3. Inconsistencies when WDK calls ESAPI security validators as it sends the string to validate in different encodings (escaped/non-escaped, UTF-8 “native”, UTF-8 “converted”, etc.).

So, in order to fix this problems we need to fix the ESAPI library to be aware of the character encoding:

1. Get ESAPI 2.1.0 sources (Downloads – owasp-esapi-java – OWASP Enterprise Security API (Java Edition) – Google Project Hosting)

2. Add the following entries to webtop/WEB-INF/ESAPI.properties:

Validator.InputEncoding=UTF-8 //App server encoding
Validator.OutputEncoding=ISO-8859-1 //Locale encoding

3. Modify Validator.HTTPHeaderValue, Validator.FileName and Validator.DirectoryName with your special characters:

Validator.HTTPHeaderValue=^[a-zA-Z0-9<strong>áéíóúÁÉÍÓÚñÑ</strong>()\\-=\\*\\.\\?;,+\\/:&amp;_ ]*$
Validator.FileName=^[a-zA-Z0-9<strong>áéíóúÁÉÍÓÚñÑ</strong>!@#$%^&amp;{}\\[\\]()_+\\-=,.~'` ]{1,255}$
Validator.DirectoryName=^[a-zA-Z0-9<strong>áéíóúÁÉÍÓÚñÑ</strong>:/\\\\!@#$%^&amp;{}\\[\\]()_+\\-=,.~'` ]{1,255}$

4. Modify Validator.WDKHTTPURI adding a blank (because EMC assumes that nobody uses blanks when naming a folder):

Validator.WDKHTTPURI=^/([a-zA-Z0-9. \\-_]*/?)$

5. Add the following lines to org.owasp.esapi.reference.DefaultSecurityConfiguration

    private String inputEncoding = null;
    private String outputEncoding = null;  

    public String getInputEncoding() {
        return getESAPIProperty(INPUT_ENCODING, "UTF-8"); //UTF-8 is the default value returned if Validator.InputEncoding is not found
    }  

    public String getOutputEncoding() {
        return getESAPIProperty(OUTPUT_ENCODING, "ISO-8859-1"); //ISO-8859-1 is the default value returned if Validator.OutputEncoding is not found
    }  

    private String getEncodedESAPIProperty(String key){
        try {
            if (inputEncoding!=null && outputEncoding!=null){
                return new String(((String)properties.get(key)).getBytes(outputEncoding), inputEncoding);
            }else{
                return (String)properties.get(key);
            }
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
        return key;
    }

6. Modify DefaultSecurityConfiguration(), getESAPIProperty(String key, String def) and getESAPIProperty(String key, boolean def) methods:

    public DefaultSecurityConfiguration() {
        // load security configuration
        try {
            loadConfiguration();
            this.setCipherXProperties();  

            //deal with encoding
            inputEncoding=getInputEncoding();
            outputEncoding=getOutputEncoding();  

        } catch( IOException e ) {
          logSpecial("Failed to load security configuration", e );
          throw new ConfigurationException("Failed to load security configuration", e);
        }
    }  

    protected String getESAPIProperty( String key, String def ) {
        //String value = properties.getProperty(key);
        String value=getEncodedESAPIProperty(key);
        if ( value == null ) {
              logSpecial( "SecurityConfiguration for " + key + " not found in ESAPI.properties. Using default: " + def, null );
              return def;
        }
        return value;
    }  

    protected boolean getESAPIProperty( String key, boolean def ) {
        //String property = properties.getProperty(key);
        String property=getEncodedESAPIProperty(key);
        if ( property == null ) {
              logSpecial( "SecurityConfiguration for " + key + " not found in ESAPI.properties. Using default: " + def, null );
              return def;
        }
        if ( property.equalsIgnoreCase("true") || property.equalsIgnoreCase("yes" ) ) {
            return true;
        }
        if ( property.equalsIgnoreCase("false") || property.equalsIgnoreCase( "no" ) ) {
            return false;
        }
        logSpecial( "SecurityConfiguration for " + key + " not either \"true\" or \"false\" in ESAPI.properties. Using default: " + def, null );
        return def;
    }

7. Now that you are modifying this class, you can comment the logSpecial calls from loadPropertiesFromStream,loadConfiguration and getResourceFile and skip the messages thrown by ESAPI when loading the libraries (Best Practices – Review before releasing)

8. Modify org.owasp.esapi.reference.validation.StringValidationRule:

private String checkWhitelist(String context, String input, String orig) throws ValidationException{
  // check whitelist patterns
  //deal with encoding
  DefaultSecurityConfiguration sec=(DefaultSecurityConfiguration)DefaultSecurityConfiguration.getInstance();  

  Charset inputcharset = Charset.forName(sec.getInputEncoding());
  Charset outputcharset = Charset.forName(sec.getOutputEncoding());  

  ByteBuffer inputBuffer=null;
  try {
      inputBuffer = ByteBuffer.wrap(URLDecoder.decode(input,sec.getOutputEncoding()).getBytes());
  } catch (UnsupportedEncodingException e) {
      e.printStackTrace();
  }  

  CharBuffer data = inputcharset.decode(inputBuffer);  

  ByteBuffer outputBuffer = outputcharset.encode(data);
  byte[] outputData = outputBuffer.array();
  input=new String(outputData);  

    for (Pattern p : whitelistPatterns) {
        if ( !p.matcher(input).matches() ) {
            throw new ValidationException( context + ": Invalid input. Please conform to regex " + p.pattern() + ( maxLength == Integer.MAX_VALUE ? "" : " with a maximum length of " + maxLength ), "Invalid input: context=" + context + ", type(" + getTypeName() + ")=" + p.pattern() + ", input=" + input + (NullSafe.equals(orig,input) ? "" : ", orig=" + orig), context );
        }
    }  

    return input;
}

9. Generate jar file or replace the classes in the bundled esapi.jar.

This changes will work for virtual links and http transfer, however you may need to modify additional methods/validators depending on your customizations or case uses, but you get the idea.

This “patch” works with webtop 6.8 latest patch in every browser I’ve tested (ie, firefox, chrome, opera).

One has to wonder what is EMC thinking when they take some decisions: In Webtop 6.8 EMC introduced a “new” library in order to improve protection against XSS vulnerabilities. They decided to use OWASP’s ESAPI library, a project that hasn’t been updated in over 2 years (check ESAPI/esapi-java-legacy · GitHub and ESAPI/esapi-java · GitHub) and doesn’t show any sign of being currently developed (quite the contrary: Off-the-Wall Security)

Anyway, with the introduction of this library they have broken the virtuallinks component, at least for everyone that uses non ASCII characters. To sum up what has been discussed in Webtop 6.8 Download Servlet Escaped Umlaut Problem:

In a “standard” environment:

• Tomcat running with -Dfile.encoding=UTF-8
• Default ESAPI.properties
• Webtop version: 6.8 SP6

Any “weird” character (in spanish: áéíóúñ) will break the virtuallinks functionality in two cases:

1. The user is logged in webtop:
VirtualLink404Handler.getServletErrorRequestUri method has an url conversion

virtualLinkPath = URLDecoder.decode(virtualLinkPath, "UTF-8");

that has the following behaviour (in Chrome and Firefox, IE seems to work just fine):

BEFORE URLDecoder: /folder%20name/%C3%A1%C3%A9%C3%AD%C3%B3%C3%BA
AFTER URLDecoder: /folder name/áéíóú
BEFORE URLDecoder: /folder%20name/%C3%A1%C3%A9%C3%AD%C3%B3%C3%BA
AFTER URLDecoder: /folder name/áéíóú
BEFORE URLDecoder: /folder%20name/%E1%E9%ED%F3%FA
AFTER URLDecoder: /folder name/?????

that “????” string will return “?????” or a sequence of square icons or simply a blank depending of the character encoding. In order to fix it, you need to rewrite the class changing that method to something like:

    String vLink=URLDecoder.decode(virtualLinkPath, "UTF-8");
    if (vLink.indexOf("?")!=-1){//broken UTF-8 encoding
        virtualLinkPath = URLDecoder.decode(virtualLinkPath, "ISO-8859-1");
    }else{
        virtualLinkPath = URLDecoder.decode(virtualLinkPath, "UTF-8");
    }

This will make virtuallinks that have special characters to be correctly opened from any browser.

2. The user hasn’t logged in webtop:
Virtuallink component will redirect to the login component (check url-addressable-components & virtuallinkconnect first) and then you’ll get a popup saying

“There’s been an error. Contact with your administrator”

without any additional info or stacktrace in the logs.

What is happening is that ESAPI is forbidding the redirection to the login compoenent due to the presence of “unsafe” characters as you can see in a the following phantom exception:

HTTP header value: Header Manipulation: FormProcessor: Invalid input. Please conform to regex ^[a-zA-Z0-9()\-=\*\.\?;,+\/:&_ ]*$ with a maximum length of 1024

Obviously, your next step would be adding the special characters to the Validator.HTTPHeaderValue regular expression in ESAPI.properties:

Validator.HTTPHeaderValue=^[%a-zA-Z0-9áéíóúÁÉÍÓÚñÑ()\\-=\\*\\.\\?;,+\\/:&_ ]*$

Restart the server, retry the virtuallink url and:

HTTP header value: Header Manipulation: FormProcessor: Invalid input. Please conform to regex ^[%a-zA-Z0-9áéÃóúÁÉÍÓÚñÑ()\-=\*\.\?;,+\/:&_ ]*$ with a maximum length of 1024

WTF!!?? ESAPI reads the properties file as a stream and doesn’t care about the encoding

You can fix this by overwriting WDKESAPIValidator.getValidHeader (check Re: Webtop 6.8 Download Servlet Escaped Umlaut Problem) or you can do it in a nicer way (considering it is unlikely that the ESAPI jar will be ever updated):

1. Download ESAPI 2.1.0 source
2. Modify org\owasp\esapi\reference\DefaultSecurityConfiguration.java:

protected String getESAPIProperty( String key, String def ) {
  String value = properties.getProperty(key);

with

    String value=null;
    try {
      value = new String(((String)properties.get(key)).getBytes("ISO-8859-1"), "UTF-8");
    } catch (UnsupportedEncodingException e) {
      e.printStackTrace();
    }

And

protected boolean getESAPIProperty( String key, boolean def ) {
  String property = properties.getProperty(key);

with

    String property=null;
    try {
      property = new String(((String)properties.get(key)).getBytes("ISO-8859-1"), "UTF-8");
    } catch (UnsupportedEncodingException e) {
      e.printStackTrace();
    }

3. Replace the class in webtop/WEB-INF/lib/esapi-2.1.0.jar

And now your special characters in ESAPI.properties will be read correctly and you can use again the virtuallink component.