ESAPI vs character codification

Webtop Patch notes state that problems with esapi have been fixed:

WEBTOP-32460
Opening a document with accent in name using http mode with Webtop 6.8 using IE11 results in a security exception and the contents of the file are not displayed.

Well, if you check HttpTransportManager.class you can check the “fix” by yourself:

Prepatch:

    try{
       contentDisposition.append(SECURITY.validator().getValidHeader("HttpTransportManager", makeSafeHeaderValue(strCleanFileName), "Header Manipulation"));
    }catch (UnsupportedEncodingException e){
        throw new WrapperRuntimeException(e);
    }

Patch “fix”:

    try{
         ClientInfo localClientInfo = ClientInfoService.getInfo();
         if ((localClientInfo.isPlatform(ClientInfo.WIN)) && (localClientInfo.isBrowser(ClientInfo.MSIE))) {
            contentDisposition.append(makeSafeHeaderValue(str));
          } else {
            contentDisposition.append(SECURITY.validator().getValidHeader("HttpTransportManager", makeSafeHeaderValue(str), "Header Manipulation"));
          }
    }catch (UnsupportedEncodingException localUnsupportedEncodingException){
          throw new WrapperRuntimeException(localUnsupportedEncodingException);
    }

This is obviously a lazy way to “fix” it, as the problem is in the ESAPI library and skipping the security check is not really a way to fix it.

There are basically 3 problems with the character enconding in wdk:

  1. The way WDK handles characters: This is a minor problem, as even if WDK does weird things such as converting back and forth the charecters, it “works”
  2. ESAPI is not character encoding aware, this makes useless adding your “special” characters to the ESAPI.properties validators, as this would be read incorrectly by ESAPI (more info here: Virtuallinks vs character encoding vs ESAPI)
  3. Inconsistencies when WDK calls ESAPI security validators as it sends the string to validate in different encodings (escaped/non-escaped, UTF-8 “native”, UTF-8 “converted”, etc.).

So, in order to fix this problems we need to fix the ESAPI library to be aware of the character encoding:

1. Get ESAPI 2.1.0 sources (Downloads – owasp-esapi-java – OWASP Enterprise Security API (Java Edition) – Google Project Hosting)

2. Add the following entries to webtop/WEB-INF/ESAPI.properties:

Validator.InputEncoding=UTF-8 //App server encoding
Validator.OutputEncoding=ISO-8859-1 //Locale encoding

3. Modify Validator.HTTPHeaderValue, Validator.FileName and Validator.DirectoryName with your special characters:

Validator.HTTPHeaderValue=^[a-zA-Z0-9<strong>áéíóúÁÉÍÓÚñÑ</strong>()\\-=\\*\\.\\?;,+\\/:&amp;_ ]*$
Validator.FileName=^[a-zA-Z0-9<strong>áéíóúÁÉÍÓÚñÑ</strong>!@#$%^&amp;{}\\[\\]()_+\\-=,.~'` ]{1,255}$
Validator.DirectoryName=^[a-zA-Z0-9<strong>áéíóúÁÉÍÓÚñÑ</strong>:/\\\\!@#$%^&amp;{}\\[\\]()_+\\-=,.~'` ]{1,255}$

4. Modify Validator.WDKHTTPURI adding a blank (because EMC assumes that nobody uses blanks when naming a folder):

Validator.WDKHTTPURI=^/([a-zA-Z0-9. \\-_]*/?)$

5. Add the following lines to org.owasp.esapi.reference.DefaultSecurityConfiguration

    private String inputEncoding = null;
    private String outputEncoding = null;  

    public String getInputEncoding() {
        return getESAPIProperty(INPUT_ENCODING, "UTF-8"); //UTF-8 is the default value returned if Validator.InputEncoding is not found
    }  

    public String getOutputEncoding() {
        return getESAPIProperty(OUTPUT_ENCODING, "ISO-8859-1"); //ISO-8859-1 is the default value returned if Validator.OutputEncoding is not found
    }  

    private String getEncodedESAPIProperty(String key){
        try {
            if (inputEncoding!=null && outputEncoding!=null){
                return new String(((String)properties.get(key)).getBytes(outputEncoding), inputEncoding);
            }else{
                return (String)properties.get(key);
            }
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
        return key;
    }

6. Modify DefaultSecurityConfiguration(), getESAPIProperty(String key, String def) and getESAPIProperty(String key, boolean def) methods:

    public DefaultSecurityConfiguration() {
        // load security configuration
        try {
            loadConfiguration();
            this.setCipherXProperties();  

            //deal with encoding
            inputEncoding=getInputEncoding();
            outputEncoding=getOutputEncoding();  

        } catch( IOException e ) {
          logSpecial("Failed to load security configuration", e );
          throw new ConfigurationException("Failed to load security configuration", e);
        }
    }  

    protected String getESAPIProperty( String key, String def ) {
        //String value = properties.getProperty(key);
        String value=getEncodedESAPIProperty(key);
        if ( value == null ) {
              logSpecial( "SecurityConfiguration for " + key + " not found in ESAPI.properties. Using default: " + def, null );
              return def;
        }
        return value;
    }  

    protected boolean getESAPIProperty( String key, boolean def ) {
        //String property = properties.getProperty(key);
        String property=getEncodedESAPIProperty(key);
        if ( property == null ) {
              logSpecial( "SecurityConfiguration for " + key + " not found in ESAPI.properties. Using default: " + def, null );
              return def;
        }
        if ( property.equalsIgnoreCase("true") || property.equalsIgnoreCase("yes" ) ) {
            return true;
        }
        if ( property.equalsIgnoreCase("false") || property.equalsIgnoreCase( "no" ) ) {
            return false;
        }
        logSpecial( "SecurityConfiguration for " + key + " not either \"true\" or \"false\" in ESAPI.properties. Using default: " + def, null );
        return def;
    }

7. Now that you are modifying this class, you can comment the logSpecial calls from loadPropertiesFromStream,loadConfiguration and getResourceFile and skip the messages thrown by ESAPI when loading the libraries (Best Practices – Review before releasing)

8. Modify org.owasp.esapi.reference.validation.StringValidationRule:

private String checkWhitelist(String context, String input, String orig) throws ValidationException{
  // check whitelist patterns
  //deal with encoding
  DefaultSecurityConfiguration sec=(DefaultSecurityConfiguration)DefaultSecurityConfiguration.getInstance();  

  Charset inputcharset = Charset.forName(sec.getInputEncoding());
  Charset outputcharset = Charset.forName(sec.getOutputEncoding());  

  ByteBuffer inputBuffer=null;
  try {
      inputBuffer = ByteBuffer.wrap(URLDecoder.decode(input,sec.getOutputEncoding()).getBytes());
  } catch (UnsupportedEncodingException e) {
      e.printStackTrace();
  }  

  CharBuffer data = inputcharset.decode(inputBuffer);  

  ByteBuffer outputBuffer = outputcharset.encode(data);
  byte[] outputData = outputBuffer.array();
  input=new String(outputData);  

    for (Pattern p : whitelistPatterns) {
        if ( !p.matcher(input).matches() ) {
            throw new ValidationException( context + ": Invalid input. Please conform to regex " + p.pattern() + ( maxLength == Integer.MAX_VALUE ? "" : " with a maximum length of " + maxLength ), "Invalid input: context=" + context + ", type(" + getTypeName() + ")=" + p.pattern() + ", input=" + input + (NullSafe.equals(orig,input) ? "" : ", orig=" + orig), context );
        }
    }  

    return input;
}

9. Generate jar file or replace the classes in the bundled esapi.jar.

This changes will work for virtual links and http transfer, however you may need to modify additional methods/validators depending on your customizations or case uses, but you get the idea.

This “patch” works with webtop 6.8 latest patch in every browser I’ve tested (ie, firefox, chrome, opera).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s