What I expect for Documentum from ECD/OpenText

What I think OpenText should keep:

  • Support Community
  • ContentHub
  • EMC Support (at least the search engine with the knowledge base)
  • Download Center (After so many changes, I really don’t want to change again :P)
  • EMC GitHub
  • Webtop (It’s the only client that works OOTB)

What I think OpenText should change:

  • Sales policy (Stop pushing products that customers don’t need, and yes, that means D2 and xCP for customers that simply want a “library”)
  • Sort libraries/dependencies in the Documentum stack (if 7.x is released, make it that every product has the same version of every library). Rebrand products accordingly (yeah, Webtop 7.x if it is using DFC 7.x)
  • Detailed changelog of the products (don’t force us to go through every single customization to check if some component we’re extending has changed and broke the customizations)
  • Kill lockbox with fire (or at least, make it optional for both CS and D2). Nobody understands it, nobody knows how to set it up, so it is more a pain in the ass than a security feature
  • Oh, and remove D2 dependency on JMS

I’ve lost any hope to see dmbasic gone for good, or JBOSS replaced with something “lighter”, that’s why those didn’t make it into the list (as probably many others, that I don’t remember now)

(And yes, I’m not a huge fan of D2 :P)

WebTop’s new content transfer mechanism

Yesterday I saw Webtop 6.8.2 was released, and as this was great news because this version was supposed to remove Java applets once and for all. While most of us though that removing applets would imply moving to HTML5, EMC has decided to implement an intermediate solution 😦

Webtop 6.8.2 will prompt users to install a browser extension first time you log in:

webtop-warning

In Firefox the extension will be installed, and in Chrome user will be redirected to the Chrome Web Store:

webtop-chrome-extension

After installing the extension, the first time users try to download something they’ll be prompted to install a local application that will handle the transfers (WebSockets I presume).

Good news: Now you don’t need to fight with Java/browser combination

Bad news: Users have to perform operations. Not the cleanest solution (IMHO).

Bonus: Now importing files is really not-user friendly:

  1. File -> import
  2. Ugly Java window will pop-up over the import container asking for files to import
  3. Select files/folders, press ok and… nothing happens. You are kept in a blank container window with “Next” and “Finish” buttons and no indication whatsoever if the files are been/have been uploaded or what is going on.

ESAPI vs character codification

Webtop Patch notes state that problems with esapi have been fixed:

WEBTOP-32460
Opening a document with accent in name using http mode with Webtop 6.8 using IE11 results in a security exception and the contents of the file are not displayed.

Well, if you check HttpTransportManager.class you can check the “fix” by yourself:

Prepatch:

    try{
       contentDisposition.append(SECURITY.validator().getValidHeader("HttpTransportManager", makeSafeHeaderValue(strCleanFileName), "Header Manipulation"));
    }catch (UnsupportedEncodingException e){
        throw new WrapperRuntimeException(e);
    }

Patch “fix”:

    try{
         ClientInfo localClientInfo = ClientInfoService.getInfo();
         if ((localClientInfo.isPlatform(ClientInfo.WIN)) && (localClientInfo.isBrowser(ClientInfo.MSIE))) {
            contentDisposition.append(makeSafeHeaderValue(str));
          } else {
            contentDisposition.append(SECURITY.validator().getValidHeader("HttpTransportManager", makeSafeHeaderValue(str), "Header Manipulation"));
          }
    }catch (UnsupportedEncodingException localUnsupportedEncodingException){
          throw new WrapperRuntimeException(localUnsupportedEncodingException);
    }

This is obviously a lazy way to “fix” it, as the problem is in the ESAPI library and skipping the security check is not really a way to fix it.

There are basically 3 problems with the character enconding in wdk:

  1. The way WDK handles characters: This is a minor problem, as even if WDK does weird things such as converting back and forth the charecters, it “works”
  2. ESAPI is not character encoding aware, this makes useless adding your “special” characters to the ESAPI.properties validators, as this would be read incorrectly by ESAPI (more info here: Virtuallinks vs character encoding vs ESAPI)
  3. Inconsistencies when WDK calls ESAPI security validators as it sends the string to validate in different encodings (escaped/non-escaped, UTF-8 “native”, UTF-8 “converted”, etc.).

So, in order to fix this problems we need to fix the ESAPI library to be aware of the character encoding:

1. Get ESAPI 2.1.0 sources (Downloads – owasp-esapi-java – OWASP Enterprise Security API (Java Edition) – Google Project Hosting)

2. Add the following entries to webtop/WEB-INF/ESAPI.properties:

Validator.InputEncoding=UTF-8 //App server encoding
Validator.OutputEncoding=ISO-8859-1 //Locale encoding

3. Modify Validator.HTTPHeaderValue, Validator.FileName and Validator.DirectoryName with your special characters:

Validator.HTTPHeaderValue=^[a-zA-Z0-9<strong>áéíóúÁÉÍÓÚñÑ</strong>()\\-=\\*\\.\\?;,+\\/:&amp;_ ]*$
Validator.FileName=^[a-zA-Z0-9<strong>áéíóúÁÉÍÓÚñÑ</strong>!@#$%^&amp;{}\\[\\]()_+\\-=,.~'` ]{1,255}$
Validator.DirectoryName=^[a-zA-Z0-9<strong>áéíóúÁÉÍÓÚñÑ</strong>:/\\\\!@#$%^&amp;{}\\[\\]()_+\\-=,.~'` ]{1,255}$

4. Modify Validator.WDKHTTPURI adding a blank (because EMC assumes that nobody uses blanks when naming a folder):

Validator.WDKHTTPURI=^/([a-zA-Z0-9. \\-_]*/?)$

5. Add the following lines to org.owasp.esapi.reference.DefaultSecurityConfiguration

    private String inputEncoding = null;
    private String outputEncoding = null;  

    public String getInputEncoding() {
        return getESAPIProperty(INPUT_ENCODING, "UTF-8"); //UTF-8 is the default value returned if Validator.InputEncoding is not found
    }  

    public String getOutputEncoding() {
        return getESAPIProperty(OUTPUT_ENCODING, "ISO-8859-1"); //ISO-8859-1 is the default value returned if Validator.OutputEncoding is not found
    }  

    private String getEncodedESAPIProperty(String key){
        try {
            if (inputEncoding!=null && outputEncoding!=null){
                return new String(((String)properties.get(key)).getBytes(outputEncoding), inputEncoding);
            }else{
                return (String)properties.get(key);
            }
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
        return key;
    }

6. Modify DefaultSecurityConfiguration(), getESAPIProperty(String key, String def) and getESAPIProperty(String key, boolean def) methods:

    public DefaultSecurityConfiguration() {
        // load security configuration
        try {
            loadConfiguration();
            this.setCipherXProperties();  

            //deal with encoding
            inputEncoding=getInputEncoding();
            outputEncoding=getOutputEncoding();  

        } catch( IOException e ) {
          logSpecial("Failed to load security configuration", e );
          throw new ConfigurationException("Failed to load security configuration", e);
        }
    }  

    protected String getESAPIProperty( String key, String def ) {
        //String value = properties.getProperty(key);
        String value=getEncodedESAPIProperty(key);
        if ( value == null ) {
              logSpecial( "SecurityConfiguration for " + key + " not found in ESAPI.properties. Using default: " + def, null );
              return def;
        }
        return value;
    }  

    protected boolean getESAPIProperty( String key, boolean def ) {
        //String property = properties.getProperty(key);
        String property=getEncodedESAPIProperty(key);
        if ( property == null ) {
              logSpecial( "SecurityConfiguration for " + key + " not found in ESAPI.properties. Using default: " + def, null );
              return def;
        }
        if ( property.equalsIgnoreCase("true") || property.equalsIgnoreCase("yes" ) ) {
            return true;
        }
        if ( property.equalsIgnoreCase("false") || property.equalsIgnoreCase( "no" ) ) {
            return false;
        }
        logSpecial( "SecurityConfiguration for " + key + " not either \"true\" or \"false\" in ESAPI.properties. Using default: " + def, null );
        return def;
    }

7. Now that you are modifying this class, you can comment the logSpecial calls from loadPropertiesFromStream,loadConfiguration and getResourceFile and skip the messages thrown by ESAPI when loading the libraries (Best Practices – Review before releasing)

8. Modify org.owasp.esapi.reference.validation.StringValidationRule:

private String checkWhitelist(String context, String input, String orig) throws ValidationException{
  // check whitelist patterns
  //deal with encoding
  DefaultSecurityConfiguration sec=(DefaultSecurityConfiguration)DefaultSecurityConfiguration.getInstance();  

  Charset inputcharset = Charset.forName(sec.getInputEncoding());
  Charset outputcharset = Charset.forName(sec.getOutputEncoding());  

  ByteBuffer inputBuffer=null;
  try {
      inputBuffer = ByteBuffer.wrap(URLDecoder.decode(input,sec.getOutputEncoding()).getBytes());
  } catch (UnsupportedEncodingException e) {
      e.printStackTrace();
  }  

  CharBuffer data = inputcharset.decode(inputBuffer);  

  ByteBuffer outputBuffer = outputcharset.encode(data);
  byte[] outputData = outputBuffer.array();
  input=new String(outputData);  

    for (Pattern p : whitelistPatterns) {
        if ( !p.matcher(input).matches() ) {
            throw new ValidationException( context + ": Invalid input. Please conform to regex " + p.pattern() + ( maxLength == Integer.MAX_VALUE ? "" : " with a maximum length of " + maxLength ), "Invalid input: context=" + context + ", type(" + getTypeName() + ")=" + p.pattern() + ", input=" + input + (NullSafe.equals(orig,input) ? "" : ", orig=" + orig), context );
        }
    }  

    return input;
}

9. Generate jar file or replace the classes in the bundled esapi.jar.

This changes will work for virtual links and http transfer, however you may need to modify additional methods/validators depending on your customizations or case uses, but you get the idea.

This “patch” works with webtop 6.8 latest patch in every browser I’ve tested (ie, firefox, chrome, opera).

Virtuallinks vs character encoding vs ESAPI

One has to wonder what is EMC thinking when they take some decisions: In Webtop 6.8 EMC introduced a “new” library in order to improve protection against XSS vulnerabilities. They decided to use OWASP’s ESAPI library, a project that hasn’t been updated in over 2 years (check ESAPI/esapi-java-legacy · GitHub and ESAPI/esapi-java · GitHub) and doesn’t show any sign of being currently developed (quite the contrary: Off-the-Wall Security)

Anyway, with the introduction of this library they have broken the virtuallinks component, at least for everyone that uses non ASCII characters. To sum up what has been discussed in Webtop 6.8 Download Servlet Escaped Umlaut Problem:

In a “standard” environment:

  • Tomcat running with -Dfile.encoding=UTF-8
  • Default ESAPI.properties
  • Webtop version: 6.8 SP6

Any “weird” character (in spanish: áéíóúñ) will break the virtuallinks functionality in two cases:

1. The user is logged in webtop:
VirtualLink404Handler.getServletErrorRequestUri method has an url conversion

virtualLinkPath = URLDecoder.decode(virtualLinkPath, "UTF-8");

that has the following behaviour (in Chrome and Firefox, IE seems to work just fine):

BEFORE URLDecoder: /folder%20name/%C3%A1%C3%A9%C3%AD%C3%B3%C3%BA
AFTER URLDecoder: /folder name/áéíóú
BEFORE URLDecoder: /folder%20name/%C3%A1%C3%A9%C3%AD%C3%B3%C3%BA
AFTER URLDecoder: /folder name/áéíóú
BEFORE URLDecoder: /folder%20name/%E1%E9%ED%F3%FA
AFTER URLDecoder: /folder name/?????

that “????” string will return “?????” or a sequence of square icons or simply a blank depending of the character encoding. In order to fix it, you need to rewrite the class changing that method to something like:

    String vLink=URLDecoder.decode(virtualLinkPath, "UTF-8");
    if (vLink.indexOf("?")!=-1){//broken UTF-8 encoding
        virtualLinkPath = URLDecoder.decode(virtualLinkPath, "ISO-8859-1");
    }else{
        virtualLinkPath = URLDecoder.decode(virtualLinkPath, "UTF-8");
    }

This will make virtuallinks that have special characters to be correctly opened from any browser.

2. The user hasn’t logged in webtop:
Virtuallink component will redirect to the login component (check url-addressable-components & virtuallinkconnect first) and then you’ll get a popup saying

“There’s been an error. Contact with your administrator”

without any additional info or stacktrace in the logs.

What is happening is that ESAPI is forbidding the redirection to the login compoenent due to the presence of “unsafe” characters as you can see in a the following phantom exception:

HTTP header value: Header Manipulation: FormProcessor: Invalid input. Please conform to regex ^[a-zA-Z0-9()\-=\*\.\?;,+\/:&_ ]*$ with a maximum length of 1024

Obviously, your next step would be adding the special characters to the Validator.HTTPHeaderValue regular expression in ESAPI.properties:

Validator.HTTPHeaderValue=^[%a-zA-Z0-9áéíóúÁÉÍÓÚñÑ()\\-=\\*\\.\\?;,+\\/:&_ ]*$

Restart the server, retry the virtuallink url and:

HTTP header value: Header Manipulation: FormProcessor: Invalid input. Please conform to regex ^[%a-zA-Z0-9áéÃóúÁÉÍÓÚñÑ()\-=\*\.\?;,+\/:&_ ]*$ with a maximum length of 1024

WTF!!?? ESAPI reads the properties file as a stream and doesn’t care about the encoding

You can fix this by overwriting WDKESAPIValidator.getValidHeader (check Re: Webtop 6.8 Download Servlet Escaped Umlaut Problem) or you can do it in a nicer way (considering it is unlikely that the ESAPI jar will be ever updated):

1. Download ESAPI 2.1.0 source
2. Modify org\owasp\esapi\reference\DefaultSecurityConfiguration.java:

protected String getESAPIProperty( String key, String def ) {
  String value = properties.getProperty(key);

with

    String value=null;
    try {
      value = new String(((String)properties.get(key)).getBytes("ISO-8859-1"), "UTF-8");
    } catch (UnsupportedEncodingException e) {
      e.printStackTrace();
    }

And

protected boolean getESAPIProperty( String key, boolean def ) {
  String property = properties.getProperty(key);

with

    String property=null;
    try {
      property = new String(((String)properties.get(key)).getBytes("ISO-8859-1"), "UTF-8");
    } catch (UnsupportedEncodingException e) {
      e.printStackTrace();
    }

3. Replace the class in webtop/WEB-INF/lib/esapi-2.1.0.jar

And now your special characters in ESAPI.properties will be read correctly and you can use again the virtuallink component.

Video streaming from Webtop

We have stored in our repository some video tutorials to help users. Usually they double click on them, the UCF applet/http download launches and they’ll open them with some video player than may or may not have the rigth codecs to play the video.

So in order to improve the “user experience” we decided to try using some HTML5 video player to play the content from the browser.

In this case we are using MediaElements.js (mediaelementjs.com), but any player will do.

1. Create a hidden cabinet with mp4/webm videos.
2. Add a menu action to load the hidden cabinet in an objectlist (I’m simulating a click on the browsertree):

      <actionmenuitem dynamic="generic" name="videohelp"  
           nlsid="MSG_VIDEOHELP" action="videohelp_action"  
           showifinvalid="false" showifdisabled="true" />  
        @Override  
        public boolean execute(String paramString, IConfigElement paramIConfigElement,  
              ArgumentList paramArgumentList, Context paramContext, Component paramComponent,  
              Map paramMap) {  
            String pathIds=FolderUtil.getFolderIdsFromPath("/VIDEOS");  
          
            ArgumentList args=new ArgumentList();  
            args.add("objectId",pathIds);  
          
            ClientSessionState.setAttribute("customlocation", "/VIDEOS");  
          
            return ActionService.execute("view", args, paramContext, paramComponent, this);  
        }  

NOTE: I’m using a custom scope that allows to scope by folder, that’s why I’m setting the customlocation attribute.

3. Create a new objectlist component which:

  • Overrides the default action when double clicking a document so it launches the video player instead of opening.
  • Generates a virtual link for the video content.
    private String getVideoUrl(IDfDocument video) throws DfException, IOException{  
            String url="";  
      
            IDfFolder fold=(IDfFolder)getDfSession().getObject(video.getFolderId(0));      
            url=url+getPageContext().getRequest().getServerName()+":"  
                   +getPageContext().getRequest().getServerPort()  
                   +getPageContext().getServletContext().getContextPath();  
            url=url+fold.getFolderPath(0)+"/"+video.getObjectName();  
          
            return url;  
    }  

objectlist_videos.jsp:

    //add javascript to show/hide player in a cool way.  
    //Add the parameter passed in the click event to the src attribute of the video tag.  
  
       <dmf:form>  
          <video width="640" height="360" src="" type="video/mp4"  
            id="player1" controls="controls" preload="none"></video>  
          
         <%@ include file='/custom/jsp/objectlist/doclist_body_videos.jsp' %>  
      </dmf:form> 

doclist_body_videos.jsp:

        //modify object id celltemplate:  
      
         <dmf:datagridRowEvent eventname="dblclick">  
             <dmf:link onclick='show' name='objectLink'  
                    runatclient='true' datafield='title'>  
                    <dmf:argument name='id' datafield='r_object_id' />  
                    <dmf:argument name='type' datafield='r_object_type' />  
                    <dmf:argument name='isFolder' datafield='isfolder' />  
                    <dmf:argument name="isVirtualDoc" datafield='r_is_virtual_doc' />  
                    <dmf:argument name='assembledFromId' datafield='r_assembled_from_id' />  
                    <dmf:argument name="linkCount" datafield='r_link_cnt' />  
                    <% String url=form.getVideoUrl(dataProvider.getDataField("r_object_id")); %>  
                    <dmf:argument name="url" value="<%=url %>" />  
             </dmf:link>  

This works, but is not really streaming, as it first downloads the video, and then plays it. But (I never though I would say this) ACS comes to rescue!

ACS links allow content streaming, so instead of getting the virtual url link to the video, we’ll get the ACS url:

    try{  
        IDfClientX clientx = new DfClientX();  
        IDfDocument video=(IDfDocument)getDfSession().getObject(new DfId(objectId));  
      
        IDfAcsTransferPreferences atp = clientx.getAcsTransferPreferences();  
        atp.preferAcsTransfer(true);  
      
        IDfExportOperation exportOp = clientx.getExportOperation();  
        exportOp.setAcsTransferPreferences(atp);  
        IDfExportNode exportNode = (IDfExportNode) exportOp.add(video);  
        boolean result = exportOp.execute();  
      
        if (result) {  
            IDfList nodes = exportOp.getNodes();  
            for (int i = 0, size = nodes.getCount(); i < size; i++) {  
                IDfExportNode node = (IDfExportNode) nodes.get(i);  
                IDfEnumeration acsRequests = node.getAcsRequests();  
                while (acsRequests.hasMoreElements()) {  
                    IDfAcsRequest acsRequest = (IDfAcsRequest) acsRequests.nextElement();  
                    String docURL = acsRequest.makeURL();  
                    return docURL;                    
                }  
            }  
        }    
    }catch(DfException e){  
        e.printStackTrace();  
    }  

And the result:

videoplayer.png

Note: Fullscreen is disabled due to frameset architechture that Webtop uses makes html5 full screen work like: Put browser in full screen mode, play video in full frame (instead of full screen).

url-addressable-components & virtuallinkconnect

In recent Webtop versions EMC introduced a new secutiry feature that forbid components to be invoked through an URL:

The following configuration has been added to enable or disable components that are invoked through a URL.

<url-addressable-components>

<enabled>false</enabled>

</url-addressable-components>

Set <enabled> to true to invoke all the components through the URL.

Specific components can also be configuredforURLaddressability. This can be done by adding the <url-addressable-enabled/> tag in the corresponding component configuration file.

For example, add the following configuration file, <Web-app-root>\wdk\config\errormessage_component.xml, where <url-addressable-enabled/> enables URL addressability.

Note: EMC recommends that administrators add <url-addressable-enabled/> in the specific URL addressable-component instead of choosing to set <enabled> to true.

So when I’m getting a “401: The URL is unauthorized in WDK” when trying to use a virtual link I check app.xml:

<url-addressable-components>
<enabled>false</enabled>
</url-addressable-components>

Ok, default configuration is set, so components without <url-addressable-enabled/> cannot be invoked through an URL. Let’s check virtuallinkconnect component default configuration:

<!–XSRF component whitelisting change–>
<url-addressable-enabled/>

Ok, so if the component is whitelisted by default, why it is being blocked? Well, what does any component do when you invoke it through an URL? Yes, it redirects to the login component, which ,of course, it’s not whitelisted by default.

So by adding <url-addressable-enabled/> to the login definition, the virtuallinkconnect component will work with the component blocking enabled.

Chrome dropping NPAPI (Java) support and UCF

Since Chrome 42 was released, NPAPI plugins are disabled by default and the support for these plugins will be completely removed in Chrome 45 (expected to be released in September). This means no more UCF in Chrome. So, what will EMC do?

Considering that in the latest Webtop release (6.8) only 3 browsers are supported (IE 10/11, Firefox 24.6 and Safari 6.1.4/7.0.4) most likely nothing. Besides, this move from Google can “help” EMC to keep pushing D2/xCP to customers and reinforce the “Webtop won’t be developed anymore” message. But, what does this move means for current users?

I think sooner or later Java will be removed from every browser. Spartan/Edge won’t even support Microsoft’s own ActiveX, and considering I don’t see Oracle developing a new plugin based on PPAPI (which by the way is not a standard protocol and it’s only supported by Chrome and Opera), so it wouldn’t surprise me to see Mozilla announcing they’re dropping support for NPAPI in future releases.

This means that by the end of this year, there will be most likely 6 browsers (Chrome, IE, Spartan/Edge, Firefox, Opera, Safari) and EMC will provide support only for 3 of them (2 being “obsolete”, IE and Firefox 24.x). And even though we know that (until now) even without being officialy supported webtop (mostly) works with any browser, this won’t be the case much longer (as long as you need to transfer content).

In the end, we’ll have to deal with angry users that like Chrome because it’s cool and because they don’t want to use the corporative IE6/7/8 (not supported also) or the company-branded browser they’ve come to hate through the years.

Until know, my “usual” hack for matching IE/Firefox performance with Chrome’s was mainly disabling the ucfinvoker in the main component and any plugin not used in app.xml. This makes webtop to load the ucf libraries only when needed instead of doing it every time the user login so it seems that the main component loads faster (yes, I’m cheating :D). And I’m also considering disabling UCF transfer for view action and leaving UCF enabled only for edition, so read-only users won’t be disturbed with error messages/warnings when using Webtop.

However, there’s still the problem of dealing with the users that cannot use their preferred browser to do their job and convincing them that is not Documentum’s fault…

Edit: Even better solution (removing UCF in chrome)

Extend the evaluator class defined in the action (LaunchViewEvaluator, LaunchEditEvaluator, ContentTransferLaunchComponentEvaluator, etc.) with:

    public class ContentTransferLaunchComponentEvaluator extends com.documentum.web.contentxfer.ContentTransferLaunchComponentEvaluator {
        public String evaluate(String strName, String strAction, IConfigElement config, ArgumentList arg, Context context, Component component){
            HttpServletRequest request = (HttpServletRequest)component.getPageContext().getRequest();
            String userAgent = request.getHeader("User-Agent");  

            if (userAgent!=null && userAgent.length()>0 && userAgent.indexOf("Chrome")>=0) {
                    return "http";
            }
            return super.evaluate(strName, strAction, config, arg, context, component);
        }
    }

Edit v2: The best solution:

1. Check PanfilovAB post about Component Qualifiers and check the request listener class: Component qualifier

2. Create the following class:

    public class CustomContentTransferConfig extends com.documentum.web.contentxfer.ContentTransferConfig {
        @Override
        public String getContentTransferMechanism(){
            String userAgent=ComponentRequestListener.getBrowserUserAgent();  

            if (userAgent!=null && userAgent.length()>0 && userAgent.indexOf("Chrome")>=0) {
                return "http";
            }
            return super.getContentTransferMechanism();
        }
    }

3. Modify webtop\WEB-INF\classes\com\documentum\web\contentxfer\ContentTransferConfig.properties with:

configReaderClass=CustomContentTransferConfig

Done!

Webtop Recycle Bin using Aspects

When you talk with a customer for the first time, there’s always the issue with deleting documents. Do we allow delete? If so, who can delete? and then, what happens if a document is deleted by accident?

1. Restore a previous backup: You’re limited to the last backup (DB+Filestore), so you’ll lost everything done from that moment on. Recovery of individual documents is painful.

PROS: Should be the “default” as every organization is supposed to have backups of their systems.

CONS: Incomplete recovery (only until the last backup was done)

2. Use some backup/recovery tool such as CYA SmartRecovery: I’ve used this on the past. I wouldn’t use it again unless the recovery of individual documents is an essential business requirement.

PROS: You can restore “easily” individual documents. In case of full disaster you can restore the last backup and then use SmartRecovery to restore the newer documents.

CONS: Initial backup can take forever depending on the size of the repository. You’ll duplicate the required space for backups/Filestore

3. Develop some kind of customization:

PROS: Avoid license/Filestore costs. Enjoy developing for a while

CONS: Mostly client dependant, something will probably break when migrating to a newer version

I’ve usually seen the 3rd option implemented by overriding the destroy/destroyAllVersions/doDestroy methods or the delete action in Webtop to do anything else (or doing nothing at all). D2 implements this functionality out-of-the-box, so I though Aspects where a nice tool to implement this in a WDK-client, so here’s a little step by step guide:

1. Define the aspect type and the aspect attributes: In this case I’ve use “trash_aspect” for the aspect name and defined an “path” attribute to store the source folder of the document.

2. Implement the aspect:

ITrashAspect.java:

package es.test.trashaspect;  
  
import com.documentum.fc.common.DfException;  
  
public interface ITrashAspect {  
    public String getTrashPathName() throws DfException;  
    public void setTrashPathName(String path) throws DfException;  
}  

TrashAspect.java:

package es.test.trashaspect;  
  
import com.documentum.fc.client.DfDocument;  
import com.documentum.fc.common.DfException;  
  
public class TrashAspect extends DfDocument implements ITrashAspect {  
  
    private static final String ASPECT_PATH_NAME = "trash_aspect.pathname";  
    
    public String getTrashPathName() throws DfException {  
        return this.getString(ASPECT_PATH_NAME);  
    }  
  
    public void setTrashPathName(String path) throws DfException {  
        this.setString(ASPECT_PATH_NAME path);        
    }    
}  

3. Package the classes into jar files, and create the jar definitions in Composer. Create the trash_aspect module and configure the aspect module to use the trash_aspect type.

4. Modify your custom base type TBO with:

    public boolean checkTrashAspect(){  
        boolean isTrash=false;  
        
        try {  
            IDfList aspectList = getAspects();  
            
            for (int i=0; i < aspectList.getCount() && !isTrash; i++) {  
                if (((String) aspectList.get(i)).equalsIgnoreCase("trash_aspect")) {  
                    isTrash=true;  
                }  
            }  
            
            if (!isTrash){  
                String folderPath="";  
                String folderIdPath="";  
                IDfId processingFolderId=getFolderId(0);  
                while (!folderIdPath.startsWith("/0c")){  
                    IDfFolder folder=(IDfFolder)getSession().getObject(processingFolderId);  
                    folderPath="/"+folder.getObjectName()+folderPath;  
                    processingFolderId=folder.getFolderId(0);  
                }  
                
                CallBackTrashAspect callback=new CallBackTrashAspect();  
                callback.setFolderPath(folderPath);  
                
                attachAspect("trash_aspect",callback);  
                
                unlink(folderPath);  
                link("/TRASH");  
                
                save();  
            }  
        } catch (DfException e) {  
            e.printStackTrace();  
        }  
        
        
        return isTrash;  
    }  
    @Override  
    public void destroy() throws DfException {  
        if (checkTrashAspect()){  
            super.destroy();  
        }  
    }  
    
    @Override  
    public void destroyAllVersions() throws DfException {  
        if (checkTrashAspect()){  
            super.destroyAllVersions();  
        }  
    }  
    
    @Override  
    protected void doDestroy(boolean arg0, Object[] arg1) throws DfException {  
        if (checkTrashAspect()){  
            super.doDestroy(arg0, arg1);  
        }  
    } 

CallBackTrashAspect.java:

    import com.documentum.fc.client.IDfPersistentObject;  
    import com.documentum.fc.client.aspect.IDfAttachAspectCallback;  
    import com.documentum.fc.common.DfException;  
      
    import es.test.trashaspect.ITrashAspect;  
      
    public class CallBackTrashAspect implements IDfAttachAspectCallback {  
      
        private String folderPath="";  
        
        public String getFolderPath() {  
            return folderPath;  
        }  
      
        public void setFolderPath(String folderPath) {  
            this.folderPath = folderPath;  
        }  
      
        public void doPostAttach(IDfPersistentObject currentObj) throws Exception {  
            ((ITrashAspect)currentObj).setTrashPathName(folderPath);  
            currentObj.save();  
        }  
    }  

5. The restore functionality, using a Webtop action:

    package es.test.trashaspect;  
      
    import java.util.Map;  
      
    import com.documentum.fc.client.IDfDocument;  
    import com.documentum.fc.client.aspect.IDfAspects;  
    import com.documentum.fc.common.DfException;  
    import com.documentum.fc.common.DfId;  
    import com.documentum.fc.common.IDfList;  
    import com.documentum.web.common.ArgumentList;  
    import com.documentum.web.formext.action.IActionExecution;  
    import com.documentum.web.formext.component.Component;  
    import com.documentum.web.formext.config.Context;  
    import com.documentum.web.formext.config.IConfigElement;  
      
    public class RestoreAction implements IActionExecution{  
      
        @Override  
        public boolean execute(String arg0, IConfigElement arg1, ArgumentList arg2, Context arg3, Component arg4, Map arg5) {        
            try {  
                IDfDocument restoreObj=(IDfDocument)arg4.getDfSession().getObject(new DfId(arg2.get("objectId")));  
                IDfList aspectList = ((IDfAspects) restoreObj).getAspects();  
                
                boolean isTrash=false;  
                
                for (int i=0; i < aspectList.getCount() && !isTrash; i++) {  
                    if (((String) aspectList.get(i)).equalsIgnoreCase("trash_aspect")) {  
                        isTrash=true;  
                    }  
                }  
                
                if (isTrash){  
                    String sourceFolder=((ITrashAspect)restoreObj).getTrashPathName();  
                    restoreObj.unlink("/TRASH");  
                    restoreObj.link(sourceFolder);  
                    
                    ((IDfAspects) restoreObj).detachAspect("trash_aspect", null);  
                    restoreObj.save();  
                }  
                
            } catch (DfException e) {  
                e.printStackTrace();  
            }  
            
            return false;  
        }  
      
        @Override  
        public String[] getRequiredParams() {  
            return null;  
        }  
    }  

Documentum Email Notification improvements

Following PanfilovAB‘s post regarding Documentum email notification system and his customization (read it here: blog.documentum.pro/2014/08/04/e-mail-notifications) I decided to improve my own mail customization by adding the Groovy library in order to get advantage of the customization possibilities.

So, what does Documentum provide by default? Let’s see: Depending on the value stored in the mail_method attribute found in dm_server_config, CS will use:

  • dm_event_sender: Notifications will use dm_event_sender method. This method uses dmbasic to run $DM_HOME/bin/dm_event_sender.ebs. This EBS defines a plain-text email template for each of the possible events generated by the CS and sends the email by running the binary mail.exe bundled with the CS.
  • dm_html_event_sender: Same behaviour than dm_event_sender, but the email templates are HTML templates. In order to use it, dm_event_sender must be modified to use the $DM_HOME/bin/dm_html_event_sender.ebs script.
  • [BPM] dm_template_event_sender: When BPM is intalled in the CS, a new servlet (DO_METHOD) is deployed in JMS. mail_method attribute is updated with this placeholder to use the servlet instead of running a method. When an event generates a notification, a POST is send to the do_method servlet and the notification is processed. If the event is a workflow event, the servlet will process it applying (if has been previously defined) a JSP template and sending the email with Java Mail. Otherwise it will forward the notificacion to the dm_event_sender method.
  • mail method: There’s also a method that allows a “direct call” to mail.exe, as long as you provide the necessary parameters.

So, what main problems/limitations do we have?

  • If you have BPM installed, you have 3 different entry points to a new mail:
    • event_sender
    • html_event_sender
    • JSP templates

So if you update one template, you have to update every file to keep consistency

  • When migrating CS, you have to be careful not to overwrite/remove your customized EBS. And if “something” has changed you’ll have to recheck your existing templates. Meaning you’ll have to enjoy the “power” of dmbasic and its great debugging capabilities (yes, that means writing to an output file)

So, when I read Andrey’s solution using Groovy I realized that if you have to modify your email templates “often”, that’s the way to go. So, what do we need?:

Documentum container with two components and a custom method:

  • List templates
  • Edit template
  • Custom mail method

List template will be a listbox showing the documents returned by a query that fetches our custom templates located in a special folder:

Untitled.png

For ease of use, the name of the document indicates the event that uses the template and, if applicable, the r_object_id of the activity that uses the template, allowing further customization.

Edit template will be a component that loads the content of the selected template into an editor (codemirror, check Andrey’s updated post to see a “nice” implementation as WDK control), a sortablelistbox that will allow the user to put temporal values to be replaced in the template for preview, and the preview button that will generate the final email applying the submitted values to the template. This component has a <inputvars> element in its definition that defines default values to be already filled (docbase_name, user_description, server_url, etc).

Untitled2.png

Now, I only need to improve my mail designing skills

Finally, a custom Java method that will replace dm_template_event_sender and that will run a java class that generates the email body and sends the email.

What do we get with this customization?

  1. Get rid of dmbasic email notificacions
  2. Unify email notifications (it will only use the defined templates)
  3. Simplify email management (my “default” template now has 200 lines, while my “default” dm_event_sender has 1400)
  4. Easier upgrade process