OTEW 2018 fun

I’m not attending the event being held in Toronto, but I found through twitter this nice url:

http://hol-host05.eastus.cloudapp.azure.com:81/d2-unity-web/ui/app.html -> This is the new D2 UI (and yes, you can use the you-know-which-default-user(s) to log in and check it by yourself) deployed on Azure (which is weird, considering Opentext has its own cloud…)

But, the really funny thing here, are these urls:

http://hol-host05.eastus.cloudapp.azure.com:81/da -> da 7.3 (but with CS 16.4/SQL Server)

http://hol-host05.eastus.cloudapp.azure.com:81/D2 -> hello old D2 vulnerabilities 🙂

http://hol-host05.eastus.cloudapp.azure.com:81/d2-unity-web/repositories -> and you can log in with you-know-which-default-user(s), and you have a nice DQL tool provided by REST services 🙂

Documentum 16.3 delayed until Feb 2018

If you access the new support page, there is a section where you can find roadmaps for the products, and if you check the Documentum related ones, you’ll see the target date for ths new version listing new features. Without going into much detail (As I’m not sure how public this info is), these caught my atention:

  1. Ongoing exposure of D2 APIs through REST (goodbye DFS?)
  2. Native support for S3 object storage protocols to alleviate storage costs (goodbye OnDemand, hello AWS?)
  3. New REST Services (see #1, no mention of DFS anywhere to be found)
  4. Webtop will get a 16.3 version (yeah!)


Support adventures II (REST Services edition)

I wasn’t expecting this week to be so “productive” 🙂

I had the chance to engage again with Documentum support, which is always an adventure 😉

We’re currently porting a framework from DFS to REST services. A couple of days ago, one of my colleagues had a problem and ask me if I knew why a query through the REST services was failing.

The query is using DATETOSTRING to retrieve a time field and returning a column with the attribute name using the alias, so, if a user requests expiration_date we execute a query like:

select DATETOSTRING(expiration_date,’dd/mm/yyyy’) as expiration_date from…

which returns the requested attribute in the desired (configurable) format. Easy, right? Well, the problem was that using r_creation_date or r_modify_date was throwing an exception:

 {    “status”: 400,
“message”: “There are illegal arguments provided.”,
“details”: “Invalid format: \”12/04/2013\” is malformed at \”/04/2013\””}

We also observed that changing the alias to something else than r_creation_date/r_modify_date would work (but it was breaking our use case).

This was weird, because I was quite sure that those kind of queries worked previously, so I checked against the Content Server:

Connected to Documentum Server running Release 7.1.0210.0328  Linux64.Oracle
1> select DATETOSTRING(r_creation_date,’dd/mm/yyyy’) as r_creation_date from dm_document enable (return_top 1);
2> go
(1 row affected)

As I though, it was working just fine. I suspected it had something to do with the custom date format, as EMC/OpenText tends to forget that not everyone uses ANSI or the american date format, so I decided to open a SR.

After some exchange of emails, we were told that both r_creation_date and r_modify_date where reserved words and that it was expected to fail. What???

After asking support to either fill this as a product limitation or fixing it, I decided to take a look at the code myself.

As I suspected, the query works just fine, and results are returned to the Query controller. The problem here is with the way REST generates the response. If you run the query with a different alias to see the result page you’ll get this:

"entries": [  {
"id": ",%27dd/mm/yyyy%27)%20as%20rr_creation_date%20from%20dm_document%20enable%20(return_top%201)&index=0",
"title": "12/04/2013",
"updated": "2017-06-16T11:04:38.284+00:00",
"published": "2017-06-16T11:04:38.284+00:00",
"content": {
"json-root": "query-result",
"definition": "",
"properties": {
"rr_creation_date": "12/04/2013"

Do you see those fields before the content element? That’s where everything breaks, why? Because of this:

public Date entryUpdated() {
Date updated = null;
Object modifyDate = ((QueryResultItem)getDataInternal())
if (modifyDate == null) {
updated = new Date();
} else if ((modifyDate instanceof Date)) {
updated = (Date)modifyDate;
} else {
updated = DateFormatter.parse(modifyDate.toString());
return updated;

public Date entryPublished()
Date published = null;
QueryResultItem queryResultItem=getDataInternal();
Object modifyDate = ((QueryResultItem)getDataInternal())
if (modifyDate == null) {
published = new Date();
} else if ((modifyDate instanceof Date)) {
published = (Date)modifyDate;
} else {
published = DateFormatter.parse(modifyDate.toString());
return published;

As you can see (besides the obvious copy/paste from one method to another changing the name of one variable), the standard view looks in the results from the query for a column with those names, and if it founds a column matching that name, tries to parse it with a forced format (which of course, it’s miserably failing with our custom date format).

This, in my opinion, is a bug, because the behaviour of the query functionality it is inconsistent between the Documentum stack, in fact, this query only fails with REST services, so I’ll keep pushing the SR to be treated as a bug and fixed by the talented team, and not as a “product limitation” or a “feature request”.

And if you face the same problem, and it is still not fixed, you have three options:

  • Keep waiting for a fix
  • Extend com.emc.documentum.rest.view.impl.QueryResultItemView and handle the IllegalArgumentException that throws DateFormatter.parse
  • Extend the controller adding a custom view for the results and removing/overriding the updated/published fields.

Filename in Documentum REST Services

If you have used Document REST Services you most likely have realized that downloading any content from an object return a “document”/”response” + dot + file extension.

You can check William Zhou’s answer here:

This had been discussed in the initial implementation but hadn’t been implemented since Content-Disposition response header is neither mandatory nor handled by all HTTP clients. Besides, it has some overhead to map the format/mime to a filename extension. But I think it has values helping for the download experience. It is appreciated if you can file a feature request CR so that we can discuss this with the product manager.

I’ve alredy raised a SR to support in order to get OpenText to consider adding this “feature”.

However, if you can’t/don’t want to wait, just “extend” com.emc.documentum.rest.controller.ContentMediaController adding the following line to the getContent method just before returning the response:

headers.setContentDispositionFormData("attachment", (String)co.getAttributeByName("object_name"));

and you’re good to go.


Updated roadmap, webinar and FAQ from OpenText Documentum

If you registered for the webinar OpenText held a few weeks ago (if you didn’t, maybe you can check Andrey’s post on the subject), you should have received an invitation to some site from OT with a FAQ about the Documentum stack. I’m not going to paste it here, just in case it’s not public, but IMHO, the highlights are:

  • Content Server: Aligment with ECD. No new features but trying to move to a microservices architechture. It probably means CS won’t evolve anymore.
  • Webtop: Several points on this. Will keep being supported and updated to support latest CS.
  • Rest: Every new product based on this. No mention about DFS (I hope it’s dead, unless they decide to update with libraries from this decade)
  • xCP: It looks like xCP 1.x support will be over by the end of 2019.
  • Support: Still through EMC’s site until the end of the year.

Customizing/Extending REST services (II)

This is an improved version of the “hack” posted in the previous post (Customizing/Extending REST services)

  • Decompile com.emc.documentum.rest.controller.QueryController
  • Look for method executeQuery and modify the following line as you wish:

Preconditions.checkArgument(DQLChecker.isSafeQuery(dql), MessageBundle.INSTANCE.get(“E_UNSAFE_QUERY_DQL”, new Object[] { dql }));

You can use something like:

Preconditions.checkArgument(true, MessageBundle.INSTANCE.get(“E_UNSAFE_QUERY_DQL”, new Object[] { dql }));

if you want everyone to be able to run write queries or you can allow these queries to be executed only by some user(s) with something like:

if (RepositoryContextHolder.getLoginName().equalsIgnoreCase(“dmadmin”)){
//skip DQLChecker check for select statements

  • Finally place the new class in dctm-rest/WEB-INF/classes/com/emc/documentum/rest/controller.

CMIS vs REST comparison with code examples (Custom WebDAV server for Documentum)

Although My Documentum for Desktop is a nice product when you want a deeper integration with Windows, if you only need WebDAV access for browsing and creating documents it can be too much for the users. So I decided to develop a custom WebDAV server using a 1.x version of milton.io’s WebDAV library (newer 2.x free version doesn’t seem to work well with Office documents).

This library provides a WebDAV Servlet that you can customize for providing your own objects as WebDAV elements. I used CMIS (with Apache Chemistry) to query the repository and provide a WebDAV access to a Documentum repository. I wasn’t happy with the performance so I tried with the REST API as well.

The following is a comparison of features and code between those two implementations:

Compatibility Multiple CMS Documentum only
Memory footprint * Big Small
Ease of implementation ** Very easy Easy
Speed/Performance Fast Slightly faster (noticeable difference)

* Memory footprint: Although I’m still not sure of the source of the problem, CMIS consumes every Documentum session specified in dfc.properties. So if you have 100 max concurrent sessions, it’s going to use every single one. This is less noticeable when using a WebDAV client (session usage is between 7-20 sessions) but when using Windows integrated WebDAV client it goes crazy after 2nd level of folders. Not sure if it is due to windows querying every single object displayed or something related to CMIS specification or the implementation in emc-cmis.war

** Ease of implementation: When using CMIS with Apache Chemistry, you can code every single operation with a couple of lines of code.

Using the REST API will require a little bit more coding (I used org.json library to parse the responses):

Create document

  • CMIS:
Folder tempFolder=(Folder)session.getObjectByPath(path);
Map<String, Object> properties = new HashMap<String, Object>();
properties.put(PropertyIds.OBJECT_TYPE_ID, "dm_document");
properties.put(PropertyIds.NAME, newName);  

ContentStream contentStream = new ContentStreamImpl(newName,
          BigInteger.valueOf(length), contentType, in);
Document newDoc = tempFolder.createDocument(properties, contentStream, VersioningState.MAJOR);  

return newDoc.getId();
  • REST:
HttpPost httpPost=new HttpPost(pathId.replace("/objects/","/folders/")

HttpClient client = new DefaultHttpClient();  

String json = "{\"properties\" : {\"object_name\" : \"" + newName +
          "\", \"r_object_type\" :\"dm_document\"}}";  

File temp = File.createTempFile(newName,".tmp");
byte[] data = IOUtils.toByteArray(in);
FileUtils.writeByteArrayToFile(temp, data);  

MultipartEntity reqEntity = new MultipartEntity();
reqEntity.addPart("data", new StringBody (json,
reqEntity.addPart("content", new FileBody(temp, newName, contentType, null));  

HttpResponse response = client.execute(httpPost);

String result=getResponseAsString(response.getEntity().getContent());
JSONObject obj= new JSONObject(result);
JSONArray jsonArray = (JSONArray) obj.getJSONArray("links");  

return (String)jsonArray.getJSONObject(0).get("href");

Create Folder:

  • CMIS:
    Folder tempFolder=(Folder)session.getObjectByPath(path);
    Map<String, Object> properties = new HashMap<String, Object>();
    properties.put(PropertyIds.OBJECT_TYPE_ID, "cmis:folder");
    properties.put(PropertyIds.NAME, newName);  

    Folder newFolder=tempFolder.createFolder(properties);  

    return newFolder.getId();
  • REST:
    HttpPost httpPost=new HttpPost(pathId.replace("/objects/","/folders/")

    HttpClient client = new DefaultHttpClient();  

    String json = "{\"properties\" : {\"object_name\" : \"" + newName +
              "\", \"r_object_type\" :\"dm_folder\"}}";  

    MultipartEntity reqEntity = new MultipartEntity();
    reqEntity.addPart("data", new StringBody (json,

    HttpResponse response = client.execute(httpPost);  

    String result=getResponseAsString(response.getEntity().getContent());
    JSONObject obj= new JSONObject(result);
    JSONArray jsonArray = (JSONArray) obj.getJSONArray("links");  

    return (String)jsonArray.getJSONObject(0).get("href");

Version object:

  • CMIS:
Document newDoc = (Document) session.getObject(session.createObjectId(objectId));  

ObjectId newVersionId = newDoc.checkOut();
Document newVersion=(Document)session.getObject(newVersionId);
ContentStream cstream = new ContentStreamImpl(newDoc.getContentStream().getFileName(),
               BigInteger.valueOf(length), newDoc.getContentStream().getMimeType(), in);
newVersionId=newVersion.checkIn(true, null, cstream, null);  

return newVersionId.getId();


HttpGet httpGet=new HttpGet(pathId);

HttpClient client = new DefaultHttpClient();  

//get lock-checkout url
HttpResponse response = client.execute(httpGet);  

String result=getResponseAsString(response.getEntity().getContent());
JSONObject obj=new JSONObject(result);
JSONArray jsonArray = (JSONArray) obj.getJSONArray("links");  

String lockUrl=null;
for (int i=0; i<jsonArray.length(); i++){
     if (jsonArray.getJSONObject(i).get("rel").toString().endsWith("checkout")){

//lock-checkout object
HttpPut httpPut=new HttpPut(lockUrl);

response = client.execute(httpPut);
obj=new JSONObject(result);
jsonArray = (JSONArray) obj.getJSONArray("links");  

String objName=obj.getJSONObject("properties").get("object_name").toString();
String checkinUrl=null;  

//get checkin url
for (int i=0; i<jsonArray.length(); i++){
     if (jsonArray.getJSONObject(i).get("rel").toString().endsWith("checkin-next-major")){

String json = "{\"properties\" : {}}";  

File temp = File.createTempFile(objName,".tmp");
byte[] data = IOUtils.toByteArray(in);
FileUtils.writeByteArrayToFile(temp, data);  

//cheking new version
HttpPost httpPost=new HttpPost(checkinUrl);

MultipartEntity reqEntity = new MultipartEntity();
reqEntity.addPart("data", new StringBody (json,
reqEntity.addPart("content", new FileBody( temp , objName, contentType, null));  


response = client.execute(httpPost);
obj=new JSONObject(result);
jsonArray = (JSONArray) obj.getJSONArray("links");  

return (String)jsonArray.getJSONObject(0).get("href");

Consuming RestFul services with GSON

If you want to consume the RestFul services you can check Wei Zhou’s posts in the documents section of the documentum developer network (Documentum) or check EMC RestFul Services — Clients JAVA by astone to see an example of how to consume the JSON directly from JAVA.

However, while developing the Android app I found that it was way easier to go the old Java-objects way, and you can do this by using GSON (or any other similar library).

First, you’ll need to create the POJOs/Beans for the classes; as we don’t have those (thanks EMC…) you’ll have to code them by yourself from the JSON responses (or use something like http://www.jsonschema2pojo.org/ to autogenerate them).

Once you have those (and the Gson libraries in your project) it is quite straightforward:

Reading a JSON response as an object:

DefaultHttpClient httpClient = new DefaultHttpClient();  
//documentum user and password  
String authorizationString = "Basic " + Base64.encodeToString((strUser + ":" + strPass).getBytes(), Base64.NO_WRAP);  
//request the page, here we get the main repository page  
HttpGet getRequest = new HttpGet(currentServer +"/repositories");  
//don't forget the credentials  
getRequest.setHeader("Authorization", authorizationString);  
//get the response  
HttpResponse getResponse = httpClient.execute(getRequest);  
HttpEntity getResponseEntity = getResponse.getEntity();  
Reader reader = new InputStreamReader(getResponseEntity.getContent());  
//use gson to get the object  
Gson gson=new Gson();  
RepositoryList repoList = (RepositoryList)gson.fromJson(reader, RepositoryList.class);  

Modify some property by sending a POST request:

DefaultHttpClient httpClient = new DefaultHttpClient();  
//documentum user and password  
String authorizationString = "Basic " + Base64.encodeToString((strUser + ":" + strPass).getBytes(), Base64.NO_WRAP);  
//setting up the POST request  
HttpPost postRequest = new HttpPost(currentServer+"/repositories/"+currentRepo+"/objects/"+strId);  
//don't forget the credentials  
//Create the object with the property you want to modify  
RepositoryObject ro=new RepositoryObject();  
Attribute props=new Attribute();  
props.setSubject("new Subject");  
//set the header to the POST request  
postRequest.setHeader("Content-Type", "application/vnd.emc.documentum+json; charset=utf-8");  
//convert the object to json using gson  
Gson gson=new Gson();  
postRequest.setEntity(new StringEntity(gson.toJson(ro)));  
//do the POST  
HttpResponse postResponse = httpClient.execute(postRequest);

Note: As you can see, I’m not using service discovery although that would be the right way to do it, however, this was faster and it works as a how-to

Customizing/Extending REST services

In another post (Android app using REST services (II)) I said that by default the REST services only allow to run read queries on the repository. This is obviously a setback when you want to do anything else than querying the repository. So, what can we do about this?

First thing that comes to mind is to decompile the classes and see what’s going on. Basically, there’s a “checker” class that checks if the DQL should be executed or not. You can decompile, modify and replace this class to be able to run write queries on the repository.

However, there’s a cleaner (and I guess not so illegal) way to do this:

Create a new class named com.emc.documentum.rest.utils.DQLChecker containing the following:

package com.emc.documentum.rest.utils;

public class DQLChecker {
  public static boolean isSafeQuery(String dql){
    //do your stuff to check whatever you want to check in the sql string
    return true; //or false if you don't want the query to be run

Then put this class in webapps/dctm-rest/WEB-INF/classes/com/emc/documentum/rest/utils (or repackage the war file) and restart the server

What happens here is, whatever class you drop on WEB-INF/classes will take precedence over the same class present in any jar file in the lib folder.

PS: You can use this with any class in the REST services in case you want to modify some behaviour.



Android app using REST services (II)

Some more info on this:

  • Environment:
    • Documentum 6.6P27/SQLServer
    • REST Services 7.0
  • App Development info:
    • App developed with AndroidStudio
    • Compatible with Android 4.x

Some tips/thoughts:

  • Stick to eclipse + android SDK instead of using AndroidStudio (unless until AndroidStudio is mature enough). You’ll save time and problems.
  • Programming in Android is a pain in the ass with so many different devices/versions. Apple and xCode wins there.
  • Mobile emulators requiere a huge amount of resources. Either find a test mobile phone or get a good development computer with at least 8gb RAM. I think the iPhone emulator runs better in iMacs than the android emulator in PC
  • The REST Services package doesn’t come with a POJO/BEAN implementation, so if you plan to use Java you’ll have to code those classes from scratch (boring) to parse the responses (maybe there’s a better/automatic way to do it)
  • The query service (used as a workaround to get the job info, as there is no job service) doesn’t allow write queries, so forget about updating objects/executing jobs on demand (unless you want to decompile some class and change some “startsWith(“select “)” to something else)
  • The REST services are quite fast, much much faster than DFS, consider this if you need to develop something new.

App updates:

  • Uses SQLite to store preferences
  • Remembers connection details
  • Added user list
  • You can choose what job / user attributes want to show in the details window


  • Login:


  • Job options:


  • User options:


  • Job details:


  • User details:



  • Modify user properties
  • Show/update document properties