Category: REST

Customizing/Extending REST services (II)

Alvaro de AndresLeave a comment

This is an improved version of the “hack” posted in the previous post (Customizing/Extending REST services)

  • Decompile com.emc.documentum.rest.controller.QueryController
  • Look for method executeQuery and modify the following line as you wish:

Preconditions.checkArgument(DQLChecker.isSafeQuery(dql), MessageBundle.INSTANCE.get(“E_UNSAFE_QUERY_DQL”, new Object[] { dql }));

You can use something like:

Preconditions.checkArgument(true, MessageBundle.INSTANCE.get(“E_UNSAFE_QUERY_DQL”, new Object[] { dql }));

if you want everyone to be able to run write queries or you can allow these queries to be executed only by some user(s) with something like:

if (RepositoryContextHolder.getLoginName().equalsIgnoreCase(“dmadmin”)){
//skip DQLChecker check for select statements
}

  • Finally place the new class in dctm-rest/WEB-INF/classes/com/emc/documentum/rest/controller.

CMIS vs REST comparison with code examples (Custom WebDAV server for Documentum)

Alvaro de AndresLeave a comment

Although My Documentum for Desktop is a nice product when you want a deeper integration with Windows, if you only need WebDAV access for browsing and creating documents it can be too much for the users. So I decided to develop a custom WebDAV server using a 1.x version of milton.io’s WebDAV library (newer 2.x free version doesn’t seem to work well with Office documents).

This library provides a WebDAV Servlet that you can customize for providing your own objects as WebDAV elements. I used CMIS (with Apache Chemistry) to query the repository and provide a WebDAV access to a Documentum repository. I wasn’t happy with the performance so I tried with the REST API as well.

The following is a comparison of features and code between those two implementations:

  CMIS REST
Compatibility Multiple CMS Documentum only
Memory footprint * Big Small
Ease of implementation ** Very easy Easy
Speed/Performance Fast Slightly faster (noticeable difference)

* Memory footprint: Although I’m still not sure of the source of the problem, CMIS consumes every Documentum session specified in dfc.properties. So if you have 100 max concurrent sessions, it’s going to use every single one. This is less noticeable when using a WebDAV client (session usage is between 7-20 sessions) but when using Windows integrated WebDAV client it goes crazy after 2nd level of folders. Not sure if it is due to windows querying every single object displayed or something related to CMIS specification or the implementation in emc-cmis.war

** Ease of implementation: When using CMIS with Apache Chemistry, you can code every single operation with a couple of lines of code.

Using the REST API will require a little bit more coding (I used org.json library to parse the responses):

Create document

  • CMIS:
Folder tempFolder=(Folder)session.getObjectByPath(path);
Map<String, Object> properties = new HashMap<String, Object>();
properties.put(PropertyIds.OBJECT_TYPE_ID, "dm_document");
properties.put(PropertyIds.NAME, newName);  

ContentStream contentStream = new ContentStreamImpl(newName,
          BigInteger.valueOf(length), contentType, in);
Document newDoc = tempFolder.createDocument(properties, contentStream, VersioningState.MAJOR);  

return newDoc.getId();
  • REST:
HttpPost httpPost=new HttpPost(pathId.replace("/objects/","/folders/")
          .replace(".json","/objects.json"));
httpPost.addHeader("Authorization",authorizationString);  

HttpClient client = new DefaultHttpClient();  

String json = "{\"properties\" : {\"object_name\" : \"" + newName +
          "\", \"r_object_type\" :\"dm_document\"}}";  

File temp = File.createTempFile(newName,".tmp");
byte[] data = IOUtils.toByteArray(in);
FileUtils.writeByteArrayToFile(temp, data);  

MultipartEntity reqEntity = new MultipartEntity();
reqEntity.addPart("data", new StringBody (json,
          "application/vnd.emc.documentum",Charset.forName("UTF-8")));
reqEntity.addPart("content", new FileBody(temp, newName, contentType, null));  

httpPost.setEntity(reqEntity);
HttpResponse response = client.execute(httpPost);
temp.delete();  

String result=getResponseAsString(response.getEntity().getContent());
JSONObject obj= new JSONObject(result);
JSONArray jsonArray = (JSONArray) obj.getJSONArray("links");  

return (String)jsonArray.getJSONObject(0).get("href");

Create Folder:

  • CMIS:
    Folder tempFolder=(Folder)session.getObjectByPath(path);
    Map<String, Object> properties = new HashMap<String, Object>();
    properties.put(PropertyIds.OBJECT_TYPE_ID, "cmis:folder");
    properties.put(PropertyIds.NAME, newName);  

    Folder newFolder=tempFolder.createFolder(properties);  

    return newFolder.getId();
  • REST:
    HttpPost httpPost=new HttpPost(pathId.replace("/objects/","/folders/")
                   .replace(".json","/objects.json"));
    httpPost.addHeader("Authorization",authorizationString);  

    HttpClient client = new DefaultHttpClient();  

    String json = "{\"properties\" : {\"object_name\" : \"" + newName +
              "\", \"r_object_type\" :\"dm_folder\"}}";  

    MultipartEntity reqEntity = new MultipartEntity();
    reqEntity.addPart("data", new StringBody (json,
              "application/vnd.emc.documentum",Charset.forName("UTF-8")));  

    httpPost.setEntity(reqEntity);
    HttpResponse response = client.execute(httpPost);  

    String result=getResponseAsString(response.getEntity().getContent());
    JSONObject obj= new JSONObject(result);
    JSONArray jsonArray = (JSONArray) obj.getJSONArray("links");  

    return (String)jsonArray.getJSONObject(0).get("href");

Version object:

  • CMIS:
Document newDoc = (Document) session.getObject(session.createObjectId(objectId));  

ObjectId newVersionId = newDoc.checkOut();
Document newVersion=(Document)session.getObject(newVersionId);
ContentStream cstream = new ContentStreamImpl(newDoc.getContentStream().getFileName(),
               BigInteger.valueOf(length), newDoc.getContentStream().getMimeType(), in);
newVersionId=newVersion.checkIn(true, null, cstream, null);  

return newVersionId.getId();

REST:

HttpGet httpGet=new HttpGet(pathId);
httpGet.addHeader("Authorization",authorizationString);  

HttpClient client = new DefaultHttpClient();  

//get lock-checkout url
HttpResponse response = client.execute(httpGet);  

String result=getResponseAsString(response.getEntity().getContent());
JSONObject obj=new JSONObject(result);
JSONArray jsonArray = (JSONArray) obj.getJSONArray("links");  

String lockUrl=null;
for (int i=0; i<jsonArray.length(); i++){
     if (jsonArray.getJSONObject(i).get("rel").toString().endsWith("checkout")){
         lockUrl=jsonArray.getJSONObject(i).get("href").toString();
     }
}  

//lock-checkout object
HttpPut httpPut=new HttpPut(lockUrl);
httpPut.addHeader("Authorization",authorizationString);  

response = client.execute(httpPut);
result=getResponseAsString(response.getEntity().getContent());
obj=new JSONObject(result);
jsonArray = (JSONArray) obj.getJSONArray("links");  

String objName=obj.getJSONObject("properties").get("object_name").toString();
String checkinUrl=null;  

//get checkin url
for (int i=0; i<jsonArray.length(); i++){
     if (jsonArray.getJSONObject(i).get("rel").toString().endsWith("checkin-next-major")){
         checkinUrl=jsonArray.getJSONObject(i).get("href").toString();
     }
}  

String json = "{\"properties\" : {}}";  

File temp = File.createTempFile(objName,".tmp");
byte[] data = IOUtils.toByteArray(in);
FileUtils.writeByteArrayToFile(temp, data);  

//cheking new version
HttpPost httpPost=new HttpPost(checkinUrl);
httpPost.addHeader("Authorization",authorizationString);  

MultipartEntity reqEntity = new MultipartEntity();
reqEntity.addPart("data", new StringBody (json,
             "application/vnd.emc.documentum",Charset.forName("UTF-8")));
reqEntity.addPart("content", new FileBody( temp , objName, contentType, null));  

httpPost.setEntity(reqEntity);  

response = client.execute(httpPost);
result=getResponseAsString(response.getEntity().getContent());
obj=new JSONObject(result);
jsonArray = (JSONArray) obj.getJSONArray("links");  

return (String)jsonArray.getJSONObject(0).get("href");

Consuming RestFul services with GSON

Alvaro de AndresLeave a comment

If you want to consume the RestFul services you can check Wei Zhou’s posts in the documents section of the documentum developer network (Documentum) or check EMC RestFul Services — Clients JAVA by astone to see an example of how to consume the JSON directly from JAVA.

However, while developing the Android app I found that it was way easier to go the old Java-objects way, and you can do this by using GSON (or any other similar library).

First, you’ll need to create the POJOs/Beans for the classes; as we don’t have those (thanks EMC…) you’ll have to code them by yourself from the JSON responses (or use something like http://www.jsonschema2pojo.org/ to autogenerate them).

Once you have those (and the Gson libraries in your project) it is quite straightforward:

Reading a JSON response as an object:

DefaultHttpClient httpClient = new DefaultHttpClient();  
      
//documentum user and password  
String authorizationString = "Basic " + Base64.encodeToString((strUser + ":" + strPass).getBytes(), Base64.NO_WRAP);  
      
//request the page, here we get the main repository page  
HttpGet getRequest = new HttpGet(currentServer +"/repositories");  
//don't forget the credentials  
getRequest.setHeader("Authorization", authorizationString);  
//get the response  
HttpResponse getResponse = httpClient.execute(getRequest);  
HttpEntity getResponseEntity = getResponse.getEntity();  
Reader reader = new InputStreamReader(getResponseEntity.getContent());  
      
//use gson to get the object  
Gson gson=new Gson();  
RepositoryList repoList = (RepositoryList)gson.fromJson(reader, RepositoryList.class);

Modify some property by sending a POST request:

DefaultHttpClient httpClient = new DefaultHttpClient();  
  
//documentum user and password  
String authorizationString = "Basic " + Base64.encodeToString((strUser + ":" + strPass).getBytes(), Base64.NO_WRAP);  
  
//setting up the POST request  
HttpPost postRequest = new HttpPost(currentServer+"/repositories/"+currentRepo+"/objects/"+strId);  
//don't forget the credentials  
postRequest.setHeader("Authorization",authorizationString);  
  
//Create the object with the property you want to modify  
RepositoryObject ro=new RepositoryObject();  
Attribute props=new Attribute();  
  
props.setSubject("new Subject");  
ro.setProperties(props);  
  
//set the header to the POST request  
postRequest.setHeader("Content-Type", "application/vnd.emc.documentum+json; charset=utf-8");  
//convert the object to json using gson  
Gson gson=new Gson();  
postRequest.setEntity(new StringEntity(gson.toJson(ro)));  
  
//do the POST  
HttpResponse postResponse = httpClient.execute(postRequest);

Note: As you can see, I’m not using service discovery although that would be the right way to do it, however, this was faster and it works as a how-to

Customizing/Extending REST services

Alvaro de Andres2 Comments

In another post (Android app using REST services (II)) I said that by default the REST services only allow to run read queries on the repository. This is obviously a setback when you want to do anything else than querying the repository. So, what can we do about this?

First thing that comes to mind is to decompile the classes and see what’s going on. Basically, there’s a “checker” class that checks if the DQL should be executed or not. You can decompile, modify and replace this class to be able to run write queries on the repository.

However, there’s a cleaner (and I guess not so illegal) way to do this:

Create a new class named com.emc.documentum.rest.utils.DQLChecker containing the following:

package com.emc.documentum.rest.utils;

public class DQLChecker {
  public static boolean isSafeQuery(String dql){
    //do your stuff to check whatever you want to check in the sql string
    return true; //or false if you don't want the query to be run
  }
}

Then put this class in webapps/dctm-rest/WEB-INF/classes/com/emc/documentum/rest/utils (or repackage the war file) and restart the server

What happens here is, whatever class you drop on WEB-INF/classes will take precedence over the same class present in any jar file in the lib folder.

PS: You can use this with any class in the REST services in case you want to modify some behaviour.

 

 

Android app using REST services (II)

Alvaro de AndresLeave a comment

Some more info on this:

  • Environment:
    • Documentum 6.6P27/SQLServer
    • REST Services 7.0
  • App Development info:
    • App developed with AndroidStudio
    • Compatible with Android 4.x

Some tips/thoughts:

  • Stick to eclipse + android SDK instead of using AndroidStudio (unless until AndroidStudio is mature enough). You’ll save time and problems.
  • Programming in Android is a pain in the ass with so many different devices/versions. Apple and xCode wins there.
  • Mobile emulators requiere a huge amount of resources. Either find a test mobile phone or get a good development computer with at least 8gb RAM. I think the iPhone emulator runs better in iMacs than the android emulator in PC
  • The REST Services package doesn’t come with a POJO/BEAN implementation, so if you plan to use Java you’ll have to code those classes from scratch (boring) to parse the responses (maybe there’s a better/automatic way to do it)
  • The query service (used as a workaround to get the job info, as there is no job service) doesn’t allow write queries, so forget about updating objects/executing jobs on demand (unless you want to decompile some class and change some “startsWith(“select “)” to something else)
  • The REST services are quite fast, much much faster than DFS, consider this if you need to develop something new.

App updates:

  • Uses SQLite to store preferences
  • Remembers connection details
  • Added user list
  • You can choose what job / user attributes want to show in the details window

Screenshots

  • Login:

0101b02.png

  • Job options:

03.png

  • User options:

04.png

  • Job details:

05.png

  • User details:

06.png

TODO:

  • Modify user properties
  • Show/update document properties

 

 

Android app using REST services

Alvaro de AndresLeave a comment

I’ve been learning Android development by developing an App using DCTM Rest services to expose as much functionality as possible from DA. Currently I have a working repository browser and job list/details app, that works via VPN (enabled in the phone by a 3rd app). This is just an example of what can be done with the Rest API released by EMC:

  • Login window:

001.png

  • Browser / Job choice:

002.png

  • Browser (Repository/cabinets/folders):

003.png004.png005.png

  • Open file:

006.png

  • Job list:

007.png

  • Job detail:

008.png

  • TODO:
    • Format Job details window
    • Use sqlite to store preferences/connection url
    • Cleanup code
    • Test new features:
      • User management
      • Job execution
      • Additional configurations

 

 

Testing D2 development and REST services

Alvaro de AndresLeave a comment

After the summer, I’ve found myself with some spare time between projects that I’ve used to review some EMC products:

  • D2 4.1

Over the years, we’ve developed a webtop based solution that includes several custom developed components like import from external source, TBO’s/SBO’s used to name documents/folders, etc.

So after attending several D2 demos, we decided to do exactly the same with D2 (or at least get as close as possible) as a testing project.

Custom naming/default object creation was the easiest part, as the autonaming and autolink features are functional enough to cover our needs.

Import from external source looked like it was going to be tough, as we couldn’t touch the default import, and the development documentation is… lacking. So after a little investigation thorugh the available documentation and the community, we decided to go with the external widget approach and it was quite easy actually: Take our custom applet from webtop, put it on a jsp and communicate with the D2 import component sending the files… no problem at all

Some other functionality that involved modifying default D2 behavior required to developed plugins, but after reviewing the API and the available examples we decided not even trying because currently is a waste of time trying to figure which method/libs you need.

  • Good: external widgtes are extremely easy to develop, and there is enough documentation to be able to communicate those widgets with D2 without killing yourself in the process.
  • Bad: plugins are useless as there’s no reliable documentation available, and it seems the API could change in future versions, making useless old plugins.

To sum up, I think D2 4.1 is more a beta/unfinished product (which it really is, as EMC is doing everything from scratch to made it compatible with browsers other than IE, and currently is missing features that were on previous versions). I’ve heard that D2 4.2 is a more mature product and that effors have been made regarding development documentation. If EMC wants to “kill” wdk application with D2 and xCP2 they better give us enough tools to do our customizations, as by now we all should realize that the idea of “configure only” is cool, but is far from our daily work…

  • REST Services:

I’ve been developeing different solutions with DFS (extra layers to the services, custom clients both in JAVA and .NET, bulk load applications, etc) since version 6.0. Afterwards I migrated everything that could be done with the CMIS standard to CMIS with Apache Chemistry; so it was clear the next step was trying the new REST services.

Before the summer I started developing an Android application with the REST preview just as a proof of concept (and as a way to learn how android applications work), and it worked quite well. Last week I’ve been updating the application using the REST 7.0 services (finally we have a working query service!!!) and cleaning/improving the android application that has now become an administrator tool that lets you check repository status, browse the repository, open files, check job status, etc.

I hope to keep improving the application and I hope EMC keeps the REST services, as there is way easier to develop with these REST services than using the monstruosity that DFS are. However, if CMIS meets your requirements, I still go that way, as it is a standard…