OTEW 2018 fun

I’m not attending the event being held in Toronto, but I found through twitter this nice url:

http://hol-host05.eastus.cloudapp.azure.com:81/d2-unity-web/ui/app.html -> This is the new D2 UI (and yes, you can use the you-know-which-default-user(s) to log in and check it by yourself) deployed on Azure (which is weird, considering Opentext has its own cloud…)

But, the really funny thing here, are these urls:

http://hol-host05.eastus.cloudapp.azure.com:81/da -> da 7.3 (but with CS 16.4/SQL Server)

http://hol-host05.eastus.cloudapp.azure.com:81/D2 -> hello old D2 vulnerabilities ๐Ÿ™‚

http://hol-host05.eastus.cloudapp.azure.com:81/d2-unity-web/repositories -> and you can log in with you-know-which-default-user(s), and you have a nice DQL tool provided by REST services ๐Ÿ™‚

Documentum D2 Cache

Usually when working with D2 one common problem is the cache and how/when to refresh it in order to see changes (attribute labels, dictionary values, etc.).

The common solution for every problem suggested by support is usually to delete “Tomcat’s temp folder”, where you’ll find the following files:

  • c2file-cache.data
  • folder-cache.data
  • o2attrconfig-cache.data
  • taxonomy_level-cache.data
  • skin-cache.data
  • dictionary_dql-cache.data
  • xml-cache.index
  • xml-cache.data
  • D2FileCleaningTracker_D2.ser
  • X3Image<random numbers>.png (multiple files)

This is most often a workaround for “refresh cache” option from D2-Config, which most of the times won’t work because it works by appending a /Servlet/refreshCache to the URL configured in the setting. Why does this not work? Well, it (mostly) does if you run a single server, however, if you’re running a cluster of multiple application server, which one is receiving the call and cleaning its cache? well, good luck guessing ๐Ÿ˜€

Besides, JMS also has these cached files (well, not all those files), and this cache is placed on /tmp. Is this a problem? No if you have a single Content Server on your server or if you are not running an application server (not Tomcat) with the different credentials from Documentum.

But, what happens if you happen to run, let’s say, several Content Servers on the same host, with different D2 versions in each one, and you run different application servers on your server, with different D2 versions? Well, what happens is havoc.

You’ll found the cache files on /tmp, but those will be “locked” by the user that created those files on the first place. Why this behaviour? Well, if we check support page, we’ll find KB6269196:

Summary
Because D2 does not contain any ehcache.xml configuration files, the default behavior of the ehcache library is to store the cache data files in the path specified by java.io.tmpdir.

Resolution
Append the -Djava.io.tmpdir option
Modify and change to default run time JAVA_OPTS
usage: -Djava.io.tmpdir=/home/dmadmin/temp”

So, we can get some conclusions from this support note:

1. D2 cache is stored in java.io.tmpdir, which in Tomcat is its temp folder, and in Linux is /tmp.

2. Support likes using sledgehammers to crack nuts (changing the default temporary folder of the application server for a couple of files???, really???)

3. Support/engineering/talented team just don’t bother or they can’t read

Why do I say that? Because D2 has indeed a configuration file (actually has three configuration files in JMS and two in D2/D2-Config) and the default behaviour of ehcache is not storing cache on java.io.tmpdir. Let’s take a look to the documentation of the (obsolete, as usual) version bundled in D2:

<!–
DiskStore configuration
=======================
The diskStore element is optional. To turn off disk store path creation, comment out the diskStore element below.
Configure it if you have overflowToDisk or diskPersistent enabled for any cache.
If it is not configured, and a cache is created which requires a disk store, a warning will be issued and java.io.tmpdir will automatically be used.
diskStore has only one attribute – “path”. It is the path to the directory where.data and .index files will be created.
If the path is one of the following Java System Property it is replaced by its value in the running VM. For backward compatibility these are not specified without being enclosed in the ${token} replacement syntax.

The following properties are translated:
* user.home – Userโ€™s home directory
* user.dir – Userโ€™s current working directory
* java.io.tmpdir – Default temp file path
* ehcache.disk.store.dir – A system property you would normally specify on the command line e.g. java -Dehcache.disk.store.dir=/u01/myapp/diskdir …

Subdirectories can be specified below the property e.g. java.io.tmpdir/one
–>

<diskStore path=”java.io.tmpdir”/>

So, if “nothing” is specified, ehcache will fallback to java.io.tmpdir or the value specified in the ecache-failsafe.xml (located in ehcache.jar), which happens to be pointing to java.io.tmpdir too.

So we’ve already found one configuration file in ehcache.jar, where’s the second one? Well, look into WEB-INF/lib/C6-Common.jar/com/emc/common/java/cache/d2-cache.xml in D2.war/D2-Config.war or in $JMS_DEPLOYMENT_DIR/ServerApps.ear/lib/C6-Common.jar/com/emc/common/java/cache/d2-cache.xml:

<diskStore path=”java.io.tmpdir”/>

Second configuration file (and the one actually being picked up, you can check the D2/D2-Config/JMS logs on DEBUG for confirmation) which contains a value… Actually you can change this to a hardcoded value or to a more suitable solution by setting ehcache.disk.store.dir as value and changing JAVA_OPTS in order to include -Dehcache.disk.store.dir=<your temp path for ehcache files>, which is way less aggresive than changing java.io.tmpdir.

And the third one? Take a look to your JMS ServerApps.ear/APP-INF/classes/d2-ehcache.xml, but note that this file is useless as it not picked by the classpath.

Opentext Documentum is coming next month

I didn’t realize that roadmap documents were updated last month. It looks like the February release is still going to happen (and I’ve been told a definite date, so it looks it won’t be delayed). After reviewing them (haven’t seen any changes :D), I can say:

  • Not much features regarding CS, the pattern/usage visualization and the s3 support (which, as far as I know, can be already done without official support) are the new features.
  • No word about DFS, and I know for a fact that several customers have actively asked for updates to current libs and extended support for application servers.
  • Clients get barely any changes (D2, Webtop, xCP).

I’m curious to see how many bugs are found in this first release from Opentext (and the brave customers that go first into the unknown :D), considering that some of the experienced Documentum staff left the company and the changes to the development cycle (that I guess happen when you move from a hardware company to a software company)

D2 autologin

There was a document in EDN, which now is lost, named “Useful shortcuts and urls for documentum D2”, and one of the URLs was an autologin url, quite useful for development purposes. The url shown was the following one:

D2/?_docbase=<repository>&_username=<username>&_password=<password>

However, although this works fine in D2-Config, it doesn’t work in D2 (>4.2, AFAIK) and this seems to be the case as there are several posts that seem related to this “issue”.

Well, the truth is that there’s still a way to autologin in D2:

D2/?docbase=<repository>&login=<username>&password=<password>

D2-Config DQL Editor

D2-Config (at least in D2 4.5) has a servlet (/GetData) that is used internally to run DQLs. I though when I saw it that, in the same way the REST query service is limited to run read queries, this won’t let you run write queries (you know, due to the “Get” in the name). Well, I was wrong. Any authenticated user can run any kind of query (limited by user permissions, as this is not run as superuser) by using this servlet:

D2-Config/servlet/GetData?dql1=select user_password from (select * from dm_user)&interfaceId=<my session id>&computerName=<my computer id>
D2-Config/servlet/GetData?dql1=create dm_document object set object_name='d2test' link '/Temp'&interfaceId=<my session id>&computerName=<my computer id>
D2-Config/servlet/GetData?dql1=delete dm_document object where folder('/Temp') and object_name='d2test'&interfaceId=<my session id>&computerName=<my computer id>

D2 configuration vs. Doclist widget

One weird requirement customer may want is to show different columns for different types. This kind of requirement is very difficult to meet with obsolete, difficult to customize and migrate webtop, because you’ll need to create a configuration such as:

<config version="1.0">
<scope type="custom_type">
<component id="objectlist" extends="objectlist:webtop/config/objectlist_component.xml">
<columns>
<column>
...
</column>
</columns>
</component>
</scope>
</config>

and be done with it.

Now, how can we achieve this difficult customization in our beloved, configurable, easy to migrate D2? We can’t.

Stupid Useless Limited doclist widget will store a single column preference for the widget, meaning that if you want to apply certain column configuration, it will be applied for every location, no matter the type. Besides, the configuration is stored in the format object_type.attribute. This means that, if you have inherited attributes, you can easily find yourself in the following situations:

  • If you added subtype1.attribute, supertype and other subtypes won’t show any value, and it will only work for subtype1 objects.
  • If you have different labels for the same attribute in different subtypes, only the label for the object_type.attribute you added to the column will be shown (not much of a problem yet, as you won’t see any value).

As far as I know, this has been a feature request since D2 4.5 (released on April 2015), so after almost 3 years, this hasn’t been implemented yet. Even though I’m an optimistic person, and I’m sure (not really) that the “software company” will improve D2, we can, somehow, implement this (in a very rudimentary way, to be honest).

The current behaviour of the doclist widget is basically located in com.emc.d2fs.dctm.content.FolderContent. This class receives a request to retrieve and return the contents of a folder.

The two main methods that are interesting to our requirement would be getContent and getFilteredContent.

These methods receive some parameters, and they generate a query to retrieve objects from the folder, and then return a resulset (DocItems object) with the objects from the folder.

So, in case you want to “configure” the values shown (note that you can’t, or at least I haven’t been able to, change the columns shown in the widget, so data won’t match columns names), you’ll need to:

  • Modify listColNames List of String values with only the attributes you want to retrieve.
  • Take the List of Items returned by the ContentBuilder class and, for each attribute stored, customize the attribute name with the column you want to show the value in.
  • Replace OOTB FolderContent class with your own.

Of course, you have to have this in mind when migrating to a new D2 versions, because I’m quite sure this won’t work as expected in every D2 version…