Experimental D2-SmartView SDK

The experimental preview of the D2 SmartView SDK is finally available to download. This comes packaged as a zip file containing the SDK, which is a combination of Maven, NPM and NodeJS (not the most attractive combination for Documentum old-timers :D)

So, once we get the zip file, we can do the following to install the SDK:

mkdir sviewsdk
cd sviewsdk/
mv ../d2/smartviewsdk.zip .
unzip smartviewsdk.zip

chmod u+x *.sh
sudo apt-get update
sudo apt install maven
mvn -version

sudo apt install nodejs
node -v

sudo apt install npm
curl -sL https://deb.nodesource.com/setup_14.x | sudo bash -
sudo apt-get install -y nodejs
node -v

sudo npm install -g npm@latest
sudo npm install -g grunt-cli

./ws-init.sh

As the “supported” NPM versions are 12-14, we should manually install 14. You should now run “npm update” to make sure everything is, well, up to date. Then you can launch the documentation by running “npm run documentation”

d2sv-sdk@22.4.0 documentation
node ./utils/doc-server.js

Starting documentation server at http://0.0.0.0:7777

And if you open localhost:7777/sdk:

After reading through the documentation, we should try to run the “workspace assistant”. For this, I had to manually run the following command as the ws-init script didn’t work properly: “sudo npm run postinstall”. After this, you can run the start command “npm start”

d2sv-sdk@22.4.0 start
node ./utils/run-generator-cli.js interface

And you’ll see the “assistant”:

You can browse through the options to see what’s available. For this first test I opted for using the included examples and then compiling them:

After this, I copied the resulting jar file artifact (on the “target” folder, and not the “dist” folder, as the documentation wrongly states) to Smartview and… Smartview no longer starts 😀 So I guess I’ll have to keep investigating… good luck

Documentum D2 container image (or how not to build container images)

We already saw that OpenText clearly fails to understand the concept of container (TIP: a container is a process, not a VM) so it keeps providing “D2/da/webtop/rest” containers, when what they should provide is an “application server” container where end users would mount the custom war file on /webapps or whatever (by the way, this would simplify the yaml/helm charts from hell with the million options to try to configure D2/da/etc. via yaml).

Anyway, it seems that understanding that concept is a lost battle, however, you would expect that at least OpenText would know by now how to properly build a Docker image. Well, to anyone’s surprise (except to OpenText engineers I guess), latest D2 (22.2) image is 3.65GB!!!!!! (This is bigger than Content Server itself)

registry.opentext.com/dctm-d2pp-classic-ol 22.2 1eee2a974793 5 weeks ago 3.65GB

Let’s investigate this wizardry… If we open the image we can see several “big” folders:

Wonder what’s going on here… let’s check that 700mb folder:

Not only we have D2 exploded on Tomcat’s webapps folder, we also have the D2.war on the image… What else do we have on those +100mb folders?

Yum-update cache… (several times)

Python? on a Tomcat (D2) application server??

But, what’s going on here? Well, when you run a multi-staged container build, you need to understand that every command creates a new layer (something clearly explained in the Docker documentation) so you should be extra-careful and delete everything you copy/create in the same step if it doesn’t have to be present on the final image (as explained in the Docker best-practices documentation). So basically, when creating the image, OpenText is first copying D2.war image, then in another stage they extract the files, and then on another step the war file is deleted (or that’s what they think they are doing, but they’re just creating a new layer without the file, not really deleting it). Also, instead of running a single yum-update and deleting the cache on the same stage, the just keep running yum-updates as they need it, effectively creating multiple cache folders…

This image can be easily squashed and you’ll end up with this:

dctm-d2222 22.2 d3d968108687 13 days ago 1.83GB

Exactly the same image, but nearly 2GB smaller, and this is without even bothering to remove the cache/unnecesarry files:

We’ll see if for 22.3 OpenText learns to deliver proper containers or we still have to deal with these +3GB images…

D2 Video Preview Widget

A while ago I did this for webtop (Video streaming from Webtop) which was later published by EMC as a white paper. This is a similar attempt using an external widget on D2.

In this case, I’ve used video.js, a free Javascript video player. The widget itself is quite straightforward, just create a normal external widget, register the D2_EVENT_SELECT_OBJECT and configure the external widget sending via parameter the user name, the repository and the session ticket. Once this is done, you just have to get the r_object_id of the selected document, set that value on the video object via JS, and just enjoy.

You can check the full on code on GitHub.

D2 slow loading login screen

You may have realized that D2 takes some time to load the login screen when you want to log in. The usual behaviour is that after typing D2’s url in your browser, you’ll see something that looks like a loading screen (after d2 spin wheel):

and after a few moments the login screen in shown.

This behaviour is not something Documentum users are used to when using Webtop so, why this happens? Well, by now you’ve probably realized that D2 does not show the repository name, but the repository description. And in a questionable design decision, this processing is done every time the login screen is loaded (=every time a user access).

This happens on C6-Common\com\emc\common\dctm\objects\DfDocbaseMapEx class and the effect is quite clear as it is shown in this log trace:

c.e.c.d.o.DfDocbaseMapEx[] : Load docbases from docbrocker 8.925s

Yes, here we have users waiting for 9 seconds before the login screen is shown, and this happens every single time they try to access D2.

So, what wizardry is behind this odd behaviour? Let’s check it:

public DfDocbaseMapEx(String attrName, String direction) throws DfException {
    StopWatch stopWatch = new StopWatch(true);
    this.m_docbases = new TreeSet(new DfDocbaseComparator(this, attrName, direction));
    IDfClient client = DfClient.getLocalClient();
    IDfDocbaseMap docbaseMap = client.getDocbaseMap();

    int count = docbaseMap.getDocbaseCount();
    for (int i = 0; i < count; i++) {
      try {
        String server;
        IDfTypedObject serverMap = docbaseMap.getServerMap(i);
        String name = docbaseMap.getDocbaseName(i);
        String currentServer = serverMap.getString("i_host_name");

        if (serverMap.findString("r_host_name", currentServer) == -1) {
          server = serverMap.getRepeatingString("r_host_name", 0);
        } else {
          server = currentServer;
        }
        String description = docbaseMap.getDocbaseDescription(i);
        String version = docbaseMap.getServerVersion(i);
        DfDocbaseEx docbase = new DfDocbaseEx(this, name, server, description, version);
        this.m_docbases.add(docbase);
      }
      catch (DfException e) {
        LOG.error("{}", e);
     }
    }
    LOG.debug("Load docbases from docbrocker {}", stopWatch);
  }

Well, there is no much mistery here, getting an IDfTypedObject for each repository registered in your docbroker, every single time a user logs in… it may not be very noticeable with one repository, but try this code with +10 repositories… You can add logging inside the loop and check how much time takes for each repository…

This questionable design decision, makes you also wonder about other behaviors/configurations from D2:

  • Repository filter option: This acts as a simple cosmetic filter, as you can see that there’s no filtering when the list is populated. This means that you can have one repository listed, but you still have to wait for all the repositories to be “processed”
  • If you have enabled autologin/default repository, you’ll face the same situation, users will be logged in directly… after several seconds waiting for the login screen (that they won’t see) to finish loading.

So, what can we do to fix it (besides waiting for OT to fix it)?. Well, as you can see the code is no rocket science, so there are a bunch of possibilities:

  1. Don’t process every single time the repository list (basically check if this.m_docbases == null or size == 0)
  2. Load settings.properties and actually apply the repository filter inside the for loop
  3. Process autologin setting as in #2
  4. Add an option in D2FS/settings.properties to use repository name and forget about description
  5.  Don’t instantiate the servermap and use i_host_name from docbaseMap

By implementing any of these approaches you’ll get something like this when loading the repository list:

Standard D2:
repo1:16.4.0100.0153 Linux64.Oracle -> 1.087s
repo2:16.4.0100.0153 Linux64.Oracle -> 1.093s
repo3:16.4.0100.0153 Linux64.Oracle -> 1.098s
repo4:16.4.0100.0153 Linux64.Oracle -> 1.108s
Total: 4.389s
 
Modified D2:
repo1:16.4.0100.0153 Linux64.Oracle -> 0.000s
repo2:16.4.0100.0153 Linux64.Oracle -> 0.000s
repo3:16.4.0100.0153 Linux64.Oracle -> 0.000s
repo4:16.4.0100.0153 Linux64.Oracle -> 0.000s
Total: 0.002s

And users will be much happier than before 😀

D2 Config zip export without D2-Config

In a previous post I explained how to perform a full import of the D2 configuration without using D2-Config (D2-Config without ActiveX). You can also export the D2-Configuration to a XML file by using the bundled d2configutils in D2-Config. But how about exporting a full D2-Config as a zip file (including images, etc.)?. Well, here you go:

getSManager();
session=sm.getSession("repository");

D2Session.initTBOEx(session,false);
D2Config config=new D2Config(session);

//this will place the zip file in your temp folder
File zipConfigfile = config.getZipExport(null, null, false, new ArrayList());

sm.release(session);

If you want to export just a single application/configuration, just change the second parameter to the name of your application/configuration.

 

Custom labels for column values in D2 queries

Problem: We have a query that is using aliases (r_object_type as cod) that shows documents on D2’s objectlist. We want to use a custom dictionary for those values, but the dictionary values are not applied.

Solution: This mappings are stored in d2_attribute_mapping_config, this table contains a list of repeating attributes that store those mappings:

  • type_name: type with the attribute to map
  • attr_name: name of the attribute that contains the values to map
  • dictionary_name: name of the dictionary with the values to show
  • alias_locale_name: auto/language of the values to show

So, this table stores in the repeating attributes the values of the type, attribute and dictionary. This means that you can only map columns that have attribute names.

Solution? Workaround? Append new values to those attributes, indicating the alias you are using, the type used in the “from” clause of the query and the dictionary, and you’re good to go.

Of course, this is completely unsupported by Opentext 🙂

D2-Config without ActiveX

If somehow you find yourself working for a customer with very strict security policies that don’t allow ActiveX components, you can still work with D2-Config even if you cannot install the activex component.

You can access /D2-Config/ConnectDialog.html, ignore the popups and login as usual, then you’ll be redirected to /D2-Config/interface.html?interfaceId=null# which is the page with D2-Config matrix.

Every time you click somewhere you’ll get a popup stating the C6 ActiveX is not loaded but you can simply ignore it, as most of the application will work.

If you need to perform a full import of a configuration, you can do so with the following code (note that this will reset D2-Config configurations):

getSManager();
session=sm.getSession("repository");

D2Session.initTBOEx(session,false);
D2Config config=new D2Config(session);

config.importZip(new File("full path to Zip file with configuration"),
true, null, null, true, true, null);

Map arguments = new HashMap();
arguments.put("-callerUrl", "http://server/D2/");
arguments.put("-all", Boolean.TRUE);
D2Method.start(session, D2RefreshCacheMethod.class, arguments);

sm.release(session);

You’ll need to add to your project d2-api.jar and c6-common.jar, mark as approved the dfc instance used by the program’s dfc, and set the java.security parameter as explained in D2 4.7/16.4 configuration in eclipse

D2 4.7/16.4 configuration in eclipse

This post is quite similar to the Debugging D2 4.5/4.6 previously posted, explaining how to set up D2 without lockbox, but much more simple due to the missing lockbox.

Extract all files from D2/D2-Config to a local folder in your computer, configure a new (tomcat) server in Eclipse, and configure the following parameters:

  • JVM arguments:
    • -Djava.io.tmpdir=”<absolute path to some temp folder>\d2″ (make sure this folder exists)
    • -Djava.security.policy=file:///<absolute path to java.policy file> (the documentation states to change java.policy in your JRE, but I rather use this cleaner solution: create a local java.policy file with a single line:

      grant { permission com.documentum.fc.client.impl.bof.security.RolePermission “*”, “propagate”;}; and reference it with the java.security.policy parameter)

    • Additional memory/custom parameters

Now, start the server, take note of the dfc.keystore id, mark it as approved privilege client in DA and you are good to go.

OTEW 2018 fun

I’m not attending the event being held in Toronto, but I found through twitter this nice url:

http://hol-host05.eastus.cloudapp.azure.com:81/d2-unity-web/ui/app.html -> This is the new D2 UI (and yes, you can use the you-know-which-default-user(s) to log in and check it by yourself) deployed on Azure (which is weird, considering Opentext has its own cloud…)

But, the really funny thing here, are these urls:

http://hol-host05.eastus.cloudapp.azure.com:81/da -> da 7.3 (but with CS 16.4/SQL Server)

http://hol-host05.eastus.cloudapp.azure.com:81/D2 -> hello old D2 vulnerabilities 🙂

http://hol-host05.eastus.cloudapp.azure.com:81/d2-unity-web/repositories -> and you can log in with you-know-which-default-user(s), and you have a nice DQL tool provided by REST services 🙂