Previously on DCTM: The Next Generation

Yesterday’s Docbase

There has been a growing disconnect between how we design for the back end and the front end. The front end developer can apply agile methodologies, use new patterns like composition over inheritance, and organize data based on tags and facets instead of deep hierarchies.  Back-end developers are getting these benefits from new systems like NoSQL databases, but Documentum architects are stuck in the 90s with the current content server–in no small part because it runs on relational databases like Oracle.  Documentum did an excellent job marrying objects and tables before ORM (object-relational mapping) was a catchy initialism; the inherent disparity between the models that was inconvenient before, but now it actively hinders new ideas on the back end.  Basing NGIS on EMC’s XML database xDB changes all that, and traits are a windfall from that decision.

The latest details on traits comes from a video by Jeroen van Rotterdam, IIG’s Chief Architect.  There are two basic constructs for designing with traits:  Trait definitions group data definitions, services, and event models into packages that are attached to objects at runtime.  Type definitions control which traits can or must be attached to objects and how those traits resolve conflicts; e.g., order of precedence or mutual exclusion.  Objects end up being truly lightweight–only an ID attribute for sure and very probably a type attribute as well.  This will be a very different world than the present-day Documentum schema of data-only inheritance, tacked-on type-aware behavior, and a bloated base object type with every possible attribute stuffed into it.

For Documentum architects and server-side programmers, this is the first exciting thing to happen to Documentum since, well, forever.  The model is one of the things that made Documentum stand out; it painted a rich, complex picture of what a document could be. Instead of document as file, a document included versions, renditions, and complex structures with flexible binding (i.e., virtual documents).  A document was a different thing (or collection of things) based on context and function.  It also stepped beyond the straight jacket of the relational model to include multi-value (repeating) attributes and SQL extensions that recognized the world is more like objects than tables.  Although this perspective on document management is still valid today, the methodologies to realize it feel outdated. Traits represent almost two decades of lessons learned since the first docbase schemas struggled out of the primordial scanned-document/shared-folder ooze.

Changes of this magnitude don’t happen often in software systems.  A mature market doesn’t take well to fundamental shifts that aren’t backward compatible, so there’s significant risk here assuming EMC wants to keep its current customers happy. Discussion about migration paths from the current content server to NGIS are still little more then speculation. Given how many customers are turning to other options, being bold may be the solution if EMC can make a better product with a less onerous migration path.  Let’s put aside those messy details for now and consider what traits may mean for the Documentum architect.

Will this become my new Linkedin profile picture?

A  Few of My Favorite Things

It’s hard to say if the implementation of traits will live up to the potential from what we know so far.  I had similar high hopes for DBOF, light-weight objects, Aspects, and DFS when talked about in theory, but they all fell short in implementation either in outright design or by being born prematurely.  Letting my inner optimist out of his cage for a moment, these are some things I hope to see the implementation of traits bring about:

Attributes become rich data rather than columns in a database. Defining and constraining attributes using XSD is an easy win since NGIS sits atop EMC’s XML-based xDB instead of an RDBMS, and it gets rid of the stapled-on data dictionary in the current Documentum toolbox.  I hope this means that attributes can contain complex data–perhaps small XML documents and certainly associative arrays–but removing size constraints and character set problems (odd escape characters, special characters, unicode, etc.) is a huge win regardless. Maybe this will help people realize that attributes aren’t all metadata.  Sometimes, they are the data; e.g., non-content objects.

The broken promises of DBOF and Aspects can finally be fully realized.  A weakness in Documentum’s original model was the separation of data and behavior.  Being object-based, there were familiar ways to organize data (e.g., inheritance) that fell short of real OOP because they didn’t equally apply to behavior.  DBOF was the first attempt to relate code to the type hierarchy; it was hobbled by being duct-taped onto the side of the client libraries (DFC) instead of integrated into the server directly.  Then Aspects repeated DBOF’s same fundamental mistakes.  Traits appear to marry data and behavior nicely, and I can’t imagine them being handled anywhere but together on the server.

Traits eliminate the need for inheritance and fat objects.  Inheritance as a design principle has been under fire for years:  Single inheritance is restricting; multiple inheritance introduces pitfalls along with greater flexibility; both are difficult to refactor.  The single inheritance nature of the Documentum object model wasn’t as restrictive then because it only included data.  With behavior coming along for the ride in NGIS, it becomes a bigger issue.  Just like in programming, it looks like EMC is favoring composition over inheritance.  I haven’t seen anything showing inheritance for types or traits in NGIS yet, and I won’t be surprised if it doesn’t.  Another consequence of this composition-based approach is the object becomes really, really lightweight–little more than identity on one end and trait container on the other.  That looks surprisingly like the really, really lightweight document in MongoDB with it’s single _ID default attribute.

The Documentum architect gets a whole new (NoSQL) toolbox to build and extend systems. Although Documentum grew out of the relational database culture of the time, it broke major relational conventions because the document management problem space felt more object-oriented than table-oriented; i.e., repeating attributes, virtual documents, object-ish extensions to SQL.  Now that NoSQL is here, it’s obviously a better fit:  Documentum architects taking a look at 10Gen’s webinar on schema design in MongoDB should quickly recognize the familiar ground.  What’s new is how dynamic, real-time the data model becomes with traits because they are only attached as needed at runtime, and objects can have different versions of a trait at the same time allowing for upgrades in a conditional or rolling manner.  The docbase will no longer be where content goes to die; it becomes a Livin’ Thing. Hopefully sizing, deploying, and scaling NGIS systems will also look more NoSQL than relational, but traits don’t give us any hints there.

Me if dm_history repeats itself

Cautious Optimism

I’ll admit to some cautious optimism here; traits may be a sign that NGIS will make Documentum architecture interesting again.  Beyond just updating the toolbox, the new capabilities may inspire organizations to start solving new, interesting problems again. Documentum did a good job of getting people to think about document management the first time around, but now everybody “knows” what document management is and what Documentum is “good for”.  Traits hint that NGIS could be a game changer on the much-less-talked-about back end; it won’t directly delight the New User, but a pretty front end on a shaky server foundation won’t be particularly useful to the New User. It’s up to EMC to get the implementation right and get the word out.

Based on the blog post Next Generation Information Server: Traits Explained | Jeroen’s Crazy Content and video:

Don't google "dead horse" for images.

My response to Pie’s Quality of Documentum Over the Years bears repeating, even if I am beating a dead dmHorse:

I started with version 2, back when I was just a newly-minted UNIX geek. One thing you missed with the transition to 4i was the introduction of the DFC. DMCL had a very UNIX feel; a simple, open API designed to be glued into any programming language. DFC was just Java then, with a COM layer growing over it later. That was also the point where EMC became more marketing-driven and started chasing the Internet bubble at the expense of their existing clients.

Both were attempts to capitalize on hot topics of the time, Java and the Web. I never bought that the DFC would make a whole pool of talent available; Documentum’s about the model, not the means. However, the marketroids successfully reframed it. Hiring managers now believe they can take Java people and mold them into Documentum people, and I hear gasps of disbelief when I say Java or Visual Studio aren’t requirements to do Documentum–a good Java programmer is not necessarily a good Documentum developer.

This Java mentality did increase the number of people with Documentum on their resumes, but the talent didn’t increase as much as the volume. It just diluted (maybe also tainted) the pool. It became harder to find good people in the now-mirky waters.

The lack of focus then is what brings us to the lack of quality now. Innovation at the model and server level is rare, and frankly I don’t give Documentum much geek cred anymore because of it. Great ideas like BOF and Aspects are stapled into an API rather than made an inherent part of the product. Too much work up the stack (and on vertical solutions) has made the product top-heavy and tottery. EMC continues to chase markets (i.e., case management) rather than concentrate on making a solid core product.

Noted rumor-monger  Brilliant Leap speculates about Documentum in a world without EMC. Tongues were already wagging at EMC World about the SAP-EMC partnership leading to something a little more intimate. It has enough of the smell of truth to make an irresistible rumor.

As rumors go, I still prefer the Microsoft angle because of the obscene anatomy kissing that IIG is still doing.  Truth is it’s just to easy to dispel: Why buy the cow when you get the milk for free? I doubt Microsoft will be stamping shrink-wrapped boxes of SharePoint with Documentum Inside! anytime soon, but I’d bet they have an infinite number of code monkeys banging away to make their own document management Hamlet. Once they do, it’s bye-bye Documentum! Then all those monkeys will get down to business and start flinging feces IIG’s way.

Part of Documentum’s doom was being a software company bought by a hardware company; however, it won’t be saved if another software company buys it next. Such companies (SAP included) would buy Documentum to augment their flagship product, not eclipse it.  With no Fairy Godmother rescue from being passed around from one wicked step-mother to the next, this story’s ending will be more Le Boheme than Cinderella. Or worse, a more-jackal-than-wolf company that hasn’t innovated for decades might gobble it up to suck the last trickle of marrow from its cracked bones.  *cough* computer associates *cough*

I am no Wall Street cheerleader, especially after my time in Big Finance, but the closest thing to a Fairy Godmother out there is a technology-oriented private equity fund. Such a fund buys troubled companies to turn them around and sell them for a profit. Unlike most of Wall Street, they take the long view of years rather than a quarter or the milliseconds around a stock’s uptick.

Their methods can be harsh, but their goal unlike any step-mother’s would be to make Documentum the best product and most profitable (and saleable) brand it can be.  There may still be an ounce of brand left to save. By going private, the recuperating Documentum wouldn’t be burdened with public company regulation or the tyranny of speculative stockholders. It’s an imperfect cure for the age of gratuitous IPOs and acquisitions fueled more by irrational exuberance than smart business.

We have a test case with AOL selling Bebo to Criterion Capital Partners, LLC instead of just shutting it down. Taking Valdes’s animal shelter metaphor a little further, I’m sure Criterion will euthanize Bebo and reap their own “meaningful tax deduction” if the old dog can’t learn new tricks. Sometimes I think I’d rather see that happen to Documentum than sit through the EMC’s little opera until the consumption takes it.

I can’t find a video of the Mark Lewis keynote from EMC World 2010. Instead I’m depending on reporting from the event like Ron Miller’s article [Documentum group gets new name and new direction] and Pie’s tweets and blog posts. It’s probably for the better; I never had a taste for gruesome videos since Faces of Death, and this may be EMC finally decapitating the Documentum brand.  Rather than plunging into a pages-long diatribe about EMC’s unconditional surrender to the commoditizing of content management or the latest dish of scorn Lewis served up to Documentum veterans, let’s talk names.

Information Intelligence Group is EMC’s new moniker for the product that shall not be named. Lewis breaks down the name for us on his blog [Episode 91: EMC World 2010 - The Birth of the Information Intelligence Group]. It’s hard to read–let alone say–the name with a straight face, and this breakdown doesn’t help. At least the cumbersome and uninspired Content Management and Archiving accurately conveyed something about the product.  This new name is too broad and inherently meaningless; it will continue to erode mindshare for a product that was the de facto definition of document management. Let’s hope this new name doesn’t prove itself a compound oxymoron to boot.

The “Intelligent” product silos aren’t much better. Granted, this is a product that has to publish a separate guide with each new release to map old product names to new. Not a sheet or a few pages, a document. However, these new silos are so vertically restrictive that EMC had to toss the content server into case management.  Having done case management and having paid my dues in lines of server code, I’m perplexed. It’s like they had a very limited box of magnetic poetry to play with.

The continuing erosion of a strong brand means less mindshare among potential customers. Everybody knows SharePoint even though most don’t know what it really is. I’ve seen first-hand how good marketing trumps good product. Documentum had that name recognition–still does in many parts–and EMC seems determined to stamp it out without something sticky to replace it.

Unmentionable!

When Twitter changed their reply functionality, now called mentions, my initial reaction was unmentionable.  After a few weeks to ponder and play with it, I still think they made a big mistake.  A reply was originally a message that began with a twitter username, like this:

@zorak No, really?

Replies were public, but Twitter added a link so you could see just your replies and options to filter other people’s replies out of your friends stream.  According to the Twitter blog, the community came up with the convention that Twitter later embraced and enhanced.  Then Twitter added a separate API call and a “swoosh” button to their web site:

Just what I wanted! Twitter added metadata underneath so that a reply remembered which tweet it replies to.  Pretty soon every Twitter client included swoosh buttons and “in reply to” links.  This was a philosophical break for Twitter–whether they know it or not–because there was no way to distinguish swoosh and “@user …” via SMS.  Supporting SMS creates a larger potential user base, but it drastically limits functionality.  Until everybody has an iPhone, fledgling social networks like Brightkite must consider this trade-off.

The original reply syntax is still supported and continued to create confusion as Tweetie developer AteBits explained on his blog.  People put multiple names in the message or put the @ in the body of the message, assuming the right people would see the replies:

@me @myself @I Remember the milk.
Give @me some sugar, @baby!

Only @me sees the first tweet as a reply; nobody sees the second.  Search was already catching on thanks to other community-grown initiatives like hash tags; users and client developers began using search on @user instead of the reply API to catch such grammatically incorrect tweets.  Apparently this is a bad thing, or at least something Twitter discouraged, perhaps because of its impact on Twitter’s call throttling.  That and other scaling problems should make for a few good dissertations; I just hope Twitter is keeping the historical record and will be willing to share it.

This brings us to mentions which are basically just searches on @user.  Although it’s a good thing that Twitter learns from their community, the big mistake here was changing the functionality under the existing API calls.  I agree that instantly supporting new functionality in all Twitter clients is attractive to a provider, but it can–and did–create unintended consequences.  All those clients blessed with catching those malformed reply tweets were also cursed by all those side-bar mentions crowding the replies page.  Twitterati like @wilw get many more mentions than direct replies, and now there’s no easy way to sort out the two.

The lesson here is that it’s safer to create new API and UI elements for new-ish functionality and let the community migrate over than to replace the guts and hope nothing breaks.  As any API designer knows, developers will do all kinds of unexpected things once your API is released into the wild.  The Twitter community’s active, inventive role in shaping Twitter also provides for some real “They did what?” moments.  Tweaking reply functionality to support only swooshes and adding new methods for mentions would have made everybody happy.

There’s one thing I want from Twitter that they promise in the API FAQ; I want to see all replies for a given tweet.  I disagree the assertion in the Twitter blog post above that people don’t want to wander into the middle of an ongoing conversation.  Sometimes that’s the best way to discover new topics and interesting people.  When that happens, there is a need to go back and discover the source and all its tributaries.  Twitter is aware of the need, as this quote from the Twitter API Wiki FAQ shows:

How do I get all replies to a particular status?
For now, there’s not a great way to do this. We’ve heard the requests, though, and we’ll be providing a solution for it before too long.

Conversation functionality is cropping up in Twitter clients like Nambu and the soon-to-be-released Tweetie for Mac (20 April 2009).  Looks like Nambu constructs conversations based on cached tweets, building little trees as it discovers reply pointers to other already-fetched tweets.  This single-linked list structure makes it easy to find your immediate predecessor but difficult to walk up, across, and back down the tree.

Hmm, where have I seen this problem before?  Oh yeah, version trees in Documentum.  Every document remembers its immediate predecessor (i_antecedent_id) and the root of its version tree family (i_chronicle_id).  A single query on i_chronicle_id returns every version of that document.  That’s just what I want Twitter to do!

Twitter already has its own i_antecedent_id–with a better name I hope.  So add an equivalent to i_chronicle_id and a new getAllReplies API call.  I suggest topic_id since that’s what the root tweet of a tree of replies becomes.  It would be nice to go back and stitch up all the previous replies-of-replies, but I would understand if the hit on the database would be too big.  How many tweets are in there anyway?

Let’s deal with the underlying issue here:  The ESF article is talking about the other 99% of EMC that bought Documentum to sell disk.  EMC friends, I’m just kidding!  I can joke, right?  Since storage is not my expertise beyond some hacking around with Centera as primary content stores, I suppose EMC might really knock their customers’ socks off in the storage arena, but I doubt it.

Network disk isn’t something you see until it isn’t there, just like any good support technology.  It’s not sexy.  Apple’s products are shameless show-boaters meant to hog the spotlight.  They are meant to be seen, to be touched, and–dare I admit–even licked. Maybe that’s not a recommended way to unlock an iPhone, but what else can you do about winter, thick gloves, and a touch screen?

Would the average EMC storage user know the EMC logo on sight?  Would the average Apple user?

OK, so the EMC logo actually says “EMC” in it.  You get the point though, right?

Enderle’s article talks about quality, metrics, and customer loyalty.  All those things are important to Apple, although the often-excellent quality of Apple products is marred on a regular basis with things like incendiary power supplies and the worst product launch ever: iPhone 3G + MobileMe.  WORST!  LAUNCH!  EVER!

Only Starbucks matches Apple’s skill at selling Lifestyle. The synergy of Apple’s well-designed, well-integrated components had this caged bird singing gaily a few posts ago despite a healthy fear of monoculture coming from a science background.  That all misses the point, the one reason the whole discussion is apples and oranges:  Innovation.

Apple sets the bar for technology after technology:  operating systems, mp3 players, online music retailing, and of course smart phones.  The integration, the cool, the marketing are all icing on the cake because Apple does something better than anybody else:  They innovate, and they do it where they can define the market rather than chase it.

I don’t think Big Disk lives or dies on such radical innovation. In fact, their customers probably fear change more than most.  Change is not good for 24/7 availability.  I can hear the compliance officers, archivists, and system admins shrieking in terror at the thought of something that might as likely store their entire repositories on a postage stamp or burst into flames if looked at the wrong way.

There is a relentless integrity of concept, simplicity, and message spanning all Apple products that likely has a single source, Steve Jobs.  Such single-mindedness is what makes big, ambitious, risky, not-for-the-faint-of-heart products succeed or fail spectacularly.  Apple’s done both regularly.  It allows org-wide turn-on-a-dime changes, something that another industry titan *cough*Bill Gates*cough* executed brilliantly after completely missing the Internet as the Next Big Thing.

That conceptual integrity, that vision from the top is also why Apple clung to its single-button mouse a decade too long and why the iPod Touch and iThingThatWillNotBeNamed are missing the aesthetically unpleasing extra two or three buttons needed for touch-without-sight operation.  Because that’s how Steve Jobs sees it, end of story.

I’m sure Joe could give Steve a few helpful hints on running disk farms for MobileMe or handling eDiscovery for the next options scandal, but that’s not the point.  It’s what Jobs teaches his successor and if that successor has the Right Stuff to wield Apple as a single instrument of innovation, lest Apple repeat the recent catastrophes of their rivals to the North.

I have some long-time draft Documentum posts that should see daylight soon as well, and the new smart container feature in D6.5 has me pondering the fate of virtual documents and EMC’s folder fetish. Thanks to Brilliant Leap for sending me the link and distracting me from iPhone adoration for a few minutes.

Finally, kudos to the makers of WordPress 2.7.  Constant interface changes make me a little crazy (glares at Lotus Notes), but this truly was a big step forward for the project.  I’m looking forward to what they do in 2009.

Have you ever retired a Mac by mistake? No.

I bought a unibody MacBook Pro. Aside from being a thing of beauty, the improved GPU means all my games run well under Boot Camp; the Dell XPS 600 may get an early retirement (in the Blade Runner sense).  I love the trackpad and found myself forgoing a mouse while traveling–except under Windows where it’s squirrelly and inconsistent.  I hope Apple starts squeezing out Boot Camp updates as fast as iPhone updates to fix the track pad problem and take full advantage of the dual GPU.  Please cross all appropriate appendages.

I ended my contract in Rockville, MD.  The commute got to me and I ended my contract with F. after a one month extension to see one of my projects through its first deployment to prod. I’m making a pattern of very long separations to help with transition, and that’s both good and bad: Good this time because I think it really made a difference in the deployment; bad because I missed out on the before-end-of-year hiring cycle.  I hope it reboots as usual in the beginning of the new year despite the grim economic situation.

I got the flu.  My last day in Rockville was when the fever kicked in.  The following three weeks were pretty miserable, and I still have a bit of a cough after another two.  Next year I’m definitely getting a flu shot, and I’m wondering if I should still get one this year since they typically treat a handful of the most common strains.  Any medical professionals care to comment?

I drank the Kool Aid and bought an iPhone.  Expect a subsequent post with some of my favorite apps.  I’m testing how long it can last in stand-by mode with power consumption optimization (WiFi and 3G only when needed, no push, hourly pull) and it looks like the answer’s 72 hours.  I may have to bump my SMS messages up to 1500 from 200 which irks me to no end, but that’s one of a handful of gripes that are more about AT&T than the device itself.  Otherwise, it’s a Good Thing ™.

I attended MJD‘s talk at Philadelphia Linux Users Group (PLUG) about strong typing.  The slides for Atypical Types are on MJD’s site.  We agree that Java 1.5 is the first usable version of the 1970s-style programming language but for different reasons:  He asserted at the talk that Java typing got better because a Haskell guy rewrote the 1.5 compiler and force fed Java some good medicine like generics. Is Haskell the new programming language incubator?  Given things like type inference, how a smart compiler removes all strong-type clutter that stupid compilers require, could be.

I corrected a long-standing mistake about Documentum Composer in my I’d Rather Be in Philadelphia blog post.  Despite being a newer Documentum customer, F. had enough invested in its 5.3 architecture and how it integrated into their corp-wide build process to make Composer a non-starter.  I’m looking forward to a contract in the future where I get to use all the new 6.5 stuff without the drag of a 5.x (or even 6.0) install base.

Resume of John Kominetz (20080903)

1. Both folders did indeed have identical object names without leading or trailing whitespace.
2. They did NOT have any values in common between their i_folder_ids, so they weren’t really in the same place. But Desktop Client though otherwise.
3. One had r_folder_path values with trailing whitespace in one of its containing folders.
   e.g. “/cabinet/folder1 /folder2/folder3″ — Hard to eyeball the space after “folder1″ unless you’re looking for it!

After that, I couldn’t reproduce the problem. This docbase was well known for having “issues”, so nobody (including Documentum) was surprised that something impossible and inexplicable was happening. I mentally filed it under “Documentum moving in mysterious ways”, gave the business administrators a tedious but effective work-around, and got back to more urgent concerns.

The next year I ran into this problem again from the engineering side instead of the business unit. As a particular project wrapped up or budget ran out, the company passed me around to different groups. I’d worked in engineering back in 2001, and my knowledge of the company’s custom environment made letting me go difficult.

I ended up helping out with the 4.2.8/5.2.5 upgrades by developing a triage procedure and fixing some of the more dire problems. Ever see those pictures of tumorous, sooty lungs the Cancer Society uses to scare people away from cigarettes? That’s what some of these docbases looked like on the inside. Not fun. Not pretty.

After triaging a hundred docbases, I discovered some of them also had corrupt r_folder_paths, although in smaller numbers than the original-problem docbase. Thankfully it didn’t interfere with the upgrade procedure. I made a few more notes before continuing onto my next engineering task, the arduous quest to get Documentum to fix the horribly-broken-under-Sybase content replication process.

Another year passed, I was passed along to another business unit, and a new folder-related problem emerged. Business users (in webtop) reported empty folders and missing documents. The document management group (in desktop client) reported the documents were right where they filed them. If you guessed those empty folders had corrupt r_folder_paths, you were right. Webtop uses a slightly different query to determine folder contents than desktop client and as a result couldn’t “see” the contents of such a corrupt folder.

There Will Be Pain

You may want to take some ibuprofen before continuing …

This particular client as a rule doesn’t allow anybody to talk directly to the Sybase DBAs. Don’t ask why. Healthy docbases require attentive, knowledgeable DBAs working with developers and docbase administrators. I would probably have figured this problem out by year one if I’d had that kind of healthy relationship, but I didn’t.

Some suggested I master Sybase. Frankly, that’s wasted effort since it’s such a small part of Documentum’s market. If this were Oracle or SQL Server, sure. Thing is, the client desperately needs DBAs cross-trained in Documentum if they were going to tame the growing performance and stability problems. Despite getting a few people in management to shake their heads yes when I said this, nothing ever came of it. Sigh.

So I found the next-best thing to a current Sybase DBA, an ex-DBA in my part of the organization. I started asking questions, and he casually mentioned the “obvious” fact that the Sybase API strips all leading and trailing whitespace from VARCHARs passed to it. I was dumbfounded.

Call me a purist, but this is data corruption by design. If I programmatically pass a string to a database, I expect the exact same string back when I ask for it. That’s just common sense–unless you live on Planet Sybase–but it didn’t really explain the problems.

Most of the affected docbases either had custom scripts or Input Accel adding content, and they all had several layers of folders that related to content metadata. This was the second big clue; both methods of adding content had to sometimes create several layers of new folders to file the first document for a particular client, counterparty, or deal. That’s still didn’t explain it all.

The DMCL caches metadata for newly-created, fetched, and modified objects. That includes folders. It turns out that the cache doesn’t automatically refresh from the database after a save. Now we have the three components needed for the impossible to happen. Let’s walk through the process.

1. Open iapi and connect to a docbase.
2. Create a new folder “B ” whose name has a space at the end (OCR, bad typing, etc.), link it into cabinet “A”, and save it. The cache has object_name as “B ” and r_folder_path as “/A/B “. Sybase stores them as “A” and “/A/B”.
3. Create another new folder “C” in the same session, link it to “B”, and save it. It fetches “B”‘s r_folder_path from the cache and generates its own r_folder_path as “/A/B /C”. Note the space after “B”, and also note that sybase doesn’t have anything to trim because the space is within the string.
4. Put some stuff in “C” and close this session.
5. Navigate down through the new folders in desktop client and “C” has content.
6. Navigate down through the new folders in webtop and “C” appears empty.

Both client applications use “r_folder_path” in the query to find a folder’s content. One remembers and builds the path with some whitespace trimming as it goes along, the other uses literal values from r_folder_path. Can you figure out which is which? That difference is the root of the disappearing content mystery.

The duplicate folders mystery arises from the fact that the DMCL won’t remove corrupt r_folder_path entries when relinking a parent folder triggers a cascade update on its children. That junk stayed in the r_folder_path of the moved “Reports” folder but didn’t block scripts from creating another “Reports” folder there since the validation is on i_folder_id. Both end up with the same corrupt path, as would a third after moving the second, and so on.

There you go. A combination of the DMCL cache, Sybase trimming whitespace, a DMCL application building several levels of folders, one of the non-leaf folders having trailing whitespace, and relinking predecessor folders causes data corruption that manifests itself in several different ways. Fighting other fires, Sybase strangeness, and the improbability of it all kept everybody guessing for years.

By the way, I couldn’t retrieve the API code from the client that demonstrates the problem, but savvy readers should be able to reproduce it.

What should developers do about this?

In custom DMCL code, either trim the whitespace yourself before setting the folder’s name or do a revert/refresh on each folder object right after saving it. I seem to recall that was advice often ignored in the early days of DMCL programming.

In Input Accel, if you can’t hack it with VB or something to do the same, it’s time for a dm_job running dm_fix_folders as often as necessary. Running it regularly would be a good thing for everybody to do actually.

I think the DFC didn’t manifest the problem, and that may explain why I couldn’t reproduce the problem with webtop or Desktop Client–that or those apps trim whitespace along with other client-side validations.

What should Documentum and Sybase do about this?

It’s probably unreasonable to expect Sybase to simply stop trimming whitespace since it may break many systems depending on the behavior. They could however add configuration options to turn off trimming at the database and session levels. Then Documentum could provide options for both on their end, maybe in server.ini. Some installations may depend on standard Sybase behavior when sharing or exposing registered tables to other Sybase instances, so a session option would be polite.

Until and unless Sybase gives the docbase server or DBA some way to deactivate the offending behavior, Documentum could do the same things a DMCL programmer would do explicitly in the DMCL. I still can’t stand the idea of an API trimming whitespace, so I’d prefer they force a cache update on a folder object after a save. The representation in the persistent store should take precedence over a cache to assure consistency with other processes that may have pulled a fresh copy of their own.

Does SQL Server have the same problem?

I never got around to testing SQL Server. Being a Sybase derivative, I wonder. Oracle definitely doesn’t have the problem, but I’d appreciate if somebody with a SQL Server docbase would test this out and post a comment.