Posted by James Kobielus on February 20, 2008
As I may have mentioned before, I cover complex event processing (CEP) as it intersects with information and knowledge management (I&KM). Or, more specifically, as it supports real-time business intelligence (BI). Or, perhaps more pedantically, as it enables decision support systems (DSS) to facilitate business agility in response to dynamic conditions.
DSS is the term by which BI was known from the late 70s until the early 90s (fun fact: my first position as an IT industry analyst was in the late 80s, under Peter G.W. Keen, who, along with Michael Scott Morton, published a seminal work on DSS in 1978). DSS is a useful framework for thinking about BI, because it places your focus on the decision "agent," on the flow of that agent's decision-making process, and on the collection of static and dynamic evidence and events that fed that process.
In looking at CEP for I&KM/BI, I'm often compelled to take a DSS point of view. As an analyst, that perspective helps me model the phenomena in the proper context -- i.e.., the many layers of complexity:
- Each event may be quite complex in its own right, standing for a linked set of data updates, application state transitions, and process status changes.
- Each event stream is a complex, linked, ordered, transactional sequence of time-stamped objects.
- Each event-processing engine may consolidate, cleanse, correlate, filter, and persist objects across a complex set of concurrent event streams.
- Each event-reliant decision agent (e.g., end user) may access, interact with, and/or consume events through a complex interface (dashboards, analytics, semantic layer, etc.), across multiple devices (desktop, laptop, Blackberry, etc.) and have a complex event-enriched streaming "experience" (see previous post authored by myself).
- Each event-reliant decision agent may be a complex creature in its own right with its own complex, convoluted, squishy decision-making methodology -- i.e., an individual human being with their own habits and cognitive/psychological dispositions; or a group making decisions collectively and collaboratively through workflow, or social networking; or a half-human/half-automated workflow behaving in the herky-jerky manner one would expect from a split-personality decision agent; or a completely automated orchestration of applications triggered by rules engines, etc.).
In fact, the organization/team/group/collective is the principal uber-decision agent in most businesses. The BI market is beginning to understand that basic fact, and to address it in their solutions. As you all may recall from a previous post of mine, BI solutions are growing more collaboration-oriented, in terms of vendors adding more workflow, instant messaging, social networking, and Web 2.0 functionality to their offerings.
As CEP increasingly permeates vendors' real-time BI/DW portfolios, I fully expect that real-time, event-triggered, under-deadline, continuously refreshing dashboard-sharing collaboration applications will come to the fore. Maybe we'll rely on social networking to make it happen on the fly (per Rob Koplowitz' great coverage of all this). Maybe we'll lean on our online avatars (per Erica Driver’s work in this area) to model various collective responses before we go "live" with it all.
Clearly, collaboration adds another level of complexity to an organization's ability to respond to events. Ask any news organization trying to coordinate the real-time efforts of diverse reporters, editors, photographers, and others on a breaking story. Or any analyst organization trying to whip together a collaborative blogpost laying out their collective position on some fresh industry development. Under deadline, events must be absorbed, responsibilities divvied up, and actions taken -- and pronto, by whoever's holding the baton. Decisions must be made under time- and resource-stress.
Where real-time event-triggered DSS is concerned, collaboration could be a showstopper, if we don't watch out. Collaboration adds complexity, hence the potential for more decision latency. But collaboration also builds buy-in, and, as such, can be worth the muss and fuss of versions, reviews, revisions, and provisional decisions.
Indeed, having your organization fully behind whatever action was ultimately taken is the best support system any decision could possibly have.