BI, Analytics, And CEP: Some Fruitful Potential Follow-Ons From Software AG’s Acquisition Of IDS Scheer

 by James Kobielus

Yes, of course, Software AG is buying IDS Scheer primarily for the latter’s ARIS family of business process management (BPM) tools. I’ll leave it to my Forrester colleagues who focus on BPM--on both the IT and TI sides of the house--to call out the ramifications for Software AG’s positioning in that market.

But, believe it or not, this deal will also launch Software AG into the growing markets for business intelligence (BI), analytics, and complex event processing (CEP) solutions. We bet you didn’t realize that IDS Scheer has ARIS solutions in these fast growing markets, but in fact they do--and they’re continue to evolve those offerings.

It’s no surprise that IDS Scheer’s BI, analytics, and CEP offerings supplement and extend its BPM portfolio. Its CEP solution, ARIS Process Event Monitor, supports business activity monitoring (BAM). Its analytics offerings, ARIS Process Performance Management and ARIS Performance Dashboard, support visualization, dashboarding, scorecarding, drilldown, and alerting on process key performance indicators (KPIs), both historical and real-time. And its forthcoming BI offering, ARIS MashZone, will support self-service user development of reports, dashboards, and other views of process and business metrics.

IDS Scheer has little market share in these non-core segments. And the vendor is no immediate threat, by itself or under its future corporate parent, to the leaders in the BI, analytics, and CEP segments. Indeed, its forthcoming mashup-oriented BI offering only provides a subset of the features available from market leaders such as SAP Business Objects, IBM Cognos, and MicroStrategy. But the fact that Software AG will soon be able to provide its own offerings in those segments, rather than rely wholly on partners, represents an important step in its attempt to field a full service oriented architecture (SOA) solution stack.

As noted in a blog entry a year and a half ago, BI is the crown jewel in any comprehensive SOA solution portfolio. SOA suites cannot be considered feature-complete unless they incorporate a comprehensive range of BI features. This acquisition continues the ongoing SOA solution build-out strategy that motivated Software AG to acquire webMethods in 2007.

But it’s not clear yet whether Software AG plans to flesh out its BI, analytics, and CEP strategies going forward and thereby confront SAP, Oracle, IBM, Microsoft, and other SOA full-stack vendors head-on in these segments. It is also unclear how much effort or expense Software AG would incur in extricating the IDS Scheer offerings from the larger ARIS portfolio in order to make them more general-purpose and less BPM-centric. Nevertheless, Software AG will at the very least have a strong set of enabling technologies to support any such strategy in the near future.

What’s most exciting, and potentially differentiating, about the Software AG/IDS Scheer BI portfolio is the combination of CEP with mashup and an in-memory architecture to support truly real-time, interactive analytics. In other words, Software AG/IDS Scheer could take a page out of the book of another SOA full-stack vendor: TIBCO and its Spotfire product group. In doing so, Software AG/IDS Scheer would also be well-positioned to duke it out with SAP, IBM, Microsoft, and Oracle, all of which are beginning to emphasize in-memory CEP-enabled BI strategies. As we noted in a report from late 2008, in-memory architectures are coming to dominate the BI arena. Likewise, Forrester has called attention in to the growing adoption of CEP for truly real-time BI.

Whether Software AG capitalizes on the opportunity to expand its SOA solution stack into BI remains to be seen. Considering that it took Oracle more than a year to publicly declare how it will position BEA’s CEP and data federation technologies within its own SOA stack, we may have to wait a while before Software AG and IDS Scheer craft an equivalent roadmap--if they ever do.

But if they wait too long, the newly merging vendors may find that the dynamic SOA, BI, and CEP markets have passed them by.

After So Many Years Of Ballyhoo, Semantic Web Still Searching For Killer App

By James Kobielus

Cynics might call Semantic Web a technology looking for a solution. And they might have a point.

Semantic Web refers to a long-running World Wide Web Consortium (W3C) initiative that is working toward an ambitious--some might say hopelessly Utopian--goal. At heart, it is a vision for how the World Wide Web should evolve to realize its full interoperability potential.

People vary widely in how they interpret the scope of the Semantic Web initiative. The tech industries are swarming with a wide range of projects, products, and tools that implement different variants of this vision. What vision is that? In the broadest sense, Semantic Web refers to an all-encompassing metadata, description, and policy layer that can, potentially, support universal, automatic, comprehensive end-to-end interoperability across every macro or micro entity—including data, components, services, applications, and services—on every conceivable level.

Whew!!! If that’s not the working definition of “pie in the sky” or “boil the ocean” (pick your metaphor), I don’t know what is. In fact, I’m hard-pressed to refer to Semantic Web as a definable market or solution segment. However, it’s not entirely vacuous.

For starters, organizations can implement W3C-developed semantic description standards—such as Resource Description Framework (RDF) and Web Ontology Language (OWL)--to make the meaning of content unambiguously comprehensible to services, applications, bots, and other automated components. Second, there is a reasonably robust market for “ontology” tools to support RDF/OWL-based modeling of application semantics. Finally, there is some incremental adoption of these tools and concepts in established IT segments, such as:

• Enterprise content management (ECM): Semantic approaches can support more powerful discovery, indexing, search, classification, commentary, and navigation across heterogeneous stores of unstructured and semi-structured content. Semantic search—driven by concepts, not mere text strings--is regarded by some as a primary Semantic Web application. Indeed, many Semantic Web vendors are primarily implementing the technology in search engines that leverage ontology-based concepts to improve search accuracy and reduce spurious hits.

• Enterprise information integration (EII):Semantic approaches enable consolidated viewing, query, and update of structured data that has been retrieved from diverse sources. Indeed, most commercial EII environments present an abstract semantic layer that mediates access to heterogeneous data, such as enterprise resource planning and customer relationship management applications, converging it all to a common presentation-side schema. A handful of those EII vendors have begun to support Semantic Web standards, primarily through third-party software plug-ins

• Enterprise service bus (ESB):Semantic approaches can facilitate multilayered application, process, and service interoperability across disparate environments. To date, there has been little production implementation of Semantic Web standards in the ESB arena, though some vendors have adopted semantics, ontologies, and RDF to describe the conceptual models implemented by application endpoints, agents, and intermediary nodes within ESB-like middleware approaches such as event stream processing.

But Semantic Web approaches are still on the periphery of these markets. 10+ years into its inception, Semantic Web still has no clear killer app. It’s not clear if or when that app will emerge.

Meet One-On-One With Forrester Analysts At Our Business & Technology Leadership Forum 2008

Consistently rated as one of the most popular features of Forrester Events, one-on-one meetings give you the opportunity to discuss the unique technology issues facing your organization with Forrester analysts. Business & Technology Leadership Forum attendees may schedule up to two 20-minute one-on-one meetings with the Forrester analysts of their choice, depending on availability. Registered attendees will be able to schedule one-on-one meetings starting on Monday September 15, 2008. Book early!

• a
• b
• c
• d
• e
• f
• g
• h
• i
• j
• k
• l
• m
• n
• o
• p
• q
• r
• s
• t
• u
• v
• w
• x
• y
• z

B

William Band
Research coverage for Business Process & Applications professionals
Customer relationship management applications, customer experience management, stakeholder alignment, enterprise CRM
See all »

Matthew Brown
Research coverage for Information & Knowledge Management professionals
Marketing and advertising, enterprise portals, intranets and extranets, information and knowledge management
See all »

Peter Burris
Research coverage for Technology Product Management & Marketing professionals
Enterprise marketing platforms, marketing automation, high-tech, application development
See all »

Back to top »

C

Bobby Cameron
Research coverage for CIOs
IT governance, risk, and compliance; the marketing of IT; serving the business; security and risk
See all »

Marc Cecere
Research coverage for CIOs
Designing IT organizations, changing the culture of an IT organization, IT strategic planning
See all »

Patrick M. Connaughton
Research coverage for Business Process & Applications professionals
Supply chain management services, supply chain management applications, enterprise mobility, RFID
See all »

Alex Cullen
Research coverage for CIOs
IT organization; IT strategy, planning, and governance; organizational design and change management, IT management
See all »

Back to top »

E

Boris Evelson
Research coverage for Information & Knowledge Management professionals
Information and knowledge management, business intelligence, OLAP, data warehousing
See all »

Back to top »

G

Chip Gliedman
Research coverage for Business Process & Applications professionals
Customer relationship management, help desk/service desk, customer service and support, packaged applications
See all »

Back to top »

H

Paul D. Hamerman
Research coverage for Business Process & Applications professionals
ERP, human capital management, financial management, business performance solutions
See all »

Brian W. Hill
Research coverage for Information & Knowledge Management professionals
eDiscovery, archiving, records and retention management, enterprise content management (ECM)
See all »

Bradford J. Holmes
Research coverage for Vendor Strategy professionals
Tech marketing tools and best practices; government, high-tech, tech marketing strategies
See all »

Back to top »

K

Rob Karel
Research coverage for Information & Knowledge Management professionals
Information and knowledge management, integration technologies, metadata management, extract
See all »

Rob Koplowitz
Research coverage for Information & Knowledge Management professionals
Information Workplace, collaboration strategy, collaborative platforms, SharePoint
See all »

Back to top »

L

George Lawrie
Research coverage for Business Process & Applications professionals
Retail information technology; consumer goods supply chain; pricing, promotions, and revenue optimization; collaborative processes such as trade promotions management and sales; and operations planning
See all »

Sharyn Leaver
Research coverage for Business Process & Applications professionals
Packaged applications, business process management, ERP, application strategy and selection
See all »

Craig Le Clair
Research coverage for Information & Knowledge Management professionals
ECM, BPM, output management, document processing services
See all »

Back to top »

M

Pete Marston
Research coverage for Business Process & Applications professionals
Customer relationship management, sales force management, software-as-a-service, outsourcing
See all »

Kyle McNabb
Research coverage for Information & Knowledge Management professionals
Information and knowledge management, document imaging, eForms and information capture, enterprise content management
See all »

Thomas Mendel, Ph.D.
Research coverage for Vendor Strategy professionals
Product portfolio strategies, mobile services, business service management, data center management
See all »

Connie Moore
Research coverage for Information & Knowledge Management professionals
Information and knowledge management, business process optimization, IT organization, enterprise content management
See all »

Back to top »

O

Leslie Owens
Research coverage for Information & Knowledge Management professionals
Information and knowledge management, taxonomy and classification, enterprise search platforms, text mining and analytics
See all »

Back to top »

P

Natalie L. Petouhoff, Ph.D.
Research coverage for Business Process & Applications professionals
Customer service and support, customer experience, customer experience management, business strategy for customer experience
See all »

Lisa Pierce
Research coverage for IT Infrastructure & Operations professionals
Voice services, telecommunications services by region, remote access infrastructure, networking
See all »

Tom Pohlmann
Research coverage for CIOs
Business models, high-tech, corporate strategy, tech sector economics, product and solutions strategies
See all »

Stephen Powers
Research coverage for Information & Knowledge Management professionals
Information and knowledge management, digital asset management, enterprise content management, Web content management
See all »

Back to top »

R

Stefan Ried, Ph.D.
Research coverage for Vendor Strategy professionals
Enterprise architecture, Service-oriented architecture, application platforms and programming strategy; application development
See all »

Back to top »

S

Ted Schadler
Research coverage for Information & Knowledge Management professionals
Real-time collaboration tools (instant messaging, presence, document sharing, etc.), cloud-based collaboration and email, mobile collaboration tools and applications, virtual worlds for the enterprise
See all »

Claire Schooley
Research coverage for Information & Knowledge Management professionals
eLearning, information and knowledge management, videoconferencing, Web conferencing, enterprise collaboration, new workforce, retiring workforce
See all »

Back to top »

T

Scott Tiazkun
Research coverage for Business Process & Applications professionals
Financial management; governance, risk, and compliance; financial management applications; security and risk
See all »

Zach Thomas
Research coverage for Business Process & Applications professionals
Human resources management applications, compensation, recruitment strategies, packaged applications
See all »

Back to top »

W

Tim Walters, Ph.D.
Research coverage for Information & Knowledge Management professionals
Web content management, enterprise content management, digital asset management, information and knowledge management
See all »

R "Ray" Wang
Research coverage for Business Process & Applications professionals
Enterprise apps and ERP, software contract negotiations, software partnerships and ecosystems, customer data integration
See all »

Doug Washburn
Research coverage for IT Infrastructure & Operations professionals
Green IT, IT organization, IT infrastructure and operations, IT management
See all »

Back to top »

Y

Gil Yehuda
Research coverage for Information & Knowledge Management professionals
Enterprise Web 2.0 and Social Computing; collaboration strategy, tools, and culture; virtual communities of practice; virtual team collaboration
See all »

Back to top »

CEP For Real-Time BI: Vendor Announcement Events Come In Threes, Apparently

By James Kobielus

As is clear from the general pattern of my Forrester I&KM blog posts, I'm closely tracking the penetration of complex event processing (CEP) into the world of real-time BI and data warehousing (DW). I'm working on a report now providing best practices for real-time DW, which will be available later in Q2. I'll follow up with a trends document on CEP for real-time BI in Q3.

One thing I've noticed is that the BI industry doesn't fully realize how deeply and quickly CEP is beginning to permeate reporting, dashboarding, analytics, and other decision-support apps. Case in point: I was interviewed yesterday by Mary Hayes Weier of InformationWeek, a reporter who definitely knows the BI market. She asked me about areas where BEA might complement Oracle's BI portfolio. I mentioned operational BI (leveraging the semantic federation/virtualization/abstraction layer in BEA AquaLogic Data Services Platform) and CEP (leveraging BEA WebLogic Event Server). She understood the operational BI side of it right away, but stopped me mid-sentence with the question, "What's CEP? I've never heard that acronym before."

Not to single out Mary — who, as I stressed, is definitely one of the best tech reporters in the BI arena — but I've also encountered this same lack of CEP familiarity among BI vendors and users. What I say to everybody is that CEP, primarily associated with real-time SOA apps such as business activity monitoring (BAM), is an enabler for truly real-time (i.e., end-to-end streaming subsecond-latency) BI — as a complement and/or alternative to traditional historical/thru-the-EDW BI. The fact that CEP familiarity is low in the BI space right now is because few of the leading BI vendors have much of a BI-integrated CEP capability — yet.

Interestingly, right after I got off the phone with Mary, I checked the day's news releases and encountered three product announcements ("events"!) from vendors who have staked their BI strategy purely on CEP, both on the middleware side and in the front-end presentation, interaction, and analytic layer. I'm speaking of these vendors/announcements (the titles of which I've edited down to avoid propagating gratuitous marketing boilerplate in this, the Forrester blog):

• Altosoft Introduces Altosoft Insight 3.0: This announcement concerns enhancements to Altosoft Insight. First, the new version 3.0 includes enhancements for continuous event-based business activity monitoring (BAM) and rule-driven workflow-exception alerting across diverse operational systems — i.e., one of CEP's traditional core markets. Second, it includes new functionality for user-driven, code-free, AJAX-enabled visual development of continually refreshable browser-based reports — i.e, for CEP-infused low-latency BI. Third, it beefs up the low-latency event-streaming middleware features of the product, with tweaks to its high-speed, 64-bit in-memory calculation engine for real-time data aggregation, transformation, and optimization, plus cache acceleration for complex operations like multi-source outer joins with intelligent calculation and compression for optimized memory management, while minimizing loading on operational event/data sources. Note that Altosoft can also do dashboarding, reporting, and analytics on data retrieved from third-party DWs, and join/analyze this data alongside continuously refreshed-via-CEP operational BI data from many sources.

• Truviso Delivers Next-Generation Analytics Solution: This announcement concerns enhancements in the new Truviso v2.5. This version adds functionality for high availability; event-based real-time integration with enterprise data sources; event-driven integration with user interfaces (e.g, browsers, portals); and interactive visualization of continuously refreshed CEP/BI analytics. It also supports development of models for customized event-based data flows, including stream-staged data-integration and changed data propagation/visibility. Clearly, changed data capture and low-latency end-to-end propagation is a middleware area where CEP/BI overlaps with best practices for near-real-time/"active" DW.

• TIBCO Spotfire Delivers Visual, Interactive Business Mashups: This announcement concerns Spotfire 2.1, the latest set of user-driven development enhancements to the browser-oriented CEP/BI solution family that TIBCO acquired almost a year ago. This version continues to enhance Spotfire's impressive capabilities for interactive visualization of myriad live event-data feeds in BI/analytics apps, such as reports, dashboards, trending graphs, and comparison charts, which display inside the TIBCO Spotfire Web Player. Spotfire 2.1 implements a Web 2.0 mashup-style development approach.

Of course, this is just a convenience sampling of CEP/BI product announcements on one day. There are of course other vendors going deep on CEP, for BAM and/or BI (re BI, Agent Logic, Aleri, Coral8, GemStone Systems, and Syndera also spring to mind). I will cover them in greater depth in my forthcoming report in Q3. As I noted above, the leading BI vendors are conspicuous, so far, by their absence from this hot new niche. But I suspect that, by the time I publish that report, one or more of the bigger BI vendors will have their own CEP roadmaps mapped out to a greater degree.

For I&KM professionals in the here and now, though, if you want truly real-time BI, you should seriously explore the solutions from these and other CEP/BI pioneers. You should consider these offerings either as an alternative to real-time DW, or as an adjunct to the traditional historical/static EDW for real-time operational BI.

In my upcoming real-time DW report, I'll provide pointers for identifying deployment scenarios for going with real-time DW vs. CEP for real-time/operational BI. The distinctions between these approaches are rapidly blurring. I wouldn't be surprised to see the established EDW vendors add CEP-like deterministic-subsecond-latency features to their offerings.

Oh... I pointed Mary to these CEP/BI vendor announcements. Whether she follows up by speaking with them is up to her. But three such announcements in rapid succession on a single morning tends to catch my attention.

Important, but not earth-shattering, events in a steady and growing stream.

Jim

Competitive Business Intelligence, Harnessed Through Collaboration And CEP, Harvested Across The Cloud

By James Kobielus

Sometimes ideas for blog posts flow out of everyday conversations with colleagues. I want to thank Leslie Owens and Matt Brown for stimulating the following thought train.

The external competitive environment is the cloud where opportunities and threats hang, sometimes latent, sometimes looming. So it only makes sense that enterprises will outsource more of the competitive surveillance to the cloud of external resources, such as analyst firms, third-party market intelligence subscription feeds, social networking, Web 2.0, etc.

Of course, enterprises realize they don't dare outsource the competitive intelligence function entirely. That explains why they maintain research staff, tools, portals, and informational resources in-house. Mostly, these competitive intelligence teams monitor the prevailing market conditions that impact on their companies' core businesses. But, to a great degree, they also serve as an early-warning system helping their organization respond to specific breaking events -- i.e., the "disrupters" -- that threaten to capsize the corporate boat.

Recognizing this perennial "disrupter pre-emption" requirement, enterprises are concerned with best practices for setting up event-driven competitive intelligence operations. These best practices should help them survey the external horizon more comprehensively and proactively. Best practices should also help them foster, harness, and harvest internal collaboration among competitive-intelligence subject matter experts.

Essentially, competitive intelligence operations of this sort practice CEP in the following senses that I described in a previous post:

• Each event may be quite complex in its own right, standing for a linked set of data updates, application state transitions, and process status changes (NEW NOTE: a "disrupter" is any extremely complex, perhaps way too vague, but still undeniably important "event" -- disrupters aren't "tagged" as such -- and they may not be easily identified before, during, or after the fact -- maybe, hey Leslie, on-the-fly social tagging is the best way to approach this squishiness).
• Each event-reliant decision agent (e.g., end user) may access, interact with, and/or consume events through a complex interface (dashboards, analytics, semantic layer, etc.), across multiple devices (desktop, laptop, Blackberry, etc.) and have a complex event-enriched streaming  "experience" (NEW NOTE: competitive intelligence groups make use of the full range of portals, e-mail, IM, search engines, social networking, wikis, blogs, podcasts, workflow, alerts/notifications, etc.).
• Each event-reliant decision agent may be a complex creature in its own right with its own complex, convoluted, squishy decision-making methodology -- i.e., an individual human being with their own habits and cognitive/psychological dispositions; or a group making decisions collectively and collaboratively through workflow, or social networking; or a half-human/half-automated workflow behaving in the herky-jerky manner one would expect from a split-personality decision agent; or a completely automated orchestration of applications triggered by rules engines, etc.) (NEW NOTE: competitive intelligence teams are very human teaming environments, at heart -- everybody is a "sentinel" on the "lookout" for critical events while others are "sleeping" or attending to something else).

Tying in another observation from that earlier post, I expect that CEP for I&KM (i.e., real-time, event-triggered, under-deadline, continuously-refreshing dashboard-sharing collaboration applications) will play a key role in event-driven competitive intelligence everywhere. CEP, hence BI, will be used to beef up organizations' in-house competitive intelligence/surveillance function, supplementing (hopefully not replacing) the outsourced competitive intelligence/surveillance they get from analyst firms such as Forrester.

In such an environment, the self-service, in-house research portal's the chief presentation layer, and BI operates as an intelligence source and/or target accessible via the portal, with real-time/near-real-time data integration approaches (e.g., ESP, CDC, MOM, trickle-feed ETL, etc.) providing the low-latency plumbing to deliver those feeds to the DW/BI/portal/preso front-end.

The BI vendors are laying the foundation for this emerging best practice. If you look at what vendors such as Business Objects are doing, they're making more external, commercial, competitive intelligence feeds accessible, via partnerships with content aggregators/publishers, from their platforms (e.g., http://www.businessobjects.com/news/press_release.asp?id=20070521_006524). They're also providing text mining/analytics-integrated tools (e.g., http://www.businessobjects.com/news/press_release.asp?id=20070924_006494) for searching across internal and external, unstructured/semi-structured data sources. And they're expanding the social networking and other collaborative features, and mashup offerings, for bringing together real-time feeds of internal/external data/events (e.g., http://www.businessobjects.com/news/press_release.asp?id=20080313_00001).

Business Objects is a bit ahead of the industry curve on all these things. But it's clear that, as market leader, they've laid down the chief challenge for all BI vendors, to make their offerings more pervasive in competitive intelligence use cases, and also to harvest the informational resources of the Web 2.0 cloud to the max.

Complex Decisions Driven, But Not Overtaken, By Events

By James Kobielus

As I may have mentioned before, I cover complex event processing (CEP) as it intersects with information and knowledge management (I&KM). Or, more specifically, as it supports real-time business intelligence (BI). Or, perhaps more pedantically, as it enables decision support systems (DSS) to facilitate business agility in response to dynamic conditions.

DSS is the term by which BI was known from the late 70s until the early 90s (fun fact: my first position as an IT industry analyst was in the late 80s, under Peter G.W. Keen, who, along with Michael Scott Morton, published a seminal work on DSS in 1978). DSS is a useful framework for thinking about BI, because it places your focus on the decision "agent," on the flow of that agent's decision-making process, and on the collection of static and dynamic evidence and events that fed that process.

In looking at CEP for I&KM/BI, I'm often compelled to take a DSS point of view. As an analyst, that perspective helps me model the phenomena in the proper context -- i.e.., the many layers of complexity:

• Each event may be quite complex in its own right, standing for a linked set of data updates, application state transitions, and process status changes.
• Each event stream is a complex, linked, ordered, transactional sequence of time-stamped      objects.
• Each event-processing engine may consolidate, cleanse, correlate, filter, and persist objects across a complex set of concurrent event streams.
• Each event-reliant decision agent (e.g., end user) may access, interact with, and/or consume events through a complex interface (dashboards, analytics, semantic layer, etc.), across multiple devices (desktop, laptop, Blackberry, etc.) and have a complex event-enriched streaming "experience" (see previous post authored by myself).
• Each event-reliant decision agent may be a complex creature in its own right with its own complex, convoluted, squishy decision-making methodology -- i.e., an individual human being with their own habits and cognitive/psychological dispositions; or a group making decisions collectively and collaboratively through workflow, or social networking; or a half-human/half-automated workflow behaving in the herky-jerky manner one would expect from a split-personality decision agent; or a completely automated orchestration of applications triggered by rules engines, etc.).

In fact, the organization/team/group/collective is the principal uber-decision agent in most businesses. The BI market is beginning to understand that basic fact, and to address it in their solutions. As you all may recall from a previous post of mine, BI solutions are growing more collaboration-oriented, in terms of vendors adding more workflow, instant messaging, social networking, and Web 2.0 functionality to their offerings.

As CEP increasingly permeates vendors' real-time BI/DW portfolios, I fully expect that real-time, event-triggered, under-deadline, continuously refreshing dashboard-sharing collaboration applications will come to the fore. Maybe we'll rely on social networking to make it happen on the fly (per Rob Koplowitz' great coverage of all this). Maybe we'll lean on our online avatars (per Erica Driver’s work in this area) to model various collective responses before we go "live" with it all.

Clearly, collaboration adds another level of complexity to an organization's ability to respond to events. Ask any news organization trying to coordinate the real-time efforts of diverse reporters, editors, photographers, and others on a breaking story. Or any analyst organization trying to whip together a collaborative blogpost laying out their collective position on some fresh industry development. Under deadline, events must be absorbed, responsibilities divvied up, and actions taken -- and pronto, by whoever's holding the baton. Decisions must be made under time- and resource-stress.

Where real-time event-triggered DSS is concerned, collaboration could be a showstopper, if we don't watch out. Collaboration adds complexity, hence the potential for more decision latency. But collaboration also builds buy-in, and, as such, can be worth the muss and fuss of versions, reviews, revisions, and provisional decisions.

Indeed, having your organization fully behind whatever action was ultimately taken is the best support system any decision could possibly have.

Complex Events, Simple Experiences

By James Kobielus

Before the week is out, a few more thoughts inspired by various things I experienced at and around Forrester's Enterprise Architecture (EA) Forum last week in beautiful (but surprisingly chilly) Coronado, California.

What stuck with me was the presentation and demo by Kevin Lynch, CTO of Adobe. Though I was already quite familiar with Adobe’s Flex technology for rich Internet applications (RIA) technology, I liked the fact that Lynch presented Flex this time around as a business tool--in other words, as an interactive visualization technology for business intelligence (BI), business performance optimization, and event-rich analytics. All of which made perfect sense in a forum for enterprise architects.

As I was watching the demo, I began to think that RIA can be a double-edge sword, where BI is concerned, in terms of RIA’s potential to deliver meaningful, actionable intelligence for decision support.

On the one hand, as Lynch demonstrated, BI application developers can easily--if they don’t restrain themselves--craft any arbitrarily complex configuration of dashboards, scorecards, graphs, reports, query boxes, tickers, widgets, pop-up windows, and other visual (and audio) UI elements to deliver every imaginable form of intelligence to the user through every device or application at their disposal. And, in fact, as more BI environments begin to deliver more real-time event-driven refreshes direct-from-the-source, the more the potential for overburdening the poor end user’s overfull cranium. Where CEP for I&KM is concerned, this can easily degenerate into a nightmare scenario of information overload.

Or the BI/CEP application developers can deliver simplicity. They can use back-end event engines, business rules engines, and other infrastructure to conceal distracting complexities. Earlier that week, I was meeting with a vendor that provides real-time DW, plus in-database analytics, plus rules engines that can deliver, triggered by streaming data, simple recommendations through an Information Worker’s BI environment. For example, the DW/CEP/BRE/analytics environment might provide account reps with crisp, time-sensitive, context-sensitive recommendations to respond to an irate customer while they’re still on the phone--to keep them from bolting to the competition.

All of which brings me to a closing thought re this particular post. The problem with the term “complex event processing” is that it seems to imply a complex UI--hence, that CEP's potential user base is limited to rocket scientists, Wall Street quants, IT industry analysts (gasp!), and other folks who are professionally obliged to handle (indeed, embrace) complexity.

But complexity is part of the problem, not the solution, where CEP for I&KM is concerned. We can’t stop the world of business from growing more multifaceted. But we can and should filter it all down to the simplest, most meaningful, most actionable experience, targeted specifically to each user, and contextualized precisely to each and every occasion.

Down to a pure and beautiful crystallization.

Complex Event Processing (CEP) For I&KM — Mouthfuls, Morsels, And Meaningful, Manageable, Multifaceted Streams of Real-Time Intelligence

By James Kobielus

This past week’s Forrester Enterprise Architecture (EA) Forum was quite an excellent experience. Being new to Forrester, it was a splendid opportunity to introduce myself to our customers, engage them in face-to-face inquiries, and present my research priorities. Not being new to the analyst space, it was also a chance for me to re-introduce — hence recontextualize — myself, and my focus areas, within the Forrester universe of research client groups, orbits, and domains.

As far as “what I cover” elevator pitches go, mine is two-thirds nailed down, not requiring any excess breath to explain. First and foremost, within the Information and Knowledge Management (I&KM) orbit, I’m Forrester’s lead analyst for Data Warehousing (DW). Secondly, I’m also the lead analyst on Predictive Analytics and Data Mining, an area that clearly plays into my DW coverage (analytic data marts, for example).

But my third focus area is more of a mouthful. In my one-on-ones at last week’s forum, I had to resort to drawing messy diagrams full of acronym balloons and wayward connecting arrows to contextualize it in the larger Forrester — and enterprise architecture — schemes of things. Here now comes another attempt, purely verbal this time, not necessarily elevator-pitch-perfect (still needs intensive nuancing), but hopefully delivered in digestible morsels that Forrester customers will find thought-provoking and inquiry-generating.

My third lead focus area is Complex Event Processing (CEP) for I&KM. At the EA Forum, I was quick to point out that Forrester already has a lead analyst for CEP: Charles Brett, who approaches the topic from an Application Development and Program Management perspective and has just published an excellent report on CEP.

As the “for I&KM” tag indicates, I’ll be looking at CEP in a slightly different context: as an enabler for truly real-time BI, predictive analytics, and business performance optimization. Indeed, most CEP applications rely on the BI portfolio of interactive visualization, dashboarding, scorecarding, reporting, query, predictive analytics, and data mining tools to support agile response and proactive coordination around real-time, emerging, or breaking business opportunities and threats. Behind it all is a low-latency middleware fabric that enables continuous monitoring, aggregation, correlation, and filtering of event data captured from operational applications, business process management systems, databases, and other sources.

So far, CEP has had only a minimal footprint in the BI arena, mostly because BI applications still rely primarily on historical data that has been consolidated in DWs, hence provide, at best, "near real-time" refreshes. Where enterprises have ventured into truly subsecond-latency CEP, they have traditionally implemented is as a stovepipe separate from their BI environments. CEP infrastructures typically incorporate their own, distinct, event-optimized service layers for interactive visualization, dashboarding, modeling, repository, rules engine, resource connection, and administration.

However, it’s only a matter of time before most BI vendors partner with CEP pure-plays, or acquire them outright, in order to strengthen their real-time functionality. We expect to see SAP/Business Objects, SAS, IBM/Cognos, Oracle/Hyperion, Microsoft, Information Builders, and MicroStrategy venture into the CEP arena in the next 1-2 years. Likewise, it’s very likely that a now-independent Teradata, which has taken the lead in real-time DW, will snatch up a CEP vendor to build out its real-time BI/DW portfolio. IBM's recent acquisition of CEP pure-play AptSoft shows that it is serious about CEP for I&KM, as does TIBCO's  acquisition of Spotfire.

Rest assured: I won't cover CEP as a stovepipe. You can expect to see CEP — as an enabler for real-time analytics and business optimization — incorporated as a cross-cutting theme throughout my Forrester coverage areas.