New Year Fast Approaching, and So Is the Next-Generation Enterprise Data Warehouse

By James Kobielus

This has been one of the most pivotal years in the evolution of the enterprise data warehousing (EDW) market. Every EDW vendor of note has firmly repositioned its go-to-market strategy around the appliance approach, with some also taking tentative steps into what is sure to be a key theme in 2009 and beyond: EDW in the “cloud.”

Yes, much of the recent “cloud” buzz has been bleeding-edge IT trade-paper fodder. It’s all interesting, to be sure, and there’s plenty of innovation going on. But much of the discussion seems to be a renaming, repackaging, and mashing up--with subtle twists and tweaks--of such well-established themes as service-oriented architecture (SOA), software as a service (SaaS), virtualization, and Web 2.0. And much of the trendy attention to cloud services obscures that fact that public cloud offerings from Amazon, Google, Microsoft, and others are still primarily a work in progress.

Forrester clients have only recently begun to inquire about cloud topics in earnest. But still, it’s hard to stay cynical about cloud computing for long. This solution delivery approach is coming almost certainly, inexorably, to all IT solution segments, including to EDW and business intelligence (BI). Sure, I&KM pros can safely ignore much of the cloud buzz for now, but, a year ago, they said the same about EDW appliances, and look how quickly appliances have become a dominant deployment approach in this market.

In the EDW-related inquiries I take from Forrester I&KM customers, a great many concern the appliance market. CIOs, CTOs, DBAs, and other professionals are actively considering various vendors’ appliances to replace, or at least to supplement and extend, their traditional “roll-your-own” EDWs. Typically, the I&KM pro wants me to help them select the best EDW appliance for their needs from any of several vendors, both venerable blue-chip and sexy start-up.

What validated appliances for I&KM pros this year was the fact that big-name EDW vendors--including Teradata, Oracle, IBM, Microsoft, and Sybase--have gone this route in earnest. The inflection point for the whole industry was this fall when Teradata--who effectively established the EDW appliance space years ago but had long resisted going to market under that label--embraced the approach and significantly expanded their appliance solution portfolio.

EDW cloud services are still a few years away from a similar inflection point. The leading EDW vendors have taken only the most tentative steps into the still-embryonic cloud services market. But they are all beginning to explore the cloud/SaaS channel with greater interest. They simply have to. Customers’ capital budgets are under severe pressure, and a multi-million dollar EDW solution--be it a premises-based appliance or what have you--is a tough sell. In a soft economy, any on-demand pay-as-you-go offering becomes more attractive across all customer segments. Just as important, the increasing scalability, performance, flexibility, and availability demands on the EDW and BI infrastructure are spurring many users to consider managed, hosted, outsourced offerings with fresh interest.

We’re starting to see the next-generation cloud EDW emerge. One noteworthy development this year was Oracle’s partnership with Amazon. Under that agreement, Oracle customers can license Oracle’s core EDW software stack to run in Amazon Web Services' Elastic Compute Cloud (Amazon EC2) environment. They can also use their existing software licenses on Amazon EC2 with no additional license fees.

Another important development was Microsoft’s announcement of its Windows Azure cloud initiative, of which one key component is the (still in beta) SQL Server Data Services (SSDS) subscription offering. When Microsoft SSDS goes into production in 2009, it will offer some basic DW/BI features in addition to transactional database support. Though SSDS will not initially be at a functional par with they licensed SQL Server offerings, it is clear that Microsoft plans to evolve it rapidly toward becoming a feature-competitive DW cloud offering over the next several years.

Microsoft also made a key EDW-related acquisition this year, appliance pure-play DATAllegro. Forrester expects this acquisition to figure centrally into Microsoft’s evolution of SSDS into a massively scalable cloud DW service. Though DATAllegro did not achieve much market adoption as a DW appliance pure-play, it provides Microsoft with a robust scale-out technology called “shared-nothing massively parallel processing” (MPP). By the way, Microsoft is playing catch-up in this regard, since most of its closest competitors implement shared-nothing MPP in their EDW premises-oriented solutions, and such DW appliance pure plays as Netezza, Greenplum, and Vertica also implement it to varying degrees.

When Microsoft ultimately ships a DATAllegro-powered SQL Server EDW appliance under its “Project Madison” in a year or two, we would not be surprised to see it adopted first in SSDS. The cloud EDW offering would benefit immensely from shared-nothing MPP’s ability to manage petabytes of analytic data and parallelize queries and other transactions seamlessly across a grid of hundreds or thousands of nodes.

Indeed, the industry consensus is largely in favor of shared-nothing MPP across the storage and compute tiers, coupled with flexible information-as a-service (IaaS) and server virtualization, as the principal platform for cloud computing. In the next-generation EDW, shared-nothing MPP allows the infrastructure to become more fluid, flexible, and virtualized, while managing ever more massive data sets and providing the agility to handle more complex mixed workloads of reporting, query, OLAP, data mining, data cleansing, transformations, and other functions.

As 2009 approaches, we’ll see pure-play DW cloud vendors come to the fore, appealing both to the early adopters among I&KM pros as well as to those under severe budgetary, headcount, and data center constraints. The established EDW vendors will not come to the cloud in a big way till 2010 at the earliest, it appears. But they will come, and with all guns blazing.

In 2-3 years’ time, the established vendors will own the EDW cloud space just as they’re starting to own the rapidly maturing appliance segment--in part, by acquiring the most promising cloud startups. The longer the economy stays drab and dreary, the faster the cloud EDW segment will expand, mature, and consolidate.

As that happens, commercial EDW cloud offerings will become as diverse and feature-rich as appliances have become, and I&KM pros will almost certainly ramp up their cloud EDW inquiries.

Forrester’s EDW analyst stands ready to serve.

Predicting The Battle Over Collaboration Infrastructure In 2009

by Gil Yehuda

It’s always the short questions that make my job interesting. Like this one.

Gil, do you think companies will cut back on Enterprise Web 2.0 in light of the economy?

First reaction--it depends. I’m an analyst, that’s always our first answer. But what does it depend on? What are all the factors at play and how will this impact your decisions?  So, here’s my read of the Enterprise Web 2.0 trends based on many conversations with my clients and vendors. I will focus specifically on wiki and social networking tools used to improve internal collaboration and knowledge sharing. These are gaining momentum and acceptance within the enterprise. (See my TechRadar report for the details on what Forrester sees in scope for Enterprise Web 2.0.)

There will be a slowdown of IT-driven collaboration projects in 2009. But there will be increased interest in business-driven collaboration projects. Why? There is a technology populist movement, and has been for a while. Small and medium-sized businesses (SMBs) typically operate with little IT support and rely upon vendors for collaboration services – nothing new here.  But we find that business units in enterprises, especially those in companies with politically weak IT departments, are increasingly behaving like SMBs, and they are going out and provisioning technology on their own. This is a form of institutional Tech Populism.

IT departments react by trying to block the business from getting software services from the cloud. And for good reason.  It’s hard to manage disconnected islands of infrastructure. Furthermore, it circumvents IT, adding complexity and risk to their job--especially if they were not involved in selecting the service. Anecdotally, we find that workers are now spending more time working from their home computing environment in order to access the blocked online productivity sites.

In response, IT says “Ok, we’ll provide you what you need so you don’t have to go behind our backs." In parallel, the traditional brand name ISVs are bolting collaboration features onto their existing enterprise platforms.  IT departments want to minimize operational complexity, so they will be attracted to wikis and social networks provided by vendors with familiar names--regardless of their functional adequacy.

Will it work? In an ideal economy where IT has a budget, maybe.  But what about now?

I predict that IT-driven internal collaboration initiatives will be squeezed tight: 1. they are usually more expensive than the Tech Populist options. 2. IT is being asked to sacrifice projects, and they would rather cut fat, not bone. Meaning, they’d rather protect their bread-and-butter IT infrastructure from being outsourced. And 3.The business considers projects initiated by IT to be less vital. Remember who pays the bills.

However, for business-driven internal enterprise Web 2.0 collaboration projects, I see growth. Why?  Because the business will find their collaboration needs to grow in 2009, while they see IT providing them with fewer services. Collaboration needs grow as a result of layoffs, mergers, and deepening external partnerships (requiring new infrastructure to collaborate outside the firewall with trusted, external partners).  And this happens while IT’s services shrink as a result of layoffs, a focus on streamlining operational costs, while not taking on new projects.

Who wins? The SaaS based collaboration vendors: folks like Box.net, GroupSwim, Jive, OneHub, PBwiki, SocialCast, Socialtext, and others who provide collaboration services in the cloud for about $5-$15 per user per month, give or take. These products range in functionality, where some focus on the wiki, others on the social network, and still others are more suited for file sharing within trusted groups. But these are easy pickings for business that are looking to circumvent IT and set up a small departmental solution. Especially in departments that are looking to collaborate with a few external partners.

The good news is that when the economy picks up, IT and the business can have a heart-to-heart talk and make some decisions about the future of the SaaS based content.  Some will leave it in the cloud if they continue to like the prices and features.  Others will revisit the brand-name collaboration options that are provided by the ECM and portal vendors. By this time, many of these options will catch up in functionality, providing a solution that will make both IT and the business happy.

And now, the battle. The story above works for companies that are willing to move to SaaS based products to address near term collaboration need in 2009. But many organizations cannot, or will not, allow themselves to house their intellectual property on someone else’s servers – no matter what the vendor says to assure them. This means that organizations with hard-line IT shops will face a battle between IT and the business for a collaboration solution that integrates with IT’s existing infrastructure, but requires little IT involvement.

Now, there are good on-premise collaboration tools in the market today that are poised to solve this battle – depending on what your IT infrastructure is. The challenge I see is that many of these vendors are not sure to whom they are selling their solution – to the business, IT, or the fragile partnership between the two. Remember, partnerships encounter stress during tough economies. It’s hard for IT and the business to work together when they are eying each other’s budgets.

You see my thoughts, so please share yours.  Given the current economy:

• Will you cut back on Enterprise Web 2.0?
• Will you deepen the divide between IT and the business?
• Or will you try to form a stronger partnership between the two?

Mark Twain, US Politics, And The State Of Enterprise BI

By Boris Evelson and Norman Nicolson

If you haven’t yet heard the latest news on the American political scene, let me fill you in: Illinois Governor Rod Blagojevich has been arrested on charges of conspiracy to commit fraud and soliciting bribery. Among the alleged offenses is that the Governor planned to sell the Senate seat of President-elect Barack Obama to the highest bidder, or, if no offers met his expectations, to take the seat for himself for personal gain. One is reminded of the remark, often attributed (perhaps incorrectly) to Mark Twain, that the United States has “the best politicians money can buy.”

Putting witticisms aside, this news does remind us all that spending more money is no guarantee of success. When we say someone has “the best X money can buy,” we don’t mean they have the best, period. There’s always room for using what you buy more intelligently, effectively, and efficiently, or finding a hidden gem at bargain prices. Information and Knowledge Management professionals often delude themselves into thinking that spending more money on enterprise applications, especially business intelligence, is the key to success, but that’s simply not the case. Our latest survey (the results of which will be published soon) shows that most large businesses have at least three, and often many more, BI tools in house, often with no plans to consolidate - and larger enterprise deals for BI can run well over six figures. Add to those license costs and maintenance fees the price of associated professional services (which can be, anecdotally, up to five times as much as license fees for a given project), and it becomes abundantly clear that enterprises are spending plenty on “the best BI tools money can buy.”

However, are they really getting the best BI, or just expensive BI tools? User satisfaction is still mediocre at best: our latest survey shows that BI applications are hard to use, trust in the data is poor, and IT is too slow to respond to requests for new reports or data sources. Additionally, in spite of the best efforts of analytics evangelists, BI is still mostly relegated to back-office efficiency gains, instead of being used to drive the strategy of the business.

Our client interactions constantly show that, while the best BI tools are a necessary component for the best BI implementation, other components are equally as or even more important than the BI tools themselves. Business led data governance, integrated MDM and metadata processes, finely tuned Business Intelligence Solutions Centers, and end user BI self service, are among just a few non-technical components that make up the best BI money can’t just simply buy.

What do you think? How can we move from “the best BI tools money can buy” to “the best BI?”

Extranet Collaboration Platforms. Coming Soon, But Then There's That Pesky Many-To-Many Problem

by Ted Schadler

In our conversations with many information and knowledge management professionals, it's clear that their distributed and multicompany teams need better extranet collaboration tools.

And they feel the problem is only getting worse as companies go virtual, global, distributed, outsourced, green, travel-less, and partnered, thus driving the need for ever-better collaboration tools that work outside the firewall.

Trouble is, the messaging and collaboration services that  companies have implemented are designed primarily for internal teams.

For example, it's bloody difficult to set up a secure instant messaging connection with every partner you might want to work with. Such interoperability between IM platforms is technically possible, but operationally nightmarish.

So clever employees do what they must: Use public IM and calendaring services, cobble together conferences from piece parts, and fall back on endless scheduling and sharing emails and voice conferencing. Ugh. Ugly. And scary.

Well, the solution's just around the corner say vendors new and old. After all, many are on the cusp of major product releases that promise much better extranet connections and capabilities:

• IBM Bluehouse promises a new extranet collaboration platform.
• Google already offers an extranet collaboration toolkit in its Google Apps Premier Edition.
• Cisco is adding extranet collaboration capabilities to WebEx.
• Microsoft is moving its services into the cloud for easier extranet access.
• PBwiki is already cloud-based and ready for extranet collaboration.
• The extranet collaboration toolkit list goes on: Veodia, Forterra, Dimdim, Qwaq.

That may be true. I surely hope so. But I fear that we have a few more hurdles to clear before extranet collaboration becomes as straightforward as internal collaboration. The basic problem is the many-to-many combinatorial problem. (The mathematicians I know call this a combinatorial problem, and its solution scales exponentially with the number of companies, nee people, involved.)

In addition to all the important stuff to support multicompany teams -- conferencing, video, shared calendars, team sites, persistent chat, search, shared documents, unified communications, structured processes, etc. -- these structural problems must be solved:

1. An extranet collaboration platform will have to be set up in the cloud. Using a cloud-based provider turns a many-to-many exponential problem into a many-to-one linear problem. But one of the partnering company must still own and control the platform services.
2. The many-to-many permission problem must be solved. How do to manage access control when teams form and disband and companies sometimes partner and sometimes compete? The access control tools need to get easy to use and integrate well with existing corporate directories.
3. IT will have to grow comfortable with new access control dashboards. Thomson Reuters Messaging has solved this problem for some industries, giving individual companies control over what their employees can do on the extranet collaboration platform that it hosts.

Agree? Disagree? Please comment.