“Just Right” Customer Analytics - update

Why, oh, why is it that every time I hear about a BI project from an IT person, or from a business stakeholder describing how IT delivered it, with few exceptions, these are the stories plagued with multiple challenges? And why is it that when I hear a BI story about an application that was installed, built, and used by a business user, with little or no support from IT, it’s almost always a success story?

I think we all know the answer to that question. It’s all about IT/business misalignment. A business user wants flexibility, while an IT person is charged with keeping order and controlling data, applications, scope, and projects. A business user wants to react to ever-changing requirements, but an IT person needs to have a formal planning process. A businessperson wants to have a tool best-suited for the business requirements, and an IT person wants to leverage enterprise standard platforms.

Who’s right and who’s wrong? Both. The only real answer is somewhere in the middle. There’s also a new emerging alternative, especially when applied to specific domains, like customer analytics. As I have repeatedly written in multiple research documents, front-office processes are especially poorly-suited for traditional analytics. Front office processes like sales and marketing need to be infinitely more agile and reactive, as their back office cousins from finance and HR for obvious reasons.

But there’s a dilemma. Large enterprise BI platforms are function-rich, powerful, scalable, and robust. But they are not agile. Excel and Microsoft Access, the most ubiquitous BI tools, while ultimately agile, are not scalable and do not have rich BI functionality. They also have minimal data integration and data scrubbing capabilities and no capability to tie multiple steps in a single process. Now there’s an answer for that something in between the two. Just like with the story of Goldilocks And The Three Bears, there are BI tools that are “just right” for the Agile enterprise BI.

We’ve already written about “Agile BI out of the box” where data-centric BI solutions combine end-to-end BI processes into a single BI application, which makes it more agile (easier to design, build, implement, and change). There’s now an emergence of new technologies that I would classify as process- centric BI, since there’s a strong emphasis on a workflow engine (which controls parts of a processes that change frequently — through process modeling, rule management, and process reporting), that ties all of the BI steps such as ETL, data cleansing, SQL, and building reports and dashboards. Workflow engine is key here, since you not only automate processes using modeling (very little coding, more like assembling), you can change the processes very quickly. This allows you to do continuous process improvement, making small changes every week, or even every day, and making bigger process version changes every three months.

BI workflow is the core capability of the two emerging vendors in this space: www.quiterian.com and www.alteryx.com. Their focus is on customer and marketing analytics where workflow is especially important. Every process involves getting raw data, integrating and cleansing it, running the data against certain rules to come up with segmentation and scoring, and inserting data into marketing campaigns and other downstream applications. But even though these two vendors mostly specialize in customer analytics, there’s no reason why their technology can’t be deployed in other BI use cases.

All of the customer success stories that I heard about from these two vendors come from marketing and sales, not IT professionals. Other than infrastructure support (and in the case of Alteryx that can also be bypassed since they have a cloud-based offering), IT plays little to no role in the applications built using these products. Marketing and sales operations pros install the software, keep it up to date, design and build their apps. Then they run them and distribute the results to their colleagues.

Yes, of course, I know what you are going to say. What about enterprise standards and single version of the truth? That argument works well in the back office. Most of the sales and marketing pros we talked to tell us that they’ll take quick over accurate (well, they still need the data that’s mostly — just not 100%— accurate) most of the time. So don’t apply “single version of the truth” to all enterprise stakeholders equally.

April 6, 2011 addition. I actually view this as an emergence of a new - well, maybe not new, but definitely not previously well defined - space in BI. If I include two other vendors - www.logixml.com and www.jackbe.com, who I think fit well into that category - I can see this space defined as follows. Comments?

Sept 20, 2011. Just got a briefing from www.altosoft.com - another addition to the segment.

Comments

Comment to "Just Right" Customer Analytics

Boris, again you go directly to the right point.

IT should be the enabler while users the owners of making decission process.

However, for the traditional BI, IT plays the role of the owner while in the new Agile BI is the user who owns the project, supported by IT.

New BI or Agile BI o Just In Time BI plays a complementary function to the traditional BI into the prganization and the balance, now goes to the user side.

It's a power game in the company led by the corporate open culture and analytical attitude.

Somenthing is changing very fast and some CIOs are still thinking about controlling corporate data.

Best regards

Josep Arroyo
CEO - Quiterian

Single Version of the Truth and Agility

Boris,

Thanks for the read.

1) Having a single version of the truth and agility do not contradict each other. But if I had to choose between each department building its own solution and a data warehouse, I would go for the data warehouse. Not for a traditional relational one, but I would definitely prefer different departments share common data repositories. The question is whether this data warehouse can be built incrementally and easily, as requirements arise, and not force you to launch a 6 month preliminary project before business users see a real number.

2) There is always friction between IT and business users, but it's more apparent and BI than in other industries. The main reason is that traditional corporate BI solution have an IT task in between almost every stage of a business process involving BI. If IT could simply deliver the data to the business users in a controlled streamlined manner, business users could go while doing their own data analysis using end-user friendly tools without getting bottlenecked at IT.

3) Just an opinion, but I think most projects 'not involving IT' were in fact projects out sourced to a 3rd party consultant/integrator that is paid by the hour. If you paid your internal IT by the hour, I'm pretty sure you'd get some pretty happy business users (and some annoyed CFOs) :-)

The problem is not IT or the business users, the problem is the crappy tools they get.

Elad
SiSense

Another player

JackBe just convinced me that their Presto product http://www.jackbe.com/products/ fits in the same category.

Great post! "Who’s right and

Great post!
"Who’s right and who’s wrong? Both. The only real answer is somewhere in the middle."

Could not agree more. When it comes to BI, Business and IT needs has been wildly disjointed in the past. But, how can these two groups meet in the middle? I was wondering the same in my recent blog: http://blog.pentaho.com/2011/03/22/it-vs-business/

A new approach to BI has arrived. While IT plays a vital role for change management of the BI applications, business users can get started and get going a lot faster that they could before.

Regards,
Farnaz Erfan
Pentaho

Forget single version of the truth- give me interop

Boris,

After an early success as entrepreneur (modest), I became a consultant specializing in marketing-- audits, plans, and execution, including multiple turn arounds as temp CMO, and on early BP with some of the giants today. In total companies I discovered earlier than peers are today worth about $600 billion. While I did not profit greatly from most-- requires significant capital to play the early stage VC and in those days I had little, it was based on a combination of marketing, business, and technical intelligence. Most people who spend lots of time traveling and in mtgs don't stand a chance against someone like me frankly-- they cannot invest the time required-- among the great challenge of institutionalization of venturing.

Later I founded a pioneering incubator -- spin off from consultancy, then converted, with multiple market leaders, and then an early stage VC firm. For years now I have been founder of Kyield, which is at the convergence of several acronyms in the enterprise with BI in the center-- essentially a new kind of OS for the organization.

So I am in a good position to appreciate your position here. One of many challenges in the enterprise is of course providing adaptability and customer tailored differentiation at an affordable level while still being interoperable with the rest of the organization.

I totally understand why many business units, particularly in S&M, would want to go off the reservation seeking functionality that does not exist in the enterprise-- this is not new. In the 1980s the best CRMs in the world were sophisticated manual file systems and the brain of the DOS. They spent most of their time off the reservation. However, the really strong CEOs remained in the loop on "manual BI". While far from perfect, that's what separated the best from the rest.

If I still had my consultant hat on-- or head of sales or marketing, I would have no problem with teams using tools they preferred to get the job done. However, I would have an enormous problem as CEO if that system was not only compatible with enterprise management systems, but automatically integrated. The intelligence from the sales and marketing teams is absolutely essential for many other areas of the organization, and for senior level decision making, which is after all a core reason for being in BI. With interoperable data we can then automate advanced analytics and predictive modeling that isn't possible otherwise. One reason S&M folks should want to engage--with rational security measures, is that it leads to better innovation and products that deliver more value, among others.

You recently retweeted a call for BI blogs-- just minted a piece that may be of interest

Data Integrity: The Cornerstone for BI in the Decision Process
http://kyield.wordpress.com/

Cheers,

Mark Montgomery
Founder & CEO
Kyield
http://www.kyield.com

correction

Sorry -- the above should have read:

"However, I would have an enormous problem as CEO if that system was not compatible with enterprise management systems, which enable efficient automation techniques for managing the entire organization."

Who's Driving: IT or Ops?

Great thread Boris!

We have had much more success working directly with operational teams primarily, I believe, because they are not fettered with the same restrictions as IT. Some of these restrictions are real (Sox, Security), but many are perceived (heavy role related access measures, the requirement for 100% truth).

Information is Power, but shared information is Powerful. Working primarily with operations allows us to drive information to the trenches where it can be used to modify and improve performance, one individual at a time. You can't improve performance for a whole division overnight, it MUST be done at the most granular (i.e. one human being) level. IT does not have a stellar history of wanting to share information, let alone a desire to actually drive it to those who need it.

Who is right? I'd like to introduce a new player - ROI. Whoever can prove out the ROI is right!

New relevant report

A new relevant report from a customer / marketing intelligence business pro point of view http://www.forrester.com/rb/Research/make_customer-facing_bi_agile/q/id/...

Customer Intelligence is the most clear space for Agile BI

Boris,
Marketing, sales, CRM and all departments where people analyze customers behavior are the most interested ambits interested in Agile BI.
Congratullations for the approach.
Good and fresh report.
Josep Arroyo
Quiterian