What does Business Intelligence integration with R really mean

“A little prediction goes a long way” wrote Eric Siegel in his popular Predictive Analytics book. True, predictive analytics is now part and parcel of most Business Intelligence (BI), analytics and Big Data platforms and applications. Forrester Research anecdotal evidence finds that open source R is by far the most ubiquitous predictive analytics platform. Independent findings and surveys like the ones by KDNuggets and RexerAnalytics confirm our conclusions (and I quote) “The proportion of data miners using R is rapidly growing, and since 2010, R has been the most-used data mining tool.  While R is frequently used along with other tools, an increasing number of data miners also select R as their primary tool.”

To jump on this R feeding frenzy most leading BI vendors claim that they “integrate with R”, but what does that claim really mean? Our take on this – not all BI/R integration is created equal. When evaluating BI platforms for R integration, Forrester recommends considering the following integration capabilities:

  • Point and click GUI to create R scripts. Since R is a scripting language, does the BI vendor provide point-and-click GUI to generate R code? Martha Bennett suggested also asking about the capability not to just auto-generate R scripts, but also to edit, update and maintain them. Beyond just generating code, there's the whole business of model design, test, management, and execution, which is usually the realm of advanced analytics platforms. How much of these capabilities does the BI vendor provide?
  • Passing variables from BI to R. Can R routines leverage and take advantage of all of the BI metadata (data structures, definitions, etc.) without having to redefine it again just for R? A related question: does the BI vendor provide an open source utility to generate R script headers to facilitate passing variables and parameters to/from BI and R?
  • Passing variables from R to BI. Can the output from R calculations (scores, rankings) be passed back to BI platform and embedded in the BI reports and dashboards? A related question: does the BI vendor provide an open source utility to parse R script and identify variables and parameters to pass to/from BI and R?
  • Intercepting error and other messages from R. Can the BI platform intercept, interpret and act on error messages and alerts from R models/scripts?
  • PMML import/export. Can the BI vendor import/export R models based on PMML?
  • Native execution. What server are R models executed in? R server? BI server? Database server? Natively in a Hadoop server?
  • Content management. Does the BI vendor treat R scripts as part of its metadata and content, and therefore include R scripts in its content management processes such as versioning, migration (from development to test to prod environments, etc)?
  • Scalability (compliments to Mike Gualtieri). R is not designed or optimized to to process VERY large data sets. Does the BI vendor add any capabilities to parallelise R processing natively in Hadoop or in DBMS?

Find out more details including R integration capabilities of the leading BI vendors in the latest report.

Comments

Scalability

Hi Boris,

We looked into R, looked again, and then again recently in discussions with a well-known BI firm interested in an OEM partnership, which we are certainly open to for a good match.

The problem with R, as Mike touched on above, given Kyield's extension across large distributed organizations, is scale. Due to the amount of intel we collect and stream continuously in distributed fashion across largest organizations, even with distributed option I didn't see it working for us.

However, in such a case an OEM scenario makes a lot of sense as we can then integrate with partner serving more traditional BI while Kyield admittedly cherry picks specific functionality in BI, MDM, search, BPM and productivity, consolidating some old value while providing critical new.

I think an OEM type relationship may be the best way to serve customers by bringing in much needed next gen IP and systems engineering like Kyield into pre-existing ecosystems. Can provide a nice low cost option in offering a unique competitive advantage, and significant growth opportunity, shared efficient resources, etc. While some OEM exists in BI, I'm surprised we don't see more of it particularly in small to mid-size companies that don't have the capitalization to compete in R&D or classic capital consolidation via M&A.

In this regard I think the old hardware folks might be able to teach the BI software and services companies something. HW vendors don't have any choice due to the large number of components needed to provide a competitive turn key solution, so OEMs are routine norm rather than exception, with IP playing a critical role. Also worth noting that plug and play in hardware is substantially based on standards in electrical engineering which after many years of work are now finally maturing for data.

Some thoughts that came up while reading. -- MM

BI and R Integration

Boris,

WebFOCUS has some ability to dynamically generate and integrate with R bi-directionally.

See: http://bi-software.blogspot.com/2013/12/webfocus-and-r-for-dynamic-stati...

Sincerely,

Doug

crm software solutions

Top web site, I hadn't come across idlemendacity.blogspot.com earlier during my searches!
Keep up the superb work!
crm software solutions

Excellent and interesting

Excellent and interesting article. I will bookmark your blog for article updates. thanks for sharing...

BI Packages

Post new comment

If you have an account on Forrester.com, please login.

Or complete the information below to post a comment.

(Your name will appear next to your comment.)
(We will not display your email.)
Type the characters you see in this picture. (verify using audio)
Type the characters you see in the picture above; if you can't read them, submit the form and a new image will be generated. Not case sensitive.