BI and data integration professionals face a multitude of overlapping data preparation options

Ah, the good old days. The world used to be simple. ETL vendors provided data integration functionality, DBMS vendors data warehouse platforms and BI vendors concentrated on reporting, analysis and data visualization. And they all lived happily ever after without stepping on each others’ toes and benefiting from lucrative partnerships. Alas, the modern world of BI and data integration is infinitely more complex with multiple, often overlapping offerings from data integration and BI vendors. I see the following three major segments in the market of preparing data for BI:

  1. Fully functional and highly scalable ETL platforms that are used for integrating analytical data as well as moving, synchronizing and replicating operational, transactional data. This is still the realm of tech professionals who use ETL products from Informatica, AbInitio, IBM, Oracle, Microsoft and others.
  2. An emerging market of data preparation technologies that specialize mostly in integrating data for BI use cases and mostly run by business users. Notable vendors in the space include Alteryx, Paxata, Trifecta, Datawatch, Birst, and a few others.
  3. Data preparation features built right into BI platforms. Most leading BI vendors today provide such capabilities to a varying degree.
Read more

Make Your BI Environment More Agile With BI on Hadoop

In the past three decades, management information systems, data integration, data warehouses (DWs), BI, and other relevant technologies and processes only scratched the surface of turning data into useful information and actionable insights:
  • Organizations leverage less than half of their structured data for insights. The latest Forrester data and analytics survey finds that organizations use on average only 40% of their structured data for strategic decision-making. 
  • Unstructured data remains largely untapped. Organizations are even less mature in their use of unstructured data. They tap only about a third of their unstructured data sources (28% of semistructured and 31% of unstructured) for strategic decision-making. And these percentages don’t include more recent components of a 360-degree view of the customer, such as voice of the customer (VoC), social media, and the Internet of Things. 
  • BI architectures continue to become more complex. The intricacies of earlier-generation and many current business intelligence (BI) architectural stacks, which usually require the integration of dozens of components from different vendors, are just one reason it takes so long and costs so much to deliver a single version of the truth with a seamlessly integrated, centralized enterprise BI environment.
  • Existing BI architectures are not flexible enough. Most organizations take too long to get to the ultimate goal of a centralized BI environment, and by the time they think they are done, there are new data sources, new regulations, and new customer needs, which all require more changes to the BI environment.
Read more

Don't Throw Hadoop At Every BI Challenge

The explosion of data and fast-changing customer needs have led many companies to a realization: They must constantly improve their capabilities, competencies, and culture in order to turn data into business value. But how do Business Intelligence (BI) professionals know whether they must modernize their platforms or whether their main challenges are mostly about culture, people, and processes?

"Our BI environment is only used for reporting — we need big data for analytics."

"Our data warehouse takes very long to build and update — we were told we can replace it with Hadoop."

These are just some of the conversations that Forrester clients initiate, believing they require a big data solution. But after a few probing questions, companies realize that they may need to upgrade their outdated BI platform, switch to a different database architecture, add extra nodes to their data warehouse (DW) servers, improve their data quality and data governance processes, or other commonsense solutions to their challenges, where new big data technologies may be one of the options, but not the only one, and sometimes not the best. Rather than incorrectly assuming that big data is the panacea for all issues associated with poorly architected and deployed BI environments, BI pros should follow the guidelines in the Forrester recent report to decide whether their BI environment needs a healthy dose of upgrades and process improvements or whether it requires different big data technologies. Here are some of the findings and recommendations from the full research report:

1) Hadoop won't solve your cultural challenges

Read more

Hit the road running with a new BI initiative

Even though Business Intelligence applications have been out there for decades lots of people still struggle with “how do I get started with BI”. I constantly deal with clients who mistakenly start their BI journey by selecting a BI platform or not thinking about the data architecture. I know it’s a HUGE oversimplification but in a nutshell here’s a simple roadmap (for a more complete roadmap please see the Roadmap document in Forrester BI Playbook) that will ensure that your BI strategy is aligned with your business strategy and you will hit the road running. The best way to start, IMHO, is from the performance management point of view:

  1. Catalog your organization business units and departments
  2. For each business unit /department ask questions about their business strategy and objectives
  3. Then ask about what goals do they set for themselves in order achieve the objectives
  4. Next ask what metrics and indicators do they use to track where they are against their goals and objectives. Good rule of thumb: no business area, department needs to track more than 20 to 30 metrics. More than that is unmanageable.
  5. Then ask questions how they would like to slice/dice these metrics (by time period, by region, by business unit, by customer segment, etc)
Read more

Systems Of Insight: Next Generation Business Intelligence

Earlier Generation BI Needs A Tune Up

Business intelligence has gone through multiple iterations in the past few decades. While BI's evolution has addressed some of the technology and process shortcomings of the earlier management information systems, BI teams still face challenges. Enterprises are transforming only 40% of their structured data and 31% of their unstructured data into information and insights. In addition, 63% of organizations still use spreadsheet-based applications for more than half of their decisions. Many earlier and current enterprise BI deployments:

  • Have hit the limits of scalability.
  • Struggle to address rapid changes in customer and regulatory requirements.
  • Fail to break through waterfall's design limitations.
  • Suffer from mismatched business and technology priorities and languages.
Read more

Expand Your Big Data Capabilities With Unstructured Text Analytics

Beware of insights! Real danger lurks behind the promise of big data to bring more data to more people faster, better, and cheaper: Insights are only as good as how people interpret the information presented to them. When looking at a stock chart, you can't even answer the simplest question — "Is the latest stock price move good or bad for my portfolio?" — without understanding the context: where you are in your investment journey and whether you're looking to buy or sell. While structured data can provide some context — like checkboxes indicating your income range, investment experience, investment objectives, and risk tolerance levels — unstructured data sources contain several orders of magnitude more context. An email exchange with a financial advisor indicating your experience with a particular investment vehicle, news articles about the market segment heavily represented in your portfolio, and social media posts about companies in which you've invested or plan to invest can all generate much broader and deeper context to better inform your decision to buy or sell. 
 
But defining the context by finding structures, patterns, and meaning in unstructured data is not a simple process. As a result, firms face a gap between data and insights; while they are awash in an abundance of customer and marketing data, they struggle to convert this data into the insights needed to win, serve, and retain customers. In general, Forrester has found that: 
 
  • The problem is not a lack of data. Most companies have access to plenty of customer feedback surveys, contact center records, mobile tracking data, loyalty program activities, and social media feeds — but, alas, it's not easily available to business leaders to help them make decisions. 
Read more

An Approach To Converge The Worlds of Big Data And BI

Webster dictionary defines a synonym as "a word having the same or nearly the same meaning" or as "a word or expression accepted as another name for something." This is so true for popular definitions of BI and big data. Forrester defines BI as:

 

A set of methodologies, processes, architectures, and technologies that transform raw data into meaningful and useful information used to enable more effective strategic, tactical, and operational insights and decision-making.
 
While BI has been a thriving market for decades and will continue to flourish for the foreseeable future, the world doesn't stand still and:
Read more

The Forrester Wave™: Enterprise Business Intelligence Platforms, Q1 2015

The majority of large organizations have either already shifted away from using BI as just another back-office process and toward competing on BI-enabled information or are in the process of doing so. Businesses can no longer compete just on the cost, margins, or quality of their products and services in an increasingly commoditized global economy. Two kinds of companies will ultimately be more successful, prosperous, and profitable:

  • More and deeper insights will generate competitive advantage. Companies with richer, more accurate information about their customers and products than their competitors will gain substantial competitive advantage.
  • Faster access to insights will make companies more agile. Companies that have the same quality of information as their competitors but get it sooner and can turn it into action faster will outpace their peers.
Read more

It's Not Your Grandfather's Open Source BI Market Any Longer

There's never been a question on the advantages of open source software. Crowdsourcing, vendor independence, ability to see and in some cases control the source code, and lower costs are just a few benefits of open source software (OSS) and business model. Linux and Apache Hadoop are prime examples of successful OSS projects. It's a different story, however, when it comes to OSS BI. For years, OSS BI vendors struggled with growth because of:

 

  • The developer-centric nature of open source projects. The target audience for open source projects is developers, which means deals are mostly sealed by technology management. The industry, on the other hand, has gravitated toward business decision-makers within organizations over the last several years. However, business users are less interested in the opportunities that a collaborative open source community offers, and more concerned about ease of use and quick setup. Indeed, Forrester's research constantly finds evidence correlating business ownership as one of the key success factors for effective BI initiatives.
Read more

Build An Agile Business Intelligence (BI) Organization

The battle of trying to apply traditional waterfall software development life-cycle (SDLC) methodology and project management to BI has already been fought — and largely lost. These approaches and best practices, which apply to most other enterprise applications, work well in some cases, as with very well-defined and stable BI capabilities like tax or regulatory reporting. Mission-critical, enterprise-grade BI apps can also have a reasonably long shelf life of a year or more. But these best practices do not work for the majority (anecdotally, about three-quarters) of BI initiatives, where requirements change much faster than these traditional approaches can support; by the time a traditional BI application development team rolls out what it thought was a well-designed BI application, it's too late. As a result, BI pros need to move beyond earlier-generation BI support organizations to:
 
  • Focus on business outcomes, not just technologies. Earlier-generation BI programs lacked an "outcomes first" mentality. Those programs employed bottom-up approaches that focused on the project management and technology first, leaving clients without the proper outcomes that they needed to manage the business; in other words, they created an insights-to-action gap.  BI pros  should use a top-down approach that defines key performance indicators, metrics, and measures that support the business' goals and objectives. They must resist the temptation to address technology and data needs before the business requirements. 
Read more