In the past three decades, management information systems, data integration, data warehouses (DWs), BI, and other relevant technologies and processes only scratched the surface of turning data into useful information and actionable insights:
Organizations leverage less than half of their structured data for insights. The latest Forrester data and analytics survey finds that organizations use on average only 40% of their structured data for strategic decision-making.
Unstructured data remains largely untapped. Organizations are even less mature in their use of unstructured data. They tap only about a third of their unstructured data sources (28% of semistructured and 31% of unstructured) for strategic decision-making. And these percentages don’t include more recent components of a 360-degree view of the customer, such as voice of the customer (VoC), social media, and the Internet of Things.
BI architectures continue to become more complex. The intricacies of earlier-generation and many current business intelligence (BI) architectural stacks, which usually require the integration of dozens of components from different vendors, are just one reason it takes so long and costs so much to deliver a single version of the truth with a seamlessly integrated, centralized enterprise BI environment.
Existing BI architectures are not flexible enough. Most organizations take too long to get to the ultimate goal of a centralized BI environment, and by the time they think they are done, there are new data sources, new regulations, and new customer needs, which all require more changes to the BI environment.
Hello from the newest analyst serving Forrester Research’s CIO role. My name is Paul Miller, and I joined Forrester at the beginning of August. I am attached to Forrester’s London office, but it’s already clear that I’ll be working with clients across many time zones.
As my Analyst bio describes, my primary focus is on cloud computing, with a particular interest in the way that cloud-based approaches enable (or even require) organizations to embrace digital transformation of themselves and their customer relationships. Before joining Forrester, I spent six years as an independent analyst and consultant. My work spanned cloud computing and big data and I am sure that this broader portfolio of interests will continue into my Forrester research, particularly where I can explore the demonstrable value that these approaches bring to those who embrace them.
I am still working on the best way to capture and explain my research coverage, talking with many of my new colleagues, and learning about potential synergies between what they already do and what I could or should be doing. I know that the first document to appear with my name on it will be a CIO-friendly look at OpenStack, as the genesis of this new Brief lies in a report that I had to write as part of Forrester’s recruitment process. I have a long (long, long) list of further reports I am keen to get started on, and these should begin to appear online as upcoming titles in the very near future. I shall also be blogging here, and look forward to using this as a way to get shorter thoughts and perspectives online relatively quickly. I’ve been regularly blogging for work since early 2004, although too many of the blogs I used to write for are now only preserved in the vaults of Brewster Kahle’s wonderful Internet Archive.
The explosion of data and fast-changing customer needs have led many companies to a realization: They must constantly improve their capabilities, competencies, and culture in order to turn data into business value. But how do Business Intelligence (BI) professionals know whether they must modernize their platforms or whether their main challenges are mostly about culture, people, and processes?
"Our BI environment is only used for reporting — we need big data for analytics."
"Our data warehouse takes very long to build and update — we were told we can replace it with Hadoop."
These are just some of the conversations that Forrester clients initiate, believing they require a big data solution. But after a few probing questions, companies realize that they may need to upgrade their outdated BI platform, switch to a different database architecture, add extra nodes to their data warehouse (DW) servers, improve their data quality and data governance processes, or other commonsense solutions to their challenges, where new big data technologies may be one of the options, but not the only one, and sometimes not the best. Rather than incorrectly assuming that big data is the panacea for all issues associated with poorly architected and deployed BI environments, BI pros should follow the guidelines in the Forrester recent report to decide whether their BI environment needs a healthy dose of upgrades and process improvements or whether it requires different big data technologies. Here are some of the findings and recommendations from the full research report:
Even though Business Intelligence applications have been out there for decades lots of people still struggle with “how do I get started with BI”. I constantly deal with clients who mistakenly start their BI journey by selecting a BI platform or not thinking about the data architecture. I know it’s a HUGE oversimplification but in a nutshell here’s a simple roadmap (for a more complete roadmap please see the Roadmap document in Forrester BI Playbook) that will ensure that your BI strategy is aligned with your business strategy and you will hit the road running. The best way to start, IMHO, is from the performance management point of view:
Catalog your organization business units and departments
For each business unit /department ask questions about their business strategy and objectives
Then ask about what goals do they set for themselves in order achieve the objectives
Next ask what metrics and indicators do they use to track where they are against their goals and objectives. Good rule of thumb: no business area, department needs to track more than 20 to 30 metrics. More than that is unmanageable.
Then ask questions how they would like to slice/dice these metrics (by time period, by region, by business unit, by customer segment, etc)
Business intelligence has gone through multiple iterations in the past few decades. While BI's evolution has addressed some of the technology and process shortcomings of the earlier management information systems, BI teams still face challenges. Enterprises are transforming only 40% of their structured data and 31% of their unstructured data into information and insights. In addition, 63% of organizations still use spreadsheet-based applications for more than half of their decisions. Many earlier and current enterprise BI deployments:
Have hit the limits of scalability.
Struggle to address rapid changes in customer and regulatory requirements.
Fail to break through waterfall's design limitations.
Suffer from mismatched business and technology priorities and languages.
When I think about data, I can't help but think about hockey. As a passionate hockey mom, it's hard to separate my conversations about data all week with clients from the practices and games I sit through, screaming encouragement to my son and his team (sometimes to the embarrassment of my husband!). So when I recently saw a documentary on the building of the Russian hockey team that our miracle US hockey team beat at the 1980 Olympics, the story of Anatoli Tarsov stuck with me.
Before the 1960s, Russia didn't have a hockey team. Then the Communist party determined that it was critical that Russia build one — and compete on the world stage. They selected Anatoli Tarsov to build the team and coach. He couldn't see films on hockey. He couldn't watch teams play. There was no reference on how to play the game. And yet, he built a world-class hockey club that not only beat the great Nordic teams but went on to crush the Canadian teams that were the standard for hockey excellence.
This is a lesson for us all when it comes to data. Do we stick with our standards and recipes from Inmon and Kimball? Do we follow check-box assessments from CMMI, DM-BOK, or TOGAF's information architecture framework? Do we rely on governance compliance to police our data?
Or do we break the rules and create our own that are based on outcomes and results? This might be the scarier path. This might be the riskier path. But do you want data to be where your business needs it, or do you want to predefine, constrain, and bias the insight?
China Unicom demonstrated its big data analytics platform, including customer analytics, during the Shanghai World Mobile Congress last week. Huawei is helping China Unicom’s Shanghai affiliate build a big data analytics platform that can collect and analyze customer demographics and operational and behavioral data. For instance, it can estimate a consumer’s monthly income based on annual mobile fees, know whether she is walking or driving, and what routes she regularly takes. Such data is unique and even more comprehensive than that generated by Internet service giants like Baidu, Alibaba, or Tencent. China Unicom will begin to leverage this data analytics platform to monetize data in several ways:
Retain customers. China Unicom can predict which high-value customers may be thinking of dropping its services and target marketing based on the customers’ context to retain them. For instance, the system can automatically send a targeted offering when a customer passes by a China Unicom office. In 2014, the telco performed A/B testing among 200,000 Unicom subscribers in Shanghai who were thinking of changing telecom service providers. This helped China Unicom retain RMB 10 million in revenue from those users receiving targeted marketing offerings from the system.
Enhance public security. China Unicom uses the platform to help the Shanghai city government to monitor people’s location in the city in real time. The system provides a real-time heat map and automatically sends an alert when it discovers that too many people are crowded into one area. This can help the government avoid accidents such as the one that occurred in the Waitan district of Shanghai last year.
It is easy to get ahead of ourselves with all the innovation happening with data and analytics. I wouldn't call it hype, as that would imply no value or competency has been achieved. But I would say that what is bright, shiny, and new is always more interesting than the ordinary.
And, to be frank, there is still a lot of ordinary in our data management world.
In fact, over the past couple of weeks, discussions with companies have uncommonly focused on the ordinary. This in some ways appeared to be unusual because questions focused on the basic foundational aspects of data management and governance — and for companies that I have seen talk publicly about their data management successes.
"Where do I clean the data?"
"How do I get the business to invest in data?"
"How do I get a single customer view of my customer for marketing?"
What this tells me is that companies are under siege by zombie data.
Data is living in our business under outdated data policies and rules. Data processes and systems are persisting single-purpose data. As data pros turn over application rocks and navigate through the database bogs to centralize data for analytics and virtualize views for new data capabilities, zombie data is lurching out to consume more of the environment, blocking other potential insight to keep the status quo.
The questions you and your data professional cohorts are asking, as illustrated above, are anything but basic. The fact that these foundational building blocks have to be assessed once again demonstrates that organizations are on a path to crush the zombie data siege, democratize data and insight, and advance the business.
Keep asking basic questions — if you aren't, zombie data will eventually take over, and you and your organization will become part of the walking dead.
You've probably heard about the Quantified Self (QS), a movement that aims to capture, analyze, and act upon data from the human body in the interest of better health, fitter athletes, and sharper minds. Today, QS is giving way to QW -- Quantified Workforce. A variety of technologies -- devices, software, services -- can quantify the health, fitness, mental acuity, timeliness, and collaboration of workers. Many of these services are ready for prime time, but present some challenges in implementing. These challenges aren't primarily technological; they're related to privacy, workers' rights, and human resources policies. Done right, though, quantifying the workforce can drive both top- and bottom- line growth in your company's business.
I've analyzed this trend in a new report, Smart Body, Smarter Workforce. Here are just a couple of examples of how quantifying the workforce can drive better business outcomes:
Lower the company's insurance rates. In January, 2014, Forrester predicted that insurance companies would offer lower rates to individuals who donned wearables -- and we are now seeing that response. In April, 2015, John Hancock announced an opportunity for buyers of its term and life insurance policies to earn up to 15% discount on their insurance rates by wearing a Fitbit, sharing the data with the company, and meeting certain activity levels.
In scanning through my O’Reilly Data Newsletter today, I noticed A Healthy Dose of Data, an MIT Sloan case study on the data and analytics culture at Intermountain, a healthcare network that runs 22 hospitals and 185 clinics. The study is definitely worth the read. It reviews the history of data use at Intermountain, which began way before the “big data” craze of recent years. In fact, it was back in the 1950s that one of the Intermountain cardiologists, Homer Warner, began to explore clinical data to understand why some heart patients experienced better outcomes than others. He went on to become known as the “father of medical informatics – the use of computer programs to analyze patient data to determine treatment protocols,” and with colleagues designed and launched their first decision-support tool.
The case study goes on to describe how Intermountain has cultivated a strong data and analytics culture. Over time – Rome was not built in a day, as they say – they established data maturity across the organization by investing in the capacity (new tools and technologies), developing the competencies (new skills and processes) and finally spreading the culture (awareness, understanding and best practices) of data and analytics. Their analytical approach brought results – fewer surgical infections, more effective use of antibiotics, less time in intensive care etc – contributing to lower costs, better medical outcomes, and overall patient satisfaction.