Open source big data technologies like Hadoop have done much to begin the transformation of analytics. We're moving from expensive and specialist analytics teams towards an environment in which processes, workflows, and decision-making throughout an organisation can - in theory at least - become usefully data-driven. Established providers of analytics, BI and data warehouse technologies liberally sprinkle Hadoop, Spark and other cool project names throughout their products, delivering real advantages and real cost-savings, as well as grabbing some of the Hadoop glow for themselves. Startups, often closely associated with shepherding one of the newer open source projects, also compete for mindshare and custom.
And the opportunity is big. Hortonworks, for example, has described the global big data market as a $50 billion opportunity. But that pales into insignificance next to what Hortonworks (again) describes as a $1.7 trillion opportunity. Other companies and analysts have their own numbers, which do differ, but the step-change is clear and significant. Hadoop, and the vendors gravitating to that community, mostly address 'data at rest'; data that has already been collected from some process or interaction or query. The bigger opportunity relates to 'data in motion,' and to the internet of things that will be responsible for generating so much of this.
Huawei Technologies started out nearly 30 years ago as a small private company with 14 employees and 140,000 yuan in capital. By 2015, its total revenue exceeded $60 billion. Huawei is already a global company, but its globalization journey has been a difficult one since the very beginning. Despite its continuous business growth in other regions, Huawei has faced critical censorship in the US since Day One — and last week the US government put Huawei under the microscope yet again.
National security is important, but using “national security” as an excuse for allowing unfair competition will only harm customers. It’s time for the governments of both countries to trust each other more. I’ve recently published a report focusing on Huawei’s continuous progress toward becoming a key enabler of digital transformation in the telco and enterprise spaces. Some of the key takeaways:
Huawei has holistic strategies for digital transformation. Huawei’s broad vision of digital strategy — which focuses on cloud enablement and readiness, partner enablement, and open source co-creation — has helped the firm sustain strong business growth in the telco and enterprise markets. For example, its partnerships with T-Systems on the Open Telekom Cloud in Germany and with Telefónica on public cloud in the Americas have helped carriers in local markets give cloud users on-demand, all-online, self-service experiences.
The Background – Linux as a Fast Follower and the Need for Hot Patching
No doubt about it, Linux has made impressive strides in the last 15 years, gaining many features previously associated with high-end proprietary Unix as it made the transition from small system plaything to core enterprise processing resource and the engine of the extended web as we know it. Along the way it gained reliable and highly scalable schedulers, a multiplicity of efficient and scalable file systems, advanced RAS features, its own embedded virtualization and efficient thread support.
As Linux grew, so did supporting hardware, particularly the capabilities of the ubiquitous x86 CPU upon which the vast majority of Linux runs today. But the debate has always been about how close Linux could get to "the real OS", the core proprietary Unix variants that for two decades defined the limits of non-mainframe scalability and reliability. But "the times they are a changing", and the new narrative may be "when will Unix catch up to Linux on critical RAS features like hot patching".
Hot patching, the ability to apply updates to the OS kernel while it is running, is a long sought-after but elusive feature of a production OS. Long sought after because both developers and operations teams recognize that bringing down an OS instance that is doing critical high-volume work is at best disruptive and worst a logistical nightmare, and elusive because it is incredibly difficult. There have been several failed attempts, and several implementations that "almost worked" but were so fraught with exceptions that they were not really useful in production.[i]
Businesses can obtain major benefits — including better customer experiences and operational excellence — from the internet of things (IoT) by extracting insights from connected objects and delivering feature-rich connected products.
The mobile mind shift requires businesses to proactively support these IoT benefits for nonstationary connected objects that exist as part of IoT solutions. In particular, the IoT forces businesses to acquaint themselves with the implications of mobility in the IoT context for connectivity, security, compliance with privacy and other regulations, and data management for mobility. This means that:
Mobile technologies are central to most IoT solutions. To date, technology managers have mostly focused on enterprise mobility management (EMM) as part of their mobile activities. This narrow focus is insufficient for IoT solutions.
Mobile IoT is not a technology revolution but a fundamental business process transformation. Mobility requires managers not only to deploy mobile technologies but also to exploit them to support specific business process requirements.
Mobile technologies set the framework for IoT solutions. Mobile has distinct implications for aspects like broadband availability, data management, security, and local data compliance. Ignoring these will undermine your IoT initiatives and return on investment.
My new report, Mobilize The Internet Of Things, provides advice and insights for businesses on addressing these mobile challenges in the context of planning for and implementing IoT solutions.
“We are in the business of building [FILL IN THE BLANK], why would we build an insights platform out ourselves.”
That sentiment will drive more and more companies to explore the insights services option. Many already feel like they are chasing a moving target. Data and analytics practices are evolving quickly with new tools and techniques moving the bar higher and higher. Not to mention the explosion of data sources, and the dearth of skilled talent out there. As executives become more aware of the value of data and analytics, they become increasingly dissatisfied with what their organizations can deliver: in 2014 53% of decision-makers were satisfied with internal analytics capabilities but by 2015 those satisfied fell to 42%. These are the leaders who will look for external service providers to deliver insights. They realize they might not get there themselves.
The sentiment expressed in the quote above was actually from a consumer packaged goods company. For its execs winning in cities has become paramount. As urbanization increases, cities provide big opportunities. But not all cities are alike and differentiating what they take to a specific market requires deep local knowledge – and a lot of diverse data. To create hyperlocal, timely, and contextually relevant offers, the company needs data on local news, events, and weather as well as geo-tagged social data. All of that must be combined with its own internal and partner data.
Over the past several years, Forrester's research has written extensively about the age of the customer. Forrester believes that only the enterprises that are obsessed with winning, serving, and retaining customers will thrive in this highly competitive, customer-centric economy. But in order to get a full view of customer behavior, sentiment, emotion, and intentions, Information Management professionals must help enterprises leverage all the data at their disposal, not just structured, but also unstructured. Alas, that's still an elusive goal, as most enterprises leverage only 40% of structured data and 31% of unstructured data for business and customer insights and decision-making.
So what do you need to do to start enriching your customer insights with unstructured data ? First, get your yext analysis terminology straight. For Information Management pros, the process of text mining and text analytics should not be a black box, where unstructured text goes in and structured information comes out. But today, there is a lot of market confusion on the terminology and process of text analytics. The market, both vendors and users, often uses the terms text mining and text analytics interchangeably; Forrester makes a distinction and recommends that Information Management pros working on text mining/text analytics initiatives adopt the following terminology:
Delivering broad access to data and analytics to a diverse base of users is an intimidating task, yet it is an essential foundation to becoming an insights-driven organization. To win and keep customers in an increasingly competitive world, firms need to take advantage of the huge swaths of data available and put it into the hands of more users. To do this, business intelligence (BI) pros must evolve disjointed and convoluted data and analytics practices into well-orchestrated systems of insight that deliver actionable information. But implementing digital insights is just the first step with these systems — and few hit the bull's eye the first time. Continuously learning from previous insights and their results makes future efforts more efficient and effective. This is a key capability for the next-generation BI, what Forrester calls systems of insight.
"It's 10 o'clock! Do you know if your insights support actual verifiable facts?" This is a real challenge, as measuring report and dashboard effectiveness today involves mostly discipline and processes, not technology. For example, if a data mining analysis predicted a certain number of fraudulent transactions, do you have the discipline and processes to go back and verify whether the prediction came true? Or if a metrics dashboard was flashing red, telling you that inventory levels were too low for the current business environment, and the signal caused you to order more widgets, do you verify if this was a good or a bad decision? Did you make or lose money on the extra inventory you ordered? Organizations are still struggling with this ultimate measure of BI effectiveness. Only 8% of Forrester clients report robust capabilities for such continuous improvement, and 39% report just a few basic capabilities.
Are you lost in a confusing soup of vendor-speak about what their data analytics stack actually offers? Big data, data platforms, advanced analytics, data lakes, real-time everything, streaming, the IoT, customer analytics, digital intelligence, real-time interaction, customer decision hubs, new-stuff-as-a-service, the list goes on.
Recognize the convergence happening as vendors evolve their technologies from doing just one thing like predictive analytics or search to many things together. For example, data integration, data warehouse, and BI tools are typically sold separately, but breakout vendor Looker combines data integration, model governance, basic BI, and a runtime for data applications all in one software layer that sits on your data lake. As another example, consider predictive analytics vendor Alpine Data Labs or SAS Viya from SAS. These vendors have built out a lot of data management and insight delivery tooling into their platforms because without it users struggle to maximize value. Another trend is big data search vendors like Maana that now also include hooks for predictive model execution as well as more data management functions. Lastly, systems integrators are packaging their IP and offering it as a data management and analytics integrated product — for example, Saama’s Fluid Analytics Engine or Infosys’ Information Platform.
In fact, the list of innovative vendors blending data management, analytics, and insight execution technology is growing by leaps and bounds. To address this trend, I just published a report, Insight Platforms Accelerate Digital Transformation, in which I created a broad definition that labels this trend:
The tug of war between reason and emotion has fueled contentious debate since the days of Socrates. But, Socrates and subsequent thinkers didn’t anticipate the influx of data in our contemporary world. Today, our modern media saturation, infinite social connection, and sensor-laden bodies and buildings mean that we create, consult, and critique data more than ever before. How does the vast amount of information – that is now literally at our fingertips – actually influence our daily decisions, and why?
Forrester’s Consumer Technographics® survey data proves that individuals are steeped in information and are keenly aware of it. In fact, the insight shows that US online adults increasingly lean on data to make daily choices across spheres of life:
Most enterprises aren't fully exploiting real-time streaming data that flows from IoT devices and mobile, web, and enterprise apps. Streaming analytics is essential for real-time insights and bringing real-time context to apps. Don't dismiss streaming analytics as a form of "traditional analytics" use for postmortem analysis. Far from it — streaming analytics analyzes data right now, when it can be analyzed and put to good use to make applications of all kinds (including IoT) contextual and smarter. Forrester defines streaming analytics as:
Software that can filter, aggregate, enrich, and analyze a high throughput of data from multiple, disparate live data sources and in any data format to identify simple and complex patterns to provide applications with context to detect opportune situations, automate immediate actions, and dynamically adapt.
Forrester Wave™: Big Data Streaming Analytics, Q1 2016
To help enterprises understand what commercial and open source options are available, Rowan Curran and I evaluated 15 streaming analytics vendors using Forrester's Wave methodology. Forrester clients can read the full report to understand the market category and see the detailed criteria, scores, and ranking of the vendors. Here is a summary of the 15 vendors solutions we evaluated listed in alphabetical order: