Today IBM announced its plans to acquire Vivisimo - an enterprise search vendor with big data capabilities. Our research shows that only 1% to 5% of all enterprise data is in a structured, modeled format that fits neatly into enterprise data warehouses (EDWs) and data marts. The rest of enterprise data (and we are not even talking about external data such as social media data, for example) may not be organized into structures that easily fit into relational or multidimensional databases. There’s also a chicken-and-the-egg syndrome going on here. Before you can put your data into a structure, such as a database, you need to understand what’s out there and what structures do or may exist. But in order for you to explore the data in the first place, traditional data integration technologies require some structures to even start the exploration (tables, columns, etc). So how do you explore something without a structure, without a model, and without preconceived notions? That’s where big data exploration and discovery technologies such as Hadoop and Vivisimo come into play. (There are many others vendors in this space as well, including Oracle Endeca, Attivio, and Saffron Technology. While these vendors may not directly compete with Vivisimo and all use different approaches and architectures, the final objective - data discovery - is often the same.) Data exploration and discovery was one of our top 2012 business intelligence predictions. However, it’s only a first step in the full cycle of business intelligence and
Nowadays, there are two topics that I’m very passionate about. The first is the fact that spring is finally here and it’s time to dust off my clubs to take in my few first few rounds of golf. The second topic that I’m currently passionate about is the research I’ve been doing around the connection between big data and big process.
While most enterprise architects are familiar with the promise — and, unfortunately, the hype — of big data, very few are familiar with the newer concept of “big process.” Forrester first coined this term back in August of 2011 to describe the shift we see in organizations moving from siloed approaches to BPM and process improvement to more holistic approaches that stitch all the pieces together to drive business transformation.
Our working definition for big process is:
“Methods and techniques that provide a more holistic approach to process improvement and process transformation initiatives.”
The US economy continues to show improvement – for example, today’s news that new jobless claims were near a four-year low. As the economy outlook has improved, so, too, have prospects for the US tech market. In our updated Forrester forecast for US tech purchases, "US Tech Market Outlook For 2012 To 2013: Improving Economic Prospects Create Upside Potential," we now project growth of 7.5% in 2012 and 8.3% in 2013 for business and government purchases of information technology goods and services (without telecom services). Including telecom services, business and government spending on information and communications technology (ICT) will increase by 7.1% in 2012 and 7.4% in 2013.
The lead tech growth category will shift from computer equipment in 2011 to software in 2012 and 2013, with and IT consulting and systems integration services playing a strong supporting role. Following strong growth of 9.6% in 2011, computer equipment purchases will slow to 4.5% in 2012, as the lingering effects of Thailand's 2011 floods hurt parts supply in the first half and the prospect of Windows 8 dampens Wintel PC sales until the fall. Apple Macs and iPad tablets will post strong growth in the corporate market, though, and server and storage should grow in the mid-single digits.
At IBM's Smarter Analytics event this week, clients and partners presented success stories about how organizations are driving business value out of big data, analytics, and IBM Watson technology.
- City of Dublin, Ireland using thousands of data points from local transportation and traffic signals to optimize public transit and deliver information to riders.
- Seton Healthcare mining through vast amounts of unstructured data captured in notes and dictation to get a more complete view of patients. Seton currently uses this information to construct programs that target treatments to the right patients with a goal of minimizing hospitalizations in the way that most efficiently optimizes costs with benefits. The ability to mine unstructured data gives a much more complete view of patients, including factors such as their support system, their ability to have transportation to and from appointments, and whether or not they have a primary care physician.
- WellPoint using Watson technology to improve real-time decision-making by mining through millions of pages of medical information while doctors and nurses are face-to-face with patients.
But, clients warned that as much as the technology is advancing, the biggest hurdles remained the internal ones. Clients stressed that they face a critical challenge in introducing, driving, and changing the organizational mindset to work in a new way that can take advantage of these great advances in technology. What did they suggest?
1) Executive sponsorship from the top (C-level)
2) Hiring or retraining for new roles like data scientists (schools like Syracuse are introducing and promoting new programs out of their iSchool, which can help with reskilling experienced talent from other areas)
Join us at Forrester’s CIO Forum in Las Vegas on May 3 and 4 for “The New Age Of Business Intelligence.”
The amount of data is growing at tremendous speed — inside and outside of companies’ firewalls. Last year we did hit approximately 1 zettabyte (1 trillion gigabytes) of data in the public Web, and the speed by which new data is created continues to accelerate, including unstructured data in the form of text, semistructured data from M2M communication, and structured data in transactional business applications.
Fortunately, our technical capabilities to collect, store, analyze, and distribute data have also been growing at a tremendous speed. Reports that used to run for many hours now complete within seconds using new solutions like SAP’s HANA or other tailored appliances. Suddenly, a whole new world of data has become available to the CIO and his business peers, and the question is no longer if companies should expand their data/information management footprint and capabilities but rather how and where to start with. Forrester’s recent Strategic Planning Forrsights For CIOs data shows that 42% of all companies are planning an information/data project in 2012, more than for any other application segment — including collaboration tools, CRM, or ERP.
My colleagues and I have just completed yet another engagement with a large client — one of dozens recently — who was facing a to be or not to be decision: whether to move its BI platform and applications to the cloud. It’s a very typical question that our clients are asking these days, mainly for the following two reasons:
In many cases, their current on-premises BI solutions are too inflexible to support the business now, much less in the future.
The relative success of cloud-based CRM (SFDC and others) solutions may indicate that cloud offers a better alternative.
These clients put these two statements together and make the reasonable assumption that cloud BI will solve many of the current BI challenges that cloud-based CRM solved. Reasonable? Yes. Correct? Not so fast — the only correct answer is “It depends.”
Let’s take a couple of steps back. First, let’s define applications or packaged solutions vs. platforms (because BI requires both).
Subscribe to a solution-like CRM
Provide standard business functions to all customers (which makes it different from “hosting;” see below)
Difficult to tailor to specific needs
Usually are used synonymously (but incorrectly, see below) with software-as-a-service (SaaS)
Platforms for building solutions
Subscribe to tools and resources to build solutions like CRM
Provide standard technical functions to developers
Contain limited, if any, business application functionality
Usually labeled either as platform-as-a-service (PaaS) or infrastructure-as-a-service (IaaS).
It seems everyone’s obsessed with Facebook’s IPO right now. And while CMOs are beginning to understand the possibilities of Facebook, and other social technologies, to connect and engage with customers, many CIOs remain unclear on the value of Facebook.
A question many business executives ask is this: “What’s the value of having someone like your page?”
On its own, maybe not much. But the true potential lies in the ability to collect insights about the people who like brands, products or services – be it your own or someone else’s.
For example, the chart below shows the percentage of consumers by age group who have “liked” Pepsi or Coca-Cola. These data suggest Coca-Cola is significantly more popular with 17-28 year olds than Pepsi, while Pepsi appears more popular with the 36-70 crowd. I pulled these data points directly from the Facebook likes of each of the brand pages using a free consumer tool from MicroStrategy called Wisdom. Using this tool I can even tell that Coca-Cola fans are likely to also enjoy the odd Oreo cookie and bag of Pringles.
As one of the industry-renowned data visualization experts Edward Tufte once said, “The world is complex, dynamic, multidimensional; the paper is static, flat. How are we to represent the rich visual world of experience and measurement on mere flatland?” There’s indeed just too much information out there to be effectively analyzed by all categories of knowledge workers. More often than not, traditional tabular row-and-column reports do not paint the whole picture or — even worse — can lead an analyst to a wrong conclusion. There are multiple reasons to use data visualization; the three main ones are that one:
Cannot see a pattern without data visualization. Simply seeing numbers on a grid often does not tell the whole story; in the worst case, it can even lead one to a wrong conclusion. This is best demonstrated by Anscombe’s quartet, where four seemingly similar groups of x and y coordinates reveal very different patterns when represented in a graph.
Cannot fit all of the necessary data points onto a single screen. Even with the smallest reasonably readable font, single line spacing, and no grid, one cannot realistically fit more than a few thousand data points using numerical information only. When using advanced data visualization techniques, one can fit tens of thousands data points onto a single screen — a difference of an order of magnitude. In The Visual Display of Quantitative Information, Edward Tufte gives an example of more than 21,000 data points effectively displayed on a US map that fits onto a single screen.
Demands by users of business intelligence (BI) applications to "just get it done" are turning typical BI relationships, such as business/IT alignment and the roles that traditional and next-generation BI technologies play, upside down. As business users demand more control over BI applications, IT is losing its once-exclusive control over BI platforms, tools, and applications. It's no longer business as usual: For example, organizations are supplementing previously unshakable pillars of BI, such as tightly controlled relational databases, with alternative platforms. Forrester recommends that business and IT professionals responsible for BI understand and start embracing some of the latest BI trends — or risk falling behind.
Traditional BI approaches often fall short for the two following reasons (among many others):
BI hasn't fully empowered information workers, who still largely depend on IT
BI platforms, tools and applications aren't agile enough
Emerging ARM server Calxeda has been hinting for some time that they had a significant partnership announcement in the works, and while we didn’t necessarily not believe them, we hear a lot of claims from startups telling us to “stay tuned” for something big. Sometimes they pan out, sometimes they simply go away. But this morning Calxeda surpassed our expectations by unveiling just one major systems partner – but it just happens to be Hewlett Packard, which dominates the WW market for x86 servers.
At its core (unintended but not bad pun), the HP Hyperscale business unit Project Moonshot and Calxeda’s server technology are about improving the efficiency of web and cloud workloads, and promises improvements in excess of 90% in power efficiency and similar improvements in physical density compared with current x86 solutions. As I noted in my first post on ARM servers and other documents, even if these estimates turn out to be exaggerated, there is still a generous window within which to do much, much, better than current technologies. And workloads (such as memcache, Hadoop, static web servers) will be selected for their fit to this new platform, so the workloads that run on these new platforms will potentially come close to the cases quoted by HP and Calxeda.