When it comes to data technology, are you lost in translation? What's the difference between data federation, virtualization, and data or information-as-a-service? Are columnar databases also relational? Does one use the same or different tools for BAM (Business Activity Monitoring) and for CEP (Complex Event Processing)? These questions are just the tip of the iceberg of a plethora of terms and definitions in the rich and complex world of enterprise data and information. Enterprise application developers, data, and information architects manage multiple challenges on a daily basis already, and the last thing they need to deal with are misunderstandings of the various data technology component definitions.
The tide is turning on privacy. Since the earliest days of the World Wide Web, there has been an increasing sense that the Internet would effectively kill privacy – and in the wake of the NSA PRISM program revelations, that sentiment was stronger than ever. However, by using our Forrester’s Technographics 360 methodology, which blends multiple qualitative and quantitative data sources, we found that attitudes on privacy are evolving: Consumers are beginning to shift from a state of apathy and resignation to caution and empowerment.
In our recently published report, we integrate Forrester's Consumer Technographics® survey data, ConsumerVoices Market Research Online Community qualitative insight, and social listening data to provide a holistic view of the changes in consumer perceptions and expectations of data privacy. In the past year, individuals have 1) become much more aware about the ways in which organizations collect, use, and share personal data and 2) have started to change their online behavior in response:
No self-respecting EA professional would enter into planning discussions with business or tech management execs without a solid grasp of the technologies available to the enterprise, right? But what about the data available to the enterprise? Given the shift towards data-driven decision-making and the clear advantages from advanced analytics capabilities, architecture professionals should be coming to the planning table with not only an understanding of enterprise data, but a working knowledge of the available third-party data that could have significant impact on your approach to customer engagement or your B2B partner strategy.
Data discussions can't be simply about internal information flow, master data, and business glossaries any more. Enterprise architects, business architects, and information architects working with business execs on tech-enabled strategies need to bring third-party data know-how to their brainstorming and planning discussions. As the data economy is still in its relatively early stages and, more to the point, as organizational responsibilities for sourcing, managing, and governing third-party data are still in their formative states, it behooves architects to take the lead in understanding the data economy in some detail. By doing so, architects can help their organizations find innovative approaches to data and analytics that have direct business impact by improving the customer experience, making your partner ecosystem more effective, or finding new revenue from data-driven products.
Coming back from the SAS Industry Analyst Event left me with one big question - Are we taking into account the recommendations or insights provided through analysis and see if they actually produced positive or negative results?
It's a big question for data governance that I'm not hearing discussed around the table. We often emphsize how data is supplied, but how it performs in it's consumed state is fogotten.
When leading business intelligence and analytics teams I always pushed to create reports and analysis that ultimately incented action. What you know should influence behavior and decisions, even if the influence was to say, "Don't change, keep up the good work!" This should be a fundamental function of data govenance. We need to care not only that the data is in the right form factor but also review what the data tells us/or how we interpret the data and did it make us better?
I've talked about the closed-loop from a master data management perspective - what you learn about customers will alter and enrich the customer master. The connection to data governance is pretty clear in this case. However, we shouldn't stop at raw data and master definitions. Our attention needs to include the data business users receive and if it is trusted and accurate. This goes back to the fact that how the business defines data is more than what exists in a database or application. Data is a total, a percentage, an index. This derived data is what the business expects to govern - and if derived data isn't supporting business objectives, that has to be incorporated into the data governance discussion.
Improving the use of data and analytics is a top strategic priority for many companies. But organizations face major challenges ramping up their information management capabilities — in particular due to the combination of a brutal proliferation of new or enhanced technologies, emerging data sources, and difficulty in finding skilled people with the appropriate experience. As a result, companies are increasingly looking to service providers for help.
Please note that we use the term “data services” to refer to broader engagements (including data delivery, analysis, management, or governance-related services), while “data management services” form a smaller subset of services relating to finding, collecting, migrating, and integrating data.
Here are three of the key findings from our research:
More than two-thirds of organizations expect their spending on data management services to increase; 41% stated they expect spending to increase 5% to 10% in the next 12 months.
When it comes to data investment, data management is still asking the wrong questions and positioning the wrong value. The mantra of - It's About the Business - is still a hard lesson to learn. It translates into what I see as the 7 Deadly Sins of Data Management. Here are the are - not in any particular order - and an example:
Hubris: "Business value? Yeah, I know. Tell me something I don't know."
Blindness: "We do align to business needs. See, we are building a customer master for a 360 degree view of the customer."
Vanity: "How can I optimize cost and efficiency to manage and develop data solutions?"
Gluttony: "If I build this cool solutions the business is gonna love it!"
Alien: "We need to develop an in-memory system to virtualize data and insight that materializes through business services with our application systems...[blah, blah, blah]"
Begger: "If only we were able to implement a business glossary, all our consistency issues are solved!"
Educator: "If only the business understood! I need to better educate them!."
Major industry dynamics such as digital disruption are causing chaos and upheaval even in mature industries. To help navigate the changes that result, companies are placing an ever greater premium on data-driven insights. Put simply, the management and effective utilization of data have become essential for competitive survival — but the growing volume and diversity of data leave companies scrambling to effectively and efficiently manage, govern, and utilize it. Companies face four main data management challenges at the moment:
Highly skilled and experienced resources are expensive and difficult to find.
New technologies, such as SAP HANA or Hadoop, are challenging existing capabilities.
New sources of data are rendering existing information infrastructures inadequate.
Companies still lack maturity in managing and analyzing their data. For example, few companies have a fully-fledged information management strategy (just 13%, according to Forrester’s Q2 2013 Global Information Strategy And Architecture Online Survey).
Hadoop’s momentum is unstoppable as its open source roots grow wildly into enterprises. Its refreshingly unique approach to data management is transforming how companies store, process, analyze, and share big data. Forrester believes that Hadoop will become must-have infrastructure for large enterprises. If you have lots of data, there is a sweet spot for Hadoop in your organization. Here are five reasons firms should adopt Hadoop today:
Build a data lake with the Hadoop file system (HDFS). Firms leave potentially valuable data on the cutting-room floor. A core component of Hadoop is its distributed file system, which can store huge files and many files to scale linearly across three, 10, or 1,000 commodity nodes. Firms can use Hadoop data lakes to break down data silos across the enterprise and commingle data from CRM, ERP, clickstreams, system logs, mobile GPS, and just about any other structured or unstructured data that might contain previously undiscovered insights. Why limit yourself to wading in multiple kiddie pools when you can dive for treasure chests at the bottom of the data lake?
Enjoy cheap, quick processing with MapReduce. You’ve poured all of your data into the lake — now you have to process it. Hadoop MapReduce is a distributed data processing framework that brings the processing to the data in a highly parallel fashion to process and analyze data. Instead of serially reading data from files, MapReduce pushes the processing out to the individual Hadoop nodes where the data resides. The result: Large amounts of data can be processed in parallel in minutes or hours rather than in days. Now you know why Hadoop’s origins stem from monstrous data processing use cases at Google and Yahoo.
For decades, firms have deployed applications and BI on independent databases and warehouses, supporting custom data models, scalability, and performance while speeding delivery. It’s become a nightmare to try to integrate the proliferation of data across these sources in order to deliver the unified view of business data required to support new business applications, analytics, and real-time insights. The explosion of new sources, driven by the triple-threat trends of mobile, social, and the cloud, amplified by partner data, market feeds, and machine-generated data, further aggravates the problem. Poorly integrated business data often leads to poor business decisions, reduces customer satisfaction and competitive advantage, and slows product innovation — ultimately limiting revenue.
Forrester’s latest research reveals how leading firms are coping with this explosion using data virtualization, leading us to release a major new version of our reference architecture, Information Fabric 3.0. Since Forrester invented the category of data virtualization eight years ago with the first version of information fabric, these solutions have continued to evolve. In this update, we reflect new business requirements and new technology options including big data, cloud, mobile, distributed in-memory caching, and dynamic services. Use information fabric 3.0 to inform and guide your data virtualization and integration strategy, especially where you require real-time data sharing, complex business transactions, more self-service access to data, integration of all types of data, and increased support for analytics and predictive analytics.
Information fabric 3.0 reflects significant innovation in data virtualization solutions, including: