Last year I published a reasonably well-received research document on Hadoop infrastructure, “Building the Foundations for Customer Insight: Hadoop Infrastructure Architecture”. Now, less than a year later it’s looking obsolete, not so much because it was wrong for traditional (and yes, it does seem funny to use a word like “traditional” to describe a technology that itself is still rapidly evolving and only in mainstream use for a handful of years) Hadoop, but because the universe of analytics technology and tools has been evolving at light-speed.
If your analytics are anchored by Hadoop and its underlying map reduce processing, then the mainstream architecture described in the document, that of clusters of servers each with their own compute and storage, may still be appropriate. On the other hand, if, like many enterprises, you are adding additional analysis tools such as NoSQL databases, SQL on Hadoop (Impala, Stinger, Vertica) and particularly Spark, an in-memory-based analytics technology that is well suited for real-time and streaming data, it may be necessary to begin reassessing the supporting infrastructure in order to build something that can continue to support Hadoop as well as cater to the differing access patterns of other tools sets. This need to rethink the underlying analytics plumbing was brought home by a recent demonstration by HP of a reference architecture for analytics, publicly referred to as the HP Big Data Reference Architecture.
With the employer mandate delays being the latest setback to U.S. president Obama's push for national healthcare, it's worth looking at how other countries are successfully tackling the same problem. The United Kingdom has had nationalized healthcare for years, and one of the things that makes this effort so successful is its approach to data collaboration — something Forrester calls Adaptive Intelligence.
While the UK hasn't successfully moved into fully electronic health records, it has in place today a health records sharing system that lets its over 27,000 member organizations string together patient care information across providers, hospitals, and ministries, creating a more full and accurate picture of each patient, which results in better care. At the heart of this exchange is a central data sharing system called Spine. It's through Spine that all the National Health Service (NHS) member organizations connect their data sets for integration and analysis. The data-sharing model Spine creates has been integral in the creation of summary care records across providers, an electronic prescription service, and highly detailed patient care quality analysis. As we discussed in the Forrester report "Introducing Adaptive Intelligence," no one company can alone create an accurate picture of its customers or its business without collaborating on the data and analysis with other organizations who have complementary views that flesh out the picture.
Ten years ago, open source software (OSS) was more like a toy for independent software vendors (ISVs) in China: Only the geeks in R&D played around with it. However, the software industry has been developing quickly in China throughout the past decade, and technology trends such as service-oriented architecture (SOA), business process management (BPM), cloud computing, the mobile Internet, and big data are driving much broader adoption of OSS.
OSS has become a widely used element of firms’ enterprise architecture. For front-end application architecture on the client side, various open source frameworks, such as jQuery and ExtJS, have been incorporated into many ISVs’ front-end frameworks. On the server side, OSS like Node.js is becoming popular for ISVs in China for high Web throughput capabilities. From an infrastructure and information architecture perspective, open source offerings like Openstack, Cloudstack, and Eucalyptus have been piloted by major telecom carriers including China Telecom and China Unicom, as well as information and communication solution providers like Huawei and IT service providers like CIeNET. To round this out, many startup companies are developing solutions based on MongoDB, an open source NoSQL database.
Familiarity with OSS is becoming a necessary qualification for software developers and product strategy professionals. Because of the wide usage of OSS among both vendors and end users, working experience and extensive knowledge with OSS is becoming a necessary qualification not only for software engineers, but also an important factors for product strategy professionals to establish appropriate product road maps and support their business initiatives.