Not very long ago, it would have been almost inconceivable to consider a new large-scale data analysis project in which the open source Apache Hadoop did not play a pivotal role.
Every Hadoop blog post needs a picture of an elephant. (Source: Paul Miller)
Then, as so often happens, the gushing enthusiasm became more nuanced. Hadoop, some began (wrongly) to mutter, was "just about MapReduce." Hadoop, others (not always correctly) suggested, was "slow."
Then newer tools came along. Hadoop, a growing cacophony (innacurately) trumpeted, was "not as good as Spark."
But, in the real world, Hadoop continues to be great at what it's good at. It's just not good at everything people tried throwing in its direction. We really shouldn't be surprised by this. And yet, it seems, so many of us are.
For CIOs asked to drive new programmes of work in which big data plays a part (and few are not), the competing claims in this space are both unhelpful and confusing. Hadoop and Spark are not, despite some suggestions, directly equivalent. In many cases, asking "Hadoop or Spark" is simply the wrong question.
My colleague Henry Baltazar and I have been watching the development of new systems and storage technology for years now, and each of us has been trumpeting in our own way the future potential of new non-volatile memory technology (NVM) to not only provide a major leap for current flash-based storage technology but to trigger a major transformation in how servers and storage are architected and deployed and eventually in how software looks at persistent versus nonpersistent storage.
All well and good, but up until very recently we were limited to vague prognostications about which flavor of NVM would finally belly up to the bar for mass production, and how the resultant systems could be architected. In the last 30 days, two major technology developments, Intel’s further disclosure of its future joint-venture NVM technology, now known as 3D XPoint™ Technology, and Diablo Technologies introduction of Memory1, have allowed us to sharpen the focus on the potential outcomes and routes to market for this next wave of infrastructure transformation.
I’ve recently been thinking a lot about application-specific workloads and architectures (Optimize Scalalable Workload-Specific Infrastructure for Customer Experiences), and it got me to thinking about the extremes of the server spectrum – the very small and the very large as they apply to x86 servers. The range, and the variation in intended workloads is pretty spectacular as we diverge from the mean, which for the enterprise means a 2-socket Xeon server, usually in 1U or 2U form factors.
At the bottom, we find really tiny embedded servers, some with very non-traditional packaging. My favorite is probably the technology from Arnouse digital technology, a small boutique that produces computers primarily for military and industrial ruggedized environments.
Slightly bigger than a credit card, their BioDigital server is a rugged embedded server with up to 8 GB of RAM and 128 GB SSD and a very low power footprint. Based on an Atom-class CPU, thus is clearly not the choice for most workloads, but it is an exemplar of what happens when the workload is in a hostile environment and the computer maybe needs to be part of a man-carried or vehicle-mounted portable tactical or field system. While its creators are testing the waters for acceptance as a compute cluster with up to 4000 of them mounted in a standard rack, it’s likely that these will remain a niche product for applications requiring the intersection of small size, extreme ruggedness and complete x86 compatibility, which includes a wide range of applications from military to portable desktop modules.
But Avoid Ending Up With A Zoo Of Individual Big Data Solutions
We are beyond the point of struggling over the definition of big data. That doesn’t mean that we've resolved all of the confusion that surrounds the term, but companies today are instead struggling with the question of how to actually get started with big data.
28% of all companies are planning a big data project in 2014.
According to Forrester's Business Technographics™ Global Data And Analytics Survey, 2014, 28% of the more than 1600 responding companies globally are planning a Big Data project this year. More details and how this splits between IT and Business driven projects can be found in our new Forrester Report ‘Reset On Big Data’.
Or join our Forrester Forum For Technology Leaders in London, June 12&13, 2014 to hear and discuss with us directly what Big Data projects your peers are planning, what challenges they are facing and what goals they target to achieve.
Over the years we’ve learned how to address the key business intelligence (BI) challenges of the past 20 years, such as stability, robustness, and rich functionality. Agility and flexibility challenges now represent BI’s next big opportunity. BI pros now realize that earlier-generation BI technologies and architecture, while still useful for more stable BI applications, fall short in the ever-faster race of changing business requirements. Forrester recommends embracing Agile BI methodology, best practices, and technologies (which we’ve covered in previous research) to tackle agility and flexibility opportunities. Alternative database management system (DBMS) engines architected specifically for Agile BI will emerge as one of the compelling Agile BI technologies BI pros should closely evaluate and consider for specific use cases.
Why? Because fitting BI into a row-oriented RDBMS is often like putting a square peg into a round hole. In order to tune such a RDBMS for BI usage, specifically data warehousing, BI pros usually:
Denormalize data models to optimize reporting and analysis.
Build indexes to optimize queries.
Build aggregate tables to optimize summary queries.
Build OLAP cubes to further optimize analytic queries.
SAP Has Managed A Turnaround After Léo Apotheker’s Departure
In February 2010, after Léo Apotheker resigned as CEO of SAP, I wrote a blog post with 10 predictions for the company for the remaining year. Although the new leadership mentioned again and again that this step would not have any influence on the company’s strategy, it was clear that further changes would follow, as it doesn’t make any sense to simply replace the CEO and leave everything else as is when problems were obviously growing bigger for the company.
I predicted that the SAP leadership change was just the starting point, the visible tip of an iceberg, with further changes to come. Today, one year later, I want to review these predictions and shed some light on 2010, which has become the “Turnaround Year For SAP.”
The 10 SAP Predictions For 2010 And Their Results (7 proved true / 3 proved wrong)
This year SAPPHIRE officially changed its name and became SAPPHIRE NOW. Why? Different answers from different people. Those that should know said: "The new name stresses the urgency." Urgency for whom, SAP? And will the next SAPPHIRE be named SAPPHIRE THEN? Never change a successful brand.
Another premiere for SAPPHIRE was the simultaneous show in Orlando, US and Frankfurt, Germany. With 5,000 attendees in Frankfurt, 10,500 in Orlando and 35,000 online participants, this was the biggest SAPPHIRE event ever. I must admit I was concerned going to Frankfurt while everyone in Walldorf desperately tried to escape to Orlando. Who wants to attend a second-hand event? But now I’m a believer. SAP managed to balance the important parts of the show between Orlando and Frankfurt. Keynotes were held simultaneously in both locations via virtual video connection and speakers in both cities. In general I never had the feeling I would miss anything important in Frankfurt simply because it was the smaller event overall. It didn’t make a difference if I couldn’t attend another 400 presentations in Frankfurt or 800 in Orlando from the total of 1,200+ presentations – I had a packed agenda and got all that I expected and needed, including 1:1 meetings with SAP executives like Jim Snabe. The simultaneous, virtual set-up not only helped to save a lot of cost, it created a sense of a bigger virtual community and underlined SAP’s ambitions for more sustainability. To all that traveled intercontinental: Shame on you, next year stay in your home region!
Like every show SAPPHIRE 2010 had its stars as well:
After the recent board changes the strategy will change too
After the recent board changes at SAP the message we could read in most news was like ‘new board – old strategy’. Along with the board changes SAP did not announce (yet) any significant strategic changes. But what good is it to change the board and leave everything else as is?
The recent SAP board changes are just the visible tip of the iceberg of much deeper changes SAP will and has to go through to renew itself as a leading IT vendor. Below are 10 predictions for changes in SAP’s strategic direction I expect within the next 10+ months:
1. More SAP Board Changes Will Come
Additional board changes will further strengthen the product & technology focus and competence within the SAP board. See also Forrester’s blog on the recent SAP board changes: SAP CEO Resigns – Long Live The Co-CEOs
2. Business ByDesign Will Get Back Into SAP’s Strategic Center
Business ByDesign will become again the corner stone of SAP’s growth strategy and the successful introduction will mark a ‘make it or break it’ milestone for SAP.
3. SAP Announces The Next-Generation ERP
SAP will announce a next-generation ERP solution to regain leadership in its core business area and it will likely be based on the ByDesign platform.
4. SAP Changes Its Cloud Strategy
SAP will rework its whole On-Demand strategy and will unify and align all components based on the ByDesign platform. See also Forrester’s recent blog on SAP’s On-Demand strategy: SAP Is Skydiving Into The Clouds.