Not very long ago, it would have been almost inconceivable to consider a new large-scale data analysis project in which the open source Apache Hadoop did not play a pivotal role.
Every Hadoop blog post needs a picture of an elephant. (Source: Paul Miller)
Then, as so often happens, the gushing enthusiasm became more nuanced. Hadoop, some began (wrongly) to mutter, was "just about MapReduce." Hadoop, others (not always correctly) suggested, was "slow."
Then newer tools came along. Hadoop, a growing cacophony (innacurately) trumpeted, was "not as good as Spark."
But, in the real world, Hadoop continues to be great at what it's good at. It's just not good at everything people tried throwing in its direction. We really shouldn't be surprised by this. And yet, it seems, so many of us are.
For CIOs asked to drive new programmes of work in which big data plays a part (and few are not), the competing claims in this space are both unhelpful and confusing. Hadoop and Spark are not, despite some suggestions, directly equivalent. In many cases, asking "Hadoop or Spark" is simply the wrong question.
I was in Tokyo last week, for the latest OpenStack Summit. Over 5,000 people joined me from around the world, to discuss this open source cloud project's latest - Liberty - release, to lay the groundwork for next year's Mitaka release, and to highlight stories of successful adoption.
Tokyo's Hamarikyu Gardens combine old with new (Source: Paul Miller)
And, unlike many events, this wasn't a hermetically sealed bubble of blandly anodyne mid-Atlantic content, served up to the same globe-trotting audience in characterless rooms that could so easily have been in London, Frankfurt, or Chicago. Instead, we heard from local implementers of OpenStack like Fujitsu, Yahoo! Japan, and - from just across the water - SK Telecom and Huawei.
In keynotes, case studies, and deep-dive technical sessions, attendees learned what worked, debated where to go next, and considered the project's complicated relationship to containers, software-defined networks, the giants of the public cloud, and more.
There's never been a question on the advantages of open source software. Crowdsourcing, vendor independence, ability to see and in some cases control the source code, and lower costs are just a few benefits of open source software (OSS) and business model. Linux and Apache Hadoop are prime examples of successful OSS projects. It's a different story, however, when it comes to OSS BI. For years, OSS BI vendors struggled with growth because of:
The developer-centric nature of open source projects. The target audience for open source projects is developers, which means deals are mostly sealed by technology management. The industry, on the other hand, has gravitated toward business decision-makers within organizations over the last several years. However, business users are less interested in the opportunities that a collaborative open source community offers, and more concerned about ease of use and quick setup. Indeed, Forrester's research constantly finds evidence correlating business ownership as one of the key success factors for effective BI initiatives.
Several events over the past few months in China will affect both the IT procurement strategy of Chinese organizations and the market position and development of local and foreign IT vendors, including:
A government-led push away from foreign IT vendors. Amid security concerns, the Chinese government has issued policies to discourage the use of technology from foreign IT vendors. As a result, many IT and business decision-makers at state-owned enterprises (SOEs) and government agencies have put their IT infrastructure plans — most of which involved products and solutions from foreign IT vendors — on hold. They’ve also begun to consider replacing some of their existing technology, such as servers and storage, with equivalents from domestic vendors. This is significant given that government agencies and SOEs are the key IT spenders in China.
A trend to get rid of IBM, Oracle, and EMC. Alibaba was an early mover, replacing its IBM Unix servers, Oracle databases, and EMC storage with x86 servers, open source databases like MySQL and MongoDB, and PCIe flash storage. This has evolved into replacing these foreign products and solutions with ones from local Chinese vendors. For example, Inspur launched the I2I project to stimulate customers to drop IBM Unix servers in favor of Inspur Linux servers to support business development. The Postal Savings Bank of China, China Construction Bank, and many city commercial banks have started deploying Inspur servers in their data centers. However, this only affects the x86 server and storage product market: While domestic vendors can provide x86 servers and storage, they still have no databases to replace Oracle’s.
Since Tibco acquired Jaspersoft on April 28th, 2014, I keep being asked the question: “Will this deal change the BI and analytics landscape?” (If you missed the announcement, here’s the press release.)
The short answer is: it could. The longer answer goes something like this: Jaspersoft and Tibco Spotfire complement each other nicely; Jaspersoft brings ETL and embedded BI to the table, whereas Spotfire has superior data analysis, discovery, and visualization capabilities. Jaspersoft’s open source business model provides Tibco with a different path to market, and Jaspersoft can benefit from Tibco’s corporate relationships and sales infrastructure. And with its utility-based cloud service, Jaspersoft also adds another option to Spotfire’s SaaS BI offering.
But that’s only the narrow view: once you take into consideration Tibco’s history (the hint’s in the name - “The Information Bus Company”) and the more recent string of acquisitions, a much larger potential story emerges. Starting with Spotfire in 2007, Tibco has assembled a powerful set of capabilities, including (but not limited to) analytics, data management, event processing, and related technologies such as customer loyalty management and mapping. If Tibco manages to leverage all of its assets in a way that provides enterprises with a flexible and agile integrated platform that helps them turn their data into actionable information, it will be a powerful new force that has the potential of changing enterprise BI platforms market.
To get there, Tibco has a number of challenges to address. On a tactical basis, it’s all about making the Jaspersoft acquisition work:
Retaining the talent
Making it easy for clients and prospects to engage with both companies
At the OpenStack Summit in Portland last week, the open-source cloud platform got real, to echo Forrester’s cloud team predictions for 2013. At the busy gathering attended by over 2,400, suits mingled effortlessly with hoodies and deep-tech design committee meetings were sandwiched between marquee-name customers sharing success stories. Three core themes drove the show, as outlined by Jonathan Bryce in the opening keynote: the OpenStack technology platform has matured, the ecosystem is vibrant, and the global user footprint now includes enterprise customers doing real business.
The show followed on the heels of the Grizzly release, the 7th release of the OpenStack platform. Along with stronger support for VMware and Microsoft hypervisors, Grizzly widens block storage options and includes 10+ new enterprise storage platform drivers and workload-based scheduling. A wide range of new network plugins expand the platform’s software-defined networking options and a sexier Dashboard to access, provision and automate resources.
Think of a medieval fortress: It was originally used for a small army, it has walls nine meters thick, and it’s surrounded by buildings hundreds of years old. Upon entering, you are confronted with the concept of eternity.
This fortress is located in the smallest state on earth — though it is also perhaps the best-known state in the world. The business housed within the fortress is what many might classify as a SME but with with complexity of a large enterprise, holy but busy, centralized but truly global — its work spans hundreds of countries with hundreds of currencies and hundreds of languages — and it serves very special and demanding clients.
Have a clue yet of where we are?
Zoom on Italy, then zoom on Rome, then zoom on Vatican City, and you can’t miss the round tower (Torrione Sisto V) where the Vatican Bank, or Istituto per le Opere di Religione (IOR ), is located. You won’t be allowed in if you are not a client, an employee, or part of a religious congregation. Change comes hard to institutions this steeped in tradition. To give you a clue, IOR’s previous managing director spent his entire career at IOR — 60 years — and retired at the age of 80. We all know it’s the soft and cultural aspects of transformation that are the hardest part for any organization.
Nevertheless, IOR has been going through a major change since 2008, working to replace its legacy IT system with a modern BT one. The new BT system brings more flexibility for the business, richer business functionality, and greater integration and development capabilities. Enabling fast change is the key driver for IOR’s IT transformation program from IT into BT.
We have been repeatedly reminded that the requirements of hyper-scale cloud properties are different from those of the mainstream enterprise, but I am now beginning to suspect that the top strata of the traditional enterprise may be leaning in the same direction. This suspicion has been triggered by the combination of a recent day in NY visiting I&O groups in a handful of very large companies and a number of unrelated client interactions.
The pattern that I see developing is one of “haves” versus “have nots” in terms of their ability to execute on their technology vision with internal resources. The “haves” are the traditional large sophisticated corporations, with a high concentration in financial services. They have sophisticated IT groups, are capable fo writing extremely complex systems management and operations software, and typically own and manage 10,000 servers or more. The have nots are the ones with more modest skills and abilities, who may own 1000s of servers, but tend to be less advanced than the core FSI companies in terms of their ability to integrate and optimize their infrastructure.
The divergence in requirements comes from what they expect and want from their primary system vendors. The have nots are companies who understand their limitations and are looking for help form their vendors in the form of converged infrastructures, new virtualization management tools, and deeper integration of management software to automate operational tasks, These are people who buy HP c-Class, Cisco UCS, for example, and then add vendor-supplied and ISV management and automation tools on top of them in an attempt to control complexity and costs. They are willing to accept deeper vendor lock-in in exchange for the benefits of the advanced capabilities.
I don’t understand why firms spend millions of dollars on Java application servers like Oracle Weblogic or IBM WebSphere Application Server. I get why firms spend money on Red Hat JBoss -- they want to spend less on application servers. But, why spend anything at all? Apache Tomcat will satisfy the deployment requirements of most Java web applications.
Your Java Web Applications Need A Safe, Fast Place To Run
Most Java applications don’t need a fancy container that has umpteen features. Do you want to pay for a car that has windshield wipers on the headlights? (I wish I could afford it.) Most Java applications do not need these luxuriant features or can be designed not to need them. Many firms do, in fact, deploy enterprise-class Java web applications on Apache Tomcat. It works. It is cheap. It can save tons of dough.
Expensive Java Application Servers Sometimes Add Value
There is a need for luxury. But, you probably don’t need it to provide reliable, performant, and scalable Java web applications. Application server vendors will argue that:
You need an application container that supports EJBs. EJB3 fixed the original EJB debacle, but why bother? Use Spring, and you don’t need an EJB-compliant container. Many applications don’t even need Spring. EJBs are not needed to create scalable or reliable applications.