BI professionals spend a significant portion of their time trying to instill the discipline of datadriven performance management into their business partners. However, isn’t there something wrong with teaching someone else to fly when you’re still learning to walk? Few BI pros have a way to measure their BI performance quantitatively (46% do not measure BI performance efficiencies and 55% do not measure effectiveness). Everyone collects statistics on the database and BI application server performance, and many conduct periodic surveys to gauge business users’ level of satisfaction. But how do you really know if you have a high-performing, widely used, popular BI environment? For example, you should know BI performance
Efficiency metrics such as number of times a report is used or a number of duplicate/similar reports, etc
Effectiveness metrics such as average number of clicks to find a report and clicks within a report to find an answer to a question and many others
Metric attributes/dimensions such as users, roles, departments, LOBs, regions and others
BI is used to build, report, and analyze business performance metrics and indicators. What about measuring the performance of BI itself? How do you know if you have a high-performing, widely used BI environment? Is your opinion based on qualitative “pulse checks” or is it based on quantitative metrics? BI practitioners who preach to their business counterparts to run their business by the numbers need to eat their own dog food: run their BI environment, platforms, and apps by the numbers. For example, do you know:
How many reports and queries do end users create by themselves versus how many IT creates? That's a great efficiency metric.
How many clicks within a dashboard does it take to find an answer to a question? That’/s another great efficiency metric.
How long does each user stay within each report? Do they just run and print the reports, or export the data to Excel, or do they really slice, dice, and analyze the information? That’s a good example of how effective your BI environment is.
Do you see any patterns in BI usage? User by user, department by department, or line of business by line of business?
How many reports, queries, and other objects are being used, how many are shelfware (not being used)? How often are people using the ones that are being used?
It is that dreaded time of year again where we have to report via the performance management system (PMS) on our individual performance and the value we bring to the organization. I say dreaded, because we all know that in reality the goals and objectives were set some time ago in the past, maybe a year ago, and a lot has happened since that time. The person you report to may have changed, you were redirected to other tasks, and so on. Everything seemed possible at the time of the objective setting, but now the reality hits that you were or may have been far too optimistic about your own capability. The self-assessment is difficult as you are not sure whether your manager has the same view as you. You believe you met the objective, but does their expectation meet your actual delivery? If a good performance relates to more money, the pressure and stress builds.
So whilst I was preparing for my Orlando Business Architecture Forum presentation I started to think about how business architecture teams measure and manage their performance. One of my next reports for Forrester’s business architecture playbook addresses BA performance. It was also a hot topic for the EA Council members in Orlando. I had a number of 1-on-1’s with clients who particularly asked about BA metrics and performance — in particular, “What do other business architecture teams do?”
I started listing the questions that, when answered by clients, would lead to a very valuable report for all BA leaders:
Do you measure your BA’s performance? Clients often advise me that they have fairly mature BA practices. However, very few can articulate how they measure their performance, and often comment that the business asks them to demonstrate how BA adds value. So, it would be useful to understand whether BA leaders measure their team’s performance and why they do or don’t.
I’m thrilled to see “people” talked about as a major focus of business. Company executives recognize that people are critical to sustainable organizational growth. Talent is now a C-level priority. People development is a responsibility of all managers and leaders, not just the HR department. Great to hear! Vendors see talent management as a hot space and are strategically lining up to meet business needs — enter IBM!
The rumor circulating for the past few weeks has now been confirmed: Oracle is buying Taleo, a global talent management vendor, for $1.9 billion. This is just another — albeit important — acquisition in the strategic talent management space. All companies must have core HR systems in place, but now it’s equally important to look at the strategic part of HR: the performance, succession, career development, and learning components as a layer resting on top of the core. Companies want to retain, develop, and reward their employees and need these applications in place for efficiency and effectiveness.
With this acquisition, Oracle gets a vendor with these talent management components in a pure SaaS deployment model, which provides ultimate flexibility. However, the offerings in the suite are not equally robust. Taleo is known for its recruiting app; to become a suite vendor, it added performance, which has gotten mixed reviews, and learning, which is not best in its class. Learn.com, the vendor Taleo acquired for learning, works OK for the midmarket, but its functionality does not hold up well for large global and enterprise customers.
Oracle can’t buck the SaaS tide any more. SaaS is the preferred deployment model for talent management, and the large ERP vendors like SAP (finalizing its acquisition of SuccessFactors) and Oracle are now joining the movement. Oracle offers Fusion, but a lot of work still needs to be done to develop this into a full SaaS talent suite. Once this deal closes, watch and see how Oracle positions the Taleo offerings with Fusion Talent Management.
Well actually I meant mobs of flash, but I couldn’t resist the word play. Although, come to think of it, flash mobs might be the right way to describe the density of flash memory system vendors here at Oracle Open World. Walking around the exhibits it seems as if every other booth is occupied by someone selling flash memory systems to accelerate Oracle’s database, and all of them claiming to be: 1) faster than anything that Oracle, who already integrates flash into its systems, offers, and 2) faster and/or cheaper than the other flash vendor two booths down the aisle.
All joking aside, the proliferation of flash memory suppliers is pretty amazing, although a venue devoted to the world’s most popular database would be exactly where you might expect to find them. In one sense flash is nothing new – RAM disks, arrays of RAM configured to mimic a disk, have been around since the 1970s but were small and really expensive, and never got on a cost and volume curve to drive them into a mass-market product. Flash, benefitting not only from the inherent economies of semiconductor technology but also from the drivers of consumer volumes, has the transition to a cost that makes it a reasonable alternative for some use case, with database acceleration being probably the most compelling. This explains why the flash vendors are gathered here in San Francisco this week to tout their wares – this is the richest collection of potential customers they will ever see in one place.
I have been working on a research document, to be published this quarter, on the impact of 8-socket x86 servers based on Intel’s new Xeon 7500 CPU. In a nutshell, these systems have the performance of the best-of-breed RISC/UNIX systems of three years ago, at a substantially better price, and their overall performance improvement trajectory has been steeper than competing technologies for the past decade.
This is probably not shocking news and is not the subject of this current post, although I would encourage you to read it when it is finally published. During the course of researching this document I spent time trying to prove or disprove my thesis that x86 system performance solidly overlapped that of RISC/UNIX with available benchmark results. The process highlighted for me the limitations of using standardized benchmarks for performance comparisons. There are now so many benchmarks available that system vendors are only performing each benchmark on selected subsets of their product lines, if at all. Additionally, most benchmarks suffer from several common flaws:
They are results from high-end configurations, in many cases far beyond the norm for any normal use cases, but results cannot be interpolated to smaller, more realistic configurations.
They are often the result of teams of very smart experts tuning the system configurations, application and system software parameters for optimal results. For a large benchmark such as SAP or TPC, it is probably reasonable to assume that there are over 1,000 variables involved in the tuning effort. This makes the results very much like EPA mileage figures — the consumer is guaranteed not to exceed these numbers.
Yesterday I attended the first day of SuccessFactors’ California customer conference at the Palace Hotel in San Francisco. Efficiency, speed, and good orchestration were evident throughout the day. The CEO, Lars Dalgaard, is a high-energy person who exudes confidence in the growth of his company. He is a real showman, and rather than giving a high-level company overview, his 90-minute presentation focused on product demos with touchscreen projections that worked fairly well. He clearly knows the products, has market momentum, and is driving the company forward. Lars would say, “We are about ‘Execution!’” The SuccessFactors slogan is “Success = Strategy + Execution.” The touted “new” offerings include recruiting (it’s been out for two years); a core HR data management app called Employee Central; calibration; goal execution; and the brand-new offerings through acquisitions -- Inform for workforce planning and analytics, and CubeTree for social collaboration. Acquisitions are new for SuccessFactors, so it hasn’t had experience in bringing together different company cultures and technologies, but my bet is that they’ll be successful.