Ways To Address BI Skills Shortage

Whether you are just starting on your BI journey or are continuing to improve on past successes, a shortage of skilled and experienced BI resources is going to be one of your top challenges. You are definitely not alone in this quest. Here are some scary statistics:
  • “By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills, as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions.” (Source: May 2012 McKinsey Global Institute report on Big Data)
  • “… trigger a talent shortage, with up to 190,000 skilled professionals needed to cope with demand in the US alone over the next five years.” (Source: 2012 Deloitte report on technology trends)
  • “Fewer than 25% of the survey respondents worldwide said they have the skills and resources to analyze unstructured data, such as text, voice, and sensor data.” (Source: 2012 research report by IBM and the Saïd Business School at the University of Oxford)
There are no easy short-term solutions to this challenge (with some exceptions — see the last bullet). The usual “create a competitive talent recruiting and retaining strategy and processes” is the right place to start, but it’s not going to be enough. Here are additional steps you can take as you staff your BI centers of excellence, competency centers, solution centers, programs, and projects:
  • Work with local schools, colleges, and universities. Create a data/BI scholarship fund. Sponsor BI classes. Create BI (summer) internships for students. This can help ensure a steady stream of graduates to apply for your BI positions.
  • Look inside your “shadow IT” organization. I’m sure I will not surprise you by telling you that no matter what other BI tools your organization uses, Microsoft Excel is and will continue to be your No. 1 BI tool. Find those people and entice them to come out from the shadows and work on your enterprise BI apps and platforms. These will end up being your top BI resources. In a large organization with millions of spreadsheets and hundreds of places to look for these people, consider using spreadsheet management tools to find them (Cimcon Software, Compassoft, ClusterSeven, Lyquidity, and when (or if) you upgrade to Microsoft Office 2013, Excel will come with such spreadsheet management capability, via a product that Microsoft acquired from Prodiance in 2011).
  • Work with the pros. Bring in third parties to help, but make sure you are working with consultants who have the experience, expertise, and critical mass of local talent, plus the global reach. Leverage our most recent BI Services Wave research to shortlist and select the best consultants to fit your needs.
  • If all else fails, you may truly be ready for BI outsourcing. Consider cloud BI vendors like Gooddata and Birst and some of the leading consultants mentioned in the previous bullet to not only host your BI platform, but also take on the task of building, maintaining, and delivering all of the BI artifacts: reports, dashboards, alerts, etc. 
Did I miss any other best practices? As always, curious to hear from all of you.


Not So Fast on BI Skills Shortage

Thank you for a wonderful article with my thoughts. First, to do BI and analytical work, organizations must have the ability to know "how to" ask the right questions. It is easy to become enamored with the latest new tools, but it is easy to waste money here. Having an environment and culture that allows for continuous discourse and discovery is essential.

Second, understanding unstructured data (e.g. text, voice, et al) is similar to hoarding; this is not necessarily germane to all industries in a way that would add value. Having a method to the madness is essential. People want to know "why" is this important and engage--requires strong social skills.

Third, educational systems in the US unfortunately shy away from the idea of thinking in terms of "models" or frameworks to organize information. The wonder of discovery is lost in the perceived drudgery of formulas to remember for standardized tests in secondary and post-secondary schools. Politically, we have not found a way to get away from standardized tests in favor of how to "think" and do so critically and make it fun to do so.

The skills shortage is fixable to be sure. There is a collective opportunity here to think differently and in non-linear ways to address this problem. Perhaps multi-disciplinary teams via a "global studio" of BI skills with a social networking flavor to advance the use of unstructured data as it moves through the cycle from raw information into useful insights, products and decisions that adds value.

The most important thing for

The most important thing for any enterprise today is to get started and determine how big data can most benefit your organization. There are plenty of ways to evaluate and test these technologies today with existing open source and commercial tools -- so you can start getting beyond the hype without employing experts first. When there is a need for skills, developers are savvy and they'll learn and earn those skills rapidly.

Talking about big data, Jeff Morris, Actuate's VP of Product Marketing says,"... business leaders and senior managers need to grasp the nettle. The skills gap -- if there is one -- will be short-lived, and many useful tools could help with any big data mission, easily offsetting the costs of development. Where there’s a will, if backed up by enough management support and leadership, there's surely a way."