I’ve recently conducted research on the issues of VLDB (very large databases) and how it affects BI, since the challenges of reporting and analyzing Gb data sets are very different from reporting and analyzing multi Tb data sets. Among many other interesting findings and conclusions I uncovered the following approaches to handling VLDB challenges as they relate to BI:
Generic solutions by DBMS vendors:
Share everything vs. share nothing architecture
Caching, in-memory databases
Specialized file systems
Indexing (bit-map vs. B-tree)
BI-specific solutions by BI and other vendors:
ROLAP and reporting tool specific SQL optimization
Alternate DBMS (such as search indexes and vector DBMS)
I predict that for the foreseeable future, spreadsheets will remain the most widely used enterprise application. The widespread adoption isn't hard to understand — spreadsheets are powerful and flexible, yet intuitive and easy to use and learn. Plus, ad hoc applications and spreadsheet models isolate users from constant reliance on IT and incur low costs. Since the early days of BI, spreadsheets have played a natural and major role in the BI process/architecture, including:
I've been in the BI business for over 25 years so I've seen many ups and downs in the BI cycle. We are definitely in the "up" cycle these days. I see two main reasons for it:
Enterprises can no longer stay competitive just by squeezing more efficiencies from operational applications – business intelligence applications are needed to become more effective.
Digital data (structured and unstructured) volumes are growing at 30% a year, and will be reaching zetabyte sizes by year 2010 – that’s a number with 21 zeros! Solid BI implementations will be critical to successfully turn that data into useful information.
Does anyone have any comments on where we are in the BI cycle and what are some of the more recent key drivers?