Text Analytics: A Key Trend To Watch Over The Next Three Years

My colleague Gene Leganza, who serves Enterprise Architecture Professionals, compiled the top 15 technology trends EA should watch over the next three years. He highlights technologies that are new or changing, have the potential for significant impact, and require an IT-led strategy to exploit.

He highlights text analytics technology in the report because understanding unstructured data plays a critical part in daily operations. Enterprises have too much content to review and annotate manually. Text analytics products from vendors like Temis and SAS mine, interpret, and add structure to information to reveal hidden patterns and relationships.  In my 2009 overview of text analytics, I cite the primary use cases for these tools: voice of the customer, competitive intelligence, operations improvements, and compliance and law enforcement.

But there are a few other sweet spots for text analytics tools in the enterprise:

Analytics and search: Analytics tools surface and visualize patterns; search tools return discrete results to match an expressed need. But these disciplines are blending. People want to drill in to high-level analysis to find the specific thing customers buzz about.  And many searchers don’t know how to articulate their need as a query and are looking for the big picture on a topic or trend. Forrester expects these solutions to come together, as search tools mainstream semantic features like entity extraction out of the box, and analytics vendors introduce new ways to investigate relationships and data output.  

Analytics and IT operations: IT professionals lack tools to discern what content is high value, and what content they can safely delete. My report “Take Control Of Your Content” outlines how a content professional can use text analytics tools to identify duplicate or irrelevant content before it is pushed from legacy systems to new systems. He or she can also prioritize which systems and repositories the search engine crawls, and determine which content will fill the gaps in the search index and which content to omit.

Social network analysis: My colleague Jim Kobelius published solid research on social network analysis about how companies are mining customers, employee, and stakeholder comments to find expertise and influence. Social network analysis is advanced analytics that is specifically focused on identifying and forecasting connections, relationships, and influence among individuals and groups. It mines transactions, interactions, and other behavioral information that may be sourced from social media, and/or just as often from CRM, billing, and other internal systems.

We welcome your comments on analytics as a top trend to watch in 2011.


Text Analytics Continues to Grow in 2011

Great post Leslie! - I am from Attensity (one of the vendors in the text analytics space) and if 2010 was a sign of things to come - then we whole-heartedly agree with you! (see a blog post I put up earlier today that outlines our similar view: http://blog.attensity.com/2010/12/20/what%E2%80%99s-ahead-for-customer-f...)

We believe that one of the areas that will drive the growth in text analytics is analyzing customer data. Whether it's social network/social media/online customer conversation analysis or combining that with data from surveys, emails, CRM notes, chats, etc. into the mix - large companies want, no, they NEED to know what their customers are saying to make better decisions.....

So here's to 2011!

Learn more about text/sentiment analysis

Right on!

Folks who want to learn more should look into the Text Analtyics Summit (which I chair). The 7th annual summit is slated for May 18-19, 2011 in Boston: http://www.textanalyticsnews.com/text-mining-conference/index.shtml

And if you're interested in a specialized application, check out the Sentiment Analysis Symposium, April 12 in New York: http://sentimentsymposium.com .


Text Analytics


I honestly agree, being in the analytics business on one end, we have experienced various top tools available in the space, the tools just act like robots and give you very minimal options to channelize detailed search. especially if you are sitting with bulk data, lot/extensive manual intervention is needed to make the basic or ground level analysis, later to make sense of the analytics some more time is spent to identify patterns, lot of HR is involved.

On the other end, I personally feel the API's from various social networks are not refined apart from twitter (but it gives you data only for a period of time), the tool can crawl data in a structure which can be some times confusing, there are a lot of problems and some times no scope for analyzing Geographic, Demographic and language data as its either unavailable or not so reliable.

I would definitely hope to see any kind of improvisations on my concerns which i am sure are the industry concerns in verticals like sCRM, analytics, outreach, etc.


Text Analytics

Thank You, Thank You, Thank You!

Agreed, but the landscape is confusing...

I couldn't agree more Leslie.

Companies spend millions on trying to get insights from structured data sources (think business intelligence), but these efforts are only analyzing half the data that exists.

Despite the manifest business value, the market for analytical tools and services remains very fragmented with most of the attention still being focused on Business Intelligence vendors (they are safe & well established) and Social Media analytics (the hype is overbearing). Furthermore, most companies view this topic as suited just for IT (and hence don’t take the time to articulate the business value) or have the right org structure / roles in place to enable a company view of data.

One related trend that I'm starting to see is the marriage of unstructured & structured data to drive business value; but it is still a trickle compared to the amounts that companies spend on BI.


Text and data based analysis

For us text processing with data for analytics purposes is the future. Data alone is not the entire solution, and our new service trials show that to be the case.

Text Analytics

As a small Australian text analytics company (Leximancer), we certainly see a rosy future. Customers are becoming more discerning and can discriminate between some of the smoke and mirrors offerings in text analytics and solutions that have validity and can stand-up to scientific scrutiny. Text as data demands just as much rigour as numerical data, and people with the skills to use tools well.


Would agree on Leximancer. I used it in the early days of its business life on a large fraud case in London in a package of case analysis work. Would use it again. For me, gaining customer adoption is on getting people to know what they don't know. I've just written a Vision article for IQ Journal in Australia on how records and analysis are the future. Out at the end of January. My last articles were syndicated back to the UK and USA I believe. Just going head-on for selling text analysis wont deliver the goods, we have to make it meaningful and useful to the CEO. Executives tend not to think analytically, more emotively, as they don't have the time or good people to do the research on the many small choices to be made.

Text analytics is broadly applicable to customer experience work

Thanks Leslie, very good overview. We find that text analytics is incredibly helpful in our Voice of the Customer work. However, text analytics, and data mining more generally, are important tools in addressing customer experience issues.


Beyond the Arc, Inc.

Text Analytics

This is definitely an area that researchers will have to dive into, either by choice or necessity. In my day job we are swamped with unstructured data from surveys and chat logs amongst others. Most of it sits for lack of the time and other resources needed to normalize the data and make sense out of it. I would be willing to bet there is gold in this data mine.

Greg Timpany