January 1, 2013
By Leonard Klie, Editor, CRM magazine and SmartCustomerService.com

Getting Closer to Customers Tops Big Data Agenda

Most corporate big data initiatives are aimed at improving the customer experience, according to research by IBM and the Said Business School at the University of Oxford.

Yet despite the strong focus on the customer, less than half of the organizations engaged in active big data initiatives are currently collecting and analyzing external sources of data, such as social media.

A primary reason these data sources are underused is a skills gap. Having the advanced capabilities required to analyze unstructured data—data that does not fit into traditional databases, such as text, sensor data, geospatial data, audio, images, and video, as well as streaming data—remains a major challenge for most organizations. Only 25 percent of respondents in the report, titled "Analytics: The Real-World Use of Big Data," said they have the required capabilities to analyze highly unstructured data, a major inhibitor to getting the most value from big data.

In fact, Michael Schroeck, global information management leader at IBM Global Business Services, says there are currently about 300,000 data scientist jobs available at U.S. corporations and almost no one with the skills needed to fill them.

"The analytical tools and technologies needed are evolving rapidly, but the human skills are not keeping pace," Schroeck says. Companies' demand for information is also advancing beyond their ability to process available data, Schroeck argues in the report, which is based on a survey of 1,144 business and IT professionals from 95 countries and 26 industries.

The task of effectively analyzing and making sense of all the customer data coming into companies today requires not just analytical, statistical, and mathematical skills, but also industry and business knowledge and customer service skills. Few employees have that combination, according to Schroeck.

Big data also requires the capability to analyze semistructured and unstructured data, including a variety of data types that likely are entirely new for many organizations.

In more than half of the active big data projects, respondents reported using advanced capabilities to analyze text in its natural state, such as the transcripts of call center conversations. This data can help companies understand the current mood of a customer and gain valuable insights that can be immediately used to drive customer management strategies, but that insight is hard to come by, Schroeck concludes.

Further hindering efforts are the volume and variety of data sources, the velocity with which information comes in, and the need to preserve consumer privacy.

That's why many businesses are starting slowly with their big data initiatives. This often means starting with internal data, such as point-of-sales information, sales figures, marketing campaign data, customer call volumes, and Web site views, and then only later extending to other areas, such as social media.

"Most companies recognize the potential for big data to improve decision-making and business outcomes across the enterprise. What they struggle with, however, is how to get started on their big data journey," Schroeck maintains. "Across industries and geographies, the survey found that organizations are taking a pragmatic approach to big data. While the majority of them are still in the early stages of adoption, leading organizations are beginning to derive significant value from their big data initiatives."

Of the initiatives already in place, nearly half (49 percent) have customer-centric outcomes as the top priority. This primarily involves collecting and analyzing available information related to customer behavior and preferences and using that data to better serve and sell to those customers.

To overcome the inherent skills gap, Schroeck says many companies are already creating new roles, career paths, and training programs. Universities are increasingly adding these skill sets as part of their business program curricula as well.