BETA
This is a BETA experience. You may opt-out by clicking here

More From Forbes

Edit Story

Why Don't We Dig Deeper Into the Sand of Social Data?

Following
This article is more than 10 years old.

I’m at the Dachis Group Social Business Summit 2013 in Austin listening to a great lineup of speakers on the changing trends of social business and marketing. As a whole, data analytics and BigData are on the top of minds of the speakers here, so I had to ask the same question that I have posed at other such events with analytics people: Why does data and analytics within the boundaries of the organization have to be so fine-grained and detailed while the data and information that we get from outside isn’t quite so? Does it matter?

John Hagel of Deloitte Center for the Edge spoke on the continuing pressure of seeking performance improvements in an organization and the value that data analytics can bring to ameliorate the situation or fill the gap. He also spoke of how surprises can emerge from understanding the data that makes us reevaluate some of our primary beliefs.

Deloitte Australia long held a belief: High performance and business success hinges on tightly integrated teams. Everything was driven around that view of the world. A couple of years ago, they developed a technology to analyze all their interactions by phone, email, and social networks, and were shocked to find that quite wrong.

The system created a map of all these interactions that was mashed against the performance data and key metrics of the consultants and teams involved. It turns out that tightly integrated teams, the belief they held paramount, were not the high performers. Rather, it was the teams consisting of people that reached out broadly across the organization and interacted more diversely. Asking the right question of the social data allowed them to drive improvement on a new level.

What I observed in particular is the depth of information required to discover this. This was not about large-scale aggregates and demographics but specific people, roles, and interactions. This isn’t the same depth that you find ‘beyond the firewall’ from public social networks. You simply don’t have that depth of information about identity in the public space.

Rob Bailey, CEO of DataSift (Image: DataSift)

Dr. Scott Hendrickson Gnip Rob Bailey DataSift Facebook

The challenge with public sites is that people’s identities may not match who they are in real life in any way, and there may be multiple such accounts for each person. You’d have a similar challenge if you had a customer database full of pseudonyms, and threw in thousands of other people who aren’t even your customers. And then multiply that by several times to account for each site. A data analyst would tell you that you have a lot of junk data – probably in less polite terms – in your database.

Yet, this is the trillions of grains of sand on the beach that we have to sort through to find the hidden treasures of useful information. How do companies sort through this using the tools of social media listening?

Generally, most companies are listening in broad spectrums. They look for aggregates: the name of the company or its top brands and products, broad demographics of people (e.g., under 18 year olds, those 18-34, people in a whole country), most common emotions (e.g., ‘like’, ‘hate’, ‘unhappy’, etc.)

These broad categories help us understand the what can only be described as the lowest-common denominator of information, what lots of people in a general audience may have in common. Is getting to specific information gold like what Mr. Hagel described for Deloitte Australia – per person, roles, etc. fine grained data – simply is too complex for Big Data to handle?

I sat down with Mr. Bailey of DataSift and asked about this and his point was that the market simply is not asking for it yet. The demand may come in a year or two, but right now, they aren't looking for more specific information. There are some cases where it is needed: detecting influencers. Knowing who your influencers are, in the world of marketing, can be the difference between boom and bust.

The other big challenge in finding that data is the ability to actually use it. This is the complex subjective challenge of privacy of information. It varies depending on where you are, where your information is going and how it is used. Mr. Bailey confirmed that places like France, Germany and Europe in general has looked at this issue much more carefully than the USA (where some of the top social sites happen to be based).

There is wealth out there on the beach but right now we aren’t allowed to dig too deeply, but we seem to be satisfied so far with scoops from the surface.

There is a lot of good possibilities for Social Data Analytics and BigData methods. What do you see as the possibilities? Send me your tweets @rawn. The Dachis Group is also having events in cities around the world: Shanghai, Mumbai, Sao Paulo, London, Tokyo and New York. Check them out if you're in the area.