Five Reasons Massachusetts To Lead Big Data Market

Big Data is joining the cavalcade of high tech buzzwords. Having been anointed, that means Big Data now has a study trumpeting its gigantic size and growth rate, a raft of companies going after the opportunity, and now a state initiative to capitalize on a public/private partnership. But can Massachusetts take the lead in this new industry?

Before getting into this question, let's explore what Big Data means. Based on interviews with several industry players, there is not one definition. But, as the Mass Technology Leadership Council (MassTLC) President, Tom Hopcroft, told me in a June 7th interview, all of them seem united under the broad banner of the Three Vs:

• Volume. Big data involves the extraction of useful insight from volumes of data so large that traditional database tools cannot handle the workload. Hopcroft points out that in the last year, the amount of new data created is sufficient to fill 60,000 Libraries of Congress;
• Velocity. Data analysis used to be a so-called batch processing -- managers got reports the morning after the day's transactions. Now, managers need instant analysis so they can use the insights to make decisions when they can still influence business outcomes; and
• Variety. Old-style data came in neatly formatted rows and columns. Big Data includes that as well as Twitter feeds, Facebook (FB) posts, GPS data, road sensor signals, genomic data, and videos. Big data tools try to handle them all.

Big Data companies target a $64 billion database and analytics market, according to the MassTLC's Big Data and Analytics report. Within Massachusetts, there are 120 Big Data companies.

These include hardware companies, like EMC (EMC), companies that analyze social media to help marketers pick the right messages and focus new product development, and others that analyze information driving securities prices for traders and operators of trading platforms.

And by 2018, MassTLC estimates that Massachusetts could employ 120,000 people who work in Big Data. This would represent a doubling of the current number -- 12,000 of whom work for the 120 start-ups and the balance who analyze data for big companies in health care, pharmaceuticals, and consumer products, according to the MassTLC.

One Massachusetts big data company, Crimson Hexagon (CH), is based in Boston's Seaport District. CH was co-founded by a Harvard professor and its software helps big companies to analyze social media -- using a 136 billion post database collected since 2008, according to my June 7th interview with CH's marketing VP, Wayne St. Amand.

CH helps an automobile manufacturer, for example, to mirror in its advertising copy the social media conversation about its car. So, for example, if people were talking more about a Toyota (TM) Camry's fuel efficiency on Twitter and Facebook, CH would identify this trend and Toyota might include a reference to fuel efficiency in its advertising.

St. Amand declined to provide revenue figures but he exulted that CH is "busier than ever and there is a lot of demand for what we do." He believes that CH is very popular with big companies because it does deeper analysis than do competing products.

For example, instead of merely telling you trends in specific keywords, it helps clients to identify the factors that are driving those trends. And this makes CH clients more comfortable taking action based on the findings because of the credibility that comes from understanding the factors driving the trends.

Lexington, Mass.-based StreamBase Systems helps financial institutions manage a fire hose of data to make better decisions. Mark Palmer, StreamBase CEO, told me in a June 7th interview -- citing Aite Group research -- that from 2001 and 2012, the volume of electronic trading has risen from under 30% to over 90% -- and this boosts both the volume and velocity of data that traders must analyze.

 StreamBase helps retail foreign exchange broker, Gain Capital, to give customers the best prices from among 15 different markets where currencies are traded. Palmer declined to disclose its fee -- but claimed that it's less than the $250,000 a year Gain would pay to do this itself.

Palmer argues that Gain benefits tremendously -- noting that StreamBase gives Gain Capital greater trading volume because of its reputation for finding customers the best price. That draws in new customers and delivers Gain a "dramatic return on investment" in StreamBase's software.

Here are five reasons why companies like Crimson Hexagon and StreamBase Systems are helping Massachusetts dominate big data:

• Big data software history. Cullinet, Computer Corporation of America, Phase Forward, and McCormack & Dodge are among the database companies that started in the Boston area. Their alumni started and worked for newer big data companies like Netezza -- International Business Machines (IBM) bought it in November 2010 for $1.8 billion and Endeca -- that Oracle (ORCL) acquired for $1.1 billion in 2011.
• Big data hardware. EMC, the largest stand-alone maker of data storage hardware is based in Hopkinton, Mass. -- the starting line for the Boston Marathon. The ability to work with the world's largest organizations provides insights into emerging trends.
• Data research. The leading data analysis innovators in the world work at MIT. These include Michael Stonebraker who has started several database companies and my database professor, Stan Zdonik who now teaches at Brown. Intel (INTC) decided to partner with MIT over 55 other universities for its big data research. Moreover, the state plans to offer funds for internships and matching grants for big data research in Holyoke, Mass.
• Big tech company R&D. MIT's Kendall Square has become an R&D hub for big technology companies including Google (GOOG) and Amazon (AMZN). Moreover, Hopcroft notes that IBM and Oracle employ many engineers here who focus on data analytics innovation.
• Data analysis culture. Meanwhile, Palmer believes that Massachusetts values of hard work, ingenuity, and persistence lend themselves well to building a Big Data company over a decade. In his view, this contrasts with Silicon Valley's consumer Internet built-to-flip mentality.

But Palmer believes that Massachusetts is the Avis of Big Data "we're number two, so we try harder." How so? Silicon Valley companies like Google and Facebook (FB) developed the open source database, Hadoop, project to analyze huge quantities of social media and search query data. Companies like Cloudera, about which I wrote in August 2011, take top billing in his view.

But Hopcroft believes that Massachusetts has the ingredients to "get the flywheel spinning" -- creating momentum that could finally put the state in the lead -- after decades of falling behind after its global dominance of the minicomputer industry eroded in the 1980s.