BETA
This is a BETA experience. You may opt-out by clicking here

More From Forbes

Edit Story

8 Ways To Build And Use The New Breed Of Data-Driven Applications

This article is more than 9 years old.

By Brian Ascher, partner at Venrock

Software-as-a-Service and cloud computing has been transformational for the software industry, but compared to what is coming next, you ain’t seen nothing yet. First, to appreciate where we are heading a quick review of where we’ve been is in order. Back in the olden days of business software a software company sold you an application which you installed on your servers and desktops which made business processes more efficient, facilitated workflow, and sped up information retrieval. As you used it this software accumulated data such as your customer records, financial results and manufacturing statistics. If you wanted to deeply analyze this data for trends and insights you bought Business Intelligence or Analytics packages from a different set of software vendors so you could slice and dice your data, generate reports for executives, and hopefully decipher interesting trends about your business that you would then go act on. In the early 2000s Software-as-a-Service (SaaS) companies emerged and enabled you to “rent” business applications, rather than buy them, as your employees accessed them through the Internet and their web browsers. This came with many advantages in total cost of ownership and manageability, but fundamentally most of the first SaaS applications were about workflow and data storage/retrieval just like their on-premise software forefathers. In the last few years we’ve had a “Big Data” explosion and a host of new open source technologies like Hadoop, MapReduce, and Cassandra packaged by a set of new companies that help businesses manage and manipulate their ever expanding mountains of data. Also emerging is a new generation of cloud based analytics companies that make it easy to slice, dice, and visualize big data sets.

So what’s the point of this history lesson? The point is that, for the most part, all of these business applications and more recent Big Data tools have left the burden of capturing real business insight, making decisions, and taking action on the business customers themselves. In essence, if you wanted real business value you had to create that value yourself by getting your employees to use the applications (which often means manual data input), have analysts mine and interpret the data, and ask managers and executives to make decisions based on what they see in the reports and charts. For example, if your company used Sales Force Automation software, whether it be Siebel on premise or Salesforce.com in the cloud, your sales reps had to diligently input data about their sales calls and management had to be smart about logging in to read the reports, suggest actions for each account and discern broader trends across the data. A new breed of software company is emerging, however, that combines data science expertise with deep understanding of business problems. I call them Data Driven Solutions.

These solutions use algorithmic data mining on your own data and often on external third party data accessible by cloud ecosystems and APIs. Data Driven Solutions make predictions about business functions, prescribe what to do next, and in many cases take action autonomously. Trained analysts are not required to query databases. Instead, business users get answers directly from the software. These answers typically feed seamlessly into the flow of business activity, often invisibly. While this distinction may seem subtle, I believe it is fundamental and disruptive, and represents the future of software.  This is in no way the end of SaaS, but in fact where SaaS is going next and presents massive opportunity to new SaaS innovators and a potential threat to incumbents who do not adapt.

8 Suggestions for Building Data Driven Applications

Think Moneyball, for everything Billy Beane of the Oakland A's defied the conventional wisdom of traditional baseball talent scouts by recruiting players other teams underappreciated but whom he believed represented great return on investment. He did this not by relying on his own brilliant sense of which players to recruit but by letting a math whiz run regression analysis on player statistics to figure out which lesser heralded stats were most predictive of winning baseball games. The math predicted results and told him which players to acquire, predictions which Beane followed to his team’s competitive advantage. Opportunities to apply this approach in business are practically everywhere. 6Sense* is a new SaaS company that analyzes B2B website traffic and third party data to predict which prospects are most likely to buy from you, what they will buy, when they will buy, and how much they will buy. Like Beane, they don’t rely on rules of thumb in scoring prospects, such as whether the prospect downloaded a white paper, viewed lots of product detail webpages, or has “Procurement” in their job title. 6Sense has found that these heuristics yield only about 50% accurate forecasts which is not enough to compel a sales person to trust the results. Instead, 6Sense uses a variety of machine learning statistical models to uncover the unexpected correlations which drive predictive accuracy up to 85-90% accuracy which definitely gets a sale rep’s attention. Instead of being a chore to use like salesforce automation, 6Sense tells sales reps how to close more deals and earn more commissions. InferLattice, and C9 are also innovating in the area of predictive CRM solutions.  Use your domain expertise to figure out what problems to solve for your customers, but let the data lead to new and unexpected insights.

Build in Data Learning Loops  Google enjoys a very powerful form of Network Effect. The more searches they run and resultant clicks they see the better they understand the intent of what a searcher wanted to find. This makes their search algorithms better which earns them more user searches, which keeps the search quality/volume flywheel spinning. Google also employs data learning loops on its AdWords product by selecting which ads to show for a given search result not only based on the advertisers’ bid amounts but based on Google’s prediction as to which ad is most likely to get clicked by a user. In essence, a discreet experiment is run every time a search result is served.  This notion of a “learning loop” can be applied to many business settings as long as you find a way to “close the loop” and see how your prediction or answer fared, whether it is predicting sales, customer churn, clicks on ads, or which candidates to hire. Data Learning Loops are powerful sources of competitive advantage akin to natural monopolies for those who achieve greatest scale.

Don’t Just Describe--Predict and Prescribe  Some may ask whether Data Driven Solutions are just a new name for Business Intelligence. I don’t think so. Analytics packages mostly describe what is happening by sorting and filtering your data to show sums and averages and trend lines in tabular or graphical format. Data Driven Solutions go much further by using the data to make predictions and even prescribe or execute actions. A good example is the retail industry and Point of Sale results. POS data is the basket by basket, stock-keeping unit by stock-keeping unit, store by store sales results that are collected across hundreds of thousands of retail outlets daily.  Nielsen has been compiling this data for decades and batch processes data sets for retailers and their vendors to study on a monthly basis.  How those vendors and retailers derive value from the data is up to them. Retail Solutions* is a data driven solutions company which also gets POS data from retailers and shares it with vendors, but on a near real-time or daily basis. More important than freshness of the data, however, is that Retail Solutions offers predictions and prescribes actions as solutions to business problems. RSI doesn’t just create reports, they predict when you will be out of stock on a given SKU and sends mobile alerts to shop clerks, distributors and store managers to make sure the shelves stay full. This is one of over ten predictive and prescriptive applications they provide in order to maximize return on investment for their customers. Pretty charts are not enough.

Data Is Not The Point, Focus On Solutions   Lots of companies market themselves as “Big Data” companies, but unless you are selling to IT Departments whose problems actually include managing lots of data, most business customers don’t really care about data.  They care about solving business problems. Athenahealth* helps doctors get paid faster. Turns out cash flow is really important to doctors and as a result Athena has grown very quickly and is now one of the largest SaaS companies. Doctors don’t care how Athena actually does what they do, which happens to involve statistical analysis of massive amounts of insurance claims data and heavy use of learning loops. The team at Athena deeply understands healthcare and doctors and so astutely markets themselves as a complete solution to real problems, and resist pounding their chest about how smart they are at Big Data. In fact, the word “data” does not appear even once on their homepage. Smart move.

Horizontal Strategy: Solve New Problems In New Ways   Providing applications for horizontal business functions like sales, finance, or human resources that function similarly across many industries represents very large opportunities because the market sizes are huge.  As a result there are powerful SaaS incumbents, such as Salesforce.com, Netsuite, and Workday, in each of these functional domains. As you would expect, these players are starting to add data driven application intelligence to their offerings. Fortunately for startups, the challenges businesses face are constantly changing thus creating opportunities to be the first to solve new problems with a new approach.  In the realm of marketing, for example, “Content Marketing” is the hottest new trend and is the digital marketing approach seeing the greatest increase in budget allocation. Yet marketers are highly confused as to what content to produce, how to produce it, where and how to distribute it, and especially how to measure ROI. Captora is a young startup that has jumped on this new problem with data and domain expertise and is seeing rapid growth and using their head start to establish a beachhead before direct competition comes at them. Knowing the experience of the team they won’t be resting on the side of the road but rather racing ahead to broaden their solution in synch with new challenges facing modern marketers.

Vertical Strategy: Feed the Starving  Providing deep solutions in specific industry verticals like healthcare, entertainment or education can be a huge opportunity. This is especially true in industries where data has largely been non-existent or hard to access, as has been the case in the three industries I just mentioned. If a Data Driven Solution can access, interpret, or create new data and use it to solve a big problem the market reaction can be like a starving person being offering a hot meal. Castlight Health*, for example, solves the problem that in healthcare it is generally impossible to know what a given service (an office visit or a test for example) will cost until after you’ve consumed the service and you receive your bill 30 days later. It turns out that the variance in pricing for even a commodity service like an MRI test can be 5x to 10x in a given five-mile radius. If one could know the price difference ahead of time one can consume intelligently--as we do in most other shopping situations. Large employers, which tend to be self-insured, really like the idea of helping their employees spend less on healthcare as those savings drop straight to the bottom line, and as a result some of the largest employers in America have adopted Castlight's solution. Customers like CVS Caremark, Microsoft, and Wal-Mart don’t really care about the big data blahdy blah that Castlight uses to come up with their solution, they just know they are starving for ways to lower their employee health care costs and Castlight has an effective solution.

Vertical Strategy  While some industries are just getting their first taste of Big Data, others have been sophisticated handlers and miners of Big Data for a long time, such as the investment industry, airlines, and e-commerce. In those fields a small incremental advantage afforded by a data driven vertical solution can be extremely valuable. DataMinr* is a company that transforms the full Twitter stream of public tweets using sophisticated math to discern important news events amid all the noisy babble as quickly as possible ahead of the media. Investment hedge funds will pay handsomely for incremental advantage and getting a jump on news that might move the market or a particular stock is something they are eager to buy, even amidst all their number crunching sophistication and home grown solutions. On April 23, 2013 when the stock market “Flash Crash” occurred based on a rumor that the Whitehouse was under attack, Dataminr’s algorithms figured out the attack was a hoax a full two minutes before other news outlets and its clients were able to act on the news ahead of the market’s rapid recovery from the severe dip the rumor had caused. Dataminr thus serves both a very sophisticated big data segment, investment funds, and an industry at the opposite end of the data automation curve, the news industry, with a solution that simply could not have existed until very recently.

Consumer Solutions Can Be driven By Data Too  Using Uber is a magical experience. Push a button on your phone and a car appears within minutes to take you where you want to go—no hailing, no reservations, no need to reach into your pocket for payment, and remarkably little waiting for your ride to arrive. If Uber simply sent messages to available drivers about customers needing rides the system might still be good but customers would have longer wait times, which wouldn’t be as magical. Instead, Uber uses statistical analysis on data coming from their drivers and riders to predict where demand will be highest and recommends that drivers congregate there to be ready for ride requests. Nowhere in their marketing does Uber talk about data or “quantifying” your ride patterns—consumers don’t need to know how the magic happens as long as their ride shows up quickly. PandoraAmazon.com and Zillow are each examples of consumer companies using data to invisibly enhance the customer experience. Opportunities to create consumer solutions enabled by big data are everywhere…just don’t mention the word data.

Traditional SaaS and on-premise software will be around for a long time and these vendors will add more data intelligence to their offerings.  They will be joined however, and possibly threatened by, a new generation of nimble and innovative next generation SaaS companies that will combine data and domain expertise to add massive business value to their customers.  These Data Driven Solutions represent the future of software.

*Venrock is an investor in these companies.