An engineer’s utopia is similar to the world of Wall-E; machines that do all the work, while humans sit back and relax. Unfortunately this future has not arrived yet. We yearn for the day when “Jarvis” from Iron Man is engaging in witty banter while booking our flights but instead we have Siri, which in it’s current form, is a novelty at best.
Yet, most of the startups that exist today love to automate processes and hate having humans as part of their solutions. After all, computers can do all of these mundane repetitive tasks with more efficiency and energy than a human. Whether it’s the first order of the day or the 341,553,410,016th of the day, a computer won’t tire. A human will. Humans do things differently, complain, and are too hard or expensive to debug. When engineers can create a system that automatically sells products like Amazon recommendations; who needs humans anymore?
This is the thinking that we took with our first iteration of our marketplace business, where we connect brands with bloggers: we wanted to remove the human element and give brands the ability to smart query our database and be matched to the right blogger every time. The blogger gets paid only when different binary checkpoints are passed and analytics are automatically reported. An engineer’s dream!
However, in reality we found that creating the perfect match completely automatically with algorithms was not the most elegant solution. While the matching algorithm could sift through thousands of bloggers and brands very quickly, sometimes the matches looked more than a little “off.”
For example we had a client that ran high-end resorts and the algorithm found matching bloggers that wrote about high-end travel. Unfortunately the algorithm would also throw in some backpacker blogs once in awhile – clearly a very different market! We scratched our heads for a bit wondering what could be wrong with our algorithm. Finally after reading the language in these blogs carefully we realized that often backpacking bloggers would actually be talking sarcastically about high-end resorts! Our poor little machine could not understand the subtleties of context and linguistic wordplay. This was not the algorithms fault of course – it did what it was supposed to – pick up on key words and phrases that we asked it to.
What it was missing was that secret sauce- the ability of humans to read between the lines, pick up contextual cues and insights from abstract concepts.
So how could we marry man and machine to create an efficient system? The answer was to use the power of algorithms meaningfully – i.e. by using them as a first pass to reduce the thousands of data points to a few hundred and then use human input to add the contextual piece of the puzzle – figuring out the best fit between a blogger and a brand.
Looking back, it’s almost obvious that our initial iteration would have taken much longer to build and test, we would have spent all our time trying to make the machine smarter algorithmically while ignoring the utility that a human step in the process could bring.
Don’t get me wrong - Siri,
The music industry space is another example where understanding human decision making patterns is critical to building recommendations and discovery solutions. Anshul Mathur, Director of Business Development at Senzari, a music analytics company, notices that humans “can often provide better recommendations than machines due to the ability to think contextually, creatively and adapt to complex changing inputs in real time. Humans however lack the computing power to process large amounts of data and the scale to reach larger audiences. As a technology company, we train our machines to think contextually like humans while leveraging big data science to crunch billions of information nodes."
The truth is that humans have certain abilities that machines don’t yet possess and are central to building meaningful systems. Recently
