As companies begin building the next generation of applications in the cloud, managing these applications and services is becoming increasingly challenging. For example, the retail industry is no longer just a collection of brick and mortar stores that sell goods and services during business hours. Instead, business is a 24x7x365 exchange that occurs globally and at scale. Transactions now extend beyond the purchase of goods and retailers must now analyze their customers’ social conversations across multiple channels, understand market and social trends and respond in real time, all the while, ensuring their systems are up and functioning consistently.
Web site performance has a major impact on revenue and brand recognition. Remember when Twitter was synonymous with the Fail Whale? Since fixing that problem the Twitter brand name is cast in a much more positive light. For retailers the impact of poor web performance is even more tangible. A delay of a few seconds can result in millions of dollars in lost business.
The significant impact that web performance has on success has led to larger investments in operations. There has been a surge in purchases of logging and monitoring tools, visualization tools, big data analytics, and many other technologies, in the hopes of becoming more aware and responsive to customers’ real time needs. As enterprises add more tools to their tool box to monitor and collect data for the numerous components of these highly distributed architectures, they find themselves building knowledge silos within their operations.
To put this in perspective, let’s say that a retailer is experiencing a period of poor performance during a high volume transaction period. Where do they go to investigate the problem? What is the cause? They may search their logs in Splunk, get alerts in an APM tool like New Relic, check the infrastructure metrics in Nagios, look at their favorite database monitoring tools, check their social networking streams, check the health of their cloud provider’s status page, and so on. In fact, different people within the organization may be performing all of these tasks at the same time with little to no collaboration.
What they need is a central command center that aggregates, correlates and presents data from each source, and can then be tailored for different end user groups. For example, a product owner has a different set of KPIs that they are interested in than a systems administrator or a help desk person. We need a system that ingests all of these different data sources and presents them in a way that business and IT people can react quickly to please their customers and protect their brand.
Some web companies have built some very innovative homegrown solutions, but the average enterprise does not have the desire or expertise to produce their own technology. With slim margins and fierce competition, their energy is best directed toward projects that focus on revenue, innovation, and modernization.
So what options do we have? There are many monitoring solutions that try to bring data from various sources to one platform. Many of these solutions are successfully presenting different views of data by consuming data via APIs. What they are not doing is ingesting and correlating all of this data so the customers understand how the interaction of all of these disparate data points impact the overall customer experience.
Today, SOASTA has announced a new product called DOCTM (Digital Operations Center) which is an attempt to solve the exact issues I described above. Better known as a leader in performance analytics and testing, SOASTA has leveraged their high performance big data analytics engine to ingest, aggregate, and correlate large volumes of disparate data in attempt to give customers a single view that is customizable by role. They have also deployed some amazing visualization technology that looks like a scene out of Minority Report. Check out fellow Forbes contributor Ben Kepes' article that goes into detail about this new product.
It will be interesting to see how this technology plays out. Solutions like this are in high demand as we enter the new digital era. Our systems are becoming much more horizontal, loosely coupled and complex. This allows our systems to scale accordingly and keep up with our customer demands, but it comes at the cost of manageability.
