No one likes to be kept in the dark. Every successful corporate strategy is built upon comprehensive data analysis. By capturing all the facts – and later analyzing and categorizing these points – executives can make the best informed decisions. Unfortunately, far too many business conclusions are drawn in the absence of critical facts. Executives are challenged with untapped, unknown or just “buried” data – putting effective decision-making at risk. For this reason, many CIOs are now taking a second look at their infrastructures – and shining a light on “Dark Data”.
According to Gartner Group, dark data is defined as: “information assets organizations collect, process, and store during regular business activities, but generally fail to use for other purposes. Similar to dark matter in physics, dark data often comprises most organization’s universe of information assets. Thus organizations often retain dark data for compliance purposes only.”
Dark data can be anywhere, or about anything - ranging from past digital marketing campaigns to discarded customer chat sessions. And while it holds the potential to create a holistic decision-making environment, this information is often just dragged along, going unnoticed. This is not only poor business practice, but can be very costly to store data delivering no ROI. In most cases, companies failing to exploit the entire universe of available information face three major pitfalls:
- Accelerated Storage Costs: Larger data sets mean more storage. Industry analysts predict the digital universe has doubled every two years – and should reach 44 zettabytes (44 trillion gigabytes) by 2020. Of this, upwards of 85 percent is being stored by companies rather than individuals. This translates to huge investments in storage – with IDC estimating annual worldwide shipments of storage systems will reach just under 90,000 PB next year. With an average cost of 26 cents per GB/month, businesses spend an estimated $3.12 million yearly to store one million gigabytes. It’s here the damage caused by dark data has its biggest impact, with 85 percent of all business data considered “cold” – or unused on a regular basis.
- Enhanced Security Risk: The business environment is challenged by a never-ending string of security breaches. As hackers become more sophisticated, threats to valuable data – such as financial and medical records – are now at their peak. Recent analysis by the Ponemon Institute estimates today’s average cost of just one security breach is $3.8 million – comprised of all financial outlays -- from initial breach detection and escalation through customer defections, reputation damage, and legal expenditures. And while incidents resulting from internal negligence have declined, those emanating from malicious attacks and botnets are far more severe. Dark data accelerates these costs by expanding the universe of addressable information demanding protection. While some Dark data holds value, failure to parse and exclude extraneous information means more money going out the door.
- Inhibited Compliance: All data storage falls under a banner of regulations concerning the security and availability of personal information. Increasingly stringent standards such as HIPAA and PCI DSS demand accelerated protection – meaning larger investments in compliance initiatives. With many analysts reporting the cost of regulatory security compliance at more than $3 million per year, CIOs tend to “over-retain” information even if it’s no longer relevant. As noted, this drives unnecessary costs for both information storage and management. Dark data can also accelerate legal and e-discovery expenses, as costs tend to rise alongside the volume of information to be searched and retrieved.
While the pitfalls associated with dark data are widespread, many of these issues can be minimized with the appropriate strategy and guidance. Specialized workshops offered by CenturyLink’s Data Science and Implementation Services team assist with identifying such data silos and assessing their impact on the business. Based on business objectives, data analytics strategy and relevant industry compliance requirements - CenturyLink’s data scientists prioritize the value of data and design governance strategies and architecture. This permits access and interpretation of insightful data, and secure cost-effective storage of data deemed non-vital or valuable.
While dark data holds significant potential to negatively impact organizations, data science experts help you understand and align business, technology, and data objectives.
