Companies have been embarking on big data initiatives for many years. Often, these were IT driven solutions looking for problems to solve. IT professionals understood that if they could bring many disparate data sources into one place and make that data actionable, the data would become a gold mine of valuable information. And that could lead to new revenue sources, huge cost reductions, process optimizations, and many other terrific opportunities. The challenges have always been, how to fund these big data initiatives, and how to make business executives aware of big data’s capabilities, so they can take advantage of the opportunities.
I have seen companies spend large sums of money and invest huge amounts of human capital building and operating big data solutions with very little ROI to show for it. Over the last year or two, IoT has become a frequent board room topic. Business executives have suddenly embraced IoT and are building strategies that promise to transform their companies by leveraging the power of real-time information and machine learning. IoT initiatives are appearing with robust business plans, complete with funding. IoT has become the killer app for big data.
To understand why IoT and big data go hand in hand, we must understand the underlying technologies. . There are four major categories of technology used to implement IoT. At the lowest level is called fog or edge computing. The “fog” is comprised of many types of connected devices and sensors that communicate data in real time. These devices are enabled by a variety of communication technologies and protocols.
In the fog, devices and objects with sensors can determine when a status or condition of an object changes and then make intelligent decisions based on that information. For example, a company managing a fleet of delivery trucks can detect when truck parts are performing suboptimally and schedule these trucks for preventative maintenance long before the vehicle breaks down. These types of real-time decisions greatly reduce maintenance costs and improve the overall delivery performance of the fleet. I like to refer to the data in the fog as small data.
This small data is then transmitted to a company's datacenter, often run and managed as a virtual datacenter on an Infrastructure as a Service (IaaS) cloud vendor, such as Amazon (AWS) or
Implementing all these technologies by yourself can be a very complex and time consuming endeavor. That’s why companies like Hortonworks and Cloudera have packaged all the components together, so you can focus on building applications instead of spending months working on integrating all the underlying technologies. The public cloud vendors (AWS, Google,
Once the data is stored and accessible, developers and data scientists can build applications and mine the data. This is where the real business value of IoT lies. These insights and applications enable a business to enter new markets, optimize the production of their products, streamline their processes, and personalize their consumers’ experience. Real-time data is what drives IoT.
Business value of IoT
We are seeing IoT use cases in every industry.
The power of IoT comes when many different data sources are collected from many different devices and sensors and combined in fresh ways that provide new insights and services in real time. The Smart City use case is a great example. Cities are now harvesting weather, traffic, surveillance video data, and energy consumption data to provide a new suite of services. These help them to better manage traffic routing and congestion reduction, improve service levels for emergency response and disaster response planning, and optimize the use of city power, water, and other utilities.
Manufacturing plants are reducing maintenance costs and increasing production throughput, with real-time access of data streaming from connected machines. Farmers are able to maximize their yield while reducing the usage and cost of water and fertilizers thanks to real-time information coming from smart tractors, beacons in the field monitoring weather and soil conditions, and drones flying over fields and analyzing crop health.
The use cases across industries are endless. Hospitals are providing better services at lower costs. Transportation and shipping companies are reducing maintenance costs and improving delivery service. Energy companies are producing more energy at lower cost and drastically reducing research and development costs for exploring new mining and drilling sources, and similar activities.
All these use cases are being driven by big data technologies. It is the business people, not just the IT people, who finally want big data now. We are seeing a surge in job openings for IT professionals with big data expertise, along with a huge increase in demand for data scientists and Ph.D.s. IoT is the problem that big data solutions were born to solve. We just did not know that a few years ago. Now we do.
