Streaming data unlocks new possibilities in the IoT era

In the pre-digital era, IT departments mastered a variety of technological approaches to extract value from data. Data warehouses, analytical platforms, and different types of databases filled data centres, accessing storage devices where records were safely preserved on disk for their historical value.

By contrast, says Kelly Herrell, CEO of Hazelcast, data today is being generated and streamed by Internet of Things (IoT) devices at an unprecedented rate. The "Things" in IoT are innumerable - sensors, mobile apps, connected vehicles, etc. - which by itself is explosive. Add to that the "network effect" where the degree of value is directly correlated to the number of attached users, and it's not hard to see why firms like IDC project the IoT market will reach US$745 billion (€665 billion) next year and surpass the $1 trillion (€0.89 trillion) mark in 2022.

This megatrend is disrupting the data processing paradigm. The historical value of stored data is being superseded by the temporal value of streaming data. In the streaming data paradigm, value is a direct function of immediacy, for two reasons:

Difference: Just as the unique water molecules passing through a length of hose are different at every point in time, so is the unique data streaming through the network for each window of time.
Perishability: The opportunity to act on insights found within streaming data often dissipates shortly after the data is generated.
The concepts of difference and perishability apply to this streaming data paradigm. Sudden changes detected in data streams demand immediate action, whether it's a pattern hit on real-time facial recognition or drilling rig vibration sensors suddenly registering abnormalities that could be disastrous if preventive steps aren't taken immediately.

This news was previously published on: www.iot-now.com/2019/08/12/98122-streaming-data-unlocks-new-possibilities-iot-era