Cloud computing is now increasingly used to manage petascale of data streamed from the IoT devices. Management of such a huge data is really a big challenge. Google Cloud, and Amazon AWS are providing great solutions for this. Let’s take a look at few concepts.

There are two concepts when it comes to the type of devices. One being the constrained devices and the other being standard devices. Google Cloud allows both to be managed through pub/sub model. Constrained devices are normally your home appliances are industrial sensors. Such devices like bulbs, fridge, potentiometer etc come under its belt. The other IoT devices are standard devices and these include your mobile, laptop, or smartwatch.

The cloud platforms first point of contact with these devices is the ingestion phase. In this phase, they get all the data that is coming from the devices and categorize them. E.g. logging, monitoring, pub/sub etc. Then there is a controller which makes use of this ingested pile of data. The whole process takes into the concept of streaming.

These pipelines which get merged into a single pipeline distribute the data to different engines e.g. real-time analytics and storage. Once done, they end up being used by user-interfacing applications e.g. apps running on compute clouds etc.

All these concepts can be merged into these ideas like data aggregation, transformation, enrichment, and mobility. So if your cloud provides all these things then it supports the whole ecosystem of real-time IoT data management.

In the case of Google, it uses Google Cloud Dataflow as the controller to create pipelines of ingested data, merging them, and making them available immediately for storage and analytics.

In Amazon and IBM Watson, the picture is somewhat the same. Amazon provides device SDK as well, which can use the AWS endpoints to create further service enrichment. AWS also provides the IoT API, which we can use to create applications on top of the IoT ecosystem.

All in all, the IoT real-time processing and/or streaming is getting exciting day by day. It is high time for developers to learn how to be part of such a diverse ecosystem, and make applications which are truly pervasive, at least in a good way.