IoT Messaging at scale with MQTT and Kafka

Kafka vs. MQTT for IoT:

Kafka is a very famous streaming platform. It scales well because you can cluster the brokers and it has intelligent but relatively THICK clients. These intelligent clients make it good for server to server communication and it keeps the brokers quite lightweight. However, heavy clients are not well suited for IoT where you have tiny devices with very little CPU and memory resources. For these type of environments, MQTT is an often used light weight protocol.

However, MQTT is weak when it comes to scaling it horizontally (you’d need load balances from both sides, publishers and subscribers and http which is too heavy and not reliable (subscribers must always be on)).  In this video, Tim Kellog describes a method where MQTT environments have been made salable with Kafka! Quite interesting approach to combine the strength of MQTT (lightweight on the client side) with Kafka (very scale-able streaming plattform).