Kafka: The Definitive Guide: Real-time data and stream processing at scale

Every enterprise application creates data, whether it’s log messages, metrics, user activity, outgoing messages, or something else. And how to move all of this data becomes nearly as important as the data itself. If you’re an application architect, developer, or production engineer new to Apache Kafka, this practical guide shows you how to use this open source streaming platform to handle real-time data feeds.

Engineers from Confluent and LinkedIn who are responsible for developing Kafka explain how to deploy production Kafka clusters, write reliable event-driven microservices, and build scalable stream-processing applications with this platform. Through detailed examples, you’ll learn Kafka’s design principles, reliability guarantees, key APIs, and architecture details, including the replication protocol, the controller, and the storage layer.

• Understand publish-subscribe messaging and how it fits in the big data ecosystem.
• Explore Kafka producers and consumers for writing and reading messages
• Understand Kafka patterns and use-case requirements to ensure reliable data delivery
• Get best practices for building data pipelines and applications with Kafka
• Manage Kafka in production, and learn to perform monitoring, tuning, and maintenance tasks
• Learn the most critical metrics among Kafka’s operational measurements
• Explore how Kafka’s stream delivery capabilities make it a perfect source for stream processing systems

Table of Contents

Chapter 1 Meet Kafka
Chapter 2 Installing Kafka
Chapter 3 Kafka Producers – Writing Messages to Kafka
Chapter 4 Kafka Consumers – Reading Data from Kafka
Chapter 5 Kafka Internals
Chapter 6 Reliable Data Delivery
Chapter 7 Building Data Pipelines
Chapter 8 Cross-Cluster Data Mirroring
Chapter 9 Administering Kafka
Chapter 10 Monitoring Kafka
Chapter 11 Stream Processing
Appendix A Installing Kafka on Other Operating Systems
Appendix B Additional Kafka Tools