Apache Kafka

Set up Apache Kafka clusters and develop custom message producers and consumers using practical, hands-on examples

Overview

• Write custom producers and consumers with message partition techniques
• Integrate Kafka with Apache Hadoop and Storm for use cases such as processing streaming data
• Provide an overview of Kafka tools and other contributions that work with Kafka in areas such as logging, packaging, and so on

In Detail

Message publishing is a mechanism of connecting heterogeneous applications together with messages that are routed between them, for example by using a message broker like Apache Kafka. Such solutions deal with real-time volumes of information and route it to multiple consumers without letting information producers know who the final consumers are.

Apache Kafka is a practical, hands-on guide providing you with a series of step-by-step practical implementations, which will help you take advantage of the real power behind Kafka, and give you a strong grounding for using it in your publisher-subscriber based architectures.

Apache Kafka takes you through a number of clear, practical implementations that will help you to take advantage of the power of Apache Kafka, quickly and painlessly. You will learn everything you need to know for setting up Kafka clusters. This book explains how Kafka basic blocks like producers, brokers, and consumers actually work and fit together. You will then explore additional settings and configuration changes to achieve ever more complex goals. Finally you will learn how Kafka works with other tools like Hadoop, Storm, and so on.

You will learn everything you need to know to work with Apache Kafka in the right format, as well as how to leverage its power of handling hundreds of megabytes of messages per second from multiple clients.

What you will learn from this book

• Download and build Kafka
• Set up single as well as multi-node Kafka clusters and send messages
• Learn Kafka design internals and message compression
• Understand how replication works in Kafka
• Write Kafka message producers and consumers using the Kafka producer API
• Get an overview of consumer configurations
• Integrate Kafka with Apache Hadoop and Storm
• Use Kafka administration tools

Approach

The book will follow a step-by-step tutorial approach which will show the readers how to use Apache Kafka for messaging from scratch.

Who this book is written for

Apache Kafka is for readers with software development experience, but no prior exposure to Apache Kafka or similar technologies is assumed. This book is also for enterprise application developers and big data enthusiasts who have worked with other publisher-subscriber based systems and now want to explore Apache Kafka as a futuristic scalable solution.

Table of Contents

Chapter 1: Introducing Kafka
Chapter 2: Installing Kafka
Chapter 3: Setting up the Kafka Cluster
Chapter 4: Kafka Design
Chapter 5: Writing Producers
Chapter 6: Writing Consumers
Chapter 7: Kafka Integrations
Chapter 8: Kafka Tools