Hadoop Application Architectures

Get expert guidance on architecting end-to-end data management solutions with Apache Hadoop. While many sources explain how to use various components in the Hadoop ecosystem, this practical book takes you through architectural considerations necessary to tie those components together into a complete tailored application, based on your particular use case. To reinforce those lessons, the book’s second section provides detailed examples of architectures used in some of the most commonly found Hadoop applications. Whether you’re designing a new Hadoop application, or planning to integrate Hadoop into your existing data infrastructure, Hadoop Application Architectures will skillfully guide you through the process. This book covers:

• Factors to consider when using Hadoop to store and model data
• Best practices for moving data in and out of the system
• Data processing frameworks, including MapReduce, Spark, and Hive
• Common Hadoop processing patterns, such as removing duplicate records and using windowing analytics
• Giraph, GraphX, and other tools for large graph processing on Hadoop
• Using workflow orchestration and scheduling tools such as Apache Oozie
• Near-real-time stream processing with Apache Storm, Apache Spark Streaming, and Apache Flume
• Architecture examples for clickstream analysis, fraud detection, and data warehousing

Table of Contents

Part I. Architectural Considerations For Hadoop Applications
Chapter 1. Data Modeling In Hadoop
Chapter 2. Data Movement
Chapter 3. Processing Data In Hadoop
Chapter 4. Common Hadoop Processing Patterns
Chapter 5. Graph Processing On Hadoop
Chapter 6. Orchestration
Chapter 7. Near-Real-Time Processing With Hadoop

Part II. Case Studies
Chapter 8. Clickstream Analysis
Chapter 9. Fraud Detection
Chapter 10. Data Warehouse

Appendix A. Joins In Impala