Hadoop: The Definitive Guide, 3rd Edition

Ready to unlock the power of your data? With this comprehensive guide, you’ll learn how to build and maintain reliable, scalable, distributed systems with Apache Hadoop. This book is ideal for programmers looking to analyze datasets of any size, and for administrators who want to set up and run Hadoop clusters.

You’ll find illuminating case studies that demonstrate how Hadoop is used to solve specific problems. This third edition covers recent changes to Hadoop, including material on the new MapReduce API, as well as MapReduce 2 and its more flexible execution model (YARN).

• Store large datasets with the Hadoop Distributed File System (HDFS)
• Run distributed computations with MapReduce
• Use Hadoop’s data and I/O building blocks for compression, data integrity, serialization (including Avro), and persistence
• Discover common pitfalls and advanced features for writing real-world MapReduce programs
• Design, build, and administer a dedicated Hadoop cluster—or run Hadoop in the cloud
• Load data from relational databases into HDFS, using Sqoop
• Perform large-scale data processing with the Pig query language
• Analyze datasets with Hive, Hadoop’s data warehousing system
• Take advantage of HBase for structured and semi-structured data, and ZooKeeper for building distributed systems

Table of Contents

Chapter 1. Meet Hadoop
Chapter 2. MapReduce
Chapter 3. The Hadoop Distributed Filesystem
Chapter 4. Hadoop I/O
Chapter 5. Developing a MapReduce Application
Chapter 6. How MapReduce Works
Chapter 7. MapReduce Types and Formats
Chapter 8. MapReduce Features
Chapter 9. Setting Up a Hadoop Cluster
Chapter 10. Administering Hadoop
Chapter 11. Pig
Chapter 12. Hive
Chapter 13. HBase
Chapter 14. ZooKeeper
Chapter 15. Sqoop
Chapter 16. Case Studies
Appendix A. Installing Apache Hadoop
Appendix B. Cloudera’s Distribution Including Apache Hadoop
Appendix C. Preparing the NCDC Weather Data