Hadoop: The Definitive Guide, 4th Edition

Get ready to unlock the power of your data. With the fourth edition of this comprehensive guide, you’ll learn how to build and maintain reliable, scalable, distributed systems with Apache Hadoop. This book is ideal for programmers looking to analyze datasets of any size, and for administrators who want to set up and run Hadoop clusters.

Using Hadoop 2 exclusively, author Tom White presents new chapters on YARN and several Hadoop-related projects such as Parquet, Flume, Crunch, and Spark. You’ll learn about recent changes to Hadoop, and explore new case studies on Hadoop’s role in healthcare systems and genomics data processing.

• Learn fundamental components such as MapReduce, HDFS, and YARN
• Explore MapReduce in depth, including steps for developing applications with it
• Set up and maintain a Hadoop cluster running HDFS and MapReduce on YARN
• Learn two data formats: Avro for data serialization and Parquet for nested data
• Use data ingestion tools such as Flume (for streaming data) and Sqoop (for bulk data transfer)
• Understand how high-level data processing tools like Pig, Hive, Crunch, and Spark work with Hadoop
• Learn the HBase distributed database and the ZooKeeper distributed configuration service

Table of Contents

Part I. Hadoop Fundamentals
Chapter 1. Meet Hadoop
Chapter 2. MapReduce
Chapter 3. The Hadoop Distributed Filesystem
Chapter 4. YARN
Chapter 5. Hadoop I/O

Part II. MapReduce
Chapter 6. Developing a MapReduce Application
Chapter 7. How MapReduce Works
Chapter 8. MapReduce Types and Formats
Chapter 9. MapReduce Features

Part III. Hadoop Operations
Chapter 10. Setting Up a Hadoop Cluster
Chapter 11. Administering Hadoop

Part IV. Related Projects
Chapter 12. Avro
Chapter 13. Parquet
Chapter 14. Flume
Chapter 15. Sqoop
Chapter 16. Pig
Chapter 17. Hive
Chapter 18. Crunch
Chapter 19. Spark
Chapter 20. HBase
Chapter 21. ZooKeeper

Part V. Case Studies
Chapter 22. Composable Data at Cerner
Chapter 23. Biological Data Science: Saving Lives with Software
Chapter 24. Cascading

Appendix A. Installing Apache Hadoop
Appendix B. Cloudera’s Distribution Including Apache Hadoop
Appendix C. Preparing the NCDC Weather Data
Appendix D. The Old and New Java MapReduce APIs