Big Data Made Easy: A Working Guide to the Complete Hadoop Toolset

Many corporations are finding that the size of their data sets are outgrowing the capability of their systems to store and process them. The data is becoming too big to manage and use with traditional tools. The solution: implementing a big data system.

As Big Data Made Easy: A Working Guide to the Complete Hadoop Toolset shows, Apache Hadoop offers a scalable, fault-tolerant system for storing and processing data in parallel. It has a very rich toolset that allows for storage (Hadoop), configuration (YARN and ZooKeeper), collection (Nutch and Solr), processing (Storm, Pig, and Map Reduce), scheduling (Oozie), moving (Sqoop and Avro), monitoring (Chukwa, Ambari, and Hue), testing (Big Top), and analysis (Hive).

The problem is that the Internet offers IT pros wading into big data many versions of the truth and some outright falsehoods born of ignorance. What is needed is a book just like this one: a wide-ranging but easily understood set of instructions to explain where to get Hadoop tools, what they can do, how to install them, how to configure them, how to integrate them, and how to use them successfully. And you need an expert who has worked in this area for a decade—someone just like author and big data expert Mike Frampton.

Big Data Made Easy approaches the problem of managing massive data sets from a systems perspective, and it explains the roles for each project (like architect and tester, for example) and shows how the Hadoop toolset can be used at each system stage. It explains, in an easily understood manner and through numerous examples, how to use each tool. The book also explains the sliding scale of tools available depending upon data size and when and how to use them. Big Data Made Easy shows developers and architects, as well as testers and project managers, how to:

• Store big data
• Configure big data
• Process big data
• Schedule processes
• Move data among SQL and NoSQL systems
• Monitor data
• Perform big data analytics
• Report on big data processes and projects
• Test big data systems

Big Data Made Easy also explains the best part, which is that this toolset is free. Anyone can download it and—with the help of this book—start to use it within a day. With the skills this book will teach you under your belt, you will add value to your company or client immediately, not to mention your career.

What you’ll learn

• How to install and employ Hadoop
• How to install and use Hadoop-related tools like Hive, Storm, Pig, Solr, Oozie, Ambari, and many others
• How to set up and test a big data system
• How to scale the system for the amount of data at hand and the data you expect to accumulate
• How those who have spent their careers in the SQL database world can apply their skills to building big data systems

Who this book is for

This book is for developers, architects, IT project managers, database administrators, and others charged with developing or supporting a big data system. It is also for a general IT audience, anyone interested in Hadoop or big data, and those experiencing problems with data size. It’s also for anyone who would like to further their career in this area by adding big data skills.

Table of Contents

Chapter 1. The Problem with Data
Chapter 2. Storing and Configuring Data with Hadoop, Yarn, and ZooKeeper
Chapter 3. Collecting Data with Nutch and Solr
Chapter 4. Processing Data Map Reduce
Chapter 5. Scheduling Using Oozie
Chapter 6. Moving Data with Sqoop and Avro
Chapter 7. Monitoring the System with Chukwa, Ambari, and Hue
Chapter 8. Analyzing and Querying Data with Hive and MongoDB
Chapter 9. Reporting with Hadoop and Other Software
Chapter 10. Testing with Big Top
Chapter 11. Hadoop Present and Future