Talend for Big Data

Access, transform, and integrate data using Talend’s open source, extensible tools

Overview

• Write complex processing job codes easily with the help of clear and step by step instructions
• Compare, filter, evaluate, and group vast quantities of data using Hadoop Pig
• Explore and perform HDFS and RDBMS integration with the Sqoop component

In Detail

Talend, a successful Open Source Data Integration Solution, accelerates the adoption of new big data technologies and efficiently integrates them into your existing IT infrastructure. It is able to do this because of its intuitive graphical language, its multiple connectors to the Hadoop ecosystem, and its array of tools for data integration, quality, management, and governance.

This is a concise, pragmatic book that will guide you through design and implement big data transfer easily and perform big data analytics jobs using Hadoop technologies like HDFS, HBase, Hive, Pig, and Sqoop. You will see and learn how to write complex processing job codes and how to leverage the power of Hadoop projects through the design of graphical Talend jobs using business modeler, meta-data repository, and a palette of configurable components.

Starting with understanding how to process a large amount of data using Talend big data components, you will then learn how to write job procedures in HDFS. You will then look at how to use Hadoop projects to process data and how to export the data to your favourite relational database system.

You will learn how to implement Hive ELT jobs, Pig aggregation and filtering jobs, and simple Sqoop jobs using the Talend big data component palette. You will also learn the basics of Twitter sentiment analysis the instructions to format data with Apache Hive.

Talend for Big Data will enable you to start working on big data projects immediately, from simple processing projects to complex projects using common big data patterns.

What you will learn from this book

• Know the structure of the Talend Unified Platform
• Work with Talend HDFS components
• Implement ELT processing jobs using Talend Hive components
• Load, filter, aggregate, and store data using Talend Pig components
• Integrate HDFS with RDBMS using Sqoop components
• Use the streaming pattern for big data
• Learn to reuse the partitioning pattern for big data

Approach

This book is written in a concise and easy-to-understand manner, and acts as a comprehensive guide on data analytics and integration with Talend big data processing jobs.

Who this book is written for

If you are a chief information officer, enterprise architect, data architect, data scientist, software developer, software engineer, or a data analyst who is familiar with data processing projects and who wants to use Talend to get your first big data job executed in a reliable, quick, and graphical way, then Talend for Big Data is perfect for you.

Table of Contents

Chapter 1: Getting Started with Talend Big Data
Chapter 2: Building Our First Big Data Job
Chapter 3: Formatting Data
Chapter 4: Processing Tweets with Apache Hive
Chapter 5: Aggregate Data with Apache Pig
Chapter 6: Back to the SQL Database
Chapter 7: Big Data Architecture and Integration patterns
Appendix: Installing Your Hadoop Cluster with Cloudera CDH VM