Neural Network Methods in Natural Language Processing

Neural networks are a family of powerful machine learning models. This book focuses on the application of neural network models to natural language data. The first half of the book (Parts I and II) covers the basics of supervised machine learning and feed-forward neural networks, the basics of working with machine learning over language data, and the use of vector-based rather than symbolic representations for words. It also covers the computation-graph abstraction, which allows to easily define and train arbitrary neural networks, and is the basis behind the design of contemporary neural network software libraries.

The second part of the book (Parts III and IV) introduces more specialized neural network architectures, including 1D convolutional neural networks, recurrent neural networks, conditioned-generation models, and attention-based models. These architectures and techniques are the driving force behind state-of-the-art algorithms for machine translation, syntactic parsing, and many other applications. Finally, we also discuss tree-shaped networks, structured prediction, and the prospects of multi-task learning.

Table of Contents

Chapter 1 Introduction

PART I Supervised Classification and Feed-forward Neural Networks
Chapter 2 Learning Basics And Linear Models
Chapter 3 From Linear Models To Multi-Layer Perceptrons
Chapter 4 Feed-Forward Neural Networks
Chapter 5 Neural Network Training

PART II Working with Natural Language Data
Chapter 6 Features For Textual Data
Chapter 7 Case Studies Of Nlp Features
Chapter 8 From Textual Features To Inputs
Chapter 9 Language Modeling
Chapter 10 Pre-Trained Word Representations
Chapter 11 Using Word Embeddings
Chapter 12 Case Study: A Feed-Forward Architecture For Sentence Meaning Inference

PART III Specialized Architectures
Chapter 13 Ngram Detectors: Convolutional Neural Networks
Chapter 14 Recurrent Neural Networks: Modeling Sequences And Stacks
Chapter 15 Concrete Recurrent Neural Network Architectures
Chapter 16 Modeling With Recurrent Networks
Chapter 17 Conditioned Generation

PART IV Additional Topics
Chapter 18 Modeling Trees With Recursive Neural Networks
Chapter 19 Structured Output Prediction
Chapter 20 Cascaded, Multi-Task And Semi-Supervised Learning
Chapter 21 Conclusion