**Table of Contents**

- Instructor: Yangfeng Ji
- Semester: Fall 2021
- Location: TBA
- Time: Monday and Wednesday 3:30 PM - 4:45 PM
- TA: TBA
- Office Hours: TBA

Natural language processing (NLP) seeks to provide computers with the ability to process and understand human language intelligently. Examples of NLP techniques include (i) automatically translating from one natural language to another, (ii) analyzing documents to answer related questions or make related predictions, and (iii) generating texts to help story writing or build conversational agents. This course, consisting of one fundamental part and one advanced part, will give an overview of modern NLP techniques.

This course will mainly focus on applying machine learning (particularly, deep learning) techniques to natural language processing. NLP topics covered by this course

- Text classification
- Language modeling
- Word embeddings
- Machine translation and sequence-to-sequence models
- Some advanced topics: large-scale pre-trained language modelsing (e.g. BERT), generative models, natural language generation, interpretability in NLP

**Proficiency in Python**

This course requires some programming in both homeworks and the final project. The preference of programming language for this course is Python (with some additional packages like Scipy, Sklearn, and PyTorch).**Calculus and Linear Algebra**

Multivariable derivatives, matrix/vector notations and operations; singular value decomposition, etc.**Probability and Statistics**

Mean and variance, multinomial distribution, conditional dependence, maximum likelihood estimation, Bayes theorem, etc.**Foundations of Machine Learning**

Logistic regression, cross validation, optimization with gradient descent, bias and variance decomposition, etc.

- [JE] Eisenstein, Natural Language Processing, 2018

**Supplemental materials**

- Jurafsky and Martin, Speech and Language Processing, 3rd Edition, 2019
- Shalev-Shwartz and Ben-David, Understanding Machine Learning: From Theory to Algorithms, 2014
- Goodfellow, Bengio and Courville, Deep Learning, 2016

**Last updated on May 19, 2021**