CS 4501 Machine Learning for Natural Language Processing
Table of Contents
1. Course Information
- Instructor: Yangfeng Ji
- Semester: Fall 2021
- Location: Rice 340
- Time: Monday and Wednesday 3:30 PM - 4:45 PM
- TA: Wanyu Du
- Office Hours:
- Yangfeng Ji, Tuesday 1:30 PM - 2:30 PM, Location: Rice 510/Zoom
- Wanyu Du, Thursday 1:30 PM - 2:30 PM, Location: Zoom (temporarily)
- The lectures will also be streamed and recorded on Zoom. Students can find the Zoom link on Collab.
- Schedule
- Campuswire for online discussion. By the time of our first class, students registered for this course should all receive an invitation from Campuswire. Please let the instructor know if you haven’t gotten one.
- Homework submission template for homework assignments
About final project
2. Course Description
Natural language processing (NLP) seeks to provide computers with the ability to process and understand human language intelligently. Examples of NLP techniques include (i) automatically translating from one natural language to another, (ii) analyzing documents to answer related questions or make related predictions, and (iii) generating texts to help story writing or build conversational agents. This course, consisting of one fundamental part and one advanced part, will give an overview of modern NLP techniques.
2.1 Topics
This course will mainly focus on applying machine learning (particularly, deep learning) techniques to natural language processing. NLP topics covered by this course
- Text classification
- Language modeling
- Word embeddings
- Machine translation and sequence-to-sequence models
- Some advanced topics: large-scale pre-trained language modelsing (e.g. BERT), natural language generation, interpretability in NLP
2.2 Prerequisites
- Proficiency in Python
This course requires some programming in both homeworks and the final project. The preference of programming language for this course is Python (with some additional packages like Scipy, Sklearn, and PyTorch). - Calculus and Linear Algebra
Multivariable derivatives, matrix/vector notations and operations; singular value decomposition, etc. - Probability and Statistics
Mean and variance, multinomial distribution, conditional dependence, maximum likelihood estimation, Bayes theorem, etc. - Foundations of Machine Learning
Logistic regression, cross validation, optimization with gradient descent, bias and variance decomposition, etc.
2.3 Textbooks
- [JE] Eisenstein, Natural Language Processing, 2018
Supplemental materials
- Jurafsky and Martin, Speech and Language Processing, 3rd Edition, 2019
- Shalev-Shwartz and Ben-David, Understanding Machine Learning: From Theory to Algorithms, 2014
- Goodfellow, Bengio and Courville, Deep Learning, 2016
3. Assignments and Final Project
- Homework (80%):
- There will be five homeworks and each of them is worth 16%.
- Students are allowed to discuss homework with their classmates. If you discuss with your classmates, please disclose their names in your submission.
- Directly copying answers from others is definitely considered as plagiarism.
- Project (20%):
There is only one course project and the credit breaks down to four parts- Project proposal: 6%
- Final project presentation: 7%
- Final project report: 7%
- Students should team up for this project, each group should have 2 - 3 students.
- In both homework and the final project, other than using the machine learning libraries including Sklearn, PyTorch, Tensorflow, students need to implemented the rest of the proposed model by themselves. Copying code from any resources (e.g., Github, Bitbucket, and Gitlab) is prohibited and will be considered as plagiarism.
4. Additional Information
Last updated on August 24, 2021