CSCI 5640 Natural Language Processing — Fall 2021



Wednesday, 4:30-6:15pm, Y.C. Liang Hall 101 Thursday, 4:30-5:15pm, Hui Yeung Shing Bldg G01
Wednesday, 1:30-2:15pm, William M W Mong Eng Bldg 407
Prof. Liwei WANG
Teaching Assistant
Yanyang LI Mark ZHAO
All updates will be sent to your emails. Please check your university emails.
Assignments + Paper Presentation + Final Project

Tutorial sessions would be the fastest and clearest way to discuss any problem. You are also welcomed to send emails to the instructor and the TAs.


Natural language processing (NLP) is a crucial part of artificial intelligence (AI), which aims to endow computers with the ability to process and understand human language. This course gives an overview of modern deep learning techniques for natural language processing. The course starts with basic linguistic concepts in NLP and moves from shallow bag-of-words representations to richer structural embeddings, which is the foundation for the successful use of deep learning in NLP. Then the course will guide you through three fundamental tasks of NLP: language modeling (LM), natural language understanding (NLU), and natural language generation (NLG), followed by some recent advances. Along the way we will introduce cutting-edge computational models together with insights from a linguistic perspective.


All related lecture materials will be posted on Blackboard → Lecture Notes.

The following schedule (tentative) are subject to minor modifications.

Week Date Topic Other
1 Sep 8, 9 Intro & Basic Knowledge about NLP
2 Sep 15, 16 Neural Networks
3 Sep 23 Word and Word Vectors: part 1 The day following the Chinese Mid-Autumn Festival
Release of Assignment 1
4 Sep 29, 30 Word and Word Vectors: part 2;
Language Modeling
Submission of Assignment 1
Complete team forming for projects
Assign selected paper to each team
5 Oct 6, 7 Transformers & Pretraining
6 Oct 13 More About Transformers & Pretrained Models
(Team Presentation on Selected Papers)
Double Ninth Festival
7 Oct 20, 21 Question Answering Release of Assignment 2
Release of Final Project
8 Oct 27, 28 Machine Translation Submission of Assignment 2
9 Nov 3, 4 NLG Tasks Release of Assignment 3
10 Nov 10, 11 Language + Vision Submission of Assignment 3
11 Nov 17, 18 Integrating Knowledge into Large Models
12 Nov 24, 25 Frontiers Topics in NLP
13 Dec 1, 2 Final Projects Submission of Final Project


There will be three assignments and one paper presentation in total.

Final Project

More information about Final Project will be released later.

To maximize the project achievement, students should form teams of four in general. We recommend you using the same team as in paper presentation.

You are encouraged include in your report any illustrative, concrete examples that help you understand the papers you read.

Please submit your project report (in PDF) together with related codes and material online as a single zip file at blackboard.

Course Policy