Natural Language Processing (NLP) is one of the most important fields in Artificial Intelligence (AI). It has become very crucial in the information age because most of the information is in the form of unstructured text. NLP technologies are applied everywhere as people communicate mostly in language: language translation, web search, customer support, emails, forums, advertisement, radiology reports, to name a few.

There are a number of core NLP tasks and machine learning models behind NLP applications. Deep learning, a sub-field of machine learning, has recently brought a paradigm shift from traditional task-specific feature engineering to end-to-end systems and has obtained high performance across many different NLP tasks and downstream applications. Tech companies like Google, Baidu, Alibaba, Apple, Amazon, Facebook, Tencent, and Microsoft are now actively working on deep learning methods to improve their products. For example, Google recently replaced its traditional statistical machine translation and speech-recognition systems with systems based on deep learning methods.

Optional Textbooks

• Deep Learning by Goodfellow, Bengio, and Courville free online
• Machine Learning — A Probabilistic Perspective by Kevin Murphy online
• Natural Language Processing by Jacob Eisenstein free online
• Speech and Language Processing by Dan Jurafsky and James H. Martin (3rd ed. draft)

In this course, students will learn state-of-the-art deep learning methods for NLP. Through lectures and practical assignments, students will learn the necessary tricks for making their models work on practical problems. They will learn to implement, and possibly to invent their own deep learning models using available deep learning libraries like Pytorch.

Our Approach

• Thorough and Detailed: How to write from scratch, debug and train deep neural models

• State of the art: Most lecture materials are new from research world in the past 1-5 years.

• Practical: Focus on practical techniques for training the models, and on GPUs.

• Fun: Cover exciting new advancements in NLP (e.g., Transformer, BERT).

Weekly Workload

• Every two-hour lecture will be accompanied by practical problems implemented in PyTorch.
• There will be NO office hour.
• There will be 5% marks for class participation.

Assignments (individually graded)

• There will be three (3) assignments contributing to 3 * 15% = 45% of the total assessment.
• Late day policy
  • 2 free late days; afterwards,10% off per day late
  • Not accepted after 3 late days
• Students will be graded individually on the assignments. They will be allowed to discuss with each other on the homework assignments, but they are required to submit individual write-ups and coding exercises.

Final Project (Group work but individually graded)

• There will be a final project contributing to the remaining 50% of the total course-work assessment.
  • 1–3 people per group
  • Project proposal: 5%, update: 5%, presentation: 10%, report: 30%
• The project will be a group or individual work depending on the student’s preference. Students will be graded individually. The final project presentation will ensure the student’s understanding of the project

• Proficiency in Python (using numpy and PyTorch). There is a lecture for those who are not familiar with Python.
• College Calculus, Linear Algebra
• Basic Probability and Statistics
• Machine Learning basics