Natural Language Processing | AISV.801

---

This advanced course introduces students to many aspects of natural language processing (NLP), a subfield of Artificial Intelligence (AI) focused on human language. The course includes hands-on lab work with popular open source frameworks, such as Pandas, Numpy, Keras, Hugging Face Transformers, and Pytorch and covers a wide breadth of material, ranging from traditional methods, to more recent advancements in NLP, for example ChatGPT.

Students will explore natural language understanding (NLU), natural language generation (NLG), and discuss frameworks, algorithms and supervised learning.

The course will cover deep learning (DL), how DL and NLP can be combined, modern NLP architectures and language models in the BERT family. In addition, students will learn about the amazing GPT family of language models, for example GPT, GPT3, Instruct GPT, ChatGPT, and GPT4, as well as other recent advancements in generative Large Language Models (LLMs).

Students will leave the course with a wide-breadth of experience and understanding of the diverse applications of NLP in the modern world, along with the ability to program NLP methodologies in Python.

Learning Outcomes
At the conclusion of the course, you should be able to

• Create Python code to train a supervised learning algorithm for a variety of NLP tasks
• Evaluate the Transformer Architecture
• Explain recent innovations in Large Language Models
• Analyze how ChatGPT was trained
• Create Python code to fine-tune an open source generative Large Language Model

Skills Needed: Moderate level of computer programming ability in Python, comfortable with an editor, familiarity with basic command-line operations on a laptop, and a good understanding of Machine Learning models and Deep Learning models.

Note(s): Students are required to bring laptops for classroom work. The code samples use Python 3.6.8 and TensorFlow 2/Keras, along some Jupyter notebooks in Google Colaboratory (students can optionally pre-register for a free account).  Students also have the option of installing the Python 3+ version of Anaconda distribution on their laptops from the following link: https://www.anaconda.com/ on their machines.