CS 5642 Advanced Natural Language Processing (Spring 2020)

[Ted Pedersen]

This spring I'll be offering CS 5642, Advanced Natural Language Processing (MWF 12-12:50). This is the first time the course has been offered so I wanted to tell you a little bit about it. 

Natural Language Processing is what computers are doing when they spell check, translate, correct, summarize, or auto complete our writing. It's what computers do when we carry out a Google Search, and it's running in the background as we use social media, checking to make sure we aren't using hate speech and to make sure we see ads that will appeal to us. NLP is that big space where computers and human language meet, and one tries to do something with the other. 

What makes CS 5642 Advanced? In reality I think a better name for the class would be "Recent Developments in NLP". It will be focused on what's been happening in NLP in the last five years or so, which is a lot. 

Machine Learning, Artificial Intelligence, and in particular Deep Learning have swept through NLP in the last few years and fundamentally changed things in big ways. Google, Facebook, Microsoft, etc. have invested heavily in Deep Learning and NLP, to the point where these ideas are used in their day to day services.

Google Search : 

https://www.blog.google/products/search/search-language-understanding-bert  

Google Translate :

https://ai.googleblog.com/2016/09/a-neural-network-for-machine.html  

Facebook Content Moderation  :

https://www.wired.com/story/ai-has-started-cleaning-facebook-can-it-finish/  

Text Generation : 

https://openai.com/blog/better-language-models/  

The goal of the course is to make you able to use some of these very new and very powerful ideas. We'll focus on figuring out how to use Deep Learning to do some of what you see above, which includes generating realistic coherent text, translating from one language to another, and figuring out the meaning of words and sentences. 

We'll do our programming in a framework called PyTorch 

https://pytorch.org/ - this is a Facebook created tool for Deep Learning that is very popular in NLP and Deep Learning in general. PyTorch is a high level tool that is designed to be easy to use, so you don't need to have previous experience with Python to be successful. We'll have a few programming assignments and you'll also do your own project where you can focus on a problem that you find especially interesting. 

This course is intended for undergraduate or graduate students who have reached the point where they are able to take upper division CS electives (4xxx classes). It will be helpful if you have taken NLP, Machine Learning, or AI, but since this is the first time the course is being offered I'm open to a variety of backgrounds.

In any case, if this sounds interesting I hope you'll consider taking CS 5642, and I'd be happy to provide you with more information either in person or via email if that will help. 

Thanks!

Ted

---
Ted Pedersen
http://www.d.umn.edu/~tpederse