CS224n: Natural Language Processing with Deep Learning

Richard Socher, Christopher Manning, Stanford

Course Description

Natural language processing (NLP) is one of the most important technologies of the information age. Understanding complex language utterances is also a crucial part of artificial intelligence. Applications of NLP are everywhere because people communicate most everything in language: web search, advertisement, emails, customer service, language translation, radiology reports, etc. There are a large variety of underlying tasks and machine learning models behind NLP applications. Recently, deep learning approaches have obtained very high performance across many different NLP tasks. These models can often be trained with a single end-to-end model and do not require traditional, task-specific feature engineering. In this winter quarter course students will learn to implement, train, debug, visualize and invent their own neural network models. The course provides a thorough introduction to cutting-edge research in deep learning applied to NLP. On the model side we will cover word vector representations, window-based neural networks, recurrent neural networks, long-short-term-memory models, recursive neural networks, convolutional neural networks as well as some recent models involving a memory component. Through lectures and programming assignments students will learn the necessary engineering tricks for making neural networks work on practical problems.

This course is a merger of Stanford's previous cs224n course (Natural Language Processing) and cs224d (Deep Learning for Natural Language Processing).

Prerequisites

  • Proficiency in Python
    All class assignments will be in Python (using numpt and tensorflow). There is a tutorial here for those who aren't as familiar with Python. If you have a lot of programming experience but in a different language (e.g. C/C++/Matlab/Javascript) you will probably be fine.
  • College Calculus, Linear Algebra (e.g. MATH 51, CME 100)
    You should be comfortable taking derivatives and understanding matrix vector operations and notation.
  • Basic Probability and Statistics (e.g. CS 109 or other stats course)
    You should know basics of probabilities, gaussian distributions, mean, standard deviation, etc.
  • Foundations of Machine Learning
    We will be formulating cost functions, taking derivatives and performing optimization with gradient descent. Either cs221 or cs229 cover this background. Some optimization tricks will be more intuitive with some knowledge of convex optimization.

FAQ

Is this the first time this class is offered?
No, but this is a "new version" of the course merging in ideas from CS224D. It will cover the range of natural language processing from previous iterations of 224N but will primarily use the technique of neural networks / deep learning / differentiable programming to build solutions.
Can I follow along from the outside?
We'd be happy if you join us! We plan to make the course materials widely available: The assignments, course notes and slides will be available online. We may provide videos. We won't be able to give you course credit.
Can I take this course on credit/no cred basis?
Yes. Credit will be given to those who would have otherwise earned a C- or above.
Can I audit or sit in?
In general we are very open to sitting-in guests if you are a member of the Stanford community (registered student, staff, and/or faculty). Out of courtesy, we would appreciate that you first email us or talk to the instructor after the first class you attend.
Can I work in groups for the Final Project?
Yes, in groups of up to two people.
I have a question about the class. What is the best way to reach the course staff?
Stanford students please use an internal class forum on Piazza so that other students may benefit from your questions and our answers. If you have a personal matter, email us at the class mailing list cs224n-win1617-staff@lists.stanford.edu.
As an SCPD student, how do I make up for poster presentation component?
For the final poster presentation you can submit a video via youtube about your project.
As an SCPD student, how do I take the midterm?
For the midterm, we can use standard SCPD procedures of having your manager or somebody at your company monitor you during the exam.
Will there be virtual office hours for SCPD students
All office hours will be accesible on google hangouts. The link to the hangout is available on piazza

Schedule and Syllabus

Unless otherwise specified the course lectures and meeting times are:

Tuesday, Thursday 4:30-5:50
Location: NVIDIA Auditorium
EventDateDescriptionCourse Materials
Lecture Jan 10 Intro to NLP and Deep Learning Suggested Readings:
  1. [Linear Algebra Review]
  2. [Probability Review]
  3. [Convex Optimization Review]
  4. [More Optimization (SGD) Review]
[python tutorial]
[slides]
Lecture Jan 12 Simple Word Vector Representations Suggested Readings:
  1. [Word2Vec Tutorial - The Skip-Gram Model]
  2. [Distributed Representations of Words and Phrases and their Compositionality]
  3. [Efficient Estimation of Word Representations in Vector Space]
  4. [GloVe: Global Vectors for Word Representation]
A1 released Jan 12 Assignment #1 released
Lecture Jan 17 Neural Net Basics: Softmax Classifiers
Lecture Jan 19 More Neural Nets and Advanced Word Vector Representations
Lecture Jan 24 Project Advice, Neural Net Details and Practical Tips
Lecture Jan 26 Dependency Parsing
A1 Due Jan 26 Assignment #1 due
A2 Released Jan 26 Assignment #2 released
Lecture Jan 31 Introduction to TensorFlow
Lecture Feb 2 Recurrent Neural Networks and Language Modeling
Lecture Feb 7 RNNs/LSTMs/GRUs
Review Feb 9 Midterm Review
Project Proposal Due Feb 9 Frinal project proposal due
A2 Due Feb 9 Assignment #2 due
A3 Released Feb 9 Assignment #3 released
Midterm Feb 14 In-class midterm
Lecture Feb 16 Seq2Seq and Attention Mechanisms
Lecture Feb 21 Neural Machine Translation
Lecture Feb 23 Speech Processing
A3 Due Feb 23 Assignment #3 due
A4 Released Feb 23 Assignment #4 released Default final project
Lecture Feb 28 Convolutional and Recursive Neural Nets
Lecture Mar 2 Constituency Parsing
Lecture Mar 7 Coreference Resolution
Lecture Mar 9 Natural Language Understanding
Lecture Mar 14 Question Answering
Lecture Mar 16 Dynamic Memory Networks and Other New Architectures
Final Project Due Mar 17 Final course project / Assignment #4 due
Poster Presentation Mar 21 Final project poster presentations 12:15-3:15, location TBD
Dates:
  • 10 January 2017, 10 weeks
Course properties:
  • Free:
  • Paid:
  • Certificate:
  • MOOC:
  • Video:
  • Audio:
  • Email-course:
  • Language: English Gb

Reviews

No reviews yet. Want to be the first?

Register to leave a review

Show?id=n3eliycplgk&bids=695438
Included in selections:
Small-icon.hover Deep Learning
Good materials on deep learning.
NVIDIA
More from 'Computer Science':
Banner A Gentle Introduction to Python
Mechanical MOOC is two things: A massive open online course (MOOC), and an experiment...
M4dcoursesitebanner Mobiles for Development
Mobile devices, phones as well as Tablet computers, have been spreading fast...
Regular_c6b19e1d-17ac-4cec-aa57-fc818869b3fd Social Media in Healthcare: Opportunities and Challenges
Understand how social media is used in healthcare, to improve communication...
Regular_abf906a7-d0dd-4a3e-8bc4-0ea8861ec9f3 More Data Mining with Weka
Learn more about practical data mining, including how to deal with large data...
Regular_aa5a7351-0b70-438e-83f9-187896c71c0a Game Design and Development: Video Game Character Design
How and why do video gamers identify with the characters they control? Find...
More from 'Stanford':
Nlp-logo CS224d: Deep Learning for Natural Language Processing
Natural language processing (NLP) is one of the most important technologies...
Visionlablogo CS231n: Convolutional Neural Networks for Visual Recognition
Computer Vision has become ubiquitous in our society, with applications in search...
Databases DB: Introduction to Databases
Learn about Databases, one of the most prevalent technologies underlying internet...
Envphys_water EP101: Your Body in the World: Adapting to Your Next Big Adventure
Discover the amazing adaptability of the human body to environmental stressors...

© 2013-2019