| Date | Topic | Description | Events | Deadlines |
|---|---|---|---|---|
| 9/27 | Introduction | What can we do with LLMs? aUnderstanding LLMs (their strengths, weaknesses, how to grow it); Architecture of an agent (external corpora, NLP primitives, agent initiatives); Taxonomy of knowledge-oriented tasks; State-of-the-art results. Course design and outline. | ||
| 10/2 | Grounding Agents on Free-Text | How to create a hallucination-free conversational bot grounded on free-text? Text retrieval; Summarization; Verifying generation; Response generation; Evaluation methodology; Fine-tuning small language models. Examples: BingChat, WikiChat | Homework 1 Out | Student intro form due |
| 10/4 | Grounding Agents on Small Databases | How to create a hallucination-free conversational bot grounded on structured/unstructured data? Semantic parsing; Databases; Expressiveness of database queries; Few-shot prompting on small schemas; Handling enumerated types; Comparison with human annotations. Example: Yelp. | ||
| 10/9 | Project ideas: Research | What are the ongoing research projects that students can participate in? Chatbot for StackExchange; Semantic parsers for complex queries; Multilingual task-oriented agents; Search of structured/unstructured data; A chat-based knowledge navigator for news, medical transcripts, ArXiv papers; Persuasive agents; Social coach for autism; Cross-culture chatbots; Causal inference; Drug discovery. | Homework 2 + Project Intent Out | Homework 1 due |
| 10/11 | Project Ideas: External | What are LLM-based project outside Stanford (non-profit organizations and startups) | Mentor-Written Project Interest due | |
| 10/16 | Grounding Agents on Structured Data and Free Text | How do we handle hybrid data sources, combining free-text and structured data? Proposed techniques: classifier, linearization, combining retrieved data with COT LLM. Structured and Unstructured Query Language; Automatic schema creation; Evaluation methodology. Example: Yelp. | Project Proposal Assignment out | Homework 2 + Custom Project Intent due |
| 10/18 | Project ideas from Students | Students are invited to pitch projects needing partners. | ||
| 10/23 | Project Proposals | Groups present their proposals | Project Proposal (custom and mentor-written) due | |
| 10/25 | Project Proposals | Groups present their proposals | ||
| 10/30 | Structured / Unstructured Query Language | How to answer questions combining structured and unstructured data? SUQL language design; Automatic schema creation; Evaluation methodology. | ||
| 11/1 | Dialogue State Tracking | How to interpret a dialogue? Dialogue trees; Speech act theory; Task-oriented agents; Dialogue state tracking problem; Data acquisition methods; Experimental Results. | ||
| 11/6 | Task-Oriented Dialogue Agents | How to create an agent that helps users with transactions? Dialogue state tracking for long dialogues; neural policies; Rule-based poslicies; Full training; Few-shot training. with full labeled training | ||
| 11/8 | Advanced Dialogue Agent Topics | Persuasive agents; handling mixed initiatives; multi-lingual dialogue agents. | ||
| 11/13 | Instruction-following models | In this lecture, we'll discuss instruction following models like text-davinici and chatGPT, and the training data that goes into making these systems. | ||
| 11/15 | NLP Building Blocks |
Slide deck 1: Semantic parsing over Wikidata Slide deck 2: Entity Linking Slide deck 3: Information retrieval indexing techniques |
||
| Thanksgiving Break | ||||
| 11/27 | Multi-Modal Applications | How to build a multi-modal app that supports complex commands? Motivation; Arbitrary composition of APIs in a program by voice; Combining graphical and voice outputs; Showing voice command results in native graphical outputs; ReactGenie framework. | ||
| 11/29 | Ethics and Policies | How do we ensure that LLMs have a positive impact on the society? Public policy; Societal impact of models; Transparency; Evaluation; Open vs. Closed Models. Slide deck 1: Value-Based LLMs Slide deck 2: AI Policy: Foundation Models |
||
| 12/4 | No class | |||
| 12/6 | Final project presentation | Groups present their final projects. [2.5 hour class] | Final Project Presentation + Poster | |
| Final Project Report + Code + (recommended) video demo Due 12/12 |