Learn to build AI agents

Define what constitutes an AI agent and distinguish agents from simple chatbots

Explain the core components of an agent: perception, reasoning, action, and memory

Define tools/functions and integrate them with an LLM using function calling

Build a simple agent that can call external APIs based on user queries

Implement error handling for common agent tool execution failures (e.g., API errors, malformed outputs)

Implement conversation buffer memory for multi-turn interactions

Implement summary memory to compress long conversation histories

Explain what text embeddings are and how to generate them using an API

Build a simple vector store and implement semantic search for document retrieval

Use retrieval-augmented generation (RAG) to ground agent responses in external documents

Design a prompt template to elicit a Thought and Action from an LLM based on the ReAct framework

Implement the core ReAct agent loop: parse the action, execute the tool, and feed the observation back

Build a multi-step agent that plans and executes a sequence of tool calls for a specific task

Apply a reflection pattern where an agent critiques and improves its own output

Implement a self-correction mechanism that retries failed actions with modified parameters

Evaluate agent performance using task completion metrics

Set up a LangChain environment and build a basic agent using a pre-built tool

Add conversational memory to a LangChain agent to enable multi-turn interactions

Implement a multi-agent system using a delegation pattern (supervisor routing to specialists)

Implement a multi-agent system where agents collaborate (e.g., writer and critic agents)

Describe supervisor and hierarchical agent architectures and their use cases

Define the requirements and architecture for an autonomous research assistant agent

Implement a web search tool for the research agent

Implement a content scraping tool to extract text from web pages

Combine search, scraping, and summarization capabilities into a single research workflow

Assemble and test the complete research assistant agent on a specific query

Define supervised, unsupervised, and reinforcement learning and identify appropriate use cases

Explain the concepts of training data, features, labels, and model parameters

Describe the bias-variance tradeoff and its implications for model performance

Implement a simple linear regression model using gradient descent in Python

Evaluate model performance using train-test splits and common metrics

Explain the structure of a neuron and how it computes a weighted sum with activation

Describe common activation functions (ReLU, sigmoid, tanh) and their properties

Implement the forward pass of a feedforward neural network from scratch in Python

Explain how backpropagation computes gradients through a network

Implement the backpropagation algorithm for a two-layer neural network from scratch

Combine the forward and backward passes to train a neural network on a simple dataset

Build and train a simple feedforward neural network for classification using PyTorch

Apply stochastic gradient descent and understand learning rate and batch size

Use optimization algorithms like Adam and explain their advantages over vanilla SGD

Monitor training with loss curves and diagnose underfitting vs overfitting

Implement regularization techniques (dropout, weight decay) to prevent overfitting

Save and load trained PyTorch models for reuse

Explain how text is tokenized and converted to embeddings

Explain the core concept and intuition behind the self-attention mechanism (queries, keys, values)

Describe how the components of the transformer architecture work together (multi-head attention, positional encoding, feed-forward layers)

Explain how decoder-only transformers generate text autoregressively

Describe the pretraining and fine-tuning paradigm for LLMs

Explain instruction tuning and reinforcement learning from human feedback (RLHF)

Define the agent-environment interaction loop and the RL objective

Formalize problems as Markov Decision Processes (states, actions, rewards, transitions)

Calculate returns and discounted cumulative rewards

Distinguish between policies, value functions, and Q-functions

Derive and apply the Bellman equation for value functions

Explain the exploration-exploitation tradeoff and epsilon-greedy strategies

Implement the tabular Q-learning algorithm to solve a simple gridworld environment

Use OpenAI Gym to set up and interact with standard RL environments

Explain how a neural network can be used to approximate a Q-function (function approximation)

Implement the core components of the Deep Q-Network (DQN) algorithm, including the experience replay buffer and target network

Train a DQN agent to solve a classic control task in an OpenAI Gym environment

Derive the policy gradient theorem and implement the REINFORCE algorithm

Analyze the high variance problem in the REINFORCE algorithm

Explain the Actor-Critic architecture and how it combines value and policy learning

Describe the key ideas behind Proximal Policy Optimization (PPO), such as the clipped surrogate objective

Compare the use cases for DQN vs. Policy Gradient methods like PPO