Expert AI Roadmap

Build a custom PyTorch training loop from scratch, including separate training/validation steps, forward/backward pass, and optimizer step.

Implement and compare different optimization algorithms (e.g., SGD with momentum, Adam) within the custom training loop.

Incorporate gradient clipping into the custom training loop to mitigate exploding gradients during training.

Integrate automatic mixed precision (AMP) into the custom training loop to accelerate training speed on compatible hardware.

Implement training-run logging using TensorBoard to track metrics, visualize model graphs, and monitor hardware utilization.

Analyze the seminal paper 'Stochastic Gradient Descent Tricks' (Bottou et al.) to justify advanced optimization choices.

Explain the theoretical limitations of simple RNNs, specifically the vanishing and exploding gradient problems.

Derive the backpropagation through time (BPTT) equations for a basic Recurrent Neural Network.

Implement a custom Long Short-Term Memory (LSTM) cell from scratch, without using PyTorch's built-in RNN layers.

Explain the necessity of positional information in non-recurrent sequence models and contrast sinusoidal vs. learned positional encoding.

Implement both sinusoidal and learned positional encoding modules using PyTorch tensor operations.

Derive the mathematical formulation of Scaled Dot-Product Attention, explaining the role of query, key, and value matrices.

Extract and explain the single most important technical contribution of the paper 'Attention Is All You Need' (Vaswani et al.).

Implement the Scaled Dot-Product Attention function in PyTorch, including causal masking for decoder-style self-attention.

Build a Multi-Head Attention layer from scratch by integrating the Scaled Dot-Product Attention function and learnable Q, K, V projections.

Implement the Position-wise Feed-Forward Network and the Add & Norm (Layer Normalization + Residual Connection) components.

Assemble a complete Transformer Encoder Layer by combining the Multi-Head Attention and Feed-Forward components.

Assemble a complete Transformer Decoder Layer, including masked self-attention and encoder-decoder cross-attention mechanisms.

Construct a full Encoder-Decoder Transformer model by stacking the implemented layers and adding the final linear output layer.

Train the complete Transformer model on a sequence-to-sequence task, leveraging a previously built custom training loop.

Demonstrate the memory footprint differences when loading a PyTorch tensor in FP32, FP16, BF16, and INT8 formats.

Explain the theoretical differences and trade-offs between Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT).

Implement absolute maximum (absmax) and zero-point quantization algorithms manually on a sample tensor.

Convert a pre-trained PyTorch linear layer to 8-bit precision and measure the memory reduction and performance impact.

Explain the core mechanism of 8-bit matrix multiplication as introduced in the 'LLM.int8()' paper (Dettmers et al.).

Analyze the core methodology and contributions of the QLoRA research paper (Dettmers et al.).

Explain the concept of Intrinsic Dimensionality and its connection to the effectiveness of Low-Rank Adaptation (LoRA).

Derive the mathematical weight update rule for a LoRA layer, showing how it modifies a frozen pre-trained weight matrix.

Explain the core innovations of QLoRA, including the 4-bit NormalFloat (NF4) data type and Double Quantization.

Implement a standalone PyTorch module for a LoRA layer that performs the low-rank update (h = h + x @ W_A @ W_B).

Write a function to inject custom LoRA layers into a target nn.Linear module within a Transformer, ensuring only new matrices are trainable.

Build a script to modify a pre-trained model by injecting custom LoRA layers into its attention mechanism.

Implement a fine-tuning loop that updates only the LoRA adapter weights, demonstrating memory savings.

Analyze the core methodology and contributions of the LoRA research paper (Hu et al.).

Explain the core concepts of vector databases and the trade-offs of Approximate Nearest Neighbor (ANN) search.

Implement a chunking strategy that utilizes semantic boundary detection rather than fixed character counts.

Explain the theoretical difference and use cases for bi-encoders (retrieval) versus cross-encoders (reranking).

Implement a reranking function that re-scores retrieved documents using a pre-trained cross-encoder model.

Integrate a sparse retrieval method (e.g., BM25) and a dense retrieval method into a unified hybrid search function.

Construct an end-to-end RAG pipeline connecting hybrid search, reranking, and context augmentation steps.

Extract and explain the core dense retrieval methodology from the 'Dense Passage Retrieval' paper (Karpukhin et al.).

Design a JSON schema for a tool's arguments that enables reliable function calling by an LLM.

Explain LLM function calling as a constrained generation task and its underlying mechanism.

Implement a state management loop that handles conversation history and adheres to context window limits.

Build a custom tool executor that parses LLM outputs and reliably executes local Python functions based on the parsed action.

Implement robust error handling for malformed tool calls using try-except blocks and schema validation.

Implement a feedback loop that reports execution errors back to the LLM to enable self-correction.

Analyze the core innovation of how LLMs learn to use APIs as presented in the Toolformer paper (Schick et al.).

Explain the theoretical paradigms of Chain-of-Thought (CoT) and ReAct (Reason+Act), contrasting their approaches to complex problem-solving.

Design and implement the prompt template for a ReAct agent, including placeholders for Thought, Action, and Observation.

Implement the core control loop for a ReAct agent that calls an LLM, parses the action, executes it, and formats the observation for the next step.

Compare the architectural principles of graph-based orchestration (e.g., LangGraph) versus role-based orchestration (e.g., CrewAI).

Define the data structures for a graph-based workflow: a state object, nodes representing agent actions, and edges representing control flow.

Using LangGraph, define a multi-agent graph with nodes for distinct agent roles (e.g., researcher, writer).

Implement a conditional edge in the LangGraph structure to create a critique-revision loop between a 'writer' and 'reviewer' agent.

Analyze the core methodology of the 'ReAct' paper (Yao et al.).

Explain the core differences between a Python script and a structured REST API for serving model predictions.

Implement a basic REST API endpoint using FastAPI that accepts JSON input and returns a structured JSON response.

Explain the concept of containerization and the purpose of each major instruction in a Dockerfile (FROM, WORKDIR, COPY, RUN, CMD).

Write a Dockerfile to package the FastAPI application, ensuring dependencies are managed and the application runs in an isolated environment.

Explain the theoretical limitations of traditional NLP metrics (BLEU, ROUGE) for evaluating generative tasks.

Explain the 'LLM-as-a-Judge' concept as a solution to the limitations of traditional metrics, based on the paper by Zheng et al.

Design a detailed scoring rubric and prompt template for an LLM-as-a-judge to evaluate a specific generative task.

Implement a Python function to execute an LLM-as-a-judge evaluation, sending a model's output to a judge LLM and parsing its score.

Explain the role of the Key-Value (KV) cache in optimizing autoregressive inference in Transformers.

Explain continuous batching as a throughput optimization strategy for LLM serving, and contrast it with static batching.

Describe the PagedAttention algorithm and analyze its impact on inference throughput and memory fragmentation, based on the vLLM paper.

Set up a basic continuous integration (CI) pipeline using GitHub Actions that triggers on code changes to run automated tests.

Integrate a previously fine-tuned LoRA model into a FastAPI application to create a queryable inference endpoint.

Containerize the model API using a multi-stage Dockerfile optimized for a GPU-enabled production environment.

Analyze the high-level principles of the paper 'MLOps: Continuous delivery and automation pipelines in machine learning' (Google Research).

Deploy a containerized model API to a cloud provider's container service (e.g., AWS ECR/ECS, GCP Artifact Registry/Cloud Run).

Configure the deployed service to run on a GPU instance with a public-facing HTTPS endpoint.

Integrate the LLM-as-a-judge pipeline into the deployed API for real-time quality monitoring and logging of responses.

Implement a structured logging system to capture request/response payloads, latency, and judge scores.

Set up a basic monitoring dashboard (e.g., using Grafana, Datadog) to visualize key service metrics like latency and request rate.

Load test the deployed API endpoint using a tool like `locust` to measure throughput (tokens/second) and identify performance bottlenecks.

Create a final project report summarizing the system architecture, deployment process, performance metrics, and evaluation results.