AI Engineering and Deep Learning Course

Load a tabular dataset using Pandas and perform initial exploration to understand its structure and summary statistics.

Apply common data preprocessing techniques, including handling missing values and scaling numerical features, using Pandas and Scikit-learn.

Train and evaluate a linear regression model using Scikit-learn.

Train and evaluate a logistic regression classifier using Scikit-learn.

Train and evaluate a decision tree and random forest model using Scikit-learn.

Perform k-means clustering on an unlabeled dataset using Scikit-learn.

Apply cross-validation and grid search to tune model hyperparameters.

Build a Scikit-learn pipeline that combines a preprocessing step and a model estimator.

Explain the structure of a single neuron and compute its output given weights, bias, and an activation function.

Describe and compare common activation functions (sigmoid, tanh, ReLU) and their properties.

Implement a forward pass for a multi-layer perceptron using only NumPy.

Derive the gradient of the loss function with respect to the weights and biases of the output layer.

Apply the chain rule to derive the gradient of the loss function with respect to the weights and biases of a hidden layer.

Implement the weight update step using the computed gradients and a learning rate.

Create and manipulate tensors in PyTorch, including reshaping and basic operations.

Use PyTorch's autograd to compute gradients automatically.

Define a neural network using nn.Module and nn.Linear layers.

Select and instantiate an appropriate loss function and optimizer for a defined model.

Implement the sequence for a single training step: zero gradients, forward pass, compute loss, backward pass, and weight update.

Move tensors and models to a GPU device in PyTorch to accelerate training.

Use PyTorch Dataset and DataLoader to wrap, batch, and shuffle training data.

Train and evaluate a multi-class classifier on a benchmark dataset using PyTorch.

Explain and compare SGD, momentum, RMSProp, and Adam optimizers.

Explain the purpose of weight initialization strategies such as Xavier and He initialization.

Apply learning rate scheduling strategies during training.

Identify overfitting by analyzing training and validation loss curves.

Apply batch normalization to stabilize and accelerate training.

Implement an early stopping mechanism in a training loop based on validation performance.

Explain the convolution operation and the role of filters, stride, and padding in a CNN.

Describe the function of pooling layers and their effect on spatial dimensions.

Define a CNN architecture for image classification using PyTorch nn.Conv2d and nn.MaxPool2d layers.

Train a CNN on a standard image dataset and monitor its loss and accuracy.

Evaluate a CNN using an accuracy score, confusion matrix, and per-class metrics.

Apply data augmentation techniques to improve generalization on image tasks.

Explain the architecture of a vanilla Recurrent Neural Network (RNN) and the vanishing gradient problem.

Describe how LSTM and GRU cells address long-range dependencies in sequences.

Preprocess text data for sequence models using tokenization.

Implement an LSTM-based model in PyTorch for sequence classification.

Train a sentiment analysis model using an LSTM network in PyTorch.

Explain the limitations of RNNs, such as sequential computation, that motivated the development of Transformers.

Explain the self-attention mechanism and how scaled dot-product attention is calculated.

Describe multi-head attention and its advantage over single-head attention.

Explain the role of positional encoding and why it is needed in the Transformer architecture.

Describe the architecture of a single Transformer encoder block, including multi-head attention and feed-forward layers.

Describe the architecture of a single Transformer decoder block, including masked self-attention and cross-attention.

Explain how the encoder and decoder stacks work together in the full Transformer model.

Differentiate between encoder-only (BERT), decoder-only (GPT), and encoder-decoder (T5) model architectures.

Describe the differences between word, character, and subword tokenization methods (e.g., BPE).

Compute scaled dot-product attention step-by-step in PyTorch on a small example.

Load a pre-trained model and its corresponding tokenizer from the HuggingFace Transformers library.

Perform text classification using a pre-trained model with HuggingFace pipelines.

Explain the concept of transfer learning in NLP and when fine-tuning is appropriate.

Fine-tune a pre-trained Transformer model on a custom text dataset using the HuggingFace Trainer API.

Explain the architecture and purpose of a standard autoencoder for dimensionality reduction and reconstruction.

Explain the architecture and objective of a Variational Autoencoder (VAE).

Explain the generator-discriminator framework of a Generative Adversarial Network (GAN) and its training objective.

Implement the GAN training loop with separate updates for the generator and discriminator.

Train a GAN, generate synthetic images, and compare GANs vs. VAEs in terms of output quality and training stability.

Explain the forward (noising) and reverse (denoising) processes in diffusion models.

Describe the training objective of a denoising diffusion probabilistic model (DDPM).

Compare diffusion models with GANs in terms of image quality, diversity, and computational cost.

Explain how autoregressive language models (GPT-style) generate text token by token.

Apply prompt engineering techniques such as few-shot prompting, chain-of-thought, and system prompts.

Explain the Retrieval-Augmented Generation (RAG) architecture and its components.

Explain the difference between fine-tuning and in-context learning for adapting LLMs.

Explain the concept of parameter-efficient fine-tuning (PEFT) and the LoRA method.

Apply parameter-efficient fine-tuning (LoRA) to adapt an LLM to a specific task using HuggingFace.

Explain the concept of a REST API and build a basic endpoint using FastAPI.

Serve a trained model as a REST API using FastAPI.

Explain the purpose of containerization and write a basic Dockerfile for a Python application.

Build a Docker image and run the containerized ML application.

Explain the purpose of model monitoring and common metrics to track in production.