AI: Basic to Advanced

Define vector spaces and subspaces, and verify their axiomatic properties.

Determine linear independence/dependence of vectors and define the span of a set of vectors.

Define and construct a basis for a vector space and determine its dimension.

Perform matrix operations including addition, multiplication, and transposition.

Define linear transformations and represent them using matrices.

Solve systems of linear equations using matrix methods (e.g., Gaussian elimination).

Compute the rank and nullity of a matrix and apply the rank-nullity theorem.

Compute the determinant and inverse of a square matrix and describe their properties.

Define eigenvalues and eigenvectors and explain their geometric interpretation in relation to linear transformations.

Derive the characteristic polynomial of a matrix and use it to compute eigenvalues.

Compute the eigenvectors corresponding to each eigenvalue by solving the system (A - λI)x = 0.

Perform the diagonalization of a matrix and prove the conditions under which a matrix is diagonalizable.

Use eigendecomposition of a covariance matrix to derive the formulation of Principal Component Analysis (PCA).

Apply PCA to a data matrix to perform dimensionality reduction.

Define the Singular Value Decomposition (SVD) and the properties of its constituent matrices (U, Σ, V).

Use SVD to perform low-rank approximation of a data matrix.

Compute partial derivatives of multivariable functions.

Compute the gradient of a scalar-valued multivariable function.

Apply the chain rule for composite functions in multiple variables.

Compute the Jacobian matrix for a vector-valued function.

Define and compute the Hessian matrix for a scalar-valued multivariable function.

Use the gradient and Hessian to construct a second-order Taylor approximation of a function.

Identify critical points of a function and use the Hessian matrix to classify them as local minima, maxima, or saddle points.

Define convex sets and convex functions, and state their key properties in optimization.

Prove that for a convex function, any local minimum is also a global minimum.

Derive the gradient descent update rule for unconstrained optimization.

Implement the gradient descent algorithm to find the minimum of a simple convex function.

Define smoothness (Lipschitz continuity) and strong convexity of a function.

Derive the convergence rate of gradient descent for smooth and strongly convex functions.

Formulate an optimization problem with equality constraints.

Solve a simple equality-constrained optimization problem using Lagrange multipliers.

Formulate an optimization problem with inequality constraints.

Derive the Karush-Kuhn-Tucker (KKT) conditions as necessary conditions for optimality in problems with inequality constraints.

Define probability spaces, sample spaces, events, and the axioms of probability.

Apply probability rules including addition, multiplication, and conditional probability.

Derive and apply Bayes' theorem for updating beliefs.

Define random variables, distinguishing between discrete and continuous types.

Define and compute probability mass functions (PMFs) for discrete random variables.

Define and compute probability density functions (PDFs) for continuous random variables.

Define and compute cumulative distribution functions (CDFs) for both discrete and continuous random variables.

Compute the expected value and variance of a random variable from its definition.

Derive the mean and variance of the Bernoulli and Binomial distributions from their PMFs.

Derive the mean and variance of the univariate Gaussian distribution from its PDF.

Define joint, marginal, and conditional distributions for multiple random variables.

Compute marginal and conditional distributions from a given joint distribution.

Define covariance and correlation, and prove their key properties.

State the Law of Large Numbers (LLN) and the Central Limit Theorem (CLT) and explain their implications for statistical inference.

Define the concept of a statistical estimator and its properties (unbiasedness, consistency, efficiency).

Formulate the principle of Maximum Likelihood Estimation (MLE).

Derive the MLE for the parameters of a Bernoulli distribution.

Derive the MLE for the parameters (mean and variance) of a Gaussian distribution.

Formulate the principle of Maximum a Posteriori (MAP) estimation.

Derive the MAP estimate for a Bernoulli parameter with a Beta prior and compare it to the MLE result.

Formulate the statistical hypothesis testing framework, including null/alternative hypotheses and Type I/II errors.

Formulate the linear regression model, specifying its components and statistical assumptions.

Derive the cost function for linear regression from a least squares perspective.

Derive the Ordinary Least Squares (OLS) solution using multivariate calculus (setting the gradient to zero).

Derive the OLS solution using linear algebra (the Normal Equations).

Derive the probabilistic interpretation of linear regression using Maximum Likelihood Estimation with Gaussian errors.

Prove the unbiasedness of the OLS estimators under the classical linear model assumptions.

Derive the variance-covariance matrix of the OLS estimators.

State and explain the Gauss-Markov theorem regarding the optimality of OLS estimators.

Define residuals and derive the coefficient of determination (R-squared) as a measure of model fit.

Formulate L2 (Ridge) regularization for linear regression and derive its closed-form solution.

Explain the effect of L2 regularization on model parameters and the bias-variance trade-off.

Formulate the binary classification problem mathematically.

Derive the logistic (sigmoid) function and explain its properties for probability modeling.

Formulate the logistic regression model from a probabilistic (Bernoulli) perspective.

Derive the log-likelihood function (cross-entropy loss) for logistic regression.

Derive the gradient of the logistic regression objective function.

Apply gradient descent to optimize the parameters of a logistic regression model.

Define the decision boundary for binary logistic regression and explain how it is determined.