Introduction to Data Science (DS 215)

Manipulate N-dimensional arrays for mathematical operations using NumPy.

Perform data loading, cleaning, and transformation on tabular data using Pandas DataFrames.

Create static visualizations like histograms, scatter plots, and line charts using Matplotlib.

Outline the unified Scikit-learn API: `fit`, `predict`, and `transform` methods.

Project: Load, inspect, clean, and visualize a real-world dataset (e.g., Titanic) to prepare it for modeling.

Review and implement key probability distributions (e.g., Gaussian, Poisson, Binomial) using SciPy.

Derive the PDF of a transformed random variable using the Jacobian method.

Prove Markov's and Chebyshev's inequalities from first principles and explain their application.

Derive the covariance and correlation of linear combinations of random variables.

Construct intuitive geometric explanations for joint, marginal, and conditional probability distributions.

Derive the Weak Law of Large Numbers (WLLN) using Chebyshev's inequality.

Outline the proof of the Central Limit Theorem (CLT) using characteristic functions.

Prove that a linear combination of independent Gaussian random variables remains Gaussian.

Project: Develop a Python simulation to visually demonstrate the Central Limit Theorem and the Law of Large Numbers.

Distinguish between population parameters and sample statistics.

Define and mathematically formulate the properties of an estimator: bias, variance, and consistency.

Explain the purpose of key estimation methods: Method of Moments, Maximum Likelihood (MLE), and Bayesian Estimation.

Derive the Method of Moments estimators for the parameters of a Gamma distribution.

Explain the concept of a sufficient statistic and its role in estimation via the Rao-Blackwell theorem.

Derive the Maximum Likelihood Estimators (MLE) for parameters of Normal, Exponential, and Poisson distributions.

Derive the Normal Equations for Ordinary Least Squares (OLS) using matrix calculus.

Explain the geometric interpretation of Least Squares as an orthogonal projection onto a subspace.

Implement a numerical Maximum Likelihood Estimator for a custom probability distribution in Python.

Analyze the invariance property of Maximum Likelihood Estimators.

State the conditions for and derive the Cramer-Rao Lower Bound (CRLB).

Explain the significance of the CRLB as a benchmark for an estimator's variance.

Prove the Gauss-Markov theorem for the optimality of Ordinary Least Squares (OLS) estimators.

Derive the Best Linear Unbiased Estimator (BLUE) for a general linear model.

Compare the properties of MLE and OLS estimators for the linear model with Gaussian noise.

Derive the posterior distribution for a model with a Gaussian likelihood and Gaussian prior (conjugate priors).

Derive the Minimum Mean Square Error (MMSE) estimator as the mean of the posterior distribution.

Derive the Maximum A Posteriori (MAP) estimator as the mode of the posterior distribution.

Explain the relationship between MAP, MLE, and L2 regularization.

Project: Implement a Bayesian estimator to sequentially update beliefs about a coin's bias as new data arrives.

Formulate null and alternative hypotheses for a given research question.

Derive the Likelihood Ratio Test (LRT) statistic for simple hypotheses.

Explain the mathematical trade-off between Type-I and Type-II errors and the concept of statistical power.

Derive the one-sample t-test statistic from its definition.

Implement Linear Regression from scratch using NumPy, including the gradient descent optimization algorithm.

Mathematically decompose the generalization error into its Bias, Variance, and Irreducible Noise components.

Explain how L1 (Lasso) and L2 (Ridge) regularization affect model coefficients differently.

Project: Build and evaluate a regularized linear model to predict a continuous variable, analyzing the effect of the regularization parameter.

Derive the logistic (sigmoid) function and cost function for Logistic Regression.

Derive the update steps for the K-means algorithm and prove its convergence to a local minimum.

Explain the geometric intuition of K-means as a form of Voronoi tessellation.

Derive the Expectation (E) and Maximization (M) steps for a Gaussian Mixture Model (GMM).

Compare and contrast the use cases for K-means vs. GMM for clustering tasks.

Derive Principal Component Analysis (PCA) from first principles by maximizing variance using Lagrange Multipliers.

Explain the geometric interpretation of PCA as a rotation of the coordinate system to align with axes of maximal variance.

Relate the PCA solution to the eigendecomposition of the covariance matrix.

Implement PCA from scratch using NumPy's SVD function.

Project: Apply PCA to a high-dimensional dataset (e.g., images of faces) and visualize the principal components.

Derive the weight update rule for the Perceptron algorithm.

Explain the limitations of the Perceptron and how Multi-Layer Perceptrons (MLPs) overcome them.

Derive the backpropagation algorithm for a simple MLP using the chain rule of calculus.

Explain the Universal Approximation Theorem and its implications for neural networks.

Project: Build, train, and evaluate a simple neural network for digit classification using Scikit-learn.