Production AI/ML and Agentic Apps

Manipulate Python lists, dictionaries, sets, tuples, comprehensions, and iterators.

Use type hints, classes, data classes, modules, and packages to structure maintainable Python code.

Read and write JSON and CSV data while handling malformed input safely.

Write unit tests with pytest and use logging and an IDE debugger to locate a defect.

Use async and await for a small I/O-bound task and explain how asynchronous execution differs from CPU parallelism.

Build a small typed command-line application that reads, validates, transforms, and saves data.

Represent scalars, vectors, and matrices with NumPy arrays and interpret their shapes.

Calculate and interpret mean, median, variance, percentiles, and distributions for a dataset.

Explain probability, conditional probability, and uncertainty using everyday prediction examples.

Compute dot products and cosine similarity and relate them to semantic search.

Explain slopes, derivatives, gradients, and gradient descent using visual and code-based intuition.

Clean, filter, aggregate, and visualize a small dataset with pandas and plotting tools.

Choose suitable mathematical or statistical operations for simple AI engineering scenarios.

Produce a short exploratory data analysis that identifies data-quality issues and supports one conclusion.

Frame a business problem as a supervised, unsupervised, or non-ML task with measurable success criteria.

Split data into training, validation, and test sets without introducing data leakage.

Build a preprocessing and baseline-model pipeline with scikit-learn.

Fit a linear regression model and interpret its predictions and errors.

Fit a classification model and interpret probabilities and decision thresholds.

Evaluate a classifier using a confusion matrix, precision, recall, and F1 score.

Apply k-means clustering to an unlabeled dataset and explain when the resulting clusters are or are not useful.

Build and evaluate a reproducible classifier, identifying any leakage, overfitting, underfitting, or metric risk.

Explain how a neuron, activation function, loss function, and gradient contribute to learning.

Train a small neural network and interpret its training and validation loss curves.

Tokenize text and explain how tokenization affects context length, latency, and cost.

Create and compare text embeddings using cosine similarity.

Explain self-attention through a small example of tokens influencing one another.

Trace data through the main components of a transformer during inference.

Distinguish pretraining, instruction tuning, fine-tuning, and inference in scenario-based knowledge checks.

Build a semantic text-matching prototype using pretrained embeddings.

Call an LLM API from Python while managing configuration and secrets safely.

Design prompts with explicit instructions, context, examples, and output constraints.

Generate validated structured output with a schema and recover from invalid responses.

Manage conversation history within a token budget without relying on unlimited context.

Implement timeouts, retries, rate-limit handling, and model fallbacks for LLM calls.

Create a small test set to compare two prompt or model configurations.

Diagnose prompting, context, reliability, and cost problems in scenario-based knowledge checks.

Build a document-processing application that returns structured results with validation and error handling.

Explain when retrieval-augmented generation is preferable to prompting or fine-tuning.

Extract and chunk documents while preserving metadata needed for retrieval and citations.

Embed document chunks and store them in a vector index.

Retrieve relevant chunks using semantic search and metadata filters.

Construct a grounded answer prompt that includes retrieved evidence and citation requirements.

Measure retrieval quality separately from answer quality using a small evaluation set.

Diagnose chunking, retrieval, grounding, and citation failures in scenario-based knowledge checks.

Build a cited question-answering assistant over a small private document collection.

Decide whether a use case needs a simple LLM call, deterministic workflow, RAG pipeline, or agent.

Define a tool contract with typed inputs, validated outputs, and actionable errors.

Execute LLM-selected tool calls while validating permissions and arguments.

Model an agent workflow as explicit states, transitions, and termination conditions.

Add checkpointing, retry limits, idempotency, and failure recovery to a workflow.

Insert human approval before consequential or irreversible tool actions.

Diagnose looping, unsafe tool use, state loss, and nondeterminism in scenario-based knowledge checks.

Build a stateful assistant that selects tools and pauses for approval before a sensitive action.

Implement a bounded reason-and-act loop with a clear completion condition.

Decompose a multi-step request into a plan and update the plan after tool results.

Distinguish conversational, episodic, semantic, and application-state memory.

Store and retrieve durable agent memory with relevance, retention, and privacy controls.

Connect an agent to an external capability using either Model Context Protocol or an OpenAPI-described tool.

Choose between one agent and multiple specialized agents using reliability, latency, cost, and complexity trade-offs.

Evaluate an agent trajectory for task completion, unnecessary steps, tool errors, and policy violations.

Build a research agent that plans, uses multiple tools, records evidence, and produces a cited result.

Expose an AI workflow through a typed FastAPI endpoint with input and output validation.

Separate API, domain, model-provider, tool, and persistence concerns in an AI service.

Use asynchronous execution and bounded concurrency for slow model and tool calls.

Persist agent sessions and execution history in a relational database using a Python ORM.

Enforce API authentication and per-user authorization for sessions and sensitive tools.

Write unit and integration tests using mocked model and tool responses.

Package an AI service as a non-root Docker container with health checks and externalized configuration.

Build a Python client that calls a containerized AI service and handles authentication, timeouts, and API errors.

Create a failure taxonomy covering retrieval, generation, tool use, safety, latency, and cost.

Build a versioned evaluation dataset with expected behavior and grading criteria.

Evaluate groundedness, relevance, task success, tool selection, latency, and cost.

Trace an AI request across retrieval, model, and tool calls using structured logs and identifiers.

Implement input guardrails that detect and mitigate direct and indirect prompt-injection attempts.

Validate LLM outputs and tool arguments to prevent data exfiltration and unsafe execution.

Redact sensitive data from prompts and logs and protect model API secrets throughout the request pipeline.

Run a red-team evaluation and convert the discovered failures into automated release checks.

Design a cloud architecture for an AI service using compute, storage, database, queue, and secret-management components.

Create a continuous-integration pipeline that runs formatting, tests, security checks, and image builds.

Deploy a containerized AI service to one managed cloud runtime and verify its health endpoint.

Use a queue and background worker for long-running agent tasks.

Version prompts, evaluation data, application code, and model configuration as one release.

Define service-level indicators and alerts for quality, availability, latency, and cost.

Implement explicit model-routing rules and token or cost budgets for requests of different complexity.

Perform a monitored rollout and rollback after detecting a simulated production regression.

Select a job-relevant agentic AI problem and define users, boundaries, risks, and measurable acceptance criteria.

Create an architecture diagram and focused threat model for the capstone's RAG and tool-using workflow.

Prepare representative source data and a versioned evaluation set for the capstone.

Implement one end-to-end vertical slice from API request through retrieval or tool use to validated response.

Add state persistence and a human-approval checkpoint to the capstone workflow.

Implement automated tests and evaluation gates for the capstone application.

Deploy the capstone through the previously created delivery pipeline and verify it with health and smoke tests.

Present a portfolio-ready system walkthrough and answer system-design, ML, LLM, agent, and production-incident interview questions.