AI Architect

Classify a proposed workload as conventional software, predictive machine learning, generative AI, or a hybrid solution.

Distinguish model training, fine-tuning, and inference in terms of data, compute, and architectural responsibility.

Trace how an input prompt is transformed into an LLM response at a conceptual level.

Explain how tokenization and context-window limits affect application behavior.

Distinguish text embeddings from generated text and identify the architectural purpose of each.

Diagnose hallucination, nondeterminism, stale knowledge, and context loss in example LLM outputs.

Estimate token consumption and inference cost for a simple conversational workload.

Translate a generative-AI business idea into users, tasks, inputs, outputs, and measurable value.

Elicit AI-specific nonfunctional requirements for quality, latency, throughput, privacy, availability, and cost.

Identify use cases where generative AI should not be used because a deterministic solution is safer or simpler.

Map common solution capabilities to Azure OpenAI, Azure AI Foundry, Azure AI Search, Document Intelligence, Content Safety, and supporting Azure services.

Choose between prompting, retrieval-augmented generation, fine-tuning, and conventional code for a stated requirement.

Select an Azure-hosted model using quality, context length, latency, data residency, quota, and cost constraints.

Explain how Azure model deployments, regions, quotas, and content filtering influence an architecture.

Create a concise Azure AI service-selection decision matrix for a sample use case.

Configure a development environment for Azure-hosted LLM access without placing secrets in source code.

Send a basic model request from C# and inspect the response, usage metadata, and failure information.

Send a basic model request from Python and inspect the response, usage metadata, and failure information.

Design system and user instructions that clearly separate rules, context, and requested output.

Use model parameters appropriately to control response variability and output length.

Implement schema-constrained structured output and validate the returned data.

Implement streamed response handling in an LLM-backed API.

Handle timeouts, rate limits, transient errors, and unsafe responses in an LLM client.

Explain the roles of chunking, embeddings, indexing, retrieval, reranking, and generation in a RAG request.

Compare vector, keyword, hybrid, and semantic search for a stated retrieval problem.

Choose chunk size, overlap, and document-boundary rules for a sample knowledge base.

Design an ingestion flow that extracts, cleans, chunks, enriches, and indexes source documents.

Configure an Azure AI Search index with text, vector, metadata, and security-filter fields.

Implement a minimal retrieval pipeline and inspect the passages returned for a query.

Construct a grounded generation prompt that produces answers with verifiable citations.

Apply metadata and identity-based filters so retrieval respects user access.

Diagnose a poor RAG answer by separating ingestion, retrieval, ranking, and generation failures.

Decompose end-to-end answer quality into retrieval quality, groundedness, relevance, safety, and presentation.

Write an evaluation rubric with observable scoring criteria for a target use case.

Construct a compact test set containing normal, ambiguous, adversarial, and unanswerable questions.

Calculate precision-at-k and recall-at-k for sample retrieval results.

Evaluate groundedness and citation correctness against supplied source passages.

Run repeatable model-assisted evaluations while accounting for evaluator limitations.

Compare two prompt or retrieval configurations using consistent quality, latency, and cost measures.

Define release thresholds and regression checks for an LLM application.

Distinguish a deterministic workflow, a tool-using assistant, and an autonomous agent by control flow and risk.

Decide whether an agent is justified for a given business process.

Define a function-tool schema that is unambiguous and safe for a model to invoke.

Implement a single tool call and validate its arguments before execution.

Orchestrate a bounded multi-step workflow with explicit state and termination conditions.

Choose an appropriate strategy for conversational state and long-term memory.

Insert human approval at a high-impact action boundary.

Apply limits, permissions, and audit logging to contain agent behavior.

Evaluate whether a multi-agent design adds enough value to justify its coordination cost.

Choose among Azure Functions, Container Apps, App Service, and AKS for an AI application workload.

Design a reference architecture connecting application, API gateway, model, search, storage, identity, and observability services.

Use queues and asynchronous processing to isolate long-running ingestion or model tasks.

Select a caching strategy for prompts, retrieved content, embeddings, or model responses.

Design backpressure, retry, rate-limiting, and circuit-breaker behavior for model dependencies.

Estimate model quota and application capacity from traffic and token assumptions.

Design for regional availability, disaster recovery, and data-residency constraints.

Create a sequence diagram that shows identity, data, and failure paths through an AI request.

Create a threat model covering prompt injection, data leakage, retrieval poisoning, excessive agency, and denial of service.

Apply managed identities, role-based access control, and Key Vault to remove embedded credentials.

Design private network access for Azure model, search, storage, and application components.

Defend a RAG flow against indirect prompt injection in retrieved documents.

Enforce tenant-aware authorization at both application and retrieval layers.

Define data minimization, retention, logging, and deletion controls for sensitive prompts and outputs.

Configure layered input, output, and content-safety controls for a sample application.

Map fairness, transparency, accountability, and human-oversight risks to concrete architecture controls.

Document residual AI risks and the business owner responsible for accepting each one.

Instrument an LLM request with correlated logs, traces, token usage, retrieval details, and latency measurements.

Define service-level indicators and objectives for availability, latency, quality, and safety.

Design dashboards and alerts that distinguish application, retrieval, model, quota, and safety failures.

Version prompts, model deployments, evaluation sets, and search-index configurations as deployable artifacts.

Create a delivery pipeline with automated evaluation and security gates.

Choose a model or prompt rollout strategy that limits regression risk.

Write an incident runbook for degraded quality, unavailable models, quota exhaustion, and data exposure.

Forecast monthly model, search, compute, storage, and monitoring costs from workload assumptions.

Recommend cost and latency optimizations without violating stated quality thresholds.

Select a portfolio use case that demonstrates enterprise value without requiring unrealistic data or access.

Write a focused problem statement with users, workflow boundaries, assumptions, and exclusions.

Specify measurable functional and nonfunctional acceptance criteria for the portfolio solution.

Compare at least two solution approaches and record the selection in an architecture decision record.

Create a deployment diagram for the selected Azure solution.

Create request and ingestion sequence diagrams for the selected Azure solution.

Produce a threat model and residual-risk register for the selected design.

Estimate capacity and monthly Azure cost for low, expected, and high usage.

Plan an incremental delivery roadmap from proof of concept to production.

Create a structured repository containing application, evaluation, infrastructure, and architecture artifacts.

Provision a minimal set of Azure resources with a reusable infrastructure-as-code template.

Build and test the document ingestion and Azure AI Search indexing path.

Build an authenticated C# API that returns grounded answers and citations.

Create a Python evaluation harness for the portfolio test set.

Add access controls, safety checks, and prompt-injection defenses to the prototype.

Add correlated telemetry and a dashboard for quality, latency, errors, and token use.

Use evaluation results to make and document one measurable architecture improvement.

Produce a concise README and recorded-demo outline that explain the problem, decisions, evidence, and trade-offs.

Deliver a two-minute explanation of the portfolio architecture for both technical and nontechnical interviewers.

Answer common LLM, embeddings, RAG, agent, and evaluation questions using precise architectural language.

Explain Azure AI service choices and credible alternatives without relying on product-name memorization.

Whiteboard a secure, scalable RAG solution from an unfamiliar case-study prompt.

Identify missing requirements and ask high-value discovery questions during a system-design interview.

Quantify latency, capacity, availability, and cost trade-offs using back-of-the-envelope calculations.

Diagnose quality and reliability failures in an interview troubleshooting scenario.

Defend architecture decisions while acknowledging risks, alternatives, and conditions that would change the choice.

Construct concise STAR stories that connect prior .NET backend experience to AI architecture responsibilities.