Llama Local Models and Training

Calculate scaled dot-product attention with a causal mask for a toy token sequence.

Compare standard multi-head attention with Llama-style rotary positional embeddings and grouped-query attention, identifying their inference implications.

Trace tensor shapes through a Llama decoder block while identifying the roles of residual connections, RMS normalization, and SwiGLU layers.

Load a small Llama-family model with Hugging Face Transformers on the RTX 3080 Ti and verify its device placement and numerical precision.

Run a quantized GGUF model with llama.cpp using the correct chat template and controlled temperature, top-p, stop, and seed settings.

Run a compatible local model with MLX on Apple Silicon and verify Metal acceleration.

Estimate model-weight and KV-cache memory from parameter count, precision, context length, and attention configuration.

Compare common GGUF quantization levels by memory use, speed, and expected quality.

Use time to first token, tokens per second, and peak memory measurements to tune context size, batching, and device offloading for a local workload.

Serve a model with Ollama and call its chat interface from Python while keeping conversation history within a fixed context budget.

Validate structured JSON output from a local model and recover from invalid responses.

Implement a constrained local tool call and safely return its result to the model.

Chunk a document collection, generate local embeddings, and create an index that preserves source metadata.

Retrieve and rerank context passages for a user question.

Generate an answer with citations restricted to retrieved evidence.

Evaluate retrieval recall and grounded-answer quality on a small question set.

Create a repeatable baseline evaluation set for the intended custom behavior.

Convert raw examples into a consistent instruction-tuning dataset using the model's chat format.

Audit a dataset for duplication, evaluation leakage, licensing conflicts, private information, and exposed secrets, then produce safe training and validation splits.

Calculate how LoRA rank and target-module selection affect the number of trainable parameters.

Configure a QLoRA supervised fine-tuning run that fits within the RTX 3080 Ti's memory budget.

Run a short supervised fine-tuning job, save reproducible checkpoints and metadata, and diagnose its training and validation loss curves.

Compare a fine-tuned adapter with the base model on the held-out evaluation set.

Load, merge, and export a LoRA adapter for quantized local inference.

Train and evaluate a tokenizer on a small domain corpus.

Complete and test a small decoder-only transformer composed of embeddings, Llama-style blocks, and an output head.

Implement a pretraining loop with batching, validation, gradient clipping, and checkpoints.

Run a small pretraining job, measure validation loss and hardware throughput, and use a standard scaling rule to estimate full-scale Llama data, compute, and wall-clock requirements.

Implement a minimal private coding assistant that supplies selected repository context to a local model.

Implement a minimal RPG assistant that maintains validated character and world state across interactions.

Compare prompting, RAG, and fine-tuning on the same custom-domain task and justify the best approach.

Package a local assistant with reproducible model settings, privacy boundaries, and performance measurements.