Linear Algebra and PyTorch Instrumentation for Model Internals

Compute vector norms, cosine similarities, and orthogonal projections with PyTorch tensors.

Track tensor shapes through batched matrix multiplications, transpositions, and reshaping operations.

Translate a simple indexed tensor equation into a PyTorch einsum expression and verify it against an explicit implementation.

Use singular value decomposition to identify dominant directions in an activation matrix.

Register a PyTorch forward hook that records and safely detaches an intermediate activation.

Use a PyTorch hook to replace an intermediate activation during inference.

Verify that an activation intervention changes only the intended computation and leaves model parameters unchanged.