Multimodal Fusion for Violation Detection

Extract fixed-size feature vectors from the trained image CNN in evaluation mode.

Extract CLS token embeddings from the fine-tuned BERT model in evaluation mode.

Encode categorical (e.g., school type) and temporal (e.g., season) metadata into numerical features.

Explain the rationale for using a late fusion architecture for this project.

Build and train a fusion model in PyTorch that concatenates image, text, and metadata feature vectors to predict violation severity.

Compare the multimodal model's performance against the single-modality baselines to quantify the benefit of fusion.