Rethinking Supervision Targets for Multimodal LLM Unlearning: Less Private Information, Reduced Distribution Shift

Multimodal Large Language Models can memorize and reproduce sensitive information from their training data. Supervision targets guiding model unlearning often trade forgetting effectiveness for model utility. FFN-Perturbed Self-Distillation is proposed, starting from the teacher's original output distribution as a zero-shift reference.