absorb.md — A knowledge graph of what AI thinkers are actually saying

paper / samsamoa / Jun 9

SPPE: A New Benchmark and Methods for Recovery in Surrogate-based MLLM Image Editing Privacy

This work addresses the limitations of current privacy-preserving techniques for Multimodal Large Language Model (MLLM) image editing, which often fail to recover the original image after editing sensitive regions with surrogates. The authors introduce SPPE, a new benchmark with two tasks: editability assessment and surrogate-to-source recovery. They also propose ERMA for editability prediction and C2E-S2SER for recovery, demonstrating significant improvements on both tasks.

multimodal-llmsprivacy-preserving-editingimage-editingmachine-learning-benchmarkai-safetycomputer-vision

“Existing privacy protection methods for MLLM image editing substitute sensitive regions with surrogate content, but this often results in an edited surrogate image rather than the desired edited original image, neglecting local recovery.”

paper / samsamoa / Jun 9

Adaptive Privacy Control for MLLMs Using Anchored Privacy Drifting and the AdaptShield Benchmark

This paper introduces Anchored Privacy Drifting (APD), a training-free technique to mitigate the novel privacy risks posed by Multimodal Large Language Models (MLLMs). APD addresses the limitations of existing methods by preserving contextual information while abstracting sensitive elements. A new benchmark, AdaptShield, with 22 privacy categories and MLLM-based utility assessments, was developed to evaluate APD. Experiments across multiple MLLM series demonstrate APD's effectiveness in balancing privacy enhancement and content preservation.

mllm-privacyadaptive-privacycomputer-visionprivacy-benchmarktraining-free-methodscontext-preservation

“Existing privacy methods fail to adequately protect MLLMs due to their reliance on predefined categories and fixed obfuscation strategies, which are insufficient for handling unpredictable sensitive information in user inputs and privacy-sensitive visual contexts.”

paper / samsamoa / Jun 9

CaliPPer: A Framework for Quantifying and Improving AI Binding Prediction Performance

CaliPPer is a post-hoc framework that quantifies, predicts, and improves AI model performance for binding prediction tasks. It uses a multi-chain Sample-to-Domain Distance (S2DD) coupled with Bayesian recalibration, operating at multiple resolutions to estimate generalizability, predict aggregate performance, and provide per-sample confidence. This approach addresses the unpredictability of model performance on new datasets, a common issue in therapeutic antibody and TCR discovery.

binding-predictionmodel-calibrationai-drug-discoveryconfidence-estimationcomputational-biologyprotein-modelingtherapeutic-antibodies

“CaliPPer achieves high correlation (0.80-0.92) between distance metrics and performance across various models and domains.”

paper / samsamoa / Jun 9

Novice Users' Struggles with AI-Generated Visualizations Uncover Risks and Model-Specific Failures

This study investigates the challenges faced by visualization novices when using conversational AI (ChatGPT, Gemini, Claude) to generate and interpret data visualizations. A user study with 20 novices revealed issues related to AI execution compliance, visualization quality, user misuse, and factors influencing trust and verification. The findings highlight distinct failure modes across different AI models and inform design recommendations for more effective and trustworthy AI-assisted visualization systems, underscoring the need for enhanced AI literacy.

conversational-aidata-visualizationai-interpretabilityhuman-computer-interactionuser-studyai-misinformation

“Users with no prior visualization experience struggle to identify and correct errors in AI-generated visualizations using conversational AI tools like ChatGPT.”