arxiv:2609.15635
Published on Sep 14
· Submitted by
Sebastián Andres Cajas Ordóñez on Sep 15
· Massachusetts Institute of Technology
Upvote
8
Authors:
,
,
,
,
,
,
,
,
,
Abstract
A paired image-swap audit reveals that report availability substantially reduces vision-language model sensitivity to image changes in radiology question answering.
Generated by thinkingmachines/Inkling-Small
A radiology report can already answer a clinical question, so it is hard to tell whether a vision-language model also uses the image. ModaLens, a paired image-swap audit, measures how report availability changes image sensitivity: MedGemma-27B on 3,199 paired MIMIC-CXR cases from 293 patients, all 14 questions per case (13 finding-specific and one composite), each image replaced by one from another study, usually of the same patient, with question and report fixed. Under an explicit answer instruction, the model's generated answer changes on 4.26 percent of trials with the report and 20.94 percent without it, a paired increase of 16.7 points (patient-clustered 95 percent CI 15.6 to 17.7), so report availability reduces image-swap sensitivity under this protocol; the original prompt with a lowercase first-token readout gives 4.70 percent against 17.07 percent, and substitutions also move continuous answer scores where the binary prediction does not change. The labels are derived from reports, which limits conclusions about visual correctness; the direction replicates in two further model lineages. Code, the exact prompts and a run record for every number are at https://github.com/criticaldata/[MODALENS](https://hf-mirror.com/papers?q=MODALENS).
View arXiv page View PDF Add to collection
Community
Paper submitter 8 days ago
Medical VLMs lean on the report and stop using the scan. We localize where that happens layer by layer, verbalize the behavior in plain language, and test whether targeted interventions bring the image back.
7 days ago
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
-
Source-Dependent Deference in Medical Imaging Agents Under Falsified Findings: A Pilot Audit (2026)
-
Witness Evidence Portfolios: Single-Prefill Risk Detection for Closed Multimodal Answers (2026)
-
BrainDiff: Longitudinal Report Generation for Multimodal Brain MRI (2026)
-
CANDOR: Chance-Calibrated Discordance in Frozen Foundation Encoders (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on HF Mirror checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
· Sign up or log in to comment
Upvote
8
Get this paper in your agent:
hf papers read 2609.15635
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2609.15635 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2609.15635 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2609.15635 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.