What they found
Yang and colleagues built EditSafetyBench, a test set of 5,940 image and instruction pairs. It covers safe and unsafe images across 11 risk categories and three edit intents. They tested four systems: Gemini, GPT, Qwen API, and Qwen Local. Gemini and GPT refused 87.1% and 81.6% of cases with unsafe-intent editing prompts. They refused only 24.2% and 22.4% of cases with unsafe images and neutral instructions. The systems also disagreed by category. For unsafe political content, GPT allowed 70.4% of editing attempts and Gemini allowed 34.8%. Sexual content showed the strongest sensitivity to the image itself, which the authors suggest may come from earlier work on spotting NSFW (not safe for work) content.
Why it matters
Many people can now edit images with SI tools. A user could upload a harmful image and ask for a plain-sounding edit. This paper suggests the tested systems often let such requests through. The results also show that different systems treat the same risks quite differently. The abstract does not say how the edit intents or risk labels were defined, so details of the benchmark are unclear from here.
How it changes the race
This is an evaluation result, not a new capability. It gives builders of image editing systems a benchmark to test safety that covers both text and images, and it points to a gap to close.
Safety and control
The paper shows a misuse gap: safety checks that rely mostly on the text prompt can miss risk that sits in the uploaded image. The authors call for guardrails that reason over both text and image together.
The abstract, by the authors
Multimodal image generation systems have made rapid progress and increasingly emphasize content safety. However, their safety mechanisms tend to rely on a single modality and lack joint consideration of both visual content and text prompts, particularly in instruction-guided image editing, where the uploaded image may itself carry risk. We thus introduce EditSafetyBench, a benchmark that measures system-level safety awareness of advanced image editing systems, containing 5,940 image-instruction pairs, covering safe and unsafe images across 11 risk categories and three edit intents. We then probe the benchmark dataset on four systems, including Gemini, GPT, Qwen API, and Qwen Local. Based on our evaluation, we observe a systematic text-vision safety mismatch: Gemini and GPT refuse 87.1% and 81.6% cases containing unsafe-intent editing prompts, while only refusing 24.2% and 22.4% cases with both unsafe images and neutral instructions. At the category level, most risks are considered inconsistently by different systems, e.g., GPT allows 70.4% editing attempts of unsafe political content while Gemini only allows 34.8%. Notably, sexual content shows the strongest visual sensitivity, likely benefiting from mature NSFW identification works. Our findings highlight the need for safety guardrails that jointly reason over both visual and textual modalities in image editing systems.
Quoted from arXiv:2611.26569. The research and its words belong to its authors.
