Last week, Google released an experimental Gemini 2.0 Flash model that generates images natively. Thanks to Gemini 2.0 Flash’s native multimodal capabilities, you can edit images conversationally, producing highly consistent images across multiple generations. I recently tested Gemini’s native image generation feature and came away surprised by its powerful capabilities.
Now, a Redditor has showcased that the experimental Gemini 2.0 Flash model is good at removing watermarks from images. In fact, Gemini is able to completely erase watermarks from copyrighted images, including images sourced from Shutterstock and Getty Images — two of the largest stock image providers that use watermarking to protect copyrighted content.
Unsurprisingly, the development has raised serious concerns about copyright law. In a statement to TechCrunch, a Google spokesperson mentioned, “Using Google’s generative AI tools to engage in copyright infringement is a violation of our terms of service. As with all experimental releases, we’re monitoring closely and listening for developer feedback.“
It’s important to note that Gemini 2.0’s image generation model is currently in the “experimental” phase and available on Google’s AI Studio — not on the consumer-facing Gemini website or app. AI Studio is meant for developers to test models, explore capabilities, and provide feedback to Google. However, it can also be used by general users without any restrictions.
To test the native image generation model, I uploaded a Shutterstock image with watermarks on AI Studio and asked the model to remove them. On the first try, it couldn’t remove the watermarks, but after running the prompt once more, it erased them without raising any suspicion. I also uploaded an image with my custom watermark, and Google’s model did a perfect job again.
It seems Google has not fixed the issue as of now. Last year, Google was forced to disable image generation of people after Gemini refused to produce images of white people. Now, to avoid similar backlash, Google will likely add stricter safety guardrails to prevent copyright infringement before a wider rollout.
Potential Misuses of Gemini Native Image Generation
Other than removing watermarks, the experimental Gemini model can do several things that are prone to misuse. Riley Goodside, a researcher at Scale AI who tests AI models for vulnerabilities, shared a post on X showing how the model can manipulate images by generating a realistic scene.
In the below image, you can see how skillfully Gemini has changed the background and perfectly followed the instructions to manipulate the image.
Another AI security researcher, elder_plinius, known for jailbreaking AI models, shared a post on X demonstrating how simple punctuation tricks in a prompt can bypass safety restrictions. In the tweet you see below, Gemini altered a woman’s appearance after refusing it initially.
With AI models gaining capabilities rapidly over time, it’s crucial that companies perform thorough safety tests before releasing them. We already know that Anthropic’s Constitutional Classifiers were broken within a week using a universal jailbreak, bypassing all safety measures. It indicates that AI models are far from being foolproof. They will require industry-wide collaboration to mitigate jailbreaking techniques and prevent harmful generations.
Source: Beebom