The advent of ChatGPT nearly a year ago marked the beginning of the generative AI era. However, along with this technological advancement, opposition towards AI companies has gained momentum. Artists, entertainers, performers, and even record labels have filed lawsuits against these companies, particularly targeting OpenAI, the creator of ChatGPT. The crux of these legal battles lies in the “secret sauce” behind these innovative tools: training data. These AI models heavily rely on accessing vast amounts of multimedia content, including written material and images, in order to learn and generate new content. What is distressing for many artists is that their works are being used without their knowledge or consent to train commercial AI products. The datasets used for training often include scraped material from the web, a practice that artists previously supported for search result indexing. However, with the rise of AI-generated content that competes with their own creations, many artists have now voiced their opposition to this practice.
In response to the growing concern among artists, there is now a tool in development called Nightshade. This open-source tool, conceptualized by researchers at the University of Chicago, aims to empower artists in their battle against AI. Nightshade can be added to images before they are uploaded to the web. It subtly alters the pixels in a way that is imperceptible to the human eye but has a detrimental effect on AI models attempting to train on these images. Nightshade functions as an optional setting within Glaze, another online tool developed by the same researchers, which provides artists with the ability to cloak digital artwork and confuse AI models by altering their pixel-style.
Nightshade takes the fight against AI even further by causing AI models to learn incorrect information about the objects and scenery depicted in the poisoned images. As an example, the researchers poisoned images of dogs by embedding pixel-based information that made an AI model perceive them as cats. Surprisingly, after the AI model learned from only 50 poisoned image samples, it started generating distorted images of dogs with unusual legs and unsettling appearances. With 100 poison samples, the AI consistently generated a cat when asked to generate a dog. After 300 samples, the AI produced a near-perfect image of a dog when prompted for a cat. To achieve these results, the researchers employed Stable Diffusion, an open-source text-to-image generation model that allowed Nightshade to track and manipulate the AI’s responses, even when prompted with words like “husky,” “puppy,” and “wolf.”
One of the most notable features of Nightshade is its ability to successfully poison AI models while remaining challenging to detect and defend against. The process requires developers of AI models to identify and remove images that contain the poisoned pixels. However, these pixels are purposely designed to be imperceptible to the human eye, making them difficult to identify even for software data scraping tools. Moreover, any existing poisoned images that were already included in an AI training dataset would need to be located and removed. If an AI model had already been trained on these images, it would likely require retraining. Hence, Nightshade poses both a technical and logistical challenge for AI companies to combat.
While the potential for misuse of Nightshade’s data poisoning technique does exist, the researchers’ primary aim is to empower artists and restore the balance of power between artists and AI companies. By establishing a powerful deterrent against the violation of artists’ copyright and intellectual property, Nightshade offers hope to creators. The authors of the MIT Technology Review article covering this work express the researchers’ hope that Nightshade will help shift power away from AI companies and back to artists.
The emergence of generative AI has given rise to legal battles between artists and AI companies. In response to their concerns, Nightshade offers artists a groundbreaking technological tool to fight back. By subtly altering pixels, artists can poison AI models, causing them to generate distorted or incorrect outputs. Nightshade’s ability to manipulate AI models poses challenges for detection and defense, putting the onus on AI companies to address the issue. Despite concerns about potential misuse, Nightshade has the potential to tip the balance of power, granting artists greater agency and protection for their creative works.
Leave a Reply