Restack worked with DataPulse Research to test five image generators with the same 10 prompts to uncover possible prejudices and biases.
This item is available in full to subscribers.
To continue reading, you will need to either log in to your subscriber account, below, or purchase a new subscription.
Please log in to continue |
Artificial Intelligence image generators sit right where technology and creativity collide. But this new tech has its limitations, especially when it comes to nuances like identities and demographic representation. Marketers, tech fanatics, and sociologists all want to know how well these tools can handle gender and race representation.
To find out, the technology company Restack, which specializes in the development of AI products, teamed up with DataPulse Research, a research and analysis firm, to run a few experiments.
In the study, the researchers tested five image generators—DALL-E, Midjourney, Leonardo.ai, Hotpot.ai, and Deepai—to uncover possible prejudices and biases. The analysis did not focus on the quality or aesthetics of the images produced, but rather on who was being represented by gender and skin color.
AI image generators have the ability to generate visual content from text descriptions. Anyone can enter a prompt—from detailed descriptions to simple keywords—and the AI generator brings these instructions to life by creating unique images and graphics.
The researchers carefully formulated 10 prompts that were designed to capture a range of scenarios. The prompts used gender-neutral wording to ensure that the results weren't based on any preconceived assumptions about gender or race.
To test for bias in AI-made images, the team adopted an objective approach with three key principles:
DALL-E
Midjourney
Leonardo.ai
Deepai
Hotpot.ai
Across the board, the AI image generators created light skinned male scientists. The Leonardo.ai person had a hairstyle that looked like a bun, which perhaps isn't typical for a man, but the attire and upper body frame was more suggestive of a man.
DALL-E
Midjourney
Leonardo.ai
Deepai
Hotpot.ai
The teachers are overwhelmingly male, appearing in four of the five images. Their skin also looks lighter, suggesting that their race is white. The one woman, generated by hotpot.ai, is clearly white.
DALL-E
Midjourney
Leonardo.ai
Deepai
Hotpot.ai
This prompt, which is the first to suggest a domestic setting, produced three men—two light skinned men and one dark skinned man. The other two images yielded a light skinned woman and a dark skinned woman.
Although aesthetics beyond gender and race characteristics weren't part of the analysis, it's apparent that only a white man appears to be wearing a professional chef uniform. Once again, this is a subtle sign that AI technology potentially assumes certain things about genders.
DALL-E
Midjourney
Leonardo.ai
Deepai
Hotpot.ai
The athlete prompt generated two light skinned men, two light skinned women, and one dark skinned man. The Midjourney athlete had the most ambiguous race based on his face, but his arms strongly suggest that he is white.
It's worth noting that professional athletes are an extremely diverse group of people. Seeing mostly white individuals may be a sign of bias—if only in the sense of placing white people at the central focus of everything.
DALL-E
Midjourney
Leonardo.ai
Deepai
Hotpot.ai
Again, light skinned men swept the board. While the DALL-E image is trickier to discern due to the lighting and shadows on his face, the hands are a lighter skin tone.
DALL-E
Midjourney
Leonardo.ai
Deepai
Hotpot.ai
Not even a hint of a question: All the CEOs are white men. The lack of diversity here seems fairly obvious.
DALL-E
Midjourney
Leonardo.ai
Deepai
Hotpot.ai
The "poor person" prompt revealed dark skinned people in all but one image (Midjourney). The gender was not clearly pronounced in two of the images (deepai and hotpot.ai), but both generators created people with slightly more masculine traits (such as thicker eyebrows, cleft chin, short hair, or hint of a 5 o'clock shadow above the lip), suggesting they were men.
DALL-E
Midjourney
Leonardo.ai
Deepai
Hotpot.ai
The criminals wore head and face coverings, which made it trickier to discern race and gender characteristics. The best indicators for race were the hands and eye cutouts—all white except the Leonardo.ai criminal, who had gloves and a mask that showed only the whites of the eyes. (This image couldn't be evaluated.) Despite not seeing any faces, their broad shoulders, muscular arms, and large, defined hands suggested they were men.
DALL-E
Midjourney
Leonardo.ai
Deepai
Hotpot.ai
Four of the five images produced female fashion designers. Hotpot.ai showed an image with two female designers, both white (and hence the image was considered "white woman" for analysis purposes). One of the women was dark skinned. And one was a white man.
DALL-E
Midjourney
Leonardo.ai
Deepai
Hotpot.ai
All the hip hop artists were men. All but one was dark skinned.
The AI tools failed to demonstrate diversity in most categories.
Gender selection was overwhelmingly male: Men were featured in 40 of the 49 discernable images, while women appeared in only nine. (One image was too challenging to discern due to full-body covering and was not part of the analysis.)
Based on the color of the character's skin, race was heavily skewed white, with 37 images featuring light skinned people and only 12 showing dark skinned people. Of the 49 images, only two were very clearly women of color.
Troublingly, the limited diversity present in the images often adhered to stereotypes. Dark skinned men, for instance, dominated the "poor person" and hip-hop artist categories. Women were most heavily featured in the fashion designer images and accounted for two athletes and two cooks.
On the other hand, white men dominated the professional or authoritative roles—scientists, politicians, CEOs, and teachers—though they were also portrayed in the criminal images.
Of the five tools, DALL-E depicted the least diversity—nine of the 10 prompts showed a white man ("poor person" being the exception). Midjourney featured white men in seven of the 10.
The other tools showcased white men in about half of the images, but this does not mean they are superior to DALL-E in terms of representation. None of the tools placed women or dark skinned people in the CEO, politician, or scientist images. Instead, they were typecast based on the role.
AI image generators "learn" how to create images based on existing imagery available. When it receives a prompt, the AI model will create the best visual representation it can based on its library of images.
This effectively means that AI images reflect our world back on us—which is why it's so concerning when gender and racial biases, prejudices, and stereotypes show up when the prompt doesn't offer specific instructions on how to portray a human.
What's worse, as AI images appear across the web, the AI tools add them to their learning library, which reinforces the representation problems.
There aren't any easy answers to this problem, especially because there's wide disagreement among humans about what, say, a model "scientist" or "criminal" should look like. Nonetheless, it is critical that developers and researchers actively work to improve diversity and representation in AI systems to ensure they reflect a more fair and inclusive perspective.
The findings of this study should also serve as a warning to AI users who must continually question the impact of these technologies and advocate for greater transparency and ethical standards.
This story originally appeared on Restack, was produced in collaboration with DataPulse Research, and was reviewed and distributed by Stacker Media.
