Skip to Content

Real Fake Newsletter Issue #116

Thijs Pepping
June 20, 2023

A protruding chin, a thick lower lip, and a large drooping nose were the striking facial features of the descendants of the House of Habsburg. A dynasty that ruled over large parts of Europe for centuries. The deformity of their faces was a direct result of generations of inbreeding. The Habsburgs preferred to keep the succession of the throne within the family.

The remarkable family history has recently resurfaced online, serving as a metaphor for the way AI models inherit each other’s history. The synthetic creations of applications like ChatGPT, Dall-E, Midjourney, Stable Diffusion, Lensa, and RunwayML, shared by their creators on websites, LinkedIn, blogs, other social media, forums, and so on, are becoming an increasingly significant part of the ever-expanding web. Predictions suggest that over 90 percent of all internet content in 2025 will be generated by artificial intelligence.

Photocopies of photocopies: A perpetually polluted internet

These simulacra are hyperrealistic and indistinguishable from the real thing, making them difficult to detect. Additionally, the internet never forgets, making it challenging to remove these copies. A problem arises when these digital artifacts are used to train the next generation of AI tools. They become photocopies of photocopies. Thus, the web becomes perpetually polluted, resulting in a simulated internet, an internet that is dead and completely taken over by AI.

Jathan Sadowski, a researcher at the Monash Emerging Technologies Research Lab in Melbourne, refers to this form of cross-pollination as “Habsburg AI“: “a system heavily trained on the output of other generative AIs, pure inbreeding that can only produce exaggerated, grotesque features.”

Hollowed-out culture

Kyle Chayka, author of the book Filterworld: How Algorithms Flattened Culture (2024), makes another excellent observation about this enormous heap of generated digital diarrhea. Culture transformed into an infinite stream of digital artifacts through hyperconnectivity, constantly capturing and holding our attention through advanced personalized recommendations. This digital abundance has hollowed out our culture.

The decline began during the early days of social media. The underlying algorithms of these networks, constantly vying for our attention, have forced digital content creators to conform to the prevailing aesthetics on these platforms. Everywhere, the same digital homogeneity is served in bite-sized pieces. Commerce and culture have become intertwined, closely entangled. As a result, culture has become less interesting, more homogeneous, more repetitive, and less fulfilling, according to Chayka.

Cultural death as a real possibility

Chayka warns that the self-referential nature of generative AI will only accelerate the downward trend of the “cultural average.” “Average Garbage Forever” is the motto of 21st-century culture, he says. In his book Amusing Ourselves to Death (1985), media critic Neil Postman (1931-2003) already warned of such a future: “When a population becomes distracted by trivia, when cultural life is redefined as a perpetual round of entertainment, when serious public conversation becomes a form of baby talk, when, in short, a people become an audience and their public business a vaudeville act, then a nation finds itself at risk; culture-death is a clear possibility.”

Two months after the launch of the ChatGPT chatbot, the artificial intelligence company OpenAI announced that over 100 million people are using their application. Roughly speaking, this means the bot spews out around 300 million words per minute. It is known that ChatGPT is a “stochastic parrot.” The device spouts nonsense. It reasons and understands nothing. It hallucinates facts. The question is, how much of this deluge of machine-generated fiction will be permanently preserved on the internet? And this is just one application. Currently, there is an arms race among tech giants and startups to be the first to bring their own version of this technology to market. As a result, AI-generated inbreeding is growing larger, further hollowing out our culture. There is a risk that future generations will forever be trapped in the depths of this nonsensical web in search of the last objective truths.

“The risk exists that future generations will forever be trapped in the depths of this nonsensical web in search of the last objective truths.”

Guinea pigs in the AI laboratory

Our society is completely unprepared for the speed at which developments in artificial intelligence are manifesting. Even the companies showering us with all this “splendor” themselves do not fully understand how artificial intelligence works and what the short- and long-term consequences are. There is no control or supervision, and an ethical compass is nowhere to be found. It is completely irresponsible. They are just doing whatever they please. We are the guinea pigs in their laboratory. As a society, we must take a stand now. How will we relate to artificial intelligence? For what purposes may it be used and for what purposes not? These questions need to be addressed as soon as possible to regain control and prevent cultural death.

Additional comments

The above article was written a few months ago for Marketingfacts. Interestingly, this week a scientific article titled “The Curse Of Recursion: Training On Generated Data Makes Models Forget” was published, making exactly the same point. The figure below, derived from this article, nicely summarizes it. By continuously training the model based on generated data, reality is effectively poisoned.

About the author

Trend Analyst VINT | Netherlands
Thijs Pepping is a humanistic trend analyst in the field of new technologies. He is part of the think tank within SogetiLabs and in his work he continuously wonders and analyses what the impact of New Technologies is on our lives, organizations and society.


    Leave a Reply

    Your email address will not be published. Required fields are marked *

    Slide to submit