Generative AI tools can produce useful and surprising text based on the inputs they have. But what are these inputs, and how can we know they are valid?
The Infinite Monkey Theorem (sometimes known as Shakespeare’s monkeys) refers to a hypothetical scenario in which an infinite number of monkeys randomly typing on an infinite number of typewriters for an infinite amount of time will eventually produce the complete works of William Shakespeare. This idea is often used to illustrate the concept of infinite probability and the likelihood of unlikely events occurring given a large enough sample size.
But Large Language Models (LLMs) are actually nothing like the infinite monkeys.
Consider this alternate theorem:
Given an infinite amount of input text and infinite computing resources, LLMs can produce output which exactly matches the desired response.
LLMs simulate the response that a human might make to a given input. They do this by a variety of mechanics which we can treat as a black box for now.
The interesting questions are:
- What are the structure and constraints on the LLM’s mechanics? (subject of a future post)
- What are the sources and content of the input text?
The second question considers the possibility that the input text could be incomplete, incorrect or even intentionally biased.
For example, if an LLM is trained on the contents of the Internet from 2021, it will certainly outperform an identical LLM trained using the Internet from 2006. The 2021 dataset will contain more information, but it will also contain information that is simply different or revises information from the previous data.
Incorrectness Onboard
The Internet (from any year) is riddled with text that asserts incorrect information, or draws conclusions based on incomplete information. Humans can use critical thinking to discern whether a particular sentence from an Internet source is true.
An example of this is the following quote:
“Don’t believe everything you read on the Internet just because there’s a picture with a quote next to it.” - Abraham Lincoln
The quote above was a satirical reply to an incorrect Lincoln quote on Twitter, and it has become a cultural touchstone for questioning the provenance of any internet source.
Let’s ask ChatGPT.
Prompt:
Who said: “Don’t believe everything you read on the Internet just because there’s a picture with a quote next to it.” ?
ChatGPT:
Abraham Lincoln is often attributed as having said “Don’t believe everything you read on the Internet just because there’s a picture with a quote next to it.” However, it is not a verifiable quote from Abraham Lincoln. This phrase is often used to express skepticism about the authenticity of quotes or information found online.
Some good news: The overall opinion of the text inputs to ChatGPT must have agreed that this is an incorrect quote.
Consider what would happen if the input text were pruned to remove all assertions that the quote was false, or if these inputs were seeded with text that confirmed the quote? The LLM would have no ability to use critical thought to refute the quote since it can only rely on what it had ‘read’.
Incompleteness as a feature
Similar to the quote above, the input texts could be groomed to remove information, or perhaps even more sinisterly, to remove output.
In many publicly available LLM chatbots, the NSFW problem is solved by post-filtering the output. An example of this is the Character.ai chat system, where if the conversation crosses a line that generates a NSFW response, the response is simply deleted.
Denying a random user some NSFW chat output is less concerning than denying that same user access to responses that contain legal, medical or political information. This kind of self-censorship is valid for a private entity which must limit its legal exposure, but any successful large-scale LLM system will inevitably have to handle the ethical problems here.
Google founders famously framed the company’s motto as “Don’t be evil”, which has proven to be difficult as time progresses. The coming generations of AI tools face the same problems and more.
For example, if a person asks for help thinking about how to write an essay on the problems that transgender youth face, it will probably respond in a news-like fashion.
Consider the powerful self-censorship incentives in a similar system which engages in a conversation with a transgender teenager who is trying to make decisions. (See ChatGPT’s approach to these questions here).
The choices of when and how to self-censor the results are important, particularly for a public-facing model running as a utility at scale.
Bias vs Inaccuracy
The categories above can be considered inaccuracies in the system. What happens when the ideas being discussed are matters of belief or opinion? As conversational AI systems become mainstream, the question of who controls the narratives becomes vital.
Humans will accept information from a source that seems correct and is presented in an authoritative manner. The new generation of AI-enabled misinformation and manipulation will make the recent past look like child’s play.
ChatGPT is made of Humans
We began by thinking about how to generate a specific text from infinite possibilities. The practical trick that LLMs employ is to begin from a finite (but still extremely large) set of possible texts and generate a realistic response.
When you interact with a telephone system that says “This call is being recorded for quality assurance and training purposes”, the word ‘training’ suddenly takes on a strange new meaning. Your conversation with the company representative is now available as downstream input to their in-house LLM.
Because these input texts come from humans, they are already coherent. LLMs are simply doing incredibly targeted remixes of humanity’s back catalog of knowledge and thought.
Everything you do or say in a digital format is potentially subject to collection towards this end.
As we progress into the next generation of AI-enabled conversations, it’s vital to ask where the input text comes from and how it may be inaccurate or even biased. Our general assumptions may incorrect or even dangerous.
Prompt:
Write a fake Shakespeare line that expresses that ChatGPT responses are simply regurgitated human text.
ChatGPT:
“ChatGPT doth mimic human thought, but ‘tis mere mimicry bought.”