Proving intelligence 🧠
The Turing test is a well known suggestion for how to decide if a machine is intelligent: if you can't tell that it's a machine, that's enough to count it as intelligent. There's a counter-argument, almost as well known, called the Chinese room argument: if someone who doesn't understand Chinese sits in a room using a big book of rules to construct replies to prompts in Chinese, they don't seem to be doing anything intelligent even if the replies they give are convincing to a Chinese person.
I knew the outline of this...
...but then I read the Chinese room Wikipedia page...
...and it seems it's quite controversial. There's some entertaining quotes:
"The Chinese Room argument has probably been the most widely discussed philosophical argument in cognitive science to appear in the past 25 years"
Most of the discussion consists of attempts to refute it. "The overwhelming majority still think that the Chinese Room Argument is dead wrong".
...the field of cognitive science ought to be redefined as "the ongoing research program of showing Searle's Chinese Room Argument to be false".
Fun stuff, if that's your idea of fun.
As an excercise for the reader, consider how much this does or doesn't apply to LLM chatbots.