We don’t know that for sure yet, we saw a lot of emergent intelligent properties appear as we scaled up, and we’re nowhere near done scaling LLM’s, I’m not saying it will be solved, just that we don’t know one way or the other yet.
LLMs are fundamentally different from human consciousness. It isn’t a problem of scale, but kind.
They are like your phone’s autocomplete, but very very good. But there’s no level of “very good” for autocomplete that makes it a human, or will give it sentience, or allow it to understand the words it is suggesting. It simply returns the next most-likely word in a response.
If we want computerized intelligence, LLMs are a dead end. They might be a good way for that intelligence to speak pretty sentences to us, but they will never be that themselves.
So for context, I am an applied mathematician, and I primarily work in neural computation. I have an essentially cursory knowledge of LLMs, their architecture, and the mathematics of how they work.
I hear this argument, that LLMs are glorified autocomplete and merely statistical inference machines and are therefore completely divorced from anything resembling human thought.
I feel the need to point out that not only is there no compelling evidence that any neural computation that humans do anything different from a statistical inference machine, there’s actually quite a bit of evidence that that is exactly what real, biological neural networks do.
Now, admittedly, real neurons and real neural networks are way more sophisticated than any deep learning network module, real neural networks are extremely recurrent and extremely nonlinear, with some neural circuits devoted to simply changing how other neural circuits process signals without actually processing said signals on their own. And in the case of humans, several orders of magnitude larger than even the largest LLM.
All that said, it boils down to an insanely powerful statistical machine.
There are questions of motivation and input: we all want to stay alive (ish), avoid pain, and have constant feedback from sensory organs while a LLM just produces what it was supposed to. But in an abstraction the ideas of wants and needs and rewards aren’t substantively different from prompts.
Anyway. I agree that modern AI is a poor substitute for real human intelligence, but the fundamental reason is a matter of complexity, not method.
If you truly believe humans are simply autocompletion engines then I just don’t know what to tell you. I think most reasonable people would disagree with you.
Humans have actual thoughts and emotions; LLMs do not. The neural networks that LLMs use, while based conceptually in biological neural networks, are not biological neural networks. It is not a difference of complexity, but of kind.
Additionally, no matter how many statistics, CPU power, or data you give an LLM, it will not develop cognition because it is not designed to mimic cognition. It is designed to link words together. It does that and nothing more.
A dog is more sentient than an LLM in the same way that a human is more sentient than a toaster.
In a more diplomatic reading of your post, I’ll say this: Yes, I think humans are basically incredibly powerful autocomplete engines. The distinction is that an LLM has to autocomplete a single prompt at a time, with plenty of time between the prompt and response to consider the best result, while living animals are autocompleting a continuous and endless barrage of multimodal high resolution prompts and doing it quickly enough that we can manipulate the environment (prompt generator) to some level.
Yeah biocomputers are fucking wild and put silicates to shame. The issue I have is with considering biocomputation as something that fundamentally cannot be be done by any computational engine, and as far as neural computation is understood, it’s a really sophisticated statistical prediction machine
We all want to believe that humans, or indeed animals as a whole, have some secret special sauce that makes us fundamentally distinguishable from statistical algorithms that approximate a best fit function according to some cost metric, but the fact of the matter is we don’t.
There is no science to support the idea that biological neurons are particularly special, and there are reams and reams of papers suggestin that real neural cognition is little more than an extremely powerful statistical machine.
I don’t care about what “most reasonable people” think. “Most reasonable people” don’t have an opinion about the axiom of choice, or the existence of central pattern generators. That’s not to devalue them but their opinions on things this far outside of their expertise are worth about as much as my opinions on the concept of art. I am a professional in neural computation, and I put it to you to even hypothesize about how animal neural computation is fundamentally distinct from LLM computation.
Like I said, we are wildly more capable than GPT, because our hardware is wildly more complex than any ANN, but the fundamental computing strategy is not all that different.
You’re guessing, you don’t actually know that for sure, it seems intuitively correct, but we simply do not know enough about cognition to make that assumption.
Perhaps our ability to reason exclusively comes from our ability to predict, and by scaling up the ability to predict, we become more and more able to reason.
These are guesses, all we have now are guesses, you can say “it doesn’t reason” and “it’s just autocorrect” all you want, but if that were the case why did scaling it up eventually enable it to perform basic math? Why did scaling it up improve its ability to problemsolve significantly (gpt3 vs gpt4), there’s so many unknowns in this field, to just say “nah, can’t be, it works differently from us” doesn’t mean it can’t do the same things as us given enough scale.
I’m not guessing. When I say it’s a difference of kind, I really did mean that. There is no cognition here; and we know enough about cognition to say that LLMs are not performing anything like it.
Believing LLMs will eventually perform cognition with enough hardware is like saying, “if we throw enough hardware at a calculator, it will eventually become alive.” Even if you throw all the hardware in the world at it, there is no emergent property of a calculator that would create sentience. So too LLMs, which really are just calculators that can speak English. But just like calculators they have no conception of what English is and they do not think in any way, and never will.
I’m not guessing. When I say it’s a difference of kind, I really did mean that. There is no cognition here; and we know enough about cognition to say that LLMs are not performing anything like it.
We do not know that, I challenge you to find a source for that, in fact, i’ve seen sources showing the opposite, they seem to reason in tokens, for example, LLM’s perform significantly better at tasks when asked to give a step by step reasoned explanation, this indicates that they are doing a form of reasoning, and their reasoning is limited by what I have no better term for than laziness.
It is your responsibility to prove your assertion that if we just throw enough hardware at LLMs they will suddenly become alive in any recognizable sense, not mine to prove you wrong.
You are anthropomorphizing LLMs. They do not reason and they are not lazy. The paper discusses a way to improve their predictive output, not a way to actually make them reason.
But don’t take my word for it. Go talk to ChatGPT. Ask it anything like this:
“If an LLM is provided enough processing power, would it eventually be conscious?”
“Are LLM neural networks like a human brain?”
“Do LLMs have thoughts?”
“Are LLMs similar in any way to human consciousness?”
Just always make sure to check the output of LLMs. Since they are complicated autosuggestion engines, they will sometimes confidently spout bullshit, so must be examined for correctness. (As my initial post discussed.)
You’re assuming i’m saying something that i’m not, and then arguing with that, instead of my actual claim.
I’m saying we don’t know for sure what they will be able to do when they’re scaled up. That’s the end of my assertion. I don’t have to prove that they will suddenly come alive, i’m not claiming they will, i’m just claiming we don’t know what will happen when they’re scaled, and they seem to have emergent properties as they scale up. Nobody has devised a way of predicting what emergent properties happen when, nobody has made any progress whatsoever on knowing what scaling up accomplishes.
Can they reason? Yes, but poorly right now, will that get better? Who knows.
The end of my claim is that we don’t know what’ll happen when they scale up, and that you can’t just write it off like you are.
If you want proof that they reason, see the research article I linked. If they can do that in their rudimentary form that we’ve created with very little time, we can’t write off the possibility that they will scale.
Whether or not they reason LIKE HUMANS is irrelevant if they can do the job.
And i’m not anthropomorphizing them without reason, there aren’t terms for this already, what would you call this behavior of answering questions significantly better when asked to fully explain reasoning? I would say it is taking the easiest option that still meets the qualifications of what it is requested to do, following the path of least resistance, I don’t have a better word for this than laziness.
Furthermore predictive power is just another way of achieving reasoning, better predictive power IS better reasoning, because you can’t predict well without reasoning.
LLMs are fundamentally different from human consciousness.
They are also fundamentally different from a toaster. But that’s completely irrelevant. Consciousness is something you get when you put intelligent in an agent that has to move around in and interact with an environment. A chatbot has no use for that, it’s just there to mush through lots of data and produce some, it doesn’t have or should worry about its own existence.
It simply returns the next most-likely word in a response.
So does the all knowing oracle that predicts the lotto numbers from next week. It being autocomplete does not limit its power.
LLMs are a dead end.
There might be better or faster approaches, but it’s certainly not a dead end. It’s a building block. Add some long term memory, bigger prompts, bigger model, interaction with the Web, etc. and you can build a much more powerful bit of software than what we have today, without even any real breakthrough on the AI side. GPT as it is today is already “good enough” for a scary number of things that used to be exclusively done by humans.
A chatbot has no use for that, it’s just there to mush through lots of data and produce some, it doesn’t have or should worry about its own existence.
It literally can’t worry about its own existence; it can’t worry about anything because it has no thoughts or feelings. Adding computational power will not miraculously change that.
Add some long term memory, bigger prompts, bigger model, interaction with the Web, etc. and you can build a much more powerful bit of software than what we have today, without even any real breakthrough on the AI side.
I agree this would be a very useful chatbot. But it is still not a toaster. Nor would it be conscious.
Even if they are a result of complexity, that still doesn’t change the fact that LLMs will never be complex in that manner.
Again, LLMs have no self-awareness. They are not designed to have self-awareness. They do not have feelings or emotions or thoughts; they cannot have those things because all they do is generate words in response to queries. Unless their design fundamentally changes, they are incompatible with consciousness. They are, as I’ve said before, complicated autosuggestion algorithms.
Suggesting that throwing enough hardware at them will change their design is absurd. It’s like saying if you throw enough hardware at a calculator, it will develop sentience. But a calculator will not do that because all it’s programmed to do is add numbers together. There’s no hidden ability to think or feel lurking in its design. So too LLMs.
I believe I understand everything you are saying and why you are saying it. I think you are completely missing the point, though. LLMs already do quite a few things they were not designed to do. Also, your idea of sentience seems very limited. Yes, with our biological computers we have some degree of presence over “time”, but is that critical - or is it just critical for us due to our limited faculties.
What if “the internet” developed some form of self-awareness - would we know? Our entire society could be subtly manipulated through carefully placed latency spikes, for example. I’m not saying this is happening, just that I think you are incredibly overconfident because you have an understanding of LLMs current lack of state etc.
If we added a direct feedback mechanism - realtime or otherwise - we could start seeing more compelling emergent properties develop. What about feedback and ability to self-modify?
These systems are processing information on a level we cannot even pretend to comprehend. How can you be so certain that a single training refinement couldn’t result in some sort of spark - curiosity, desire to be introspective, whatever.
Perhaps Hofstadter is losing his mind - but I think we should at least consider the possibility that his concern is warranted. We are not special.
LLMs already do quite a few things they were not designed to do.
No; they do exactly what they were designed to do, which is convert words to vectors, do math with them, and convert it back again. That we’ve find more utility in this use does not change their design.
What if “the internet” developed some form of self-awareness - would we know?
Uh what? Like how would it? This is just technomystical garbage. Enough data in one place and enough CPU in one place doesn’t magically make that place sentient. I love it as a book idea, but this is real life.
What about feedback and ability to self-modify?
This would be a significant design divergence from what LLMs are, so I’d call those things something different.
But in any event that still would not actually give LLMs anything approaching: thoughts, feelings, or rationality. Or even the capability to understand what they were operating on. Again, they have none of those things and they aren’t close to them. They are word completion algorithms.
Humans are not word completion algorithms. We have an internal existence and thought process that LLMs do not have and will never have.
Perhaps at some point we will have true artificial intelligence. But LLMs are not that, and they are not close.
You apply a reductionist view to LLMs that you do not apply to humans.
LLMs receive words and produce the next word. Humans receive stimulus from their senses and produce muscle movements.
LLMs are in their infancy, but I’m not convinced their “core loop”, so to speak, is any more basic than our own.
In the world of text: text in -> word out
In the physical word: sense stimulation in -> muscle movement out
There’s nothing more to it than that, right?
Well, actually there is more to it than that, we have to look at these things on a higher level. If we believe that humans are more than sense stimulation and muscle movements, then we should also be willing to believe that LLMs are more than just a loop producing one word at a time. We need to assess both at the same level of abstraction.
They have no core loop. You are anthropomorphizing them. They are literally no more self-directed than a calculator, and have no more of a “core loop” than a calculator does.
Do you believe humans are simply very advanced and very complicated calculators? I think most people would say “no.” While humans can do mathematics, we are different entirely to calculators. We experience sentience; thoughts, feelings, emotions, rationality. None of the devices we’ve ever built, no matter how clever, has any of those things: and neither do LLMs.
If you do think humans are as deterministic as a calculator then I guess I don’t know what to tell you other than I disagree. Other people actually exist and have internal realities. LLMs don’t. That’s the difference.
As a programmer I can confirm that LLMs definitely have loops. Look at the code, look at the algorithms, you will see the loops. The “core loop” in the LLM algorithm is “read the context, produce the next work, read the context, produce the next word”.
The core loop in animals is “receive stimulus using senses, move muscles, receive stimulus using senses, move muscles”. That’s all humans do, that’s all animals do.
I think there’s a possibility that humans are simply very advance machines. Look at the debate over whether humans have free will, it’s an interesting question and the important take away is that we still have a lot to learn about our brains and physics. I don’t want to get into that though.
You’ve ignored my main complaint. I said that you treat LLMs and humans at different levels of abstraction:
It’s not fair to say that LLMs simply predict the next word and humans have feelings and reason.
It would be fair though, to say that LLMs simply predict the next word and humans simply bounce electric-chemical signals between neurons and move muscles.
I don’t think that way about people or LLMs though. I think people have feeling and reason, and I think LLMs reason too. LLMs aren’t the same as people and aren’t as good though. But LLMs are good enough to say that they can “reason” in my experience[0].
[0]: I formed this opinion when learning linear algebra from GPT4. It was quite a good teacher. The textbook I’m using made a mistake that GPT4 caught. I encountered a proof that GPT4 wasn’t aware of, and GPT4 wouldn’t agree with me that C(A) = C(AA^T) until I explained the proof, and then GPT4 could finally reason for itself and see for itself that C(A) = C(AA^T). As an experiment, I started a new GPT4 session and repeated the experiment using a faulty proof, but I wasn’t able to convince GPT4 with a faulty proof, it was able to reason through the math concepts well enough to recognize when a mathematical proof was faulty and could not be convinced by a faulty proof. I tried this experiment 4 or 5 times. To be clear, what happened here is that GPT4 was able to learn a near math concept in one shot (within a single context window), but only if accompanied by a proper mathematical proof, and was smart enough to recognize faulty proofs as being faulty. To me, that rises to the level of “reason”.
Suppose you were saying that about me. How would I prove you wrong? How could a thinking being express that it is actually sentient to meet your standards?
If you ask ChatGPT if it is sentient, or has any thoughts, or experiences any feelings, what is its response?
But suppose it’s lying.
We also understand the math underlying it. Humans designed and constructed it; we know exactly what it is capable of and what it does. And there is nothing inside it that is capable of thought or feeling or even rationality.
It literally can’t worry about its own existence; it can’t worry about anything because it has no thoughts or feelings. Adding computational power will not miraculously change that.
Who cares? This has no real world practical usecase. Its thoughts are what it says, it doesn’t have a hidden layer of thoughts, which is quite frankly a feature to me. Whether it’s conscious or not has nothing to do with its level of functionality.
I don’t believe in scaling as a way to discover understanding. Doing that is just praying that the machine comes alive… these machines weren’t programmed to come alive in that way. That’s my fundamental argument, the design of LLMs ignores understanding of the content… it doesn’t matter how much content it’s been scaled up to.
If I teach a real AI about fishing, it should be able to reason about fishing and it shouldn’t need to have read a supplementary knowledge of mankind to do it.
What the LLMs seem to be moving towards is more of a search and summary engine (for existing content). That’s a very similar and potentially quite useful thing, but it’s not the same thing as understanding.
It’s the difference between the kid that doesn’t know much but is really good at figuring it out based on what they know vs the kid that’s read all the text books front to back and can’t come up with anything original to save their life but can quickly regurgitate and summarize anything they’ve ever read.
If I teach a real AI about fishing, it should be able to reason about fishing and it shouldn’t need to have read a supplementary knowledge of mankind to do it.
This is a faulty assumption.
In order for you to learn about fishing, you had to learn a shitload about the world. Babies don’t come out of the womb able to do such tasks, there is a shitload of prerequisite knowledge in order to fish, it’s unfair to expect an ai to do this without prerequisite knowledge.
Furthermore, LLM’s have been shown to do many things that aren’t in their training data, so the notion that it’s a stochastic parrot is also false.
Furthermore, LLM’s have been shown to do many things that aren’t in their training data, so the notion that it’s a stochastic parrot is also false.
And (from what I’ve seen) they get things wrong with extreme regularity, increasingly so as thing diverge from the training data. I wouldn’t say they’re a “stochastic parrot” but they don’t seem to be much better when things need to be correct… and again, based on my (admittedly limited) understanding of their design, I don’t anticipate this technology (at least without some kind of augmented approach that can reason about the substance) overcoming that.
In order for you to learn about fishing, you had to learn a shitload about the world. Babies don’t come out of the womb able to do such tasks, there is a shitload of prerequisite knowledge in order to fish, it’s unfair to expect an ai to do this without prerequisite knowledge.
That’s missing the forest for the trees. Of course an AI isn’t going to go fishing. However, I should be able to assert some facts about fishing and it should be able to reason based on those assertions. e.g. a child can work off of facts presented about fishing, “fish are hard to catch in muddy water” -> “the water is muddy, does that impact my chances of a catching a bluegill?” -> “yes, it does, bluegill are fish, and fish don’t like muddy water”.
There are also “teachings” brought about by how these are programmed that make the flaws less obvious, e.g., if I try to repeat the experiment in the post here Google’s Bard outright refuses to continue because it doesn’t have information about Ryan McGee. I’ve also seen Bard get notably better as it’s been scaled up, early on I tried asking it about RuneScape and it spewed absolute nonsense. Now… It’s reasonable-ish.
I was able to reproduce a nonsense response (once again) by asking about RuneScape. I asked how to get 99 firemaking, and it invented a mechanic that doesn’t exist “Using a bonfire in the Charred Stump: The Charred Stump is a bonfire located in the Wilderness. It gives 150% Firemaking experience, but it is also dangerous because you can be attacked by other players.” This is a novel (if not creative) invention of Bard likely derived from advice for training Prayer (which does have something in the Wilderness which gives 350% experience).
And (from what I’ve seen) they get things wrong with extreme regularity, increasingly so as thing diverge from the training data. I wouldn’t say they’re a “stochastic parrot” but they don’t seem to be much better when things need to be correct… and again, based on my (admittedly limited) understanding of their design, I don’t anticipate this technology (at least without some kind of augmented approach that can reason about the substance) overcoming that.
Keep in mind, you’re talking about a rudimentary, introductory version of this, my argument is that we don’t know what will happen when they’ve scaled up, we know for certain hallucinations become less frequent as the model size increases (see the statistics on gpt3 vs 4 on hallucinations), perhaps this only occurs because they haven’t met a critical size yet? We don’t know.
There’s so much we don’t know.
That’s missing the forest for the trees. Of course an AI isn’t going to go fishing. However, I should be able to assert some facts about fishing and it should be able to reason based on those assertions. e.g. a child can work off of facts presented about fishing, “fish are hard to catch in muddy water” -> “the water is muddy, does that impact my chances of a catching a bluegill?” -> “yes, it does, bluegill are fish, and fish don’t like muddy water”.
I think we might be, I remember hearing openAI was training on so much literary data that they didn’t and couldn’t find enough for testing the model. Though I may be misrememberimg.
No that’s definitely the case. However, Microsoft is now working making LLM’s more dependent on several high quality sources. For example: encyclopedias will be more important sources than random reddit posts.
There are still plenty of videos to watch and games to play. We might be running short on books, but there are many other sources of information that aren’t accessible to LLMs at the moment.
Also just because the training set contained most of the books, doesn’t mean the model itself was large enough to learn from all of them. The more detailed your questions get, the bigger the change it will get them wrong, even if that knowledge should have been in the training set. For example ChatGPT as walkthrough for games is pretty terrible, even so there should be more than enough walkthroughs in the training set to learn from, same for summarizing movies, it will do the most popular ones, but quickly fall apart with anything a little lesser known.
There is of course also the possibility that using the LLM as knowledge store by itself is a bad idea. Humans use books for that, not their brain. So an LLM that is very good at looking things up in a library could answer a lot more without the enormous models size and training cost.
Basically, there are still a ton of unexplored areas, even if we have collected all the digital books.
We don’t know that for sure yet, we saw a lot of emergent intelligent properties appear as we scaled up, and we’re nowhere near done scaling LLM’s, I’m not saying it will be solved, just that we don’t know one way or the other yet.
LLMs are fundamentally different from human consciousness. It isn’t a problem of scale, but kind.
They are like your phone’s autocomplete, but very very good. But there’s no level of “very good” for autocomplete that makes it a human, or will give it sentience, or allow it to understand the words it is suggesting. It simply returns the next most-likely word in a response.
If we want computerized intelligence, LLMs are a dead end. They might be a good way for that intelligence to speak pretty sentences to us, but they will never be that themselves.
So for context, I am an applied mathematician, and I primarily work in neural computation. I have an essentially cursory knowledge of LLMs, their architecture, and the mathematics of how they work.
I hear this argument, that LLMs are glorified autocomplete and merely statistical inference machines and are therefore completely divorced from anything resembling human thought.
I feel the need to point out that not only is there no compelling evidence that any neural computation that humans do anything different from a statistical inference machine, there’s actually quite a bit of evidence that that is exactly what real, biological neural networks do.
Now, admittedly, real neurons and real neural networks are way more sophisticated than any deep learning network module, real neural networks are extremely recurrent and extremely nonlinear, with some neural circuits devoted to simply changing how other neural circuits process signals without actually processing said signals on their own. And in the case of humans, several orders of magnitude larger than even the largest LLM.
All that said, it boils down to an insanely powerful statistical machine.
There are questions of motivation and input: we all want to stay alive (ish), avoid pain, and have constant feedback from sensory organs while a LLM just produces what it was supposed to. But in an abstraction the ideas of wants and needs and rewards aren’t substantively different from prompts.
Anyway. I agree that modern AI is a poor substitute for real human intelligence, but the fundamental reason is a matter of complexity, not method.
Some reading:
Large scale neural recordings call for new insights to link brain and behavior
A unifying perspective on neural manifolds and circuits for cognition
a comparison of neuronal population dynamics measured with calcium imaging and electrophysiology
If you truly believe humans are simply autocompletion engines then I just don’t know what to tell you. I think most reasonable people would disagree with you.
Humans have actual thoughts and emotions; LLMs do not. The neural networks that LLMs use, while based conceptually in biological neural networks, are not biological neural networks. It is not a difference of complexity, but of kind.
Additionally, no matter how many statistics, CPU power, or data you give an LLM, it will not develop cognition because it is not designed to mimic cognition. It is designed to link words together. It does that and nothing more.
A dog is more sentient than an LLM in the same way that a human is more sentient than a toaster.
In a more diplomatic reading of your post, I’ll say this: Yes, I think humans are basically incredibly powerful autocomplete engines. The distinction is that an LLM has to autocomplete a single prompt at a time, with plenty of time between the prompt and response to consider the best result, while living animals are autocompleting a continuous and endless barrage of multimodal high resolution prompts and doing it quickly enough that we can manipulate the environment (prompt generator) to some level.
Yeah biocomputers are fucking wild and put silicates to shame. The issue I have is with considering biocomputation as something that fundamentally cannot be be done by any computational engine, and as far as neural computation is understood, it’s a really sophisticated statistical prediction machine
We all want to believe that humans, or indeed animals as a whole, have some secret special sauce that makes us fundamentally distinguishable from statistical algorithms that approximate a best fit function according to some cost metric, but the fact of the matter is we don’t.
There is no science to support the idea that biological neurons are particularly special, and there are reams and reams of papers suggestin that real neural cognition is little more than an extremely powerful statistical machine.
I don’t care about what “most reasonable people” think. “Most reasonable people” don’t have an opinion about the axiom of choice, or the existence of central pattern generators. That’s not to devalue them but their opinions on things this far outside of their expertise are worth about as much as my opinions on the concept of art. I am a professional in neural computation, and I put it to you to even hypothesize about how animal neural computation is fundamentally distinct from LLM computation.
Like I said, we are wildly more capable than GPT, because our hardware is wildly more complex than any ANN, but the fundamental computing strategy is not all that different.
“most reasonable people” - indirect ad hominem is still ad hominem. You are making a fool of yourself.
You’re guessing, you don’t actually know that for sure, it seems intuitively correct, but we simply do not know enough about cognition to make that assumption.
Perhaps our ability to reason exclusively comes from our ability to predict, and by scaling up the ability to predict, we become more and more able to reason.
These are guesses, all we have now are guesses, you can say “it doesn’t reason” and “it’s just autocorrect” all you want, but if that were the case why did scaling it up eventually enable it to perform basic math? Why did scaling it up improve its ability to problemsolve significantly (gpt3 vs gpt4), there’s so many unknowns in this field, to just say “nah, can’t be, it works differently from us” doesn’t mean it can’t do the same things as us given enough scale.
I’m not guessing. When I say it’s a difference of kind, I really did mean that. There is no cognition here; and we know enough about cognition to say that LLMs are not performing anything like it.
Believing LLMs will eventually perform cognition with enough hardware is like saying, “if we throw enough hardware at a calculator, it will eventually become alive.” Even if you throw all the hardware in the world at it, there is no emergent property of a calculator that would create sentience. So too LLMs, which really are just calculators that can speak English. But just like calculators they have no conception of what English is and they do not think in any way, and never will.
We do not know that, I challenge you to find a source for that, in fact, i’ve seen sources showing the opposite, they seem to reason in tokens, for example, LLM’s perform significantly better at tasks when asked to give a step by step reasoned explanation, this indicates that they are doing a form of reasoning, and their reasoning is limited by what I have no better term for than laziness.
https://blog.research.google/2022/05/language-models-perform-reasoning-via.html
It is your responsibility to prove your assertion that if we just throw enough hardware at LLMs they will suddenly become alive in any recognizable sense, not mine to prove you wrong.
You are anthropomorphizing LLMs. They do not reason and they are not lazy. The paper discusses a way to improve their predictive output, not a way to actually make them reason.
But don’t take my word for it. Go talk to ChatGPT. Ask it anything like this:
“If an LLM is provided enough processing power, would it eventually be conscious?”
“Are LLM neural networks like a human brain?”
“Do LLMs have thoughts?”
“Are LLMs similar in any way to human consciousness?”
Just always make sure to check the output of LLMs. Since they are complicated autosuggestion engines, they will sometimes confidently spout bullshit, so must be examined for correctness. (As my initial post discussed.)
You’re assuming i’m saying something that i’m not, and then arguing with that, instead of my actual claim.
I’m saying we don’t know for sure what they will be able to do when they’re scaled up. That’s the end of my assertion. I don’t have to prove that they will suddenly come alive, i’m not claiming they will, i’m just claiming we don’t know what will happen when they’re scaled, and they seem to have emergent properties as they scale up. Nobody has devised a way of predicting what emergent properties happen when, nobody has made any progress whatsoever on knowing what scaling up accomplishes.
Can they reason? Yes, but poorly right now, will that get better? Who knows.
The end of my claim is that we don’t know what’ll happen when they scale up, and that you can’t just write it off like you are.
If you want proof that they reason, see the research article I linked. If they can do that in their rudimentary form that we’ve created with very little time, we can’t write off the possibility that they will scale.
Whether or not they reason LIKE HUMANS is irrelevant if they can do the job.
And i’m not anthropomorphizing them without reason, there aren’t terms for this already, what would you call this behavior of answering questions significantly better when asked to fully explain reasoning? I would say it is taking the easiest option that still meets the qualifications of what it is requested to do, following the path of least resistance, I don’t have a better word for this than laziness.
https://www.downtoearth.org.in/news/science-technology/artificial-intelligence-gpt-4-shows-sparks-of-common-sense-human-like-reasoning-finds-microsoft-89429
Furthermore predictive power is just another way of achieving reasoning, better predictive power IS better reasoning, because you can’t predict well without reasoning.
It’s your job to prove your assertion that we know enough about cognition to make reasonable comparisons.
May as well ask me to prove that we know enough about calculators to say they won’t develop sentience while I’m at it.
Except calculators aren’t models capable of understanding language that appear to become more and more capable as they grow. It’s nothing like that.
Are you just fucking around here? C’mon. In your hypothetical scenario, the emergent property would not be “of a calculator”.
I am picking up a hint of the autocompletion you describe, in your writing.
I think I write well :) I am not an LLM though.
They are also fundamentally different from a toaster. But that’s completely irrelevant. Consciousness is something you get when you put intelligent in an agent that has to move around in and interact with an environment. A chatbot has no use for that, it’s just there to mush through lots of data and produce some, it doesn’t have or should worry about its own existence.
So does the all knowing oracle that predicts the lotto numbers from next week. It being autocomplete does not limit its power.
There might be better or faster approaches, but it’s certainly not a dead end. It’s a building block. Add some long term memory, bigger prompts, bigger model, interaction with the Web, etc. and you can build a much more powerful bit of software than what we have today, without even any real breakthrough on the AI side. GPT as it is today is already “good enough” for a scary number of things that used to be exclusively done by humans.
It literally can’t worry about its own existence; it can’t worry about anything because it has no thoughts or feelings. Adding computational power will not miraculously change that.
I agree this would be a very useful chatbot. But it is still not a toaster. Nor would it be conscious.
You seem unfamiliar with the concept of consciousness as an emergent property.
What if we dramatically reduce the cost of training - what if we add realtime feedback mechanisms as part of a perpetual model refinement process?
As far as I’m aware, we don’t know.
How are you so confident that your feelings are not simply a consequence of complexity?
Even if they are a result of complexity, that still doesn’t change the fact that LLMs will never be complex in that manner.
Again, LLMs have no self-awareness. They are not designed to have self-awareness. They do not have feelings or emotions or thoughts; they cannot have those things because all they do is generate words in response to queries. Unless their design fundamentally changes, they are incompatible with consciousness. They are, as I’ve said before, complicated autosuggestion algorithms.
Suggesting that throwing enough hardware at them will change their design is absurd. It’s like saying if you throw enough hardware at a calculator, it will develop sentience. But a calculator will not do that because all it’s programmed to do is add numbers together. There’s no hidden ability to think or feel lurking in its design. So too LLMs.
I believe I understand everything you are saying and why you are saying it. I think you are completely missing the point, though. LLMs already do quite a few things they were not designed to do. Also, your idea of sentience seems very limited. Yes, with our biological computers we have some degree of presence over “time”, but is that critical - or is it just critical for us due to our limited faculties.
What if “the internet” developed some form of self-awareness - would we know? Our entire society could be subtly manipulated through carefully placed latency spikes, for example. I’m not saying this is happening, just that I think you are incredibly overconfident because you have an understanding of LLMs current lack of state etc.
If we added a direct feedback mechanism - realtime or otherwise - we could start seeing more compelling emergent properties develop. What about feedback and ability to self-modify?
These systems are processing information on a level we cannot even pretend to comprehend. How can you be so certain that a single training refinement couldn’t result in some sort of spark - curiosity, desire to be introspective, whatever.
Perhaps Hofstadter is losing his mind - but I think we should at least consider the possibility that his concern is warranted. We are not special.
No; they do exactly what they were designed to do, which is convert words to vectors, do math with them, and convert it back again. That we’ve find more utility in this use does not change their design.
Uh what? Like how would it? This is just technomystical garbage. Enough data in one place and enough CPU in one place doesn’t magically make that place sentient. I love it as a book idea, but this is real life.
This would be a significant design divergence from what LLMs are, so I’d call those things something different.
But in any event that still would not actually give LLMs anything approaching: thoughts, feelings, or rationality. Or even the capability to understand what they were operating on. Again, they have none of those things and they aren’t close to them. They are word completion algorithms.
Humans are not word completion algorithms. We have an internal existence and thought process that LLMs do not have and will never have.
Perhaps at some point we will have true artificial intelligence. But LLMs are not that, and they are not close.
Are we arguing semantics here?
https://www.jasonwei.net/blog/emergence https://arxiv.org/pdf/2206.07682.pdf https://arxiv.org/pdf/2304.15004.pdf
I could be wrong, obviously, but I don’t think this is as straightforward or settled as you are suggesting.
You apply a reductionist view to LLMs that you do not apply to humans.
LLMs receive words and produce the next word. Humans receive stimulus from their senses and produce muscle movements.
LLMs are in their infancy, but I’m not convinced their “core loop”, so to speak, is any more basic than our own.
In the world of text: text in -> word out
In the physical word: sense stimulation in -> muscle movement out
There’s nothing more to it than that, right?
Well, actually there is more to it than that, we have to look at these things on a higher level. If we believe that humans are more than sense stimulation and muscle movements, then we should also be willing to believe that LLMs are more than just a loop producing one word at a time. We need to assess both at the same level of abstraction.
They have no core loop. You are anthropomorphizing them. They are literally no more self-directed than a calculator, and have no more of a “core loop” than a calculator does.
Do you believe humans are simply very advanced and very complicated calculators? I think most people would say “no.” While humans can do mathematics, we are different entirely to calculators. We experience sentience; thoughts, feelings, emotions, rationality. None of the devices we’ve ever built, no matter how clever, has any of those things: and neither do LLMs.
If you do think humans are as deterministic as a calculator then I guess I don’t know what to tell you other than I disagree. Other people actually exist and have internal realities. LLMs don’t. That’s the difference.
As a programmer I can confirm that LLMs definitely have loops. Look at the code, look at the algorithms, you will see the loops. The “core loop” in the LLM algorithm is “read the context, produce the next work, read the context, produce the next word”.
The core loop in animals is “receive stimulus using senses, move muscles, receive stimulus using senses, move muscles”. That’s all humans do, that’s all animals do.
I think there’s a possibility that humans are simply very advance machines. Look at the debate over whether humans have free will, it’s an interesting question and the important take away is that we still have a lot to learn about our brains and physics. I don’t want to get into that though.
You’ve ignored my main complaint. I said that you treat LLMs and humans at different levels of abstraction:
It’s not fair to say that LLMs simply predict the next word and humans have feelings and reason.
It would be fair though, to say that LLMs simply predict the next word and humans simply bounce electric-chemical signals between neurons and move muscles.
I don’t think that way about people or LLMs though. I think people have feeling and reason, and I think LLMs reason too. LLMs aren’t the same as people and aren’t as good though. But LLMs are good enough to say that they can “reason” in my experience[0].
[0]: I formed this opinion when learning linear algebra from GPT4. It was quite a good teacher. The textbook I’m using made a mistake that GPT4 caught. I encountered a proof that GPT4 wasn’t aware of, and GPT4 wouldn’t agree with me that C(A) = C(AA^T) until I explained the proof, and then GPT4 could finally reason for itself and see for itself that C(A) = C(AA^T). As an experiment, I started a new GPT4 session and repeated the experiment using a faulty proof, but I wasn’t able to convince GPT4 with a faulty proof, it was able to reason through the math concepts well enough to recognize when a mathematical proof was faulty and could not be convinced by a faulty proof. I tried this experiment 4 or 5 times. To be clear, what happened here is that GPT4 was able to learn a near math concept in one shot (within a single context window), but only if accompanied by a proper mathematical proof, and was smart enough to recognize faulty proofs as being faulty. To me, that rises to the level of “reason”.
Suppose you were saying that about me. How would I prove you wrong? How could a thinking being express that it is actually sentient to meet your standards?
By telling me you are.
If you ask ChatGPT if it is sentient, or has any thoughts, or experiences any feelings, what is its response?
But suppose it’s lying.
We also understand the math underlying it. Humans designed and constructed it; we know exactly what it is capable of and what it does. And there is nothing inside it that is capable of thought or feeling or even rationality.
It is a word generation algorithm. Nothing more.
This is false. Read about their emergent properties. We have no way of knowing when emergent properties appear, we just notice them.
Who cares? This has no real world practical usecase. Its thoughts are what it says, it doesn’t have a hidden layer of thoughts, which is quite frankly a feature to me. Whether it’s conscious or not has nothing to do with its level of functionality.
I don’t believe in scaling as a way to discover understanding. Doing that is just praying that the machine comes alive… these machines weren’t programmed to come alive in that way. That’s my fundamental argument, the design of LLMs ignores understanding of the content… it doesn’t matter how much content it’s been scaled up to.
If I teach a real AI about fishing, it should be able to reason about fishing and it shouldn’t need to have read a supplementary knowledge of mankind to do it.
What the LLMs seem to be moving towards is more of a search and summary engine (for existing content). That’s a very similar and potentially quite useful thing, but it’s not the same thing as understanding.
It’s the difference between the kid that doesn’t know much but is really good at figuring it out based on what they know vs the kid that’s read all the text books front to back and can’t come up with anything original to save their life but can quickly regurgitate and summarize anything they’ve ever read.
This is a faulty assumption.
In order for you to learn about fishing, you had to learn a shitload about the world. Babies don’t come out of the womb able to do such tasks, there is a shitload of prerequisite knowledge in order to fish, it’s unfair to expect an ai to do this without prerequisite knowledge.
Furthermore, LLM’s have been shown to do many things that aren’t in their training data, so the notion that it’s a stochastic parrot is also false.
And (from what I’ve seen) they get things wrong with extreme regularity, increasingly so as thing diverge from the training data. I wouldn’t say they’re a “stochastic parrot” but they don’t seem to be much better when things need to be correct… and again, based on my (admittedly limited) understanding of their design, I don’t anticipate this technology (at least without some kind of augmented approach that can reason about the substance) overcoming that.
That’s missing the forest for the trees. Of course an AI isn’t going to go fishing. However, I should be able to assert some facts about fishing and it should be able to reason based on those assertions. e.g. a child can work off of facts presented about fishing, “fish are hard to catch in muddy water” -> “the water is muddy, does that impact my chances of a catching a bluegill?” -> “yes, it does, bluegill are fish, and fish don’t like muddy water”.
There are also “teachings” brought about by how these are programmed that make the flaws less obvious, e.g., if I try to repeat the experiment in the post here Google’s Bard outright refuses to continue because it doesn’t have information about Ryan McGee. I’ve also seen Bard get notably better as it’s been scaled up, early on I tried asking it about RuneScape and it spewed absolute nonsense. Now… It’s reasonable-ish.
I was able to reproduce a nonsense response (once again) by asking about RuneScape. I asked how to get 99 firemaking, and it invented a mechanic that doesn’t exist “Using a bonfire in the Charred Stump: The Charred Stump is a bonfire located in the Wilderness. It gives 150% Firemaking experience, but it is also dangerous because you can be attacked by other players.” This is a novel (if not creative) invention of Bard likely derived from advice for training Prayer (which does have something in the Wilderness which gives 350% experience).
Keep in mind, you’re talking about a rudimentary, introductory version of this, my argument is that we don’t know what will happen when they’ve scaled up, we know for certain hallucinations become less frequent as the model size increases (see the statistics on gpt3 vs 4 on hallucinations), perhaps this only occurs because they haven’t met a critical size yet? We don’t know.
There’s so much we don’t know.
https://blog.research.google/2022/05/language-models-perform-reasoning-via.html
they do this already, albeit imperfectly, but again, this is like, a baby LLM.
and just to prove it:
https://chat.openai.com/share/54455afb-3eb8-4b7f-8fcc-e144a48b6798
I think we might be, I remember hearing openAI was training on so much literary data that they didn’t and couldn’t find enough for testing the model. Though I may be misrememberimg.
No that’s definitely the case. However, Microsoft is now working making LLM’s more dependent on several high quality sources. For example: encyclopedias will be more important sources than random reddit posts.
Microsoft is also using LinkedIn to help as well, getting users to correct articles generated by AI.
Cunningham’s Law may be very helpful in this respect.
There are still plenty of videos to watch and games to play. We might be running short on books, but there are many other sources of information that aren’t accessible to LLMs at the moment.
Also just because the training set contained most of the books, doesn’t mean the model itself was large enough to learn from all of them. The more detailed your questions get, the bigger the change it will get them wrong, even if that knowledge should have been in the training set. For example ChatGPT as walkthrough for games is pretty terrible, even so there should be more than enough walkthroughs in the training set to learn from, same for summarizing movies, it will do the most popular ones, but quickly fall apart with anything a little lesser known.
There is of course also the possibility that using the LLM as knowledge store by itself is a bad idea. Humans use books for that, not their brain. So an LLM that is very good at looking things up in a library could answer a lot more without the enormous models size and training cost.
Basically, there are still a ton of unexplored areas, even if we have collected all the digital books.