The notion that the meaning is intrinsic is hogwash.
A word without someone ascribing it meaning means nothing.
A word has meaning because we give it to it, and we give it that meaning based on context.
Some of that context is remote in time and space (we learned the word ball a long time ago), some is near (that it occurs near "back yard" and "round rubbery thing" makes it more likely it's that thing we use to play games rather than a testicle or a party where people dance), but it is wholly dependent on context.
Put it alongside words in a different language, for example, and it might have a whole other set of potential meanings.
If I understand correctly, that's what the OP, and others, are claiming, that it's possible to figure out how to represent meaning by putting words next to each other, if you look for long enough, at many enough words sitting next to other words. So far this remains a matter of debate.
The objection I though you were making, with the "ball" etc, is that all these words that sit next to other words according to what the words mean, must mean something in isolation. For example, even though "ball" can be used in different contexts, it can only mean so many things, and it would never mean, say, "a cooking utensil made of aluminum where I fry my eggs each morning" no matter what other words you put it next to.
And, as Young et al (1976) have demonstrated, it is possible to move words out of their expected context for great fun and profit, but it's not clear that a word can always be moved in any context, and still make sense. I would even go so far as to say that can probably not be done, at least not without a drastic reconfiguration of all of English (where "ball" is used).
So the question is not only "what other words do we find the word 'ball' close to" but also "why do we find 'ball' next to those other words?". The latter question can't be answered just by looking at what words hang out with what other words, unless we already know what words mean on their own.
And that is, at least for me, the objection about LLMs "meaning" and "understanding" anything. No matter what text an LLM is generating, the entity that is decoding the meaning of the generating text is always a human. The LLM can't do that on its own. Because there is no mechanism that it is equipped with that could ever do that.
P.S. "words hanging out with other words" are what are known as "token collocations" in technical jargon. They are an idea as old as linguistics itself, a basal concept whose modern implementations in NLP systems are just, well, modern implementations. We still don't know how humans decode meaning from token collocations even though it's such an ancient idea. And it might even be wrong, at this point.
_________________
Bibliography
1. Angus Young, Malcolm Young, and Bon Scott (1976) Big Balls. In Harry Vanda, George Young (prod), Dirty Deeds Done Dirt Cheap, Albert productions - Atlantic Records, Sydney, Australia, side 2, track 2. URL: https://youtu.be/4WwJ6OVSwkM
> If I understand correctly, that's what the OP, and others, are claiming, that it's possible to figure out how to represent meaning by putting words next to each other, if you look for long enough, at many enough words sitting next to other words. So far this remains a matter of debate.
The fact that we can have conversations with GPT where these models are able to correctly e.g. write working code and symbolically evaluate code is sufficient proof of that assertion that it is possible to figure this out that I don't think there is any reasonable basis for debating this any more. Claiming it's not possible is just demonstrably wrong.
> And, as Young et al (1976) have demonstrated, it is possible to move words out of their expected context for great fun and profit, but it's not clear that a word can always be moved in any context, and still make sense. I would even go so far as to say that can probably not be done, at least not without a drastic reconfiguration of all of English (where "ball" is used).
I don't see what this has to do with anything. Yes, we can shift around, alter their meanings, use them in unusual contexts and still make sense of it, and yes of course they won't always make sense in every context. How is that relevant?
> The objection I though you were making, with the "ball" etc, is that all these words that sit next to other words according to what the words mean, must mean something in isolation. For example, even though "ball" can be used in different contexts, it can only mean so many things, and it would never mean, say, "a cooking utensil made of aluminum where I fry my eggs each morning" no matter what other words you put it next to.
If I use it to mean "a cooking utensil", then it means a cooking utensil when I communicate with others that have that shared context. It has no meaning separate from context.
> So the question is not only "what other words do we find the word 'ball' close to" but also "why do we find 'ball' next to those other words?". The latter question can't be answered just by looking at what words hang out with what other words, unless we already know what words mean on their own.
"What words mean on their own" makes no sense. No word has a meaning separate from context. Without context there's nothing to assign them meaning. And that context is interactions. For GPT purely words. For us, words and other sensory input. In either case, without context assigning attributes to the word "ball" it is nothing more than a meaningless a sequence of letters. What does "JMw3rfd" mean? What's its intrinsic meaning? Until I tell you that this is my new word for "ball" it has none. Once I do, it has an extrinsic meaning. It's one that isn't widespread and will soon be forgotten, but the meaning exists the moment we label it, and only once we label it, and it is purely extrinsic.
> And that is, at least for me, the objection about LLMs "meaning" and "understanding" anything. No matter what text an LLM is generating, the entity that is decoding the meaning of the generating text is always a human. The LLM can't do that on its own. Because there is no mechanism that it is equipped with that could ever do that.
We don't know enough about human reasoning, and so how close how LLMs are to how human reasoning works to be able to even begin to determine whether this is true or false.
>> The fact that we can have conversations with GPT where these models are able to correctly e.g. write working code and symbolically evaluate code is sufficient proof of that assertion that it is possible to figure this out that I don't think there is any reasonable basis for debating this any more.
Then there's no reason to continue this conversation.
A word without someone ascribing it meaning means nothing.
A word has meaning because we give it to it, and we give it that meaning based on context.
Some of that context is remote in time and space (we learned the word ball a long time ago), some is near (that it occurs near "back yard" and "round rubbery thing" makes it more likely it's that thing we use to play games rather than a testicle or a party where people dance), but it is wholly dependent on context.
Put it alongside words in a different language, for example, and it might have a whole other set of potential meanings.