This is literally the accepted term for the nodes wherein sums are produced of the set of weighted products which connect such nodes together. If you know a better term, or would prefer a better term that "neuron" to describe these points in a "neural network", by all means speak your mind, but don't pretend some offense at it.
>and that's because you restricted yourself to facts.
I restricted myself to writing from a point of view concordant with your own because that is what one does when explaining a position well, not because I agree with your point of view.
>You could not give a factual description with the word "meaning" in it (in the sense of "the meaning of...").
If we're going to play a game of semantics, than so be it.
If your argument is simply that symbols and numbers can't contain meaning because they are symbols and numbers, then you've worked your way into a tautological box that I, nor anyone else, will be able to free you from.
It is known that memories in biological neuron-based networks are not locally stored or unique to some neuron ( there is no "grandma" neuron in your head to keep track of dear old grannie ), but that instead memories are spread across the set of neurons there available.
This sort of representation in known as a "holographic" representation, which I expect our neural net will also utilize. Information will not be stored in any specific neurons in the trained model, but will be spread across the whole of them.
But even if true, that is merely data, you will no doubt point out. Where is the "meaning"?
As you want to delve philosophically into it, we must question the very nature of the word meaning here. Our model lacks any form of experiential qualia, and so cannot associate words with experienced reality, which is the normal basis for human concepts of meaning, which then underlie our development out of concrete language and into metaphoric.
This leaves meaning within the model as only being the derived relationship between words.
Words, on their own and bereft of qualia, can present our model with structure, grammar, punctuation. Words will have certain uses within these structures and grammars, they will have certain places. Words will modify words, phrases will modify larger sentences. Trues can be asserted and falsehoods. Truth and falseness itself can be demonstrated through sentences making claims about other sentences.
Would the model know what a red ball is? Of course not. But it would be aware of how the term ball is used, what sort of type it is given, the qualifications and usages of words of that type. The modifiers it is reasonable to use on it, etc.
And incredibly, with no basis in anything other than being slightly corrected for each incorrect token generated, our training harness manages to discover a mathematical neural net model of these relationships between words such that it can manipulate them as well or better than many people.
The use of backpropagation and gradient descent allows us to search the space of all possible models starting from a randomized model, and find a model that does this by nudging our model to be a little more capable at it over and over again.
However, English is not just a fixed mathematical set of symbols that can be rotely manipulated.
Our language is metaphoric from its very foundations. Our hearts soar, sink and swell in ways that have nothing to do with the reality, but only our perceptions of ourselves.
To manipulate English as it does, the traversal of the model-space must advance inexorably on a model capable of handling such non-literal language.
The model contains meanings for car related actions, for curling related actions, it contains the ability to translate between them when the only connection is a meaning that is implied by the words but not either of their literal meanings.
What is a meaning here? It is a relationship between the words. The relationship between the car words and curling words only exists as the relationship to a common implied meaning that can be transformed under the context of other words.
Where is the meaning? We don't even know for certain where the meaning is inside ourselves. I expect it to be similarly holographically stored across the set of neurons and provide the appropriate context to twist the swirl of mathematics that occurs between input vector and output.
>So I still do not know why you think all this is going on inside a language model.
Because it is the simplest explanation of what it is doing. Call it "latent spaces", "embeddings" and the assume the compression I discuss is via "manifold hypothesis".
>What do language models have to do with meaning, and why do we absolutely need to explain their behaviour by talking about meaning? Where is this information in your comment? I cannot see it.
So, where is the meaning? It is everywhere in the model once trained. It creates a holographically stored mapping of reality as presented through the corpus of human thought in the form of writing based solely on being iteratively nudged through model-space.
An amazing feat by the engineers that dreamed it up.
If you are unwilling to call this mapping model of reality built only of tokens/symbols/words "meaning", then I have no argument for you, except to again conclude that you've locked yourself in a tautological box.
If your argument is merely "ha ha, you're merely conjecturing this", then yes, of course I am. Why? Because it behooves someone to hypothesize, test, and study things which interest them.
Do you have some competing idea of how it works? It would behoove you to share it with the thread.
As it is, I only see you making demands for proof of things that you are well aware that no human has yet completely modeled, even after decades of studying such phenomena. What you gain from this incessant nay-saying, I do not understand.
>> As it is, I only see you making demands for proof of things that you are well aware that no human has yet completely modeled, even after decades of studying such phenomena.
What you gain from this incessant nay-saying, I do not understand.
I just wanted to comment on this. I've just finished a PhD (in AI, no less) and I've learned a few things about "nay-sayers", which in research are more formally critics and reviewers. What I've learned is that the "nay-sayers" are your friends. They're your friends even if they don't know it, even if their incessant nagging is driving you mad. They're your friends, because they give you the opportunity to see your work as it really is, not as the product of an unrecognised genious, but as the clumsy fumblings in the dark of a clueless and confused limited human mind. And they're your friends because they help you figure out where it is that you're wrong, and, ultimately, to improve your work.
It took me a long time to see things that way, it took many "dark tea-times of the soul" to figure out where I was wrong, when my critics were right, and how to improve my work to address the criticisms. And, you know the funny thing? Once I did that, all the frustration I felt at the "nay-saying" evaporated. The critics continued "nay-saying", but I just don't care, because I have proved to myself, and to others besides, that the criticisms have been addressed. The critics now have to find another angle of attack, and I'm disappointed as long as they can't, because that means my own understanding will stagnate and my work will not progress. Critics are worth their weight in gold.
But all that works much better in an academic environment where the critics and that "nay-sayers" are experts in their field. On the internet anyone can say anything and everybody pretends to know everything all the time, so all that we're left with is frustration. So I apologise for the frustration I discern in your use of "nay-sayers". This kind of conversation doesn't work well on the internet.
Part of why I comment is specifically to attract naysayers. I can learn from them. I've never been one to mind being wrong. I will argue vehemently for a position both because it seems correct to me, but also because it exposes my ideas to be torn down, so when I rebuild them they can be better.
I looked into things a number of people just in this thread brought up. It was a very rewarding discussion for me. I feel like my understanding is better than before it.
I became frustrated with you because your stance seemed a combination of "you can't prove it" and "it works like it does because it is what it is". You weren't expanding on your positions. It is possible that you are assuming context others in the thread do not share. It seems likely you think many here are reading into or seeing things that aren't actually present. Being tricked by the model, essentially. It is possible. But it also seemed possible that you were simply enjoying telling everyone they are wrong for fun.
With your refusal to expand on anything, it all looks much the same from this side.
I would like to understand what perspective you were arguing from, because even if I disagree with it in the end, knowing your perspective will allow me to understand where my own may be weak or flawed, and either shore it up or discard and rebuild it.
It is very frustrating to have a discussion with someone who seems to only contradict, but never explain their contradictions. They may be right. You may be right. But it was frustrating to talk to you because you didn't seem to want to say where or how people were wrong.
"It's not a brain". "It's works because it's a language model".
Okay, it's not a brain and it's just a language model. How does it perform what appears to be, whether it is or not, obvious translation of subtext in sentences between unrelated literal domains?
When I specifically asked if your position was simply that you would not attribute "meaning" to its representations, you declined to respond.
That's not an unfair position to take. An LLM has no experiential data in it. It has only the words. Philosophically, one could argue that even if it were "proven intelligent", it would still have no knowledge or contain actual meaning because it has only the words to go by. That would be a reasonable position, but you wouldn't confirm if that was what perspective you were taking.
I genuinely would enjoy knowing how you think translations on the level of the one I presented happened. Another poster discussed "latent spaces" where terms can grow closer even if not literally related. Articles I've read indicate any number of surprising results from LLms, assertions that LLMs can or cannot reason, various levels of assertion on whether LLMs can "understand" things. It appears like there are many perspectives active in this field all at once, and that they are quickly out of date as better and better LLMs are trained.
If you would be so kind, please add context to when you correct others. Pointers we can follow in order to understand your perspective.
For much of the thread, it felt like you were arguing that deep blue can't actually play chess because it has no concept of games or people or chess pieces or winning or losing, rather it only performs statistical analyses of numerically represented move sets and just outputs a move based on that statistical analysis. How can it play chess, after all, if it doesn't know what chess is?
>This kind of conversation doesn't work well on the internet
Possibly. I appreciate your having it nonetheless, as I have no one I know that cares to discuss any of this. So I have to make due with discussing it in online forums :)
Thanks for your comment! I gave my word I'd let you have the last word so I'm
not going to continue arguing. I'm just happy because I'm getting the feeling
you are not frustrated at our conversation anymore. I hope we can continue it
at some point and that I'll do a better job explaining things to you.
Especially when it comes to your big question about how can language models do
the kind of thing you illustrate with the translation example.
This conversation was interesting. I pasted it into ChatGPT (GPT-4) and asked it some questions. Here's one of its replies:
ChatGPT:
To better explain Knome's point to YeGoblynQueenne, I would provide a more concrete example and focus on the mechanisms behind LLMs that give rise to the appearance of meaning.
Large language models like GPT-3 are trained on vast amounts of text data, learning to predict the next word in a sentence based on context. This process enables the model to capture patterns, relationships, and regularities in the data, which can be seen as a form of "meaning" or understanding of language. The model's neural network, with its numerous layers, neurons, and connections, essentially forms a high-dimensional space that maps the relationships between words and their contexts.
To make Knome's argument more concrete, let's consider a specific example: the model's ability to translate the metaphor "spinning my tires" into "sweeping the ice." While it's true that the model doesn't have a direct understanding of the real-world experiences or objects the words represent, it has learned the contextual relationships between words and their usage in various situations. The model can associate "spinning my tires" with the concept of being stuck or not making progress, and "sweeping the ice" with a similar concept in the context of curling. This association is not a direct result of knowing the literal meanings of the words but rather of having learned the patterns and relationships in the data.
In essence, the "meaning" Knome refers to in LLMs is not the same as the meaning humans typically ascribe to language, which is derived from direct experiences and real-world knowledge. Instead, it is a form of abstract, statistical understanding that emerges from the relationships between words, their contexts, and the patterns in the data. The model essentially learns a high-dimensional representation of these relationships, which can be thought of as an approximation of meaning within the context of the model's training data.
Thus, when discussing meaning in the context of LLMs, it is essential to recognize that the term refers to an approximation of linguistic relationships and patterns rather than a direct understanding of real-world experiences. This approximation allows the model to manipulate language in a way that seems meaningful to humans, even though it lacks direct experiential knowledge of the world.
In summary, Knome's argument can be made more concrete and rigorous by focusing on the mechanisms behind LLMs, such as the high-dimensional representation of linguistic relationships, and by clarifying that the "meaning" in LLMs is an approximation derived from patterns and associations in the data rather than a direct understanding of real-world experiences.
>> If your argument is simply that symbols and numbers can't contain meaning because they are symbols and numbers, then you've worked your way into a tautological box that I, nor anyone else, will be able to free you from.
Too many assumptions.
>> Do you have some competing idea of how it works? It would behoove you to share it with the thread.
Yes, I do. It's a language model. It is trained to model the text in a corpus. There's no mystery there and there's no reason to assume it's doing anything else than what it was designed to do. It's a machine. People made it. It does what it was made to do. Other explanations of its behaviour are fantasies.
And computers aren't brains. Brains may be computers, or they may not, but you can't look at brains and draw conclusions about computers. For the same reason that you can't look at planes and draw conclusions about birds.
>> The use of backpropagation and gradient descent allows us to search the space of all possible models starting from a randomized model, and find a model that does this by nudging our model to be a little more capable at it over and over again.
If you haven't already, you should try implementing backpropagation, or even just gradient descent/hill climb, and observe its behaviour for a while. I suspect that, if you haven't done that, the experience will clear a lot of the rosy clouds about what backpropagation and gradient optimisation can really do.
Anyway this is getting a little too long-winded for me and I don't have time to continue the conversation. I must respectfully bow out. Last word is yours and thanks for the interaction.
> the experience will clear a lot of the rosy clouds about what backpropagation and gradient optimisation can really do.
I'm not seeing any "rosy clouds" in his description. Gradient descent is a search in model space for one that best models "meaning" relations in the data, among other things. We all know that GPT is not conscious, but it doesn't need to be. It seems obvious that it is recognizing abstract patterns and relations in the data that match our metaphorical comprehension of words and phrases, because it was designed and trained to do that, on human language that is filled with metaphorical relations. This is incredibly sophisticated mechanical pattern recognition with many layers.
> Other explanations of its behaviour are fantasies.
GPT is a type of deep neural network and the explanations I've seen in this thread have been along those lines.
>> Gradient descent is a search in model space for one that best models "meaning" relations in the data, among other things.
Not at all. Gradient optimisation "searches" for a set of parameter values that optimise (maximise or minimise) the gradient of a function. In machine learning, the function is some measure of error over a set of training data, that is to be minimised. There isn't anything in gradient optimisation that has to do with "meaning" at all.
What I think you're saying is that by training on a lot of text, a language model somehow learns to minimise its error on the meaning of the text. If you squint a bit, you'll see there's an obvious gap, there, between "text" and "meaning", and it seems to me you're trying to close that gap by alluding to how humans close it. But that's just comparing apples and oranges. As Yann LeCun pointed out recently (there was a video of a discussion with Andrew Ng posted on HN a few days ago) for a human to "train" on all the text that large language models have trained on, the human would have to read several books a day for some thousands of years. So whatever humans can do to learn how to bridge the gap between text and meaning, that's not how language models do it, and drawing an analogy between the two obscures more than it elucidates.
>> This is incredibly sophisticated mechanical pattern recognition with many layers.
It's incredibly complex. I don't know what "sophisticated" means. Is a jumbo jet "sophisticated"? I'd say it's a very complex machine, but that doesn't mean it has abilities other than the ones it was manufactured to have. Complexity is not magick, you know. We have maths for it and all. Those are even foundational maths of computer science. If we started imagining "meaning" everytime complexity cropped up in computer systems, we'd be up to our knees in it.
> There isn't anything in gradient optimisation that has to do with "meaning" at all.
That seems like a disingenuous interpretation of what I was saying. It should have been obvious and implied that I was referring to a loss function that was designed and engineered to choose a model that optimally captures meaning relations in the data (among other things). Gradient descent is directed by its loss function and that is not some random arbitrary function. For GPT it was specifically engineered for the purpose of selecting a model that captures meaning etc. These models are also trained with reinforcement learning which is a topic I don't know much about yet.
> it seems to me you're trying to close that gap by alluding to how humans close it.
I am talking about deep neural networks, not humans. GPT is a type of deep neural network. We do know something about how they work.
> It's incredibly complex. I don't know what "sophisticated" means.
I don't have time or patience for this pseudo-intellectual nitpicking. Just look it up in a dictionary. Basic English.
"Sophisticated (adjective) - (of a machine, system, or technique) developed to a high degree of complexity. E.g.
"highly sophisticated computer systems""
>> That seems like a disingenuous interpretation of what I was saying.
I hope not. It was not at all obvious to me what you were trying to say. It really doesn't help communication to use metaphors and analogies when discussing technical matters, especially when there is simple and commonly accepted terminology. Trust me, I've learned that the hard way (i.e. repeated rejections of my research articles with very frustrated comments by reviewers).
Besides, what you say is not right. Language models are not trained to optimise "a loss function that was designed and engineered to choose a model that optimally captures meaning relations in the data". Language models of the GPT line, specifically, are trained to maximise the following probability:
P(tᵢ|tᵢ₋ₖ, ..., tᵢ₋₁)
Where tᵢ is some token in the training corpus and tᵢ₋ₖ, ..., tᵢ₋₁ are the tokens in a "context window" of length k. That is actually the pre-training objective. The same probability is maximised in the supervised fine-tuning phase, but where tᵢ is replaced by a label and tᵢ₋ₖ, ..., tᵢ₋₁ is a sequence of tokens representing an instance with the given label.
You can find this information in the first GPT paper, here:
See equations (1) and (4) on page 3 (with more careful notation than mine).
Note that the above probabilities do not have anything to do with "meaning". They represent the ability of the model to, as people say, "predict the next token in a sequence". That is the only thing that GPT language models are designed and engineered to do.
You can search the entire GPT article above and you will not find any mention of "meaning". Nobody knows how to represent meaning and how to train a neural network on examples of it. It's much easier to train a neural net on text, of which there are no end of examples. That's what OpenAI has done.
>> I don't have time or patience for this pseudo-intellectual nitpicking.
Sorry, I meant that "sophisticated" is a loaded term. It's the kind of thing I hear often associated with socialites and gourmets. I find that it has no place in conversations about technology. It's the kind of expression people use when they try too hard to convince of the value of some technological artifact whose value should be clear without such exaggerations. So for example, you can talk about the complexity of large language models without having to say anything about their "sophistication", and that works well because one is measurable while the other is only a matter of opinion.
It's in the tile of the paper. "Improving Language Understanding"
So they call it "understanding". No problem. Understanding natural language requires building a representation of the meaning encoded in language. Otherwise what exactly do you think they are trying to "understand" about natural language here? GPT is not a spell-checker. It has a semantic model of language.
The stated goal is to optimize for "natural language understanding tasks", to use the exact phrasing in the article.
The article refers to "semantic similarity" which is precisely about modelling the meaning of language, by definition of the term.
Here's a definition of "semantic similarity"
"In the context of Natural Language Processing (NLP), "semantic similarity" refers to the measure of how closely related the meanings of two linguistic units, such as words, phrases, sentences, or documents, are to each other. The goal of assessing semantic similarity is to quantify the degree to which the meanings of these linguistic units overlap or share common conceptual features.
Semantic similarity plays a crucial role in various NLP tasks, including information retrieval, text summarization, document clustering, machine translation, and sentiment analysis. By understanding and quantifying the semantic relationships between linguistic units, NLP systems can provide more accurate results and better handle the complexities of natural language."
Yes, the word "meaning" is not explicitly used in the article, but it is implied. Ironically you are nitpicking over semantics that GPT can already comprehend, since it contains a model of the meaning of the word "meaning".
> They represent the ability of the model to, as people say, "predict the next token in a sequence". That is the only thing that GPT language models are designed and engineered to do.
GPT language models are designed and engineered to perform natural language understanding tasks. This is discussed in the paper that you linked to.
In order to predict or generate the next token, it uses a very super sophisticated semantic model of natural language and the previous tokens. You could also argue that a human is just "predicting the next token" when they generate a sequence of words while doing the tasks that GPT is designed to do. That misses a lot of important details. The devil is in those details.
>> It's in the tile of the paper. "Improving Language Understanding"
I have to confess I'm a little bit disappointed. I pointed you to the
OpenAI paper that has the technical details of how their models are trained,
and you chose to concentrate on the title, and the claims made about the
abilities of the model, rather than the way it actually works. The technical
details about the model's training make it clear that it is a language model,
trained to maximise the probability of a sequence of tokens, but you keep
believing that it can "understand"? Or represent meaning? I'm not even sure
what you believe.
The point is that you shouldn't believe. The OpenAI paper is not gospel. It's
a technical report. You should even be able to implement a system like the one
they describe, though at smaller scale. Maybe that will help you understand
how it really works? I don't know.
I think you don't want to know how things work. You just want to believe. And I
can't deal with faith. Faith makes no sense when you have a technical specification.
Sorry.
OK now I see your problem. You currently lack a fundamental understanding of neural networks and machine learning in general, and it seems that you didn't actually read the entire paper.
The loss function is involved in the training of a neural network (transformer).
THE LOSS FUNCTION IS NOT THE MODEL ITSELF.
The loss function is not a neural network. The loss function is not the transformer model. It is only part of the training step.
This is a major blind spot in your current technical understanding of this paper and machine learning in general.
The loss function (in GPT-3) is optimizing a 100 layer neural network with 175 billion parameters, and that is where the semantic relations emerge, in the model itself. The neural network is inferring these relations from the training data. Obviously there is no "meaning" defined in the loss function, and it makes absolutely no sense to look for it there. The structure to be learned is in the training data. Semantics are not explicitly coded into the model. The paper does not give you all of the technical details. The authors assume you have the pre-requisite knowledge of NLP, deep learning, and the transformer architecture.
"Semantic similarity" is not gospel. This is a standard NLP term and it is discussed in the paper, as it is one of the training objectives.
This emergent properties of transformers and other deep learning models is a fascinating thing to witness and to study. You should go back to basics in machine learning to properly understand how the loss function is only directing the training of a model, but is not the model itself, which can be incomprehensibly vast as in the case of GPT-3. You could not fit a full technical description of the actual trained GPT-3 model into any paper. The emergent properties of neural networks is a new field of research and we don't yet have a complete understanding of all of these properties or how to investigate them.
No I haven't taken anything personally. I am trying to explain how this stuff works. Go back and read your recent replies to me then see how condescending and ridiculous you were (accusing me of having blind faith, not wanting to know how things work, etc). I truly don't care. I'm just trying to explain something. Too bad if I have to be so blunt. No offence taken. It's actually more funny than offensive.
I have a degree in Applied Mathematics and Computer Science. But so what? You don't need a degree to understand the basics.
You're looking for an explicit semantic coding inside a loss function, which doesn't make any sense. Of course you won't find any coding for semantics described in the paper, because the entire point of machine learning is that it figures that out automatically.
How the hell can you have a PhD in machine learning without understanding that?
> Nobody knows how to represent meaning and how to train a neural network on examples of it.
We represent meaning with language models, and GPT has been trained on examples of it (semantic similarity). It learns how to represent meaning through the training process.
The meaning relations are an emergent property of the trained neural network, not explicitly programmed. The meaning relations are automatically learned by the machine. This is the magic of machine learning and neural networks. (I don't mean literal magic. These large language models are simply too large and opaque. You can't easily identify the specific neural activation pattern that encodes meaning of any given concept within a 100 layer deep 175 billion parameter neural network.)
"In essence, the "meaning" referred to in LLMs is not the same as the meaning humans typically ascribe to language, which is derived from direct experiences and real-world knowledge. Instead, it is a form of abstract, statistical understanding that emerges from the relationships between words, their contexts, and the patterns in the data. The model essentially learns a high-dimensional representation of these relationships, which can be thought of as an approximation of meaning within the context of the model's training data."
This might help get you on the right track.
Question:
Do neural networks have emergent properties?
Answer (from GPT-4):
Yes, neural networks can exhibit emergent properties. Emergent properties are characteristics or behaviors that arise from the collective interactions of the individual components in a system, which are not directly attributable to any single component. In the case of neural networks, these individual components are the neurons (or nodes) and their connections (or weights).
As neural networks learn from data through a training process, they adjust the weights between neurons, which can lead to complex patterns of activation across the network. The network's ability to recognize patterns, make predictions, and perform tasks can be seen as an emergent property that arises from the collective behavior of its interconnected neurons.
For example, in the case of language models like GPT-3, the ability to generate coherent and contextually relevant text can be considered an emergent property. This capability is not explicitly programmed into the model but arises from the learned relationships between words and phrases through the training process.
It's important to note that while neural networks can exhibit emergent properties, these properties might not always be easily explainable or interpretable, as they result from the complex interactions within the network.
Look, this is the internet. I'm sure you know what crazy things happen on the internet. In view of that, are you sure it's such a great idea going around telling people that you are "`stalking'" them, quotes or no quotes?
Could you think for a moment how some people might take your comment, including myself? Is that really how you want to interact with people online?
This is literally the accepted term for the nodes wherein sums are produced of the set of weighted products which connect such nodes together. If you know a better term, or would prefer a better term that "neuron" to describe these points in a "neural network", by all means speak your mind, but don't pretend some offense at it.
>and that's because you restricted yourself to facts.
I restricted myself to writing from a point of view concordant with your own because that is what one does when explaining a position well, not because I agree with your point of view.
>You could not give a factual description with the word "meaning" in it (in the sense of "the meaning of...").
If we're going to play a game of semantics, than so be it.
If your argument is simply that symbols and numbers can't contain meaning because they are symbols and numbers, then you've worked your way into a tautological box that I, nor anyone else, will be able to free you from.
It is known that memories in biological neuron-based networks are not locally stored or unique to some neuron ( there is no "grandma" neuron in your head to keep track of dear old grannie ), but that instead memories are spread across the set of neurons there available.
This sort of representation in known as a "holographic" representation, which I expect our neural net will also utilize. Information will not be stored in any specific neurons in the trained model, but will be spread across the whole of them.
But even if true, that is merely data, you will no doubt point out. Where is the "meaning"?
As you want to delve philosophically into it, we must question the very nature of the word meaning here. Our model lacks any form of experiential qualia, and so cannot associate words with experienced reality, which is the normal basis for human concepts of meaning, which then underlie our development out of concrete language and into metaphoric.
This leaves meaning within the model as only being the derived relationship between words.
Words, on their own and bereft of qualia, can present our model with structure, grammar, punctuation. Words will have certain uses within these structures and grammars, they will have certain places. Words will modify words, phrases will modify larger sentences. Trues can be asserted and falsehoods. Truth and falseness itself can be demonstrated through sentences making claims about other sentences.
Would the model know what a red ball is? Of course not. But it would be aware of how the term ball is used, what sort of type it is given, the qualifications and usages of words of that type. The modifiers it is reasonable to use on it, etc.
And incredibly, with no basis in anything other than being slightly corrected for each incorrect token generated, our training harness manages to discover a mathematical neural net model of these relationships between words such that it can manipulate them as well or better than many people.
The use of backpropagation and gradient descent allows us to search the space of all possible models starting from a randomized model, and find a model that does this by nudging our model to be a little more capable at it over and over again.
However, English is not just a fixed mathematical set of symbols that can be rotely manipulated.
Our language is metaphoric from its very foundations. Our hearts soar, sink and swell in ways that have nothing to do with the reality, but only our perceptions of ourselves.
To manipulate English as it does, the traversal of the model-space must advance inexorably on a model capable of handling such non-literal language.
The model contains meanings for car related actions, for curling related actions, it contains the ability to translate between them when the only connection is a meaning that is implied by the words but not either of their literal meanings.
What is a meaning here? It is a relationship between the words. The relationship between the car words and curling words only exists as the relationship to a common implied meaning that can be transformed under the context of other words.
Where is the meaning? We don't even know for certain where the meaning is inside ourselves. I expect it to be similarly holographically stored across the set of neurons and provide the appropriate context to twist the swirl of mathematics that occurs between input vector and output.
>So I still do not know why you think all this is going on inside a language model.
Because it is the simplest explanation of what it is doing. Call it "latent spaces", "embeddings" and the assume the compression I discuss is via "manifold hypothesis".
>What do language models have to do with meaning, and why do we absolutely need to explain their behaviour by talking about meaning? Where is this information in your comment? I cannot see it.
So, where is the meaning? It is everywhere in the model once trained. It creates a holographically stored mapping of reality as presented through the corpus of human thought in the form of writing based solely on being iteratively nudged through model-space.
An amazing feat by the engineers that dreamed it up.
If you are unwilling to call this mapping model of reality built only of tokens/symbols/words "meaning", then I have no argument for you, except to again conclude that you've locked yourself in a tautological box.
If your argument is merely "ha ha, you're merely conjecturing this", then yes, of course I am. Why? Because it behooves someone to hypothesize, test, and study things which interest them.
Do you have some competing idea of how it works? It would behoove you to share it with the thread.
As it is, I only see you making demands for proof of things that you are well aware that no human has yet completely modeled, even after decades of studying such phenomena. What you gain from this incessant nay-saying, I do not understand.