This is, without exaggeration, probably the fiftieth blog post or long-form comment about how someone is using an LLM for "complex learning", and I'd just really like to see at least one of these to be accompanied by a statement saying what are the kinds of problems the author can now confidently solve that they couldn't before.
In my experience, LLMs are really good for taking up your time and making you feel like you're learning, in the same way that many of the popular educational videos on YouTube are fun to watch and don't really teach you anything.
If you ask an LLM to give you a 500-word summary of quantum physics, it'll give you an oversimplification that probably leans on a hodgepodge of pop-sci metaphors. And if you start drilling down, you risk drilling down on these ELI5 metaphors, which can get you farther away from truth.
I personally don't get this attitude. For one thing, nobody on earth can give you a 500-word summary of quantum physics in a way that lets you internalize even a tiny bit of it. That's not how you learn anything.
Here's how I do it: I open Baby Rudin (3rd ed.), second chapter, and read the main text - absolute brutality. I unpack almost every sentence with Claude/GPT until I finally get what's going on. No ELI5 nonsense, just examples and counterexamples galore while absorbing the techniques and the way of thinking in analysis/topology. How do I know I've learned the material? By solving every single problem in that chapter. Here's the thing, though: the problems in Rudin can be brutal and decoupled from what's in the text, so if you can handle them, you've definitely mastered the material. No 500-word summary of analysis here.
Baby Rudin is a special textbook: it comes from a time when mathematics textbooks were not at all judged by their pedagogical value, but instead by their aesthetic appeal to mathematicians (a la Bourbaki).
If you actually want to learn analysis, there are a nearly infinite number of friendlier resources (e.g., Understanding Analysis by Abbott).
Learning to unpack difficult text on one’s own is a valuable skill. Research papers often require a similar amount of suffering, and at the frontier of knowledge, despite all the advances we’ve seen, LLMs seem to have absolutely no understanding or intuition. They are much better at things that have been expounded at length by humans before.
I would argue that if you’re going to use an LLM to make Rudin easier to understand, you are not learning how to absorb difficult material, nor are you learning analysis efficiently.
> I would argue that if you’re going to use an LLM to make Rudin easier to understand, you are not learning how to absorb difficult material, nor are you learning analysis efficiently.
I would argue that if he can solve the problems on his own, without LLM assistance, than he has mastered it.
The same could be said for someone taking a university course where Rudin is the textbook, and they rely on the professor/TA to help them understand the book.
This is needlessly nitpicky. The stark reality is that the majority of undergrad math majors do not achieve what this person has merely be reading Rudin (assuming he is actually solving most/all the problems).
I don't think so. There are many objections to methods of making learning "easier" and the invalidity of one does not imply the invalidity of all of them.
We've also seen the effects of LLMs on learning, at least when used instinctively by students. An entire generation of students seems to have atrophied their critical thinking skills this way. Talk to 21-year-olds today and you'll see what I mean.
>There are many objections to methods of making learning "easier" and the invalidity of one does not imply the invalidity of all of them.
I am pointing out a pattern of technological hubris stretching thousands of years.
>An entire generation of students seems to have atrophied their critical thinking skills this way. Talk to 21-year-olds today and you'll see what I mean.
Ah, yes, the old "kids these days" routine... quite fond of it myself sometimes. But I find myself benefitted when I take care to learn from the young.
"For the correct analogy for the mind is not a vessel that needs filling, but wood that needs igniting - no more - and then it motivates one towards originality and instils the desire for truth. Suppose someone were to go and ask his neighbours for fire and find a substantial blaze there, and just stay there continually warming himself: that is no different from someone who goes to someone else to get some of his rationality, and fails to realize that he ought to ignite his innate flame, his own intellect, but is happy to sit entranced by the lecture, and the words trigger only associative thinking and bring, as it were, only a flush to his cheeks and a glow to his limbs; but he has not dispelled or dispersed, in the warm light of philosophy, the internal dank gloom of his mind."
Is that meaningfully different from the study methods of the past? That doesn't require an LLM, and using one risks correctness in exchange for speed. You might not even get that speed if you're stuck in the weeds often enough.
A decade ago, a google search for study guides written by another professor would have been slightly slower. A decade before that, you'd be even slower fumbling through several books. Every single word could at least be trusted. You don't get that from an LLM.
> Is that meaningfully different from the study methods of the past?
The fundamental service a teacher provides is personalized feedback, quickly identifying where you are stuck and focusing the explanations and exercises on that area, drastically increasing the speed and quality of learning versus the self-supervised route.
The lack of this closed loop effectively killed the high hopes that were placed in e-learning and MOOCs 15-20 years ago, TV learning in the 1960s and many other failed revolutions, seems every generation has its own version.
It appears to me LLMs have a real potential to close this loop and become the failed educational revolution of our own generation.
> personalized feedback, quickly identifying where you are stuck and focusing the explanations and exercises on that area,
This has been a huge blocker when I tried to study advanced math myself. Many of the exercise books don't have worked out answers, so often you're either stuck or you have to hunt a variety of sources online for solutions and advice. It kills flow.
If you have an actual mathematics professor to ask questions to then sure do that, but most people don't have that luxury. Also the error rate of frontier LLMs on textbook highschool/college level mathematics is going to be extremely low.
are you referring to this section?
> Verification is different from generation: Models scoring 98 can solve problems but can't always explain why their approach works at the level a human mathematician would.
It doesn't say it can't explain why with any accuracy, it just says it can't *always* explain at the level of a mathematician, but most of us don't have such a mathematician at our beck and call to answer our questions anyways (thinking of the perspective of a self-learner outside of formal education)
It’s not just math. Anecdotally, LLMs struggle the same way with software engineering where the code they write is correct (compiles and passes tests), but reasoning is wrong often enough to eliminate most trust in these models’ ability to explain codebases or even features they themselves produce. It’s not about the _level_ of the supposed intelligence where a model struggles to summarize things succinctly or simply enough (responding to the “pocket mathematician” comment) or can’t grasp certain concepts at all (if so, how tf is it able to apply them?). It’s that by their design LLMs have no concept of truth and no concept of causality. They guess with every single inference and it’s very hard as a user to understand which guesses are more or less certain, since, you know, confidence ratings aren’t part of these models’ design either.
I wonder how long before LLMs need to cash in and even simple prompts are beyond the non-existent budget of a student. Which means it will need to be subsidized by education... and we're back in the teacher loop once again.
Or perhaps libraries. That would be neat.
>It is also significantly more engaging and fun.
That won't last long either. I remember when phone apps were the "engaging and fun ways to learn". Half life of 2 years, and we're already seeing people lose the fun factor.
When studying a mathematics text, it is good practice to question what you are reading, trying to prove things to yourself etc., which IMO makes the problem of trusted sources much less than it is in things which you are not able to verify e.g. historical accounts.
I certainly would only trust my textbook as the authoritative source, but i can see that in the absence of an expert teacher it's nice to have something that can critique a proof. Imo the fact that it's hard to verify that your own proof is correct is one of the main barriers in self-studying math, especially if one is at a level where one is not completely fluent in applying the various techniques. This also applies to judging answers to open-ended questions in any other field.
I'm a private math tutor specializing in exactly this sort of material, and I agree with this very strongly. Knowing what "counts" as a proof is one of the most common gaps I see in students who come to me after self-studying, and most students do need some back-and-forth with an expert to really get that skill down. I imagine that LLM's could be very helpful for this if they were used judiciously!
There's been research done (I know, big words -- don't have any links readily available), where it was pointing to the fact that what goes easily in (into the brain) also goes easily out, and conversely, what the brain requires effort and literal calories and energy to understand, stays there longer (investment must be rewarded / recouped somehow).
So while there's no doubt that facilitating learning is a net-positive, at some point it becomes a net-positive, I suppose -- your brain just chucks it out faster because it knows you can re-obtain the same information again since it worked so easily the first time. It doesn't know the difference between easy and hard, all it knows is how much effort it takes and how much reward (hormones) was generated for it all (to cement the habit/result).
Spaced repetition could address this, no? Also, the right example or metaphor can make a difficult concept both easier to understand and stickier in my mind which I think is different than something easy to learn.
I agree regarding metaphors, I myself aggressively utilise analogies with both myself and others, as a "digestive", but I feel like with LLMs we're treading on thin ice as it's not straightforward to classify a particular use case as one scaffolding learning with a metaphor, or just having the opposite effect where your brain "sails" in a faulty sea of "learning" while in reality there's no real work being done, not of the kind the brain needs to do in order to re-order all the synapses and own networks that ends being "knowledge" eventually.
Spaced repetition is a good way (perhaps even the optimal way) to get large amount of trivial memorization into your brain, but it not a good way to understand a complex subjects.
Memorizing one word in a foreign language is not that complex, nor hard. The tricky part about learning a language is that you have to memorize thousands of words, and the trickier part is that you have to retain most of those words over a long period of time. Spaced repetition helps by finding an optimal schedule to for the exact same activity as you would otherwise.
I also think that this broad claim (LLMs are good for learning) is also a testable hypothesis, and given the size of the AI industry I think it is reasonable to assume that AI companies would be eager to proof that LLMs are good for learning (and then pushing their own proprietary implementations into school systems and make more money).
The absence of evidence here feels very much like evidence of absence. At least my cynical view of capitalism tells me that if there was a good way to use LLMs to learn stuff, we would have research showing it, and AI companies would be waving that research all over our faces.
Your obvious use of an LLM to write this comment isn’t exactly confidence-inspiring that it’s helpful for learning to do things on your own. Seems like it’s evidence that LLM’s quickly become a crutch for even trivial efforts like posting internet comments.
I’ve found it helpful to ask LLMs specific questions about your knowledge gaps, or ask for concrete worked examples that illustrate the ideas you’re learning about.
In [0], I ask: „When applying Hidden Markov Models to POS tagging in NLP, what do the latent states and observations usually represent?” I then follow up with some specific questions and requests for walkthrough. You can’t see it from this conversations, but I have Wikipedia and a bunch of other resources open in separate pages, cross-reading, and I follow up with a handwritten toy implementation of a Viterbi-based POS tagger once my mental model crystallizes. This is very different from a 500-word summary of quantum physics, and I still had to put in effort (this is unescapable!), but I found the experience rewarding. Also note that this is relearning of a topic that was part of my uni curriculum but long forgotten.
In [1] and [2], I’m learning Spanish by reading García Lorca’s poems. Here again I’m going through the texts with a dictionary, and augmenting my learning with what a dictionary won’t tell me: given the usage of a word or phrase in this specific poem, is it something that could occur in everyday speech, or is it poetical?
100% agree. They are not good teachers. Nevertheless I've been using LLMs more and more to learn complex topics, but my learning is always anchored in something else:
- Getting through textbooks and lecture notes. LLMs have gotten very good at answering basic questions on quite advanced material (e.g. representation theory and quantum field theory). By asking a very specific question or even giving the LLM a screenshot, I can get unstuck a lot faster.
- Learning e.g. new python packages. Instead of hunting for examples on Stack Exchange, now I ask an LLM to write a minimal working example and then build off of that. By writing most of the remaining code myself and only using the LLM to answer questions, I've been able to learn new packages significantly faster.
In both cases, the LLM isn't providing the curriculum or guiding what I learn. The textbooks, papers and coding tasks are. But now I can pick these things up much more efficiently.
I think it depends on what you want to learn and what your definition of "learnt" is.
The other day I realised I had no idea how DNA and life works. I guess I studied it at high school (25 years ago), but maybe it didn't go into much detail or it just didn't click.
So I asked ChatGPT to explain it to me, I came up with my own mental model from it's explanation, told it that, then it corrected me where I misunderstood things. We went backwards and forwards for an hour, me asking questions, it correcting me, until I felt like I understood the whole picture.
Am I going to become a biologist and study the origins of life from that? Definatley not! But if my kids need help on their biology homework, I now understand the basics of it.
>So I asked ChatGPT to explain it to me, I came up with my own mental model from it's explanation, told it that, then it corrected me where I misunderstood things. We went backwards and forwards for an hour, me asking questions, it correcting me, until I felt like I understood the whole picture.
If you're only checking your understanding against the one source you used to obtain it, how can you tell whether your understanding coincides with reality (or rather, with general scientific understanding), and not just with the source you read? And I'm not asking just about ChatGPT; the same question could apply to any source. Books are not exempt from containing errors.
Because you can successfully use it to communicate with other people. That's the grounding truth. If you'd only ever read about it and never actually employed it then yes, you would not be able to make that claim.
It's not the same thing. A language is a closed system, it just requires that the participants agree on the rules in order to be successful. Biology is not a language, it's a field of study of phenomena that exist independently of minds. If you read a biology textbook (or an LLM's output about biology) and then relate to someone else what you've read, you and the other person can discuss the topic, and be no closer to ascertain whether what you two understand has any basis in reality.
> Biology is not a language, it's a field of study of phenomena that exist independently of minds.
Your personal assertion is quite wrong at a fundamental level. Biology refers to the field or study, not individual specimens. The field is comprised of the understanding that people over time compiled on nature, along with arbitrary frameworks that help people organize and reason about the topic.
Take the concept of species, and species classification. A specimens exists regardless of being classified or not. However, the same specimen can be classified differently depending on the state of the body of knowledge at the time. In some cases you had species being reclassified due to new findings, such as genetic tests.
Once you understand this, you learn that you can't claim that a field of study is anything other than abstractions and partial and incomplete observations compiled form people throughout time.
Biology is in some sense a language, how do we even agree on what the boundaries of biology vs other domains of science are without participants agreeing on the rules?
In any case I think you have an overly narrow definition of learning that we're unlikely to come to terms over.
He's talking about learning that makes you operational. If you can speak with someone, then you've learned language. If you can get a computer to do something, then you've learned some programming.
The equivalent for biology would be to grow a plant or a few plants and animals successfully. That's operational at a certain level, you could also be operational at a lower or higher level.
No, it doesn't necessarily need to be applicable, it just needs to be testable. The size of the Earth for example may not affect you directly in any way, but it's something you can verify to some degree of precision. But if you just read about it, can you really say that you know it? What if all you believe about the size and shape of the Earth is from what you've asked an LLM? Do you still know it?
EDIT: Perhaps not the best example, because the size and shape of the Earth are data that are repeated often enough that an LLM would be unlikely to quote it grossly incorrectly, but I think my point still comes across.
I’ve been burned by this enough that I no longer say that I’ve learned something if my only interaction with it is explanation from books. You can get snippets of knowledge and a framework of understanding, but true learning only comes with deep interaction of the concepts (practice, simulation, experiments, observations) and not merely reading.
>how do we even agree on what the boundaries of biology vs other domains of science are without participants agreeing on the rules?
Those boundaries are completely imaginary and don't exist in reality. In reality there's no biology, there's only elementary particles interacting physically. Whether you agree with someone else to classify a phenomenon as biological or chemical, you're not refining your knowledge of the real world, you're just performing an organizational task.
But, say, how many times the tympanic ear evolved independently is a real phenomenon that can be investigated, and it's something that you're either correct, incorrect, or ignorant about. If I tell you it evolved five times, what more can either of us gleam about the real facts by just discussing this factoid back and forth, if neither of us has access to any additional knowledge or way to put this datum to the test?
For majority of people, especially children, book is an authoritative source. Virtually no one outside of research fields and very excited enthusiasts will check sources in depth to see whether what is shown in the book is true or not.
So, I think the point OP is making is that most people don't really check sources while learning things 'the conventional way'.
We have to distinguish learning from, let's call it, recreational education. If you're learning a topic, you're not going to be using a single book. Not even schoolchildren do that. Regardless of whether you consider a book an authoritative source, if you read multiple books those books will have to agree with each other, and where they don't you'll experience some confusion until you can resolve the contradiction.
If you're only ingesting information from a single source, be that a book, a teacher, or an LLM, then you haven't really learned, because your knowledge base is unmoored. You can't learn history by studying Tolkien's mythos.
Concrete high-profile example: in "Surely You're Joking, Mr. Feynman!", Feynman told of a ball which, in a Brazilian college-level physics book, was described as having a 40% higher acceleration that it would actually have in practice.
Turns out that the author had done a thought experiment but neglected to factor in the rotational inertia.
Self-consistency and consistency with your lived experience are good heuristics. Reality is self-consistent, so anything that doesn't add up indicated an error in the source or your understanding of it (or both).
EDIT:
I think it's a kind thing you need to tune yourself into. OTOH, I've observed many (most?) people seemingly being completely oblivious to self-consistency issues of their beliefs and mental models, or even texts they're reading or instructions they're following, and yet... somehow they're generally more successful at life because of it ¯\_(ツ)_/¯.
> I've observed many (most?) people seemingly being completely oblivious to self-consistency issues of their beliefs and mental models, or even texts they're reading or instructions they're following, and yet... somehow they're generally more successful at life because of it
What do you mean by "successful at life" here? Genuine happiness, fulfillment in life? Or in the sense of doing well by what society holds as it's current interpretion of what one should strive for, and otherwise just kinda drifting through life?
Because if it's the latter, I'd say that is to be expected. It's much simpler to put your energy into fulfilling the expectations of whoever is your superior in your current group, mostly get the expected reward, and then just coast. Reflection and experimentation, which is required to get to self-consistent views, takes effort and and the willingness to question existing beliefs, which will also be uncomfortable times.
The very first thing I mention: self-consistency. It's the only thing you have if you don't have any empirical data. It's the only thing anyone has, really. QM scientists reading QM papers and experiment reports of other people, and talking with each other, are still relying on self-consistency to sniff their own (or other people's) mistakes.
>The very first thing I mention: self-consistency.
So if I consistently tell you that that lithium atoms are heavier than carbon atoms, that would make it more likely to be true?
>QM scientists reading QM papers and experiment reports of other people, and talking with each other, are still relying on self-consistency to sniff their own (or other people's) mistakes.
Physicists don't need self-consistency. They can test consistency against reality itself. That's not self-consistency, that's just plain old empiricism.
> So if I consistently tell you that that lithium atoms are heavier than carbon atoms, that would make it more likely to be true?
Nope. But if you said that, and used it as part of an explanation of some process, and every step logically checked out, and the outcome checked out too, and agreed with other things you said, and other things others said, then yes, I'd be likely to believe you.
Alas, a quick look at the periodic table raises a red flag - your explanation is inconsistent with the periodic table and what I know about its structure.
That doesn't necessarily say you are wrong - could be me. But judging by the tone of your comment vs. heaps of other things I know that are consistent with my understanding of chemistry and inconsistent with your statement...
> Physicists don't need self-consistency. They can test consistency against reality itself. That's not self-consistency, that's just plain old empiricism.
No they can't. No one has that much time or money. Physicists aren't routinely replicating every core result empirically for themselves. They rely on the descriptions of experiments and data that they read, and the self-consistency and extreme interconnectedness of reality, which means that wrong information will not add up with someone's experiment, expectations, or lived experience somewhere, and will be quickly flagged as wrong.
Empiricism is only useful because reality is self-consistent. If it weren't, you couldn't really infer anything from empirical evidence because things would just be whatever they wanted to be.
(Or more precisely: we can't prove reality is self-consistent, but if it isn't, nothing can ever make any sense, and we may just as well pack up our technological civilization and go back to the caves we crawled out of. Fortunately, empirical evidence supports the notion of reality being self-consistent to the extent we can observe it :).)
>But if you said that, and [...] I'd be likely to believe you.
That wasn't what I asked. I was very deliberate, I asked if it would be more likely to be true. What's under discussion is not your standard of evidence, but whether non-contradiction by itself is sufficient to conclude that a claim is true.
>Alas, a quick look at the periodic table raises a red flag
That's not self-consistency anymore, that's cross-corroboration. Which, yeah, good on you if you do that, but it's not the process being proposed either by the OP or by fy20.
>Physicists aren't routinely replicating every core result empirically for themselves.
OK, but that wasn't what I said. A physicist can't test every result, sure, but he can test those that are most relevant to his work. I'm not going to get into the philosophy of empiricism because it's not relevant here. My point was that an expert reading a peer's paper is not the least bit comparable to a layman reading an LLM's summary of a field of study. They're just not similar situations. One has the context and the capability to detect bullshit, while the other does not.
Not the OP, but I don't think you can. The understanding someone manufacturing silicon has will be not be attainable by reading about it etc. Similar for QM, no replacement for doing (some of) the math yourself, be involved in experiments etc.
That would be the case if our memory wouldn't be completely fluid and built on subjective perceptions. The two sort of merge together over time, you come to see what you expect. It's only when something really out there breaches the belief model when that actually happens.
Reality is most likely self-consistent, but as we can only experience a tiny part of it it‘s impossible to tell. Also, parts of reality may appear to be contradictory with each other when some pieces are not known.
> Also, parts of reality may appear to be contradictory with each other when some pieces are not known.
Exactly. This tells you where something is off. The problem may be your lack of understanding or wrong understanding, or it may be with the source, or the framing, or you may have hit a genuine lack of data - still, the puzzles don't fit in some area.
And yes, not all self-consistent understanding is correct. But all inconsistent understanding is incorrect. And the more knowledge you gain, the less likely it is that it'll all connect self-consistently, but still be very wrong.
> But all inconsistent understanding is incorrect.
Why do you believe that?
Even if we assume that reality itself is self-consistent (what does that even mean?), why would that imply that we humans are able to find a self-consistent representation of it? Maybe reality is self-consistent in some sense but cannot even be represented by the tools we use for theory building.
My point being, the ultimate target of our understanding may be self-consistent, but the way we _necessarily_ have to reduce it to lossy theories means that we can only ever approach it with a non-zero error. And a theory focusing on one aspect, minimizing representational error from one direction of approaching it, necessarily has to make assumptions that will contradict those made by another theory trying to minimize representational error coming from another direction / domain.
> but the way we _necessarily_ have to reduce it to lossy theories means that we can only ever approach it with a non-zero error.
Yes. Fortunately, we also usually don't need the error to be zero. In practice, we usually have narrow scope at any given time, and can get away with a lot of error.
E.g. people in the past found alternatives to modern germ theory, involving evil spirits and other such shenanigans, but to the extend they covered the high-level mechanisms (curse transfers through contact, hygiene and boiling water removes the harmful effects, etc.), it doesn't really matter the theories were wrong. The beliefs were consistent with each other and empirical evidence to some degree, and to that degree were useful.
> And a theory focusing on one aspect, minimizing representational error from one direction of approaching it, necessarily has to make assumptions that will contradict those made by another theory trying to minimize representational error coming from another direction / domain.
That's fair. Our brains and attentions are finite, you always have to limit the scope. If you imagine you'd have practically forever, you could sort it out and make it all consistent to arbitrary degree (subject to fundamental computational limits, which are physical limits). In practice, the heuristic of consistency works like this:
- For things within your domain of interest, inconsistent information flags an error.
- For things at the interface between your domain of interest and another domain(s), inconsistent information flags an abstraction boundary. It's where you can observe simplifications both domains make because they don't add up (and if you adjust them to make them mutually consistent, you just allowed two domains to work together).
Can you say in all honesty that a read of the Wikipedia article for DNA would have been less helpful? It's less convenient perhaps, but definitely more authoritative.
Wikipedia is a great _reference_ but not necessarily the best way to learn about a topic. Of course, this depends on the topic, on who has been writing the page, etc
A particularly bad example is higher maths - a wiki pages on a complex mathematics topic often reads like "A gruncheon is a worch in the brashation of plusters" and each of these words is a separate page or topic. Of course, you _can_ in theory 'just' click through all the tree of linked pages to understand a concept ...
For DNA the page (scanning it now) is well laid out, with images (including a spinning Rasmol? image) and lots of detail. However, the detail could be a drag on understanding for some : There are 'nucleotides' and 'nucleosides' and 'nucleobases'? There are non-canonical bases? Supercoiling? Z-DNA? While I know (most) of these things, it is because I've learned about them in other contexts, or by direct instruction.
I'm not saying it is impossible to understand DNA from that page, but it is likely to be harder (for some?) than a more conversational approach to learning.
Fair point - but I wonder how many people that have visited Wikipedia know about simple wiki. I was probably aware of it, but not enough to remember to suggest it as an alternative!
I mean - it is certainly better ... but it is still a lot of stuff. For example:
> Part of an organism's DNA is "non-coding DNA" sequences. They do not code for protein sequences. Some noncoding DNA is transcribed into non-coding RNA molecules, such as transfer RNA, ribosomal RNA, and regulatory RNAs.
Do you _need_ to know about tRNA, rRNA, and operons (?) to understand DNA? The thing about an encyclopedia/wiki entry is that it has to cover the whole topic. This is a strength for reference, as you can scan it and find the bit you need. For learning from scratch, I can see that a conversational approach (with a human or LLM) has advantages where the learner can direct the level of detail and path through the material.
Ultimately, both are worthwhile, but I can also see the strengths/weaknesses of both ways to learn.
Wikipedia is completely non-interactive so of course the experience is different. Do you want to look up something like https://en.wikipedia.org/wiki/Principal_component_analysis and try and understand that from Wikipedia? If you don't understand it, you have to try and click elsewhere. With an LLM, I can be very specific, "I understand x, y, z about PCA but I don't understand why we have to do it? What happens if I don't apply it?" and the LLM most of the time will give very approachable explanations that can be refined further if I still don't get it.
I am studying for a Masters degree in Computer Science with AI and the lecture notes are like Wikipedia sometimes. Incomplete, perhaps assume pre-knowledge that lots of Masters students won't have. All of these I have taken to ChatGPT and got great explanations, diagrams, graphs etc.
Or do you? Cause if you went back and forth with ChatGPT for an hour it definitely hallucinated and lied to you at some point. Maybe consider using something else like Brilliant.org if you want to learn a topic, yanno, so you don’t propagate whatever hallucination from ChatGPT to your kids.
These days online courses like Brilliant and such are likely largely LLM generated. Are they actually vetted by experts before publication? Who knows. My money is on no, or at least, not until someone complains.
Its easy enough to prompt ChatGPT for primary sources when doing research to validate any claims its making.
You should actually do this. Ask ChatGPT to check what it said against sources. It does correct itself. The corrections are usually not major. Just mental shortcuts.
This would be a valid point maybe 3 years ago, but most chatbots will now query and verify direct sources, especially in research mode.
This is very simple to validate and verify. You could argue it may find false primary sources.
You can condemn models for a variety of other things, but acting as if this is still reality shows a lack of understanding as to modern model capabilities
Your comment is phrased as if it somehow refutes their point but it doesn't.
> Cause if you went back and forth with ChatGPT for an hour it definitely hallucinated and lied to you at some point.
If you're asserting that this is not the case today then that's going to be require pretty extraordinary evidence. "Chatbots use Google now" is not evidence that the information they provide is in fact correct.
They don't hallucinate all the time like they used to, no, but I'd be very surprised if the majority of these sorts of conversations were free of major factual errors.
I frequently notice degradation in the model model's ability to remain coherent when it searches for information online. For example I might ask Sonnet 5 "how do I build a shed" and during its search it presumably comes across an article which talks about building a shed out of paper mache, then the model responds with something like "I caution you against your plan to build a shed out of paper mache" -- Wait, what? Who said anything about building it out of paper mache?
Firstly, I can't refute a point that's little more than an opinion - my doubt is that a modern frontier model is significantly hallucinating within a relatively short conversation that can easily be verified. There is no way to refute or accept the point without a complete conversation log. I am criticizing the hyperbolic nature of the comment.
I doubt you are getting to the context level of model degredation where it reaches context limits within a verbal hour conversation.
I've just tried to recreate your example on sonnet 5, and as someone who has done DIY projects it reads completely appropriate, but I'm happy for criticism from a shed builder. It never once tells me about paper machie or creates a silly example.
This is via a prompt requesting tools and materials, and could be further improved, unfortunately, I can't paste the markdown formatting provided.
"""
Reference size used below: 8x10 ft shed. Scale material quantities to your dimensions.
Step 1: Check Regulations & Plan
Materials: None yet — just your design/plan (graph paper or free shed-plan software)
Tools: None
Skills to find:
None required, but if your shed is large or near a boundary, a quick chat with your local planning/building department saves headaches later
What to do: Confirm permit requirements, setback distances from boundaries, and max height/size allowed without permission. Sketch your design and finalize dimensions.
Step 2: Prepare the Site
Materials:
Landscape fabric (weed barrier)
Gravel/crushed stone (for drainage base, ~4-6 in depth)
Marking spray paint or stakes + string
Tools:
Shovel & spade
Wheelbarrow
Rake
Hand tamper or plate compactor
Spirit level (4 ft) or laser level
Tape measure
Builder's square (for squaring corners)
Skills to find:
Basic site leveling — not hard, but a laser level rental helps a lot if the ground has any slope
If you have poor drainage/heavy clay soil, worth asking a landscaper for advice
What to do: Clear vegetation, mark the footprint, excavate and level, add compacted gravel base for drainage.
...
"""
I won't include the whole document, can share it further but anyone can replicate just by asking sonnet
I just don't understand the need for such hyperbole, and pretending that models are still gpt3, when you can get counter evidence in seconds.
It reminds me of the craze teachers had against trusting Wikipedia - yes, you shouldn't take all claims at face value, but arguing that nothing from Wikipedia could be useful just makes the argument silly.
Err, that wasn't meant as literal example because real conversations obviously have more than 1 turn. The only thing you could prove by getting a different result is that they don't _always_ do that, even if I shared the full conversation log.
You "don't understand the need for such hyperboles" because they're not hyperboles, I don't know how you can not pick up on these errors in your own conversations.
> and pretending that models are still gpt3
I explicitly said that the new ones are better. How's that for hyperbole?
Then this entire conversation is a pointless argument - as I agree that models aren't omniscient, godlike entities that are perfect sources of truth, and that you need to use critical thinking when using them.
I don't trust models blindly, and interrogate and verify claims that they make, but that's a basic component of being a human being.
I also never try to have massive multi step conversations to the point where I'm nearing the context limits, as if there's a subclaim i need to interrogate it's far better to clear context and just start a new chat, I have notes to join up ideas.
When learning, I'm not just doing so blindly asking a model questions, I have other material up, I can look at the answer to a example question from a textbook to verify whether I have used a model to successfully learn.
This conversation is just going to devolve further into a "well it doesn't always work" to which yes, I agree, but that doesn't mean it's not useful and doesn't help the learning process.
This is not a silver bullet that will make everyone a genius. Not even the best human tutor can do that. But it can be a massive clearer of certain roadblocks for people with no better human help available. Countless people have to resort to learning from teachers who themselves are confused and know the material only shallowly and give confused and wrong explanations that can be very hard to untie for the learner. For most undergrad level things, LLMs have a breadth and depth of knowledge and online search capability that it can make you unstuck on some misunderstanding quite effectively. You still have to work. It's not magic. But the goalpost cannot be to catapult the median person to become Einstein. It's a tool.
Luckily I had my dad available who is a scientist, during my high school and he corrected several fundamental mis-explanations of my teacher that even to my mind logically simply didn't add up. I learned not to relay this back to the teacher of course and sometimes regurgitated the wrong answer in tests. Not everyone is so lucky. This has been my frustration quite often. Textbooks are sometimes wrong, both because the author really doesn't have good expertise on that slice of the topic or out of didactic simplification reasons. Having an LLM that can consult the real grownup literature and give the full story, not the birds-and-bees is quite useful.
It is sometimes typos, but even those can be confusing. But okay I'll admit books are usually on the end of simplification, at least for hard sciences. But sometimes they do handle the edges of the authors knowledge and sometimes misconceptions creep in. Famous ones include the reason for airplanes flying or similar "let's relate the hard science to practical reality" and making mistakes there where they think they can just improvise and wing it. Even evolution is sometimes explained very wrong. History and humanities is more murky.
Of course "it's just simplified" is a defense that can be attempted to explain away all mistakes.
Dunno, I've been learning a lot of Rust in the past few days. Just dove right into a project and asked AI to teach me stuff on a need to know basis. I'm actually getting used to Rust by now.
Learning programming languages is quite trivial. Many years ago I counted 16 that I had used professionally, now they are more.
It takes way more time to master and be very comfortable with a language due to its ecosystem, though. Some languages are more likely to click with a person, yet underneath they are all the same (minus the functional languages that form their own group), e.g. some performance issues may require looking at the generate assembly code.
A programming language is a skill with an accessible source of irrefutable feedback. If the program doesn't work, then you did something wrong. What happens if you try to learn something less concrete or less testable, like quantum mechanics, as the GP suggested?
I agree with you. I'm also using AI to help me learn electronics so I can finally make some real stuff I can hold in my hands. Lots of hard engineering involved so I'm a lot less confident in my ability to spot the AI's own mistakes.
However, saying you can't learn "anything" is just too strong. I'm definitely managing to distill the AI's weights into my own brain.
It's true in a very real sense, though. Without calibration, how can you tell whether you're learning something real or something fictitious? Imagine trying to learn a language from a single teacher, without ever talking to or corresponding with a native speaker. How could you tell whether you're actually learning the language, or the teacher's unique dialect? Yes, you indeed "learned" something, it's just something that might not bear much resemblance to the real thing.
This made me realize all those benchmarks which unlike loosely acts as a person being awarded with a certificate after his doctorate is rather worthless, for you never know when it bullshit-ted.
Yep. This is the key. You need some kind of knowledge of what the end result should look like to really learn something from LLMs. Otherwise it's a deep dark forest with no way out.
Same. I setup a practice skill that has a curriculum/list of topics, teaching methodology, approach to drills, progress tracking and spaced repetition.
Additionally I asked it to also give me problems relevant in my business domain so that I learn how to directly apply the knowledge in a realistic scenario.
I am getting much more comfortable writing rust than I was barely two weeks ago. More than I was just reading tutorials.
AI is a super polarising topic. Even now there are people who are like “it’s all AI slop so it’s all useless” along with the “OMG AI!!! OMG we will have AGI soon!!!” people.
The truth, as always, is in between. There’s loads of people using it for useful things, learning, automation and getting good results. But it’s also wrong enough that you need to deploy it carefully sometimes.
Don’t worry about either group. Keep objective and use AI where it helps and do it yourself where you are better. That’s all.
yep, I think its even worth considering that some folks are internally conflicted, I am certainly an enthusiastic consumer of AI (or what I can mooch anyways) yet nothing prevents me from considering the downsides, with the usual bleeding heart guilt that comes with a perceived complicity in unfortunate happenings and I'd also probably be a nimby if I were the sort of titled landowner that anyone would listen to.
I like to use them to generate me tutorial series for technical topics I want to learn or improve. So I start a prompt like this:
> I want you to create a tutorial series about X for me. The prime objective is that I improve in topic X so never provide a solution but guide and teach. (for programming never write code).
First create a question catalog to assess my current level.
Then I would ask it to structure the tutorial challenges in the following way:
- Goal
- Concept
- Instructions
I figured that if I don't need to read any additional material on the topic the LLM is giving me too much information and I need to change the prompt.
Works for me and I used this too learn topics I feel now comfortable with, like nushell, opencyper, elisp, boot loaders etc.
But maybe you don't consider this "complex"
Curious: Why not pick up a book or two on the topic, and use the LLM to help you through it [0]? To me, prompting the LLM without grounding it sounds like a sure shot way to end up learning "pop-sci", as GP puts it, instead of the actual science?
If today's top LLMs are reliable enough (without grounding) to academically learn "complex topics" from, may be I need to adjust my priors. I must say, I do find myself chatting about other topics (without the need for grounding) that I'm trying to "absorb" (not really learn), like Behavioural Psychology & Philosophy.
[0] Products like NotebookLM are built specifically for such usecases.
I use books to accompany my learning, but I really need something that _forces_ me to think and solve problems in that space on my own.
Books can sometimes give me the illusion of learning something, but then, when needed, I've notice that I haven't really learned it.
LLMs give me structure based on my current skill level. And basically always I accompany this with books, I love reading. It's a nice combination for me.
I would love to see the "depth" of your knowledge in those topics.
You literally proved the OP's point ... thinking you're learning. More like scratching the surface, with lots of invalid data while not being able to recognize what's invalid.
It's like with latest vector of attacks being spamming Github with malware injected in proper looking code in hope of AI to index it.
Then you paste the code because you don't understand it, but you take it as working and only doing what you've asked for.
This feels like an impossible assessment - yes, a model probably can't give you the education that a advanced/expert book on a topic will, but implying that having a verifiable goal is somehow fake learning feels like an intractable problem.
What level of evidence would be sufficient for you to accept that a model may be able to teach a concept?
I'm happy to take on this challenge with a topic of your choosing, but I don't believe there will be an evidence base that satisfies you that the knowledge is earned or deep enough.
I really wonder where this assessment is coming from? It's not that I use LLM written code for something (in those exercises at least), in fact I don't let the LLM write code (see my prompt example).
It's about guiding me in _doing_ exercises so I learn and I can evaluate if I learned something if I can apply the learning myself.
> It's like with latest vector of attacks being spamming Github with malware injected in proper looking code in hope of AI to index it.
First of all "malware" which is often technology working as intended happens at the OS level, not the github repo level. You can't say a repo is malware in most cases - it depends on the deployment. Second and more importantly, the vast majority of training data would have to reflect the "injected attack" for it to have any bearing.
There are ~30 keywords in JavaScript, compared to 100s of thousands in spoken/written languages.
You cannot "attack" anyone or anything through an LLM. It's an offline, read-only model of text, that's it.
> I'd just really like to see at least one of these to be accompanied by a statement saying what are the kinds of problems the author can now confidently solve that they couldn't before.
I've spent a lot of time with LLMs for the last two years. Something I've tried, almost for decades, is to learn enough CUDA programming to be productive with it when needed. About 6 months ago, after again banging my head against it for weeks, something finally clicked and I feel like I've overcome the initial step of at least grokking the needed ideas so I know where to go next, and I can actually write + compile + use kernels made for my use cases. I won't claim to understand everything, but I couldn't do what I can today, before I learnt the things I now know.
~2 years ago, because of my very weak math foundation, I basically said "Well, CUDA looks really interesting and really fun, but it's too difficult, lets focus on other things", even after reading some starting resources and stuff. But, by asking countless of dumb questions to LLMs, forcing it to steer me in the right direction, when I'm otherwise just driving on the highway or what not, I finally feel like I have a grasp on something I earlier only dreamed about understanding, and I'm able to be productive with it now.
To be fair the entry barrier got a lot lower over the past ~5 years. Now you can write very good CUDA kernels with just a few lines of python DSL code. Zero cpp boilerplate and zero explicit compiler calls.
Stuff like Triton, nvidia warp (the language), numba, cupy jax/pallas and so many others really paved the way. You can start out really high-level, run a profiler and then dive deep into the bottlenecks.
TL,DR: Keep going, it's a great time to have fun with GPUs.
> To be fair the entry barrier got a lot lower over the past ~5 years. Now you can write very good CUDA kernels with just a few lines of python DSL code. Zero cpp boilerplate and zero explicit compiler calls.
Well, yeah, but what I've being doing is learning proper CUDA, not "Python-compiled-to-CUDA" (otherwise it'd take like a just a week to understand enough :P ) and that's looking more or less the same today (although bunch of more complicated stuff piled on top of the fundamentals) as it used to, AFAIK.
With that said, the environment is a lot simpler to setup today at least :)
I wouldn't call one proper CUDA and the other one some dumbed down version. Nvidia really seems to be pushing for these DSLs to be first class within the ecosystem. In some cases probably even more cutting edge than the nvcc frontend, since it's easier to do some experimenting on a new niche package than on the tool everyone relies on.
I believe more and more production code is running kernels which didn't originate from the traditional cuda cpp route.
> I wouldn't call one proper CUDA and the other one some dumbed down version. Nvidia really seems to be pushing for these DSLs to be first class within the ecosystem.
I wouldn't say one is dumbed down either, just different, at least the entrypoints and how you end up using the different solutions.
I'm currently experimenting with cuda-oxide for some new simulations, and managed to keep the entire simulation within just Rust essentially, while going the "traditional" (maybe better term than "proper"?) way I've ended up with a bunch of .cu files and then integrating them (via cudarc usually). Kernels themselves feel the same across both, but the integration clearly makes them different enough that I think it's worth distinguishing them, at least for clarity if nothing else.
If someone else already knew Rust but not C++, wanted to get into CUDA programming, going the cuda-oxide route would probably be easier and more familiar, than cudarc, I'd guess. Personally I'm not sure what route I prefer yet, both (as always?) have tradeoffs.
One thing I've begun to notice is that LLMs list of a bunch of interesting stuff and raise all these thing you consider but often sometimes you just want a more focused response, so these scatter responses kind of lead you to being overwhelmed and losing focus on what you really wanted to do. At least I've started noticing this.
Like I'd ask about some statical approach taken in a paper and suddenly i'm being bombard with all these potential pivots and things I really need to consider, I kind of just want to consider 1 thing at a time and come to things once I fixed the immediate issue. Sure I have no doubt these other pieces of information are useful but it's just not the most useful information I need right now.
This is less of a problem with coding agents more so putting learning related questions to an LLM, like is the method covered in this paper, yes no? instead I get an exhaustive but overwhelming and indirect response that contains part of the answer. I just wanted to know if it was worth my time going through the paper but now I'm being bombarded told all this tangential information, which is unclear to me if I need to consider right at this moment, it's really distracting.
Maybe this is something others adapted to but i've resorted speaking past it saying, "this is the question please stay on topic" or literally "one thing at a time please" and then it narrows in, but they really stretch your attention thin if you're not more aggressive with keeping them on topic. The smarter models are better, and if you use max compute it does a better job.
I think they can definitely be helpful for learning, but you got play an active role, you can't just consume what it says like content.
> many of the popular educational videos on YouTube are fun to watch and don't really teach you anything.
As a tangent, I think the concept of pop science has wasted so much time of what could be considered brilliant minds. I can't believe how much YouTube people I consider really smart consume under the guise of "learning stuff." And the videos are always designed to be addicting and to entice you to watch other of their stuff, which makes sense, because theyre a business, not a school.
I'm guilty of wasting time on YouTube as much as anyone else (I like watching stand-up routines and Red Bull extreme sports) but I am never under the guise that I'm doing anything productive with my time. Its okay to have fun learning, but I always felt that entertainment and education should be kept separate. You gotta learn something intentionally, not just get it served to you via algorithm.
Note im talking about the educational "shorts" not the 60+ minute deep dives that are basically a college level lecture.
It's the kind of high-brow entertainment that makes you feel like you learn something.
For me, most "push" things are edutainment, whether that'd be Youtube videos or public-broadcaster television programs. Things you seek out yourself are not.
In a similar vein, there's "newstertainment" (news that makes you feel like it's important to watch, but actually changes nothing tangible about your life).
Well everyone needs some downtime, and then I think it's important to acknowledge that there is a range of quality within the category of "educational entertainment." You can learn a lot of interesting things from 3BlueOneBrown and Veritasium, even though neither compares to working through a textbook.
I agree, and I dont know much about the channels you mention but im referring more to the short videos with fast cuts that are just designed to keep you watching instead of teaching you anything. Its like the overly processed vegetables you see in many stores ‐ it seems healthy because its real veggies but its actually junk food even if it is better than sugary junk food.
I’m a bit too old for the YouTube generation (can’t stand the fast cuts and permanently agitated voices), but am absolutely guilty of reading a lot of pop-sci books. I feel like they do a better job at teaching stuff for the most part, but probably also make you fall into the trap of believing you actually learnt something…
On the other hand, I’m fine with not being an expert on topics outside of my domain, as long as I retain some basic knowledge and fun party facts. So there’s that.
There's a spectrum between "barely scientific entertainment" and "dry technical reference". Also, it's not fully a zero-sum tradeoff, great authors have written serious textbooks that are quite entertaining to read, and there are pop-sci books that do a great job at covering advanced material.
I do appreciate the literal college lectures on there. Hard to get the algorithm to surface them but they’re golden for learning. Of course I would be unshocked if the authenticity signals I’m tracking (AB.203 Lecture 3 video title, general hubbub and shuffling of chairs at start of video, university affiliated channel) are all faked by grifters 6 months from now.
I’ve used the following teach skill by mattpocock to learn Java concurrency concepts in 20 bite-sized hands-on lessons starting from creating a new thread to building a Thread-safe Connection pool. I’m pretty sure I can adapt this to learn Concurrency in any language like Go or Rust.
The point is not that I’m learning Concurrency in a better way using LLMs, it’s that I can apply this style of learning using bite-sized, hands-on, visual explanations, quiz to any topic in the future. The lessons it generates are just code, and you can ask it to type check the examples, validate with recent libraries, use analogies to learn something better.
Try it before thinking it’s just ELI5 or Summarization or assuming it’ll be hallucinating without verifying facts.
If you ask an LLM to give you a 500-word summary of quantum physics, it'll give you an oversimplification that probably leans on a hodgepodge of pop-sci metaphors.
That's what you'd get if you asked a leading physicist too, so are you saying LLMs have achieved human-level intelligence?
In reality, I would just ask LLM to give me a technical explanation of quantum physics, and ask any vocab and equation I do not understand in the response.
It's like YouTube "Explainer" personalities, like Hank Green and Adam Neely. They do the kind of "learning theater" that makes you feel like you're learning something when you're actually just providing views and ad revenue. You'll come away from a video feeling like you gained knowledge, but:
1) there's a good chance it was subtly misleading (or just wrong)
2) you probably won't ever use the information in any meaningful way and will likely forget all relevant details in a few days
3) you almost certainly could have spent that time better actually doing or creating something - actually doing real learning and making real progress
The algorithm wanted these "educational" videos to be 10 minutes at first and now it seems to want them to be 20 minutes.
Once I saw that I realised just how padded the content really was and started looking elsewhere.
Other good tells the content is low quality or a bad fit for video are excessive amounts of talking head (should have been a podcast) or stock video clips (it's video, show me something).
Listening to YouTube on 2x makes it slightly bearable.
But this was also the joke about TED Talks: everyone came away smugly confident they were part of an amazing tide of progress, ignoring any engineering or scaling challenges.
Ten minutes used to be the cutoff where you could provide an interstitial advertisement. This sort of turned into a sort of wive's tale, where people thought the algorithm would push such videos. It's not the case today.
The reason that 20 minute videos are more popular these days is that more people are watching youtube on larger screens, like on their TV on the couch, and so more reliably watch longer videos than in the past.
> In my experience, LLMs are really good for taking up your time and making you feel like you're learning, in the same way that many of the popular educational videos on YouTube are fun to watch and don't really teach you anything.
You wrote a wall of text just to say you struggle with learning when using some media. That's fine, each one of us struggles with different things. However, I hardly think it's fair to extrapolate your personal struggles with learning styles to everyone in such a sweeping approach, or that this is relevant to the topic.
If you want to go back to the basics, LLMs in the very least work as chatbots that you can use to follow the Socratic method to guide your way through your learning journey. If you still struggle with learning when asking questions and getting specific answers to them then it's safe to say LLMs are not a factor.
I started learning Terraform. At the beginning i let the llm convert some of my homelab infrastructure into a tf project, because thats always the annoying part. Now I play arround with it and whenever I don't understand something or an error message appears, I ask the LLM politely e.g. why this error happened because I thought it should work this way and than it happily explains to me what wen't wrong and if my understanding was wrong or something else was the error cause.
Essentially as I did learn before LLMs appeared but now I dont have to search for some stackoverflow threads, github issues etc. to find something vaguely similar to my problem and how I exactly implement this to my project.
Learning is not about what you put in, it is about what you can take out. It is also not zero-sum, but rather exists on a wide spectrum for any given knowledge domain. A good idea could be to prompt the LLMs to keep quizzing you on what you have read and test on both recall and understanding.
Also learning is not just about truth, it is about curiosity as well. The pop-sci metaphors could actually good for satisfying the curiosity of let's say a 10-year old. What to learn and how to learn is ultimately at the judgement of the learner. The better the judgement, the more the learner can stay closer to the exact scientific details.
Idk about other people but the way I usefully use llms to help with study is to treat them as a good tutor. If you are stuck on a problem or need an explanation of a concept it helps greatly. Much of the actual process of studying Math, Physics, CS, etc in University is sitting in a room working on problem sets and now you can do that with the equivalent of a highly competent Grad Student to bug with any issue you run in to. However yes trying to get an LLM to draft a 1000 ft view of an entire field and calling that learning is a poor use case.
> many of the popular educational videos on YouTube are fun to watch and don't really teach you anything.
I really enjoyed and learn a lot of things from Karpathy's and Andrew Ng's video. sure many don't really teach you anything, but I could say many others are useful too. Maybe it depends on the way we're prompting as well? it seems useful for some like Terence's message that was shared few weeks ago
My trick is to let agents make an University Course Curriculum based of books and papers. Then if I don't understand something, I ask AI to simplify the book/paper until I get it.
Agree. Just pay attention to the follow up questions a learner is asking to see the progress. If the follow up is just "continue", "go on", "next" or a non-sequitur then it is smell of a stall. If it is challenging or filling a gap in the answer then it is progress. So production from the learner is the only signal of worth here not the quality of LLM response, the time spent or the ability of the learner to reproduce the facts given by the LLM.
The test I'd like to see: take a problem set or task you couldn't solve beforehand, learn the topic this way, then try to solve it without the LLM in the loop
I would tend to agree in the case that someone is using LLMs as their primary source for learning. But I've found a lot of use in having a claude project containing the PDF of a textbook I'm working through so that I can ask it to clarify or help me through parts I find confusing. I would definitely say that's greatly accelerated my learning - or at least greatly accelerated the speed at which I integrate information from textbooks.
I'm doing something like this for AES ciphers. I pretty much asked Cursor to write a step by step guide to the math behind everything with annotations. Then when I run into something I don't understand I can ask questions and have it elaborate. It's honestly kind of a fun way to learn about something.
Not sure how relevant my example will be but I built a spaced repetition language learning LLM wrapper app to learn Japanese and it worked really well. I went to great lengths, however, to craft prompts which resulted in more predictable and useful results. ChatGPT on its own couldn’t do this.
prompt: if a family has four boys, would a bayesian conclude that their next child would most likely be a boy?
fable taught me about the beta binomial and large observational studies that came down on both sides of the question about whether per family births are truly binomial. it also told me about countries like the uk and uae that are inching towards national genetic registries that might answer such questions definitively in time. as well as the efforts in Cyprus in this 80s to reduce beta thalassemia through voluntary testing of couples pre marriage.
as ever the real problem is formulating questions, which requires clarity on the next discrete layer of missing comprehension, which requires self-reflection and genuine insight into your own mind.
Have you actually tried? I refreshed a ton of arguments which I had studied or briefly encountered before, so it's less likely to fool me than if I knew 0. And the fact that I can frame a question precisely and as deep as I like is truly unique and incredible.
For some things you still need videos and practice but cmon, I don't get this generalised hate on LLMs, they are based on what us human wrote anyway.
> If you ask an LLM to give you a 500-word summary of quantum physics, it'll give you an oversimplification that probably leans on a hodgepodge of pop-sci metaphors
What would a double PhD in quantum physics provide differently if you asked them for a 500-word summary of such a complex field? What would the human do very differently? I have an Associate Professor from CalTech who teaches quantum physics there and I will have them review your suggestions, so don't hesitate out of concern.
> I'd just really like to see at least one of these to be accompanied by a statement saying what are the kinds of problems the author can now confidently solve that they couldn't before.
I don't get why there's this overwhelming dislike for LLMs on HN. Everytime I make a comment on how insanely productive it has made me, I get downvoted to hell that leads me to be throttled by HN for hours at which point I can't engage in the discussion anymore. I say this in advance because if you post a comment and don't see a response from me until the next day, that's what's going on.
So here's one of a dozen ways LLMs have helped me to learn, execute and deploy ideas rapidly.
MiniPCs that used to cost no more than a burrito, Raspberry Pis, and useful hardware like that are extremely expensive right now: so I have to be creative in finding replacements. I have been able to use GPT5.5/GPT5.6 Sol/Gemini 3.5 Pro/Opus 4.8 to locate cheap ($5) routers and very cheap repeaters that can be reprogrammed to run Linux on them.
This is an extremely intensive, laborious process that requires:
1. Flashing the device over Qualcomm EBL.
2. Validating that I didn't corrupt the 4GB eMMC, then partitioning it.
3. Doing multiple gated commands that verify step by step that the previous command worked correctly and had the desired effect.
4. If not, take remediation steps, failing which alert me so we can do a spike.
5. Remapping certain hardware and Flashing over a working Debian.
6. Install DropSSH + keys and validating that.
7. Installing the scripts, etc.
Since these are not meant to be used this way, and I'm repurposing them, ordering these cheap Wi-Fi routers and flashing them is not a repeatable process. Each one is slightly different from each other. You really can't script it, not reasonably.
OpenCode using the above models has cost me $5 worth of tokens so far to reflash 10 of these repeaters, at $5 each, into Microservers that do my bidding. They have 1 GB of RAM, 4 GB of eMMC, can do USB OTG and being routers have WiFi and BT. A compatible hardware today would cost me ATLEAST $50+ each.
Thanks to the LLMs for walking me through discovering this detail, holding my hands through the process and handing me over these working microservers.
In my experience, LLMs are really good for taking up your time and making you feel like you're learning, in the same way that many of the popular educational videos on YouTube are fun to watch and don't really teach you anything.
If you ask an LLM to give you a 500-word summary of quantum physics, it'll give you an oversimplification that probably leans on a hodgepodge of pop-sci metaphors. And if you start drilling down, you risk drilling down on these ELI5 metaphors, which can get you farther away from truth.