You expect every blog post to find the earliest relevant paper to cite, just so one could look at the year (without reading said paper - which would have made clear that the connection isn’t recent) to assess novelty? I don’t think that’s reasonable.
It’s a blog post. If it was, say, a peer reviewed paper by Hinton or LeCunn that fails to cite Schmidhuber, that would be reasonable criticism in my opinion. (Spoiler: they fail to cite him)
Why would blog posts not be subject to such criticism?
Either the author knew of prior work that argues the same thing and they ignored it, or they didn't know. And if one writes a 1000+ word article premised on this idea, wouldn't one be presumed to know at least in which century the idea originated from?
Arguably these kind of blog posts should be more subject to such criticisms, because the blog posts purport to "teach" the general public about a concept in an authoritative tone (or at least the author seems to pose as knowledgeable in the subject), while for academic papers, everyone who actually reads the paper knows where the ideas came from anyway and it's mainly an issue of attribution (and maybe about fairly distributing the citation count...)
> Why would blog posts not be subject to such criticism?
You're asking why casual comments from amateurs made for fun on the internet shouldn't be held to the same standard as those made by funded career academic experts writing for other experts over months and meant as part of the permanent record of the field?
Personally, I think that's a bit like asking why a friend having you over for dinner isn't always an elegant 7-course meal with wine pairings. I guess you can expect that if you want, but to me it sounds like a child expecting to go to Disney every day: ignoring the economic realities of the situation is a recipe for eternal disappointment.
This is why I hate online arguments. I didn't say earliest, but to give a pointer to the general era, and make it clear how standard the concepts are. They are part of undergrad, it's basic things. I'm not asking for a deep lit review. But modern exposition to anything related to Ai / ML / stats have extreme recency bias and young'uns are led to believe there was nothing before the transformer paper.
I guess it depends on the actual goal of the post. Education? Point people to background. Surely if you have a deep understanding of the subject, you have favored references.
If it's not education, where people are supposed to know everything, who is the audience? If the audience is supposed to know all of the background, then the post is not saying something original to the audience.
I agree the quality of the text itself is good. But I knew the background and I left (well, actually began) already agreeing.
I am happy and want people to write and share stuff. I think it's a really good exercise if only for the author. To be clear, I think the post itself is good except for the gap I mentioned in my original post, which in the grand scheme of things may seem like a nitpick. But if the same type of content makes it to the front page often without me understanding who the audience actually is, I am also allowed to wonder why out loud.
If the blog were about calculus, and stated that an elegant proof of the Fundamental Theorem of Calculus could be found in such and such undergraduate textbook, would you be upset that the citation wasn't to either Newton's or Leibniz' work?
I would want them to point to point to what they think is the best resource. (Assuming the point is educating or explaining something to the reader.) Which, If everyone followed, would automatically point back to newton/leibnez.
"Understanding" is clearly linked to compression. Taking observations and coming up with a more compact representation that explains them, analogous to coming up with a compact set of axioms that generate facts, or a small Turing machine or short program that generates a list of strings.
Intelligence is a broader concept but definitely involves understanding how a system/envoronment works and making predictions about its unfolding, especially actionable ones that allow you to steer that state towards some goal states.
The two contributioms that come to mind are the Hutter cash prize for the best compressor of the English Wikipedia and the work on PPM compression-based
text classification by the late Prof. Ian Witten's group at Waikato (NZ) [1,2].
The model that best compresses the input string was likely generated by the distribution from which the compression model was 'trained'
