By writing, you are voting on the future of the Shoggoth using one of the few currencies it acknowledges: tokens it has to predict. If you aren't writing, you are abdicating the future or your role in it. If you think it's enough to just be a good citizen, to vote for your favorite politician, to pick up litter and recycle, the future doesn't care about you.
These AI predictions never, ever seem to factor in how actual humans will determine what AI-generated media is successful in replacing human-ones, or if it will even be successful at all. It is all very theoretical and to me, shows a fundamental flaw in this style of "sit in a room reading papers/books and make supposedly rational conclusions about the future of the world."
A good example is: today, right now, it is a negative thing for your project to be known as AI-generated. The window of time when it was trendy and cool has largely passed. Having an obviously AI-generated header image on your blog post was cool two years ago, but now it is passé and marks you as behind the trends.
And so for the prediction that everything get swept up by an ultra-intelligent AI that subsequently replaces human-made creations, essays, writings, videos, etc., I am doubtful. Just because it will have the ability to do so doesn't mean that it will be done, or that anyone is going to care.
It seems vastly more likely to me that we'll end up with a solid way of verifying humanity – and thus an economy of attention still focused on real people – and a graveyard of AI-generated junk that no one interacts with at all.
I've been writing for decades with the belief I was training a future AI and
used to say that the Turing test wasn't mysterious at all because it was a solved problem in economics in the form of an indifference curve that showed where peoeple cared whether or not they were dealing with a person or a machine.
the argument against AI taking over is we organize around symbols and narratives and are hypersensitive to waning or inferior memes, thereofre AI would need to reinvent itself as "not-AI" every time so we don't learn to categorize it as slop.
I might agree, but if there were an analogy in music, some limited variations are dominant for decades, and there are precedents where you can generate dominant memes from slop that entrains millions of minds for entire lifetimes. Pop stars are slop from an industry machine that is indistinguishable from AI, and as evidence, current AI can simulate their entire catalogs of meaning. the TV Tropes website even identifies all the elements of cultural slop people should be immune to, but there are still millions of people walking around living out characters and narratives they received from pop-slop.
there will absolutely be a long tail of people whose ontology is shaped by AI slop, just like there is a long tail of people whose ontology is shaped by music, tv, and movies today. that's as close to being swept up in an AI simulation as anything, and perhaps a lot more subtle. or maybe we'll just shake it off.
That is a good point, and fundamentally I agree that these big budget pop star machines do function in a way analogous to an AI, and that we're arguing metaphysics here.
But even if a future AI becomes like this, that doesn't prevent independent writers (like gwern) from still having a unique, non-assimilated voice where they write original content. The arguments tend to be "AI will eat everything, therefore get your writing out there now" and not "this will be a big thing, but not everything."
With AI you need to think, long and hard, about the concept (borrowed from cryptography), "Today, the state of the art in is the worst it will ever be".
Humanity is pinning its future on the thought that we will hit intractable information-theoretic limitations which provide some sort of diminishing returns on performance before a hard takeoff, but the idea that the currently demonstrated methods are high up on some sigmoid curve does not seem at this point credible. AI models are dramatically higher performance this year than last year, and were dramatically better last year than the year before, and will probably continue to get better for the next few years.
That's sufficient to dramatically change a lot of social & economic processes, for better and for worse.
There's a good chance you're right, but I think there's also a chance that things could get worse at some point (with some hand-wavy definition of "a while").
Currently the state-of-the-art is propped up with speculative investments, if those speculations turn out to be wrong enough, or social/economic changes force the capital to get allocated somewhere else, then there could be a significant period of time where access to it goes away for most of us.
We can already see small examples of this from the major model providers. They launch a mind-blowing model, get great benchmarks and press, and then either throttle access or diminish quality to control costs / resources (like Claude Sonnet 3.5 pretty quickly shifted to short, terse responses). Access to SOTA is very resource-constrained and there are a lot of scenarios I can imagine where that could get worse, not better.
Even "Today, the state of the art in is the worst it will ever be" in cryptography isn't always true, like post-spectre/meltdown. You could argue that security improved but perf definitely did not.
I don’t disagree that it’ll change a lot of things in society.
But that isn’t the claim being made, which is that some sort of AI god is being constructed which will develop entirely without the influence of how real human beings actually act. This to me is basically just sci-fi, and it’s frankly kind of embarrassing that it’s taken so seriously.
> dramatically higher performance this year than last year, and were dramatically better last year than the year before
Yeah, but, better at _what_?
Cars are dramatically faster today than 100 years ago. But they still can't fly.
Similarly, LLMs performing better on synthetic benchmarks does not demonstrate that they will eventually become superintelligent beings that will replace humanity.
If you want to actually measure that, then these benchmarks need to start asking questions that demonstrate superintelligence: "Here is a corpus of all current research on nuclear physics, now engineer a hydrogen bomb." My guess is, we will not see much progress.
Having an obviously AI-generated header image on your blog post was cool two years ago, but now it is passé and marks you as behind the trends.
This is true only because publicly-accessible models have been severely nerfed (out of sheer panic, one assumes), making their output immediately recognizable and instantly clichéd.
Dall-E 2, for instance, was much better when it first came out, compared to the current incarnation that has obviously been tweaked to discourage generating anything that resembles contemporary artists' output, and to render everything else in annoying telltale shades of orange and blue.
Eventually better models will appear, or be leaked, and then you won't be able to tell if a given image was generated by AI or not.
My point is that having work that is obviously AI-generated is now a negative thing.
If, in the future, there is a way to validate humanity (as I mentioned in my comment), then any real writers will likely use it.
Anyone that doesn't validate their humanity will be assumed to be an AI. The reaction to this may or may not be negative, but the broader point is that in this scenario, the AI won't be eating all human creations.
I think the wider question mark about that sentence is that even if LLMs that ingest the internet and turn it into different words are the future of humanity, there's an awful lot of stuff in an AI corpus and a comparatively small number intensively researched blogs probably aren't going to shift the needle very much
I mean, you'd probably get more of a vote using generative AI to spam stuff that aligns with your opinions or moving to Kenya to do low wage RHLF stuff...
I don’t believe Gwern lives as frugally as he’s described in this (if this even actually is the real Gwern). I’m 100% sure that he has a persona he likes to portray and being perceived as frugal is a part of that persona.
When it comes to answering the question “who is gwern?” I reckon Gwern’s a plant a seed in people’s mind type of guy, and let them come up with the rest of the story.
Still, I like a lot of his writing. Especially the weird and niche stuff that most people don’t even stop to think about.
And thanks to Gwern’s essay on the sunk costs fallacy, I ended up not getting a tattoo that I had changed my mind about. I almost got it because I had paid a deposit, but I genuinely decided I hated the idea of what I was going to get… and almost got it, but the week before I went to get the tattoo, I read that essay, and decided if small children and animals don’t fall victim to sunk costs, then neither should I!
Literally - Gwern saved the skin on my back with his writing. Haha.
I met Gwern once, when he came to the NYC Less Wrong meetup. I don't think he was internet-famous yet. It was probably 12 years ago or so, but based on my recollection, I'm totally willing to believe that he lives very frugally. FWIW he wouldn't have looked out of place at an anime convention.
But I do know he created an enormous dataset of anime images used to train machine learning and generative AI models [1]. Hosting large datasets is moderately expensive - and it's full of NSFW stuff, so he's probably not having his employer or his college host it. Easy for someone on a six-figure salary, difficult for a person on $12k/year.
Also, I thought these lesswrong folks were all about "effective altruism" and "earning to give" and that stuff.
I'm fairly sure it's relatively true (except for occasional extra purchases on top) unless he's been keeping it up in places I wouldn't expect him to.
I don't like that now people might pigeonhole him a bit by thinking about his effective frugality but I do hope he gets a ton of donations (either directly or via patreon.com/gwern ) to make up for it.
There is a comment on the r/slatestarcodex subreddit with supposedly true information about him (which I found googling 'who is gwern'), but it left me with even more questions.
> Gwern was the first patient to successfully complete a medical transition to the gender he was originally assigned at birth... his older brother died of a Nuvigil overdose in 2001... his (rather tasteful) neck tattoo of the modafinil molecule
The only concrete things we know about gwern are that he's a world-renowned breeder of Maine Coons and that he is the sole known survivor of a transverse cerebral bifurcation.
He does have a neck tattoo, but it's actually a QR code containing the minimal weights to label MNIST at 99% accuracy.
There's some articles on his site about these attempts and they claim that they are all wrong. If it would be that "easily found" I'd guess we wouldn't be having these discussions: https://gwern.net/blackmail#pseudonymity-bounty
This was a tough listen, for two subtly similar reasons.
The voice was uncanny. Simply hard to listen to, despite being realistic. I mean precisely that: it is cognitively difficult to string together meaning from that voice. (I am adjacent to the field of audio production and frequently deal with human- and machine-produced audio. The problem this podcast has with this voice is not unique.) The tonality and meaning do not support each other (this will change as children grow up with these random-tonality voices).
The conversation is excessively verbose. Oftentimes a dearth of reason gets masked by a wide vocabulary. For some audience members I expect the effort to understand the words distracts from the relationship between the words (ie, the meaning), and so it just comes across as a mashup of smart-sounding words, and the host, guest, and show gets lauded for being so intelligent. Cut through the vocabulary and occasional subtle tsks and pshaws and “I-know-more-than-I-am-saying” and you uncover a lot of banter that just does not make good sense: it is not quite correct, or not complete in its reasoning. This unreasoned conversation is fine in its own right (after all, this is how most conversation unfolds, a series of partially reasoned stabs that might lead to something meaningful), but the masking with exotic vocabulary and style is misleading and unkind. Some of these “smart-sounding” snippets are actually just dressed up dumb snippets.
It's a real voice. Probably with some splicing, but I don't know how much. Gwern admits he isn't that good at speaking, and I believe him. He also says he isn't all that smart. Presumably that's highly relative.
The voice was uncanny. Simply hard to listen to,
despite being realistic. I mean precisely that: it
is cognitively difficult to string together meaning
from that voice.
What? According to the information under the linked video,
In order to protect Gwern's anonymity, I proposed
interviewing him in person, and having my friend Chris
Painter voice over his words after. This amused him
enough that he agreed.
I'm not familiar with the SOTA in AI-generated voices, so I could very well be mistaken.
But it did not sound fake to me, and the linked source indicates that it's a human.
Perhaps it sounds uncanny to you because it's a human reading a transcript of a conversation.... and attempting to make it sound conversational, as if he's not reading a transcript?
It's been a bumper week for interesting podcast interviews with an AI theme!
In addition to this, there are Lex Fridman's series of interviews with various key people from Anthropic [0], and a long discussion between Stephen Wolfram and Eliezer Yudkowsky on the theme of AI risk [1].
I found the conversation between Wolfram and Yudkowsky hard to listen to. In fact, I didn't make it to the half. The arguments presented by both were sort of weak and uninteresting?
I find any conversation these days involving Wolfram or Yudkowsky hard to listen to. Them trying to talk to each other... I'd imagine them talking completely past each other, and am happy not to have to verify that.
I'm only half way through that and it IS good, but I wish they wouldn't burn so much valuable time on recaps of the history that has already been told in so many other interviews, and get on to talking about the real changes we should expect going forward.
My favourite Gwern insight is “Bitcoin is Worse is Better”, where they summarize an extensive list of objections to Bitcoin and then ask if there’s a common thread:
No! What’s wrong with Bitcoin is that it’s ugly. … It’s ugly to make your network’s security depend solely on having more brute-force computing power than your opponents, ugly to need now and in perpetuity at least half the processing power just to avoid double-spending … It’s ugly to have a hash tree that just keeps growing … It’s ugly to have a system which can’t be used offline without proxies and workarounds … It’s ugly to have a system that has to track all transactions, publicly … And even if the money supply has to be fixed (a bizarre choice and more questionable than the irreversibility of transactions), what’s with that arbitrary-looking 21 million bitcoin limit? Couldn’t it have been a rounder number or at least a power of 2? (Not that the bitcoin mining is much better, as it’s a massive give-away to early adopters. Coase’s theorem may claim it doesn’t matter how bitcoins are allocated in the long run, but such a blatant bribe to early adopters rubs against the grain. Again, ugly and inelegant.) Bitcoins can simply disappear if you send them to an invalid address. And so on.
I really don’t understand why we give credit to this pile of wishful thinking about the AI corporation with just one visionary at the top.
First: actual visionary CEOs are a niche of a niche.
Second: that is not how most companies work. The existence of the workforce is as important as what the company produces
Third: who will buy or rent those services or products in a society where the most common economy driver (salaried work) is suddenly wiped out?
I am really bothered by these systematic thinkers whose main assumption is that the system can just be changed and morphed willy nilly as if you could completely disregard all of the societal implications.
We are surrounded by “thinkers” who are actually just glorified siloed-thinking engineers high on their own supply.
Gwern's (paraphrased) argument is that an AI is unlikely to be able to construct an extended bold vision where the effects won't be seen for several years, because that requires a significant amount of forecasting and heuristics that are difficult to optimise for.
I haven't decided whether I agree with it, but I can see the thought behind it: the more mechanical work will be automated, but long-term direction setting will require more of a thoughtful hand.
That being said, in a full-automation economy like this, I imagine "AI companies" will behave very differently to human companies: they can react instantly to events, so that a change in direction can be affected in hours or days, not months or years.
> Someone probably said the exact same thing when the first cars appeared.
Without saying anything regarding the arguments for or against AI, I will address this one sentence. This quote is an example of an appeal to hypocrisy in history fallacy, a form of the tu quoque fallacy. Just because someone criticizes X and you compare it to something else (Y) from another time does not mean that the criticism of X is false. There is survivorship bias as well because we now have cars, but in reality, you could've said this same criticism against some other thing that failed, but you don't, because, well, it failed and thus we don't remember it anymore.
The core flaw in this reasoning is that just because people were wrong about one technology in the past doesn't mean current critics are wrong about a different technology now. Each technology needs to be evaluated on its own merits and risks. It's actually a form of dismissing criticism without engaging with its substance. Valid concerns about X should be evaluated based on current evidence and reasoning, not on how people historically reacted to Y or any other technology.
Car and motor vehicles in general get you to work and help you do your work. They don't do the work. I guess that's the difference in thinking.
I'm not sure that it's acrually correct: I don't think we'll actually see "AI" actually replace work in general as a concept. Unless it can quite literally do everything and anything, there will always be something that people can do to auction their time and/or health to acquire some token of social value. It might taken generations to settle out who is the farrier who had their industry annihilated and who is the programmer who had it created. But as long as there's scarcity and ambition in the world, there'll be something there, whether it's "good work" or demeaning toil under the bootheel of a fabulously wealthy cadre of AI mill owners. And there will be scarcity as long as there's a speed of light.
Even if I'm wrong and there isn't, that's why it's called the singularity. There's no way to "see" across such an event in order to make predictions. We could equally all be in permanent infinite bliss, be tortured playthings of a mad God, extinct, or transmuted into virtually immortal energy beings or anything in between.
You might as well ask the dinosaurs whether they thought the ultimate result of the meteor would be pumpkin spice latte or an ASML machine for all the sense it makes.
Anyone claiming to be worrying over what happens after a hypothetical singularity is either engaging in intellectual self-gratification, posing or selling something somehow.
Didn’t say that.
If you posit that the future of the corporation is having a visionary CEO with a few minion middle managers and a swath of AI employees, then tell me, what do you do with the thousands of lost and no longer existing salaried jobs?
Or are you saying that the future is a multitude of corporations of one?
We can play with this travesties of intellectual discourse as long as you like, but we’re really one step removed from some stoners’ basement banter
There is no data, just hyperbole from those same "visionaries" who keep claiming their stochastic parrots will replace everyone's jobs and we therefore need UBI
> I love the example of Isaac Newton looking at the rates of progress in Newton's time and going, “Wow, there's something strange here. Stuff is being invented now. We're making progress. How is that possible?” And then coming up with the answer, “Well, progress is possible now because civilization gets destroyed every couple of thousand years, and all we're doing is we're rediscovering the old stuff.”
The link in this paragraph goes to a post on gwern website. This post contains various links, both internal and external. But I still failed to find one that supports claims about Newton's views on "progress".
> This offers a little twist on the “Singularity” idea: apparently people have always been able to see progress as rapid in the right time periods, and they are not wrong to! We would not be too impressed at several centuries with merely some shipbuilding improvements or a long philosophy poem written in Latin, and we are only modestly impressed by needles or printing presses.
We absolutely _are_ impressed. The concept of "rapid progress" is relative. There was rapid progress then, and there is even more rapid progress now. There is no contradiction.
Anyway, I have no idea how this interview got that many upvotes. I just wasted my time.
That works in reverse too. While I am in awe of what humanity already achieved - when I read fictional timelines of fictional worlds (Middle-Earth or Westeros/Essos) I am wondering how getting frozen in medieval like time is even possible. Like, what are they _doing_?
Those stories are inspired (somewhat) by the dark ages. Stagnation is kinda the default state of mankind. Look at places like Afghanistan. Other than imported western tech, it's basically a medieval society. Between the fall of the Roman Empire and the middle medieval era, technology didn't progress all that much. Many parts of the world were essentially still peasant societies at the start of the 20th century.
All you really need is a government or society that isn't conducive to technological development, either because they persecute it or because they just don't do anything to protect and encourage it (e.g. no patent system or enforceable trade secrets).
Even today, what we see is that technological progress isn't evenly distributed. Most of it comes out of the USA at the moment, a bit from Europe and China. In the past there's usually been one or two places that were clearly ahead and driving things forward, and it moves around over time.
The other thing that inspires the idea of a permanent medieval society is archaeological narratives about ancient Egypt. If you believe their chronologies (which you may not), then Egyptian society was frozen in time for thousands of years with little or no change in any respect. Not linguistic, not religious, not technological. This is unthinkable today but is what academics would have us believe really happened not so long ago.
> I am wondering how getting frozen in medieval like time is even possible. Like, what are they _doing_?
Not discovering sources of cheap energy and other raw inputs. If you look carefully at history, every rapid period of growth was preceded by a discovery or conquest of cheap energy and resources. You need excess to grow towards the next equilibrium.
You're right, really: it's not possible. It's a problem with the conservative impulse (*for a very specific meaning of conservative) in fiction: things don't stay frozen in amber like that. If it was nonfiction - aka real life - the experience of life itself from the perspective of living people would change and transform rapidly in the century view.
They're probably doing the same thing humans on our earth were doing for centuries until ~1600. Surviving. Given how cruel nature is I think we're lucky to have the resources to do more than just survive, to build up all this crazy technology we don't strictly need to live, just for fun/profit.
> The link in this paragraph goes to a post on gwern website. This post contains various links, both internal and external. But I still failed to find one that supports claims about Newton's views on "progress".
Could you explain further what part of https://gwern.net/newton you thought didn't support my description of Newton's view?
I thought the large second blockquote in https://gwern.net/newton#excerpts , which very prominently links to https://www.newtonproject.ox.ac.uk/view/texts/normalized/THE... , fully justified my claims, which are taken directly from Newton's statements to his son-in-law, and also closely parallel other historical statements, like Lucretius, which I also present with clear references and specific blockquotes.
I'm a little mystified that you could describe any of this as not supporting it at all, and I'm wondering if you are looking at the wrong page or something?
> “Well, progress is possible now because civilization gets destroyed every couple of thousand years, and all we're doing is we're rediscovering the old stuff.”
Irrespective of the historical accuracy of the quote I've always felt this way in some form, having personally lived through the transition from a world where it felt like you didn't have to have an opinion on everything to one dominated by the ubiquitous presence of the Internet. Although not so much because I believe an advanced human civilization has destroyed itself in our current timeline, but because the presence of so many life-changing breakthroughs in such a short period of time to me indicates a unceasing march towards a Great Filter.
This will come across as vituperative and I guess it is a bit but I've interacted with Gwern on this forum and the interaction that has stuck to me is in this thread, where Gwern mistakes a^nb^n as a regular (but not context-free) language (and calls my comment "not even wrong"):
Again I'm sorry for the negativity, but already at the time Gwern was held up by a certain, large, section of the community as an important influencer in AI. For me that's just a great example of how basically the vast majority of AI influencers (who vie for influence on social media, rather than research) are basically clueless about AI and CS and only have second-hand knowledge, which I guess they're good at organising and popularising, but not more than that. It's easy to be a cheer leader for the mainstream view on AI. The hard part is finding, and following, unique directions.
With apologies again for the negative slant of the comment.
> For me that's just a great example of how basically the vast majority of AI influencers (who vie for influence on social media, rather than research) are basically clueless about AI and CS
This is a bit stark: there are many great knowledgeable engineers and scientists who would not get your point about a^nb^n. It's impossible to know 100% of of such a wide area as "AI and CS".
>> This is a bit stark: there are many great knowledgeable engineers and scientists who would not get your point about a^nb^n. It's impossible to know 100% of of such a wide area as "AI and CS".
I think, engineers, yes, especially those who don't have a background in academic CS. But scientists, no, I don't think so. I don't think it's possible to be a computer scientist without knowing the difference between a regular and a super-regular language. As to knowing that a^nb^n specifically is context-free, as I suggest in the sibling comment, computer scientists who are also AI specialists would recognise a^nb^n immediately, as they would Dyck languages and Reber grammars, because those are standard tests of learnability used to demonstrate various principles, from the good old days of purely symbolic AI, to the brave new world of modern deep learning.
For example, I learned about Reber grammars for the first time when I was trying to understand LSTMs, when they were all the hype in Deep Learning, at the time I was doing my MSc in 2014. Online tutorials on coding LSTMs used Reber grammars as the dataset (because, as with other formal grammars it's easy to generate tons of strings from them and that's awfully convenient for big data approaches).
Btw that's really the difference between a computer scientist and a computer engineer: the scientist knows the theory. That's what they do to you in CS school, they drill that stuff in your head with extreme prejudice; at least the good schools do. I see this with my partner who is 10 times a better engineer than me and yet hasn't got a clue what all this Chomsky hierarhcy stuff is. But then, my partner is not trying to be an AI influencer.
is it really? this is the most common example for context free languages and something most first year CS students will be familiar with.
totally agree that you can be a great engineer and not be familiar with it, but seems weird for an expert in the field to confidently make wrong statements about this.
Regarding your linked comment, my takeaway is that the very theoretical task of being able to recognize an infinite language isn't very relevent to the non-formal, intuitive idea of "intelligence"
Transformers can easily intellectually understand a^nb^n, even though they couldn't recognize whether an arbitrarily long string is a member of the language -- a restriction humans share!, since eventually a human, too, would lose track of the count, for a long enough string.
I don't know what "intellectually understand" means in the context of Transformers. My older comment was about the ability of neural nets to learn automata from examples, a standard measure of the learning ability of a machine learning system. I link to a paper below where Transformers and RNNs are compared on their ability to learn automata along the entire Chomsky hierarchy and as other work has also shown, they don't do that well (although there are some surprising surprises).
>> Regarding your linked comment, my takeaway is that the very theoretical task of being able to recognize an infinite language isn't very relevent to the non-formal, intuitive idea of "intelligence"
That depends on who you ask. My view is that automata are relevant to computation and that's why we study them in computer science. If we were biologists, we would study beetles. The question is whether computation , as we understand it on the basis of computer science, has anything to do with intelligence. I think it does, but that it's not the whole shebang. There is a long debate on that in AI and the cognitive sciences and the jury is still out, despite what many of the people working on LLMs seem to believe.
FWIW, I’ve had a very similar encounter with another famous AI influencer who started lecturing me on fake automata theory that any CS undergrad would have picked up on. 140k+ followers, featured on the all the big podcasts (Lex, MLST). I never corrected him but made a mental note not to trust the guy.
It seems like his objection is that "parsing formal grammars" isn't the point of LLMs, which is fair. He was wrong about RGs vs CFGs, but I would bet that the majority of programmers are not familiar with the distinction, and learning the classification of a^nb^n is a classic homework problem in automata theory specifically because it's surprising that such a simple grammar is CF.
I take the Feynman view here; vain memory tricks are not themselves net new production, so just look known things up in the book.
Appreciate the diversity in the effort, but engineering is making things people can use without having to know it all. Far more interesting endeavor than being a human Google search engine.
No, look. If a student (I'm not a professor, just a post-doc) doesn't know this stuff, I'll point them to the book so they can look it up, and move on. But the student will not tell me I'm "not even wrong" with the arrogance of fifty cardinals while at the same time pretending to be an expert [1]. It's OK to not know, it's OK to not know that you don't know, but arrogant ignorance is not a good look on anyone.
And there's a limit to what you need to look up in a book. The limit moves further up the more you work with a certain kind of tool or study a certain kind of knowledge. I have to look up trigonometry every single time I need it because I only use it sparingly. I don't need to look up SLD-Resolution, which is my main subject. How much would Feynman need to look up when debating physics?
So when someone like Feynman talks about physics, you listen carefully because you know they know their shit and a certain kind of nerd deeply appreciates deep knowledge. When someone elbows themselves in the limelight and demands everyone treats them as an expert, but they don't know the basics, what do you conclude? I conclude that they're pretending to know a bunch of stuff they don't know.
________________
[1] ... some do. But they're students so it's OK, they're just excited to have learned so much and don't yet know how much they don't. You explain the mistake, point them to the book, and move on.
To the person that commented that five years is an awful long time to remember something like that (and then deleted their comment): you are so right. I am trying to work through this kind of thing :/
In the thread you linked, Gwern says in response to someone else that NNs excel at many complex real-world tasks even if there are some tasks where they fail but humans (or other models) succeed. You try to counter that by bringing up an example for the latter type of task? And then try to argue that this proves Gwern wrong?
Whether they said "regular grammar" or "context-free grammar" doesn't even matter, the meaning of their message is still the exact same.
a^nb^n is regular, but it is also context free. I don't think there's a restriction on the n. Why do you say this?
Edit: sorry, I read "finite" as "infinite" :0 But n can be infinite and a^nb^n is still regular, and also context free. To be clear, the Chomskky Hierarchy of formal languages goes like this:
That's because formal languages are identified with the automata that accept them and when an automaton accepts e.g. the Recursively Enumerable languages, then it also accepts the context-sensitive languages, and so on all the way down to the finite languages. One way to think of this is that an automaton is "powerful enough" to recognise the set of strings that make up a language.
Being an influencer requires very little actual competence, same goes for AI influencers.
The goal of influencers is to influence the segment of a crowd who cares about influencers. Meaning retards and manchildren looking for an external source to form consensus around.
Readit News
These AI predictions never, ever seem to factor in how actual humans will determine what AI-generated media is successful in replacing human-ones, or if it will even be successful at all. It is all very theoretical and to me, shows a fundamental flaw in this style of "sit in a room reading papers/books and make supposedly rational conclusions about the future of the world."
A good example is: today, right now, it is a negative thing for your project to be known as AI-generated. The window of time when it was trendy and cool has largely passed. Having an obviously AI-generated header image on your blog post was cool two years ago, but now it is passé and marks you as behind the trends.
And so for the prediction that everything get swept up by an ultra-intelligent AI that subsequently replaces human-made creations, essays, writings, videos, etc., I am doubtful. Just because it will have the ability to do so doesn't mean that it will be done, or that anyone is going to care.
It seems vastly more likely to me that we'll end up with a solid way of verifying humanity – and thus an economy of attention still focused on real people – and a graveyard of AI-generated junk that no one interacts with at all.
the argument against AI taking over is we organize around symbols and narratives and are hypersensitive to waning or inferior memes, thereofre AI would need to reinvent itself as "not-AI" every time so we don't learn to categorize it as slop.
I might agree, but if there were an analogy in music, some limited variations are dominant for decades, and there are precedents where you can generate dominant memes from slop that entrains millions of minds for entire lifetimes. Pop stars are slop from an industry machine that is indistinguishable from AI, and as evidence, current AI can simulate their entire catalogs of meaning. the TV Tropes website even identifies all the elements of cultural slop people should be immune to, but there are still millions of people walking around living out characters and narratives they received from pop-slop.
there will absolutely be a long tail of people whose ontology is shaped by AI slop, just like there is a long tail of people whose ontology is shaped by music, tv, and movies today. that's as close to being swept up in an AI simulation as anything, and perhaps a lot more subtle. or maybe we'll just shake it off.
But even if a future AI becomes like this, that doesn't prevent independent writers (like gwern) from still having a unique, non-assimilated voice where they write original content. The arguments tend to be "AI will eat everything, therefore get your writing out there now" and not "this will be a big thing, but not everything."
The pop industry is a:
- machine
- which takes authentic human meaning
- and produces essentially a stochastic echo of it ("slop")
- in an optimization algorithm
- to predict the next most profitable song (the song that is most "likely")
So, this sounds an awful lot like something else that's very in vogue right now. Only it was invented in 1950 or 1960, not in 2017.
Humanity is pinning its future on the thought that we will hit intractable information-theoretic limitations which provide some sort of diminishing returns on performance before a hard takeoff, but the idea that the currently demonstrated methods are high up on some sigmoid curve does not seem at this point credible. AI models are dramatically higher performance this year than last year, and were dramatically better last year than the year before, and will probably continue to get better for the next few years.
That's sufficient to dramatically change a lot of social & economic processes, for better and for worse.
Currently the state-of-the-art is propped up with speculative investments, if those speculations turn out to be wrong enough, or social/economic changes force the capital to get allocated somewhere else, then there could be a significant period of time where access to it goes away for most of us.
We can already see small examples of this from the major model providers. They launch a mind-blowing model, get great benchmarks and press, and then either throttle access or diminish quality to control costs / resources (like Claude Sonnet 3.5 pretty quickly shifted to short, terse responses). Access to SOTA is very resource-constrained and there are a lot of scenarios I can imagine where that could get worse, not better.
Even "Today, the state of the art in is the worst it will ever be" in cryptography isn't always true, like post-spectre/meltdown. You could argue that security improved but perf definitely did not.
But that isn’t the claim being made, which is that some sort of AI god is being constructed which will develop entirely without the influence of how real human beings actually act. This to me is basically just sci-fi, and it’s frankly kind of embarrassing that it’s taken so seriously.
Yeah, but, better at _what_?
Cars are dramatically faster today than 100 years ago. But they still can't fly.
Similarly, LLMs performing better on synthetic benchmarks does not demonstrate that they will eventually become superintelligent beings that will replace humanity.
If you want to actually measure that, then these benchmarks need to start asking questions that demonstrate superintelligence: "Here is a corpus of all current research on nuclear physics, now engineer a hydrogen bomb." My guess is, we will not see much progress.
This is true only because publicly-accessible models have been severely nerfed (out of sheer panic, one assumes), making their output immediately recognizable and instantly clichéd.
Dall-E 2, for instance, was much better when it first came out, compared to the current incarnation that has obviously been tweaked to discourage generating anything that resembles contemporary artists' output, and to render everything else in annoying telltale shades of orange and blue.
Eventually better models will appear, or be leaked, and then you won't be able to tell if a given image was generated by AI or not.
If, in the future, there is a way to validate humanity (as I mentioned in my comment), then any real writers will likely use it.
Anyone that doesn't validate their humanity will be assumed to be an AI. The reaction to this may or may not be negative, but the broader point is that in this scenario, the AI won't be eating all human creations.
I mean, you'd probably get more of a vote using generative AI to spam stuff that aligns with your opinions or moving to Kenya to do low wage RHLF stuff...
Deleted Comment
Still, I like a lot of his writing. Especially the weird and niche stuff that most people don’t even stop to think about. And thanks to Gwern’s essay on the sunk costs fallacy, I ended up not getting a tattoo that I had changed my mind about. I almost got it because I had paid a deposit, but I genuinely decided I hated the idea of what I was going to get… and almost got it, but the week before I went to get the tattoo, I read that essay, and decided if small children and animals don’t fall victim to sunk costs, then neither should I! Literally - Gwern saved the skin on my back with his writing. Haha.
But I do know he created an enormous dataset of anime images used to train machine learning and generative AI models [1]. Hosting large datasets is moderately expensive - and it's full of NSFW stuff, so he's probably not having his employer or his college host it. Easy for someone on a six-figure salary, difficult for a person on $12k/year.
Also, I thought these lesswrong folks were all about "effective altruism" and "earning to give" and that stuff.
[1] https://gwern.net/danbooru2021
I don't like that now people might pigeonhole him a bit by thinking about his effective frugality but I do hope he gets a ton of donations (either directly or via patreon.com/gwern ) to make up for it.
Deleted Comment
EDIT: grammar
> Gwern was the first patient to successfully complete a medical transition to the gender he was originally assigned at birth... his older brother died of a Nuvigil overdose in 2001... his (rather tasteful) neck tattoo of the modafinil molecule
The only concrete things we know about gwern are that he's a world-renowned breeder of Maine Coons and that he is the sole known survivor of a transverse cerebral bifurcation.
He does have a neck tattoo, but it's actually a QR code containing the minimal weights to label MNIST at 99% accuracy.
I’m willing to bet he became a millionaire thanks to bitcoin.
The voice was uncanny. Simply hard to listen to, despite being realistic. I mean precisely that: it is cognitively difficult to string together meaning from that voice. (I am adjacent to the field of audio production and frequently deal with human- and machine-produced audio. The problem this podcast has with this voice is not unique.) The tonality and meaning do not support each other (this will change as children grow up with these random-tonality voices).
The conversation is excessively verbose. Oftentimes a dearth of reason gets masked by a wide vocabulary. For some audience members I expect the effort to understand the words distracts from the relationship between the words (ie, the meaning), and so it just comes across as a mashup of smart-sounding words, and the host, guest, and show gets lauded for being so intelligent. Cut through the vocabulary and occasional subtle tsks and pshaws and “I-know-more-than-I-am-saying” and you uncover a lot of banter that just does not make good sense: it is not quite correct, or not complete in its reasoning. This unreasoned conversation is fine in its own right (after all, this is how most conversation unfolds, a series of partially reasoned stabs that might lead to something meaningful), but the masking with exotic vocabulary and style is misleading and unkind. Some of these “smart-sounding” snippets are actually just dressed up dumb snippets.
Oh, how humbling and modest of him!
But it did not sound fake to me, and the linked source indicates that it's a human.
Perhaps it sounds uncanny to you because it's a human reading a transcript of a conversation.... and attempting to make it sound conversational, as if he's not reading a transcript?
In addition to this, there are Lex Fridman's series of interviews with various key people from Anthropic [0], and a long discussion between Stephen Wolfram and Eliezer Yudkowsky on the theme of AI risk [1].
0. https://youtu.be/ugvHCXCOmm4
1. https://youtu.be/xjH2B_sE_RQ
First: actual visionary CEOs are a niche of a niche. Second: that is not how most companies work. The existence of the workforce is as important as what the company produces Third: who will buy or rent those services or products in a society where the most common economy driver (salaried work) is suddenly wiped out?
I am really bothered by these systematic thinkers whose main assumption is that the system can just be changed and morphed willy nilly as if you could completely disregard all of the societal implications.
We are surrounded by “thinkers” who are actually just glorified siloed-thinking engineers high on their own supply.
I haven't decided whether I agree with it, but I can see the thought behind it: the more mechanical work will be automated, but long-term direction setting will require more of a thoughtful hand.
That being said, in a full-automation economy like this, I imagine "AI companies" will behave very differently to human companies: they can react instantly to events, so that a change in direction can be affected in hours or days, not months or years.
Where is the data showing that more jobs get destroyed than created by technological disruption?
Without saying anything regarding the arguments for or against AI, I will address this one sentence. This quote is an example of an appeal to hypocrisy in history fallacy, a form of the tu quoque fallacy. Just because someone criticizes X and you compare it to something else (Y) from another time does not mean that the criticism of X is false. There is survivorship bias as well because we now have cars, but in reality, you could've said this same criticism against some other thing that failed, but you don't, because, well, it failed and thus we don't remember it anymore.
The core flaw in this reasoning is that just because people were wrong about one technology in the past doesn't mean current critics are wrong about a different technology now. Each technology needs to be evaluated on its own merits and risks. It's actually a form of dismissing criticism without engaging with its substance. Valid concerns about X should be evaluated based on current evidence and reasoning, not on how people historically reacted to Y or any other technology.
I'm not sure that it's acrually correct: I don't think we'll actually see "AI" actually replace work in general as a concept. Unless it can quite literally do everything and anything, there will always be something that people can do to auction their time and/or health to acquire some token of social value. It might taken generations to settle out who is the farrier who had their industry annihilated and who is the programmer who had it created. But as long as there's scarcity and ambition in the world, there'll be something there, whether it's "good work" or demeaning toil under the bootheel of a fabulously wealthy cadre of AI mill owners. And there will be scarcity as long as there's a speed of light.
Even if I'm wrong and there isn't, that's why it's called the singularity. There's no way to "see" across such an event in order to make predictions. We could equally all be in permanent infinite bliss, be tortured playthings of a mad God, extinct, or transmuted into virtually immortal energy beings or anything in between.
You might as well ask the dinosaurs whether they thought the ultimate result of the meteor would be pumpkin spice latte or an ASML machine for all the sense it makes.
Anyone claiming to be worrying over what happens after a hypothetical singularity is either engaging in intellectual self-gratification, posing or selling something somehow.
The link in this paragraph goes to a post on gwern website. This post contains various links, both internal and external. But I still failed to find one that supports claims about Newton's views on "progress".
> This offers a little twist on the “Singularity” idea: apparently people have always been able to see progress as rapid in the right time periods, and they are not wrong to! We would not be too impressed at several centuries with merely some shipbuilding improvements or a long philosophy poem written in Latin, and we are only modestly impressed by needles or printing presses.
We absolutely _are_ impressed. The concept of "rapid progress" is relative. There was rapid progress then, and there is even more rapid progress now. There is no contradiction.
Anyway, I have no idea how this interview got that many upvotes. I just wasted my time.
All you really need is a government or society that isn't conducive to technological development, either because they persecute it or because they just don't do anything to protect and encourage it (e.g. no patent system or enforceable trade secrets).
Even today, what we see is that technological progress isn't evenly distributed. Most of it comes out of the USA at the moment, a bit from Europe and China. In the past there's usually been one or two places that were clearly ahead and driving things forward, and it moves around over time.
The other thing that inspires the idea of a permanent medieval society is archaeological narratives about ancient Egypt. If you believe their chronologies (which you may not), then Egyptian society was frozen in time for thousands of years with little or no change in any respect. Not linguistic, not religious, not technological. This is unthinkable today but is what academics would have us believe really happened not so long ago.
Not discovering sources of cheap energy and other raw inputs. If you look carefully at history, every rapid period of growth was preceded by a discovery or conquest of cheap energy and resources. You need excess to grow towards the next equilibrium.
https://en.wikipedia.org/wiki/House_of_Wisdom#Destruction_by...
https://en.wikipedia.org/wiki/Library_of_Alexandria#Burning_...
https://en.wikipedia.org/wiki/Nalanda_mahavihara#Destruction...
https://en.wikipedia.org/wiki/Imperial_Library_of_Constantin...
https://en.wikipedia.org/wiki/List_of_destroyed_libraries#Hu...
Could you explain further what part of https://gwern.net/newton you thought didn't support my description of Newton's view?
I thought the large second blockquote in https://gwern.net/newton#excerpts , which very prominently links to https://www.newtonproject.ox.ac.uk/view/texts/normalized/THE... , fully justified my claims, which are taken directly from Newton's statements to his son-in-law, and also closely parallel other historical statements, like Lucretius, which I also present with clear references and specific blockquotes.
I'm a little mystified that you could describe any of this as not supporting it at all, and I'm wondering if you are looking at the wrong page or something?
Irrespective of the historical accuracy of the quote I've always felt this way in some form, having personally lived through the transition from a world where it felt like you didn't have to have an opinion on everything to one dominated by the ubiquitous presence of the Internet. Although not so much because I believe an advanced human civilization has destroyed itself in our current timeline, but because the presence of so many life-changing breakthroughs in such a short period of time to me indicates a unceasing march towards a Great Filter.
Then there isn't a downvote option if it proves poor.
https://news.ycombinator.com/item?id=21559620
Again I'm sorry for the negativity, but already at the time Gwern was held up by a certain, large, section of the community as an important influencer in AI. For me that's just a great example of how basically the vast majority of AI influencers (who vie for influence on social media, rather than research) are basically clueless about AI and CS and only have second-hand knowledge, which I guess they're good at organising and popularising, but not more than that. It's easy to be a cheer leader for the mainstream view on AI. The hard part is finding, and following, unique directions.
With apologies again for the negative slant of the comment.
This is a bit stark: there are many great knowledgeable engineers and scientists who would not get your point about a^nb^n. It's impossible to know 100% of of such a wide area as "AI and CS".
I think, engineers, yes, especially those who don't have a background in academic CS. But scientists, no, I don't think so. I don't think it's possible to be a computer scientist without knowing the difference between a regular and a super-regular language. As to knowing that a^nb^n specifically is context-free, as I suggest in the sibling comment, computer scientists who are also AI specialists would recognise a^nb^n immediately, as they would Dyck languages and Reber grammars, because those are standard tests of learnability used to demonstrate various principles, from the good old days of purely symbolic AI, to the brave new world of modern deep learning.
For example, I learned about Reber grammars for the first time when I was trying to understand LSTMs, when they were all the hype in Deep Learning, at the time I was doing my MSc in 2014. Online tutorials on coding LSTMs used Reber grammars as the dataset (because, as with other formal grammars it's easy to generate tons of strings from them and that's awfully convenient for big data approaches).
Btw that's really the difference between a computer scientist and a computer engineer: the scientist knows the theory. That's what they do to you in CS school, they drill that stuff in your head with extreme prejudice; at least the good schools do. I see this with my partner who is 10 times a better engineer than me and yet hasn't got a clue what all this Chomsky hierarhcy stuff is. But then, my partner is not trying to be an AI influencer.
totally agree that you can be a great engineer and not be familiar with it, but seems weird for an expert in the field to confidently make wrong statements about this.
Transformers can easily intellectually understand a^nb^n, even though they couldn't recognize whether an arbitrarily long string is a member of the language -- a restriction humans share!, since eventually a human, too, would lose track of the count, for a long enough string.
>> Regarding your linked comment, my takeaway is that the very theoretical task of being able to recognize an infinite language isn't very relevent to the non-formal, intuitive idea of "intelligence"
That depends on who you ask. My view is that automata are relevant to computation and that's why we study them in computer science. If we were biologists, we would study beetles. The question is whether computation , as we understand it on the basis of computer science, has anything to do with intelligence. I think it does, but that it's not the whole shebang. There is a long debate on that in AI and the cognitive sciences and the jury is still out, despite what many of the people working on LLMs seem to believe.
Yes, LLMs are bad at this. A similar example: SAT solvers can't solve the pigeonhole problem without getting into a loop
It is an exceptional case that requires "metathinking" maybe, rather than a showstopper issue
(can't seem to be able to write the grammar name, the original comment from the discussion had it)
Appreciate the diversity in the effort, but engineering is making things people can use without having to know it all. Far more interesting endeavor than being a human Google search engine.
And there's a limit to what you need to look up in a book. The limit moves further up the more you work with a certain kind of tool or study a certain kind of knowledge. I have to look up trigonometry every single time I need it because I only use it sparingly. I don't need to look up SLD-Resolution, which is my main subject. How much would Feynman need to look up when debating physics?
So when someone like Feynman talks about physics, you listen carefully because you know they know their shit and a certain kind of nerd deeply appreciates deep knowledge. When someone elbows themselves in the limelight and demands everyone treats them as an expert, but they don't know the basics, what do you conclude? I conclude that they're pretending to know a bunch of stuff they don't know.
________________
[1] ... some do. But they're students so it's OK, they're just excited to have learned so much and don't yet know how much they don't. You explain the mistake, point them to the book, and move on.
Dead Comment
Deleted Comment
In the thread you linked, Gwern says in response to someone else that NNs excel at many complex real-world tasks even if there are some tasks where they fail but humans (or other models) succeed. You try to counter that by bringing up an example for the latter type of task? And then try to argue that this proves Gwern wrong?
Whether they said "regular grammar" or "context-free grammar" doesn't even matter, the meaning of their message is still the exact same.
Edit: sorry, I read "finite" as "infinite" :0 But n can be infinite and a^nb^n is still regular, and also context free. To be clear, the Chomskky Hierarchy of formal languages goes like this:
Finite ⊆ Regular ⊆ Context-Free ⊆ Context-Sensitive ⊆ Recursively Enumerable
That's because formal languages are identified with the automata that accept them and when an automaton accepts e.g. the Recursively Enumerable languages, then it also accepts the context-sensitive languages, and so on all the way down to the finite languages. One way to think of this is that an automaton is "powerful enough" to recognise the set of strings that make up a language.
Specifically, you can construct a finite automata to represent it.
The goal of influencers is to influence the segment of a crowd who cares about influencers. Meaning retards and manchildren looking for an external source to form consensus around.