My master's was in Convolutional NNs for language processing. I had zero prior knowledge and my advisor recommended I watch Karpathy's lectures[1] to get up to speed
Let me say, he's a great teacher! I took a CV class with him. He should teach more, and take it seriously.
Being a popular AI influencer is not necessarily correlated with being a good researcher though. And I would argue there is a strong indication that it is negatively correlated with being a good business leader / founder.
Here's to hoping he chills out and goes back to the sorely needed lost art of explaining complicated things in elegant ways, and doesn't stray too far back into wasting time with all the top sheisters of the valley.
Edit: the more I think about it, the more I realize that it probably screws with a person to have their tweets get b-lined to the front page of hackernews. It makes you a target for offers and opportunities because of your name/influence, but not necessarily because of your underlying "best fit"
if only we compensated that knowledge properly. Youtube seems to come the closest, but Youtube educators also show how much time you have to spend attracting views instead of teaching expertise.
> It makes you a target for offers and opportunities because of your name/influence, but not necessarily because of your underlying "best fit"
That's unfortunately life in a nutshell. The best fits rarely end up getting any given position. May be overqualified, filtered out in the HR steps, or rejected for some ephemeral reason (making them RTO, not accepting their counteroffer, potentially illegal factors behind closed doors, etc).
it's a crappy game so I don't blame anyone for using whatever cards they are dealt.
> Youtube seems to come the closest, but Youtube educators also show how much time you have to spend attracting views instead of teaching expertise.
Actually for all the attention that the top Youtubers get (in terms of revenue), the reality is that it's going to be impossible to replace teaching income with popular Youtube videos alone.
Based on what I've seen, 1 million video views on Youtube gets you something like $5-10K. And that's with a primarily US audience that has the higher CPM / RPM. So your channel(s) would need to get to about 6 million views per year, primarily US driven, in order to get to earning a median US wage.
Something I've been thinking a lot about is the transition into post scarcity and how we need to dramatically alter the incentive structures and payment allocations.
I've been asking this question for about a decade and still have no good solutions: "What do you do when x% of your workforce is unemployable?" (being that x% of jobs are removed without replacement. Imagine sophisticated and cheap robots. Or if needed, magic)
This is a thought experiment, so your answer can't be "there'll be new jobs." Even if you believe that's what'll happen in real life, it's not in bounds of the thought experiment. It is best to consider multiple values of x because it is likely to change and that would more reflect a post scarcity transition. It is not outside the realms of possibility that in the future you can obtain food, shelter, and medical care for free or at practically no cost. "Too cheap to meter" if you will.
I'll give you two answers that I've gotten that I find interesting. I do not think either are great and they each have issues. 1) jobs programs. Have people do unnecessary jobs simply so they create work wherein we can compensate them. 2) Entertainment. People are, on average, far more interested in watching people play chess against one another than computers, despite the computer being better. So reasons that this ,,might,, not go away.
>The best fits rarely end up getting any given position.
This can be self-fulfilling.
In an organization beyond a certain size, there will be more almost-adequate-fits than there are leadership positions. This could be about like a recognized baseline which seems like it really needs to be scrutinized closely to see exactly who might be slightly above or below the line.
Or in a small company where there is not any almost-fit whatsoever, imagination can result in an ideal that is equally recognizable, but also might not be fully attainable.
Either way it could be OK but not exactly the best-fit.
If good fortune smiles and the rare more-than-adequate-fit appears anywhere on the horizon though, it's so unfamiliar they fly right over the radar.
I think good teachers make great researchers, because they have to understand their field very well, they anticipate and ask themselves the questions that need to be asked, they manage to always see their field with fresh eyes, they are good collaborators, and most importantly, good communicators.
My question is this, great educators like Karpathy make things from 'scratch' and explain in a way that I can understand. Is it a matter of the instructor ability to do this or it's a matter of the student(i.e. me) not having enough chops to understand material from elsewhere?
A teacher can usually adapt the content depending on its audience, I would not teach the research in my field at the same level to professionals, PhDs, master students, bachelor students, amateurs, or even school students.
If what I'm teaching is fairly complex, it requires a lot of background that I could teach, but I would not have the time to do so, because it would be to the detriment of other students. So, while I usually teach 'from scratch', depending on my audience I will obfuscate some details (that I can answer separately if a question is asked) and usually I will dramatically change the speed of the lessons depending on the previous background, because I need to assume that the student has the prerequisite background to understand at that speed fairly complex material.
As an example, I gave some explanations to a student from zero to transformers, it took several hours with lots of questions, the same presentation to a teacher not in the field took me 1h30 and to a PhD in a related field took 25 minutes, the content was exactly the same, and it was from scratch, but the background in the audience was fairly different.
Frankly, OpenAI seems to be losing its luster, and fast.
Plugins were a failure. GPTs are a little better, but I still don't see the product market fit. GPT-4 is still king, but not by that much any more. It's not even clear that they're doing great research, because they don't publish.
GPT-5 has to be incredibly good at this point, and I'm not sure that it will be.
I know things keep moving faster and faster, especially in this space, but GPT-4 is less than a year old. Claiming they are losing their luster, because they aren’t shaking the earth with new models every quarter, seems a little ridiculous.
As the popularity has exploded, and ethical questions have become increasingly relevant, it is probably worth taking some time to nail certain aspects down before releasing everything to the public for the sake of being first.
Given how fast the valuation of the company and the scope of their ambition (e.g. raising a trillion dollars for chip manufacturing) has been hyped up, I think it's fair to say "You live by the hype, you die by the hype."
You don't lose your luster only by not innovating.
Altman saga, allowing military use and other small things step by step tarnish your reputation and pushes you to the mediocrity or worse.
Microsoft has many great development stories (read Raymond Chen's blog to be awed), but what they did at the end to other competitors and how they behave removed their luster, permanently for some people.
I never bought into ethical questions. It's trained on publicly available data as far as I understand. What's the most unethical thing it can do?
My experience is limited. I got it to berate me with a jailbreak. I asked it to do so, so the onus is on me to be able to handle the response.
I'm trying to think of unethical things it can do that are not in the realm of "you asked it for that information, just as you would have searched on Google", but I can only think of things like "how to make a bomb", suicide related instructions, etc which I would place in the "sharp knife" category. One has to be able to handle it before using it.
It's been increasingly giving the canned "As an AI language model ..." response for stuff that's not even unethical, just dicey, for example.
> Claiming they are losing their luster, because they aren’t shaking the earth with new models every quarter, seems a little ridiculous.
If that's the foundation your luster is built on - then it's not really ridiculous.
GPT popularized LLMs to the world with GPT-3, not too long before GPT-4 came out. They made a lot of big, cool changes shortly after GPT-4 - and everyone in their mother announced LLM projects and integrations in that time.
It's been about 9 months now, and not a whole lot has happened in the space.
It's almost as if the law of diminishing returns has kicked in.
It sure is, but the theme in the sub-thread was about if OAI in particular can afford to do that (i.e. wait) while there are literally dozens of other companies and open-source projects showing they can solve a lot of the tasks GPT-4 does, for free, so that the OAI value proposition seems weaker and weaker by the month.
Add to that a company environment that seems to be built on money-crazed stock option piling engineers and a CEO that seems to have gotten power-crazed.. I mean they grew far too fast I guess..
Perhaps GPT-4 is losing its luster because the more people actually use it, they go from "wow that's amazing" to "amazing, yes, but..."? And the "but" looms larger and larger with more time and more exposure?
Note well: I haven't actually used it myself, so I'm speculating (guessing) rather than saying that this is how it is.
This space is growing by leaps and bounds. It's not so much the passage of time as it is the number of notable advancements that is dictating the pace.
> GPT-4 is still king, but not by that much any more
Idk, I just tried Gemini Ultra and it's so much worse than GPT4 that I am actually quite shocked. Trying to ask it any kind of coding question ends up being this frustrating and honestly bizarre waste of time as it hallucinates a whole new language syntax every time and then asks if you want to continue with non-working, in fact non-existing, option A or the equally non-existent option B until you realise that you've spent an hour trying to make it at least output something that is even in the requested language and finally that it is completely useless.
I'm actually pretty astonished at how far Google is behind and that they released such a bunch of worthless junk at all. And have the chutzpah to ask people to pay for it!
Of course I'm looking forward to gpt-5 but even if it's only a minor step up, they're still way ahead.
That's interesting, because I have had exactly the opposite experience testing GPT vs Bard with coding questions. Bard/Gemini far outperformed GPT on coding, especially with newer languages or libraries. Whereas GPT was better with more general questions.
I kind of gave up completely on coding questions. Whether it's GPT4, Anthropic, or Gemini - there's always this big issue of laziness I'm facing. Never do I get a full code, there are always stubs or TODOs (on important stuff) and when asked to correct for that.. I just get more of it (laziness). Has anyone else faced this and is there a solution? It's almost as annoying, if not more, as was incomplete output in the early days.
They seem to be steadily dumbing down GPT-4; eventually, improving performance of open source models and decreasing performance of GPT-4 will meet in the middle.
I've had plenty of dumb policy violation misfires like that with ChatGPT, and got banned from Bing (which uses OpenAI API, not GPT4 at the time I think) for it the day it launched.
Running Ollama with a 80gb mistral model works as well if not better than ChatGPT 3.5. This is a good thing for the world IMO as the magic is no longer held just OpenAI. The speed at which competitors have caught up in even the last 3 months is astounding.
> Frankly, OpenAI seems to be losing its luster, and fast.
Good.
I have no idea what's really going on inside that company but the way the staff were acting on twitter when Altman got the push was genuinely scary, major red flags, bad vibes, you name it, it reeked of it.
For me it was Ilya burning a wooden effigy that represented 'unaligned' AI. Of course the firing and twitter stuff too. Something's fucked in this company for sure.
It lost a little of its cool factor. However, they provide a nearly essential service at this point. While it is easy to underestimate, I suspect this is already have a measurable impact on global GDP.
I'll get downvoted to oblivion, but I think people underestimate the impact that their productization of the GPT in the chat format really led to a virality that likely is not entirely justified just by the underlying product alone. LLMs had been around for several years, it was just a royal pain to use. They definitely were the pioneers in democratizing it to folks, and it occupied a significant slice of mindshare of society for quite a bit. But I suspect it is only natural that it'll recede to a more appropriate level, where this is still an important and incredible piece of tech, but it will stop having the feel that "OMG THIS IS GOING TO TAKE OVER THE WORLD", because it prob. won't... at least not at the pace which popular media would have you believe.
Sam publicly asking for a 10x bigger power grid and 7 trillion dollars is a pretty clear sign that they're out of short to medium-term ideas other than "MOAR PARAMETERS".
Well, he also wanted a shit ton of money so that OpenAI coupd build its own silicon, after most of the real world money generated by the AI hype went to nVidia.
Just imagine what valuation OpenAI would have as a grid monopolist combined with nVidia, ARM, Intel and AMD! Hundreds of trillions of dollars!
I think OpenAI will do fine, but I have doubts about ChatGPT as a product. It’s just a chat UI, and I’m not convinced the UI will be chat 3 years from now.
Personally, the chat UI is the main limiting factor in my own adoption, because a) it’s not in the tool I’m trying to use, and b) it’s quicker for me to do the work than describe the work I need doing.
I interact with ChatGPT by voice pretty often, they have the best speech recognition I’ve ever seen. I can switch between languages (English, French, German) mid-sentence, think aloud, stop mid sentence, the correct what I just said, use highly technical terms (even describe code), I don’t even double check anymore because it’s almost always transcribed correctly.
They can ~easily evolve the product to a more generalized conversation UX instead of just a text based chat.
I suppose it depends what you use it for; my time in search engine has reduced massively - and so has time 'not in the tool I'm trying to use' because it's been so much faster for me to find answers to some queries with ChatGPT than a search engine.
I'm not particularly interested in having it outright program for me (other than say to sketch how to do something as inspiration, which I'll rewrite rather than copy) because I think typically I'd want to do it a certain way and it would take far longer to NLP an LLM to write it in whatever syntax than to WhateverSyntaxProgram it myself.
> GPT-5 has to be incredibly good at this point, and I'm not sure that it will be.
My guess is it isnt, these systems are hard to trust, and the rhetoric "were aiming for AGI" suggests to me that they know this and AGI might be the only surefire way out.
If you tried to replace all of a devs duties with current LLMs it would be a disaster, making sense of all that info requires focus and background thinking processes simulataneously which i dont believe we have yet.
> If you tried to replace all of a devs duties with current LLMs it would be a disaster,
Overall a chatbot like GPT-4 may be useful, but not that useful as it stands.
If you can write well, it's not really going to improve your writing.
Granted, you can automate a few tasks, but it does not give you 10X or even 2X improvement as sometimes advertised.
It might be useful here and there for coding, but it's not reliable.
gpt4 is not worth $22 a month. slow af and you get similar results with gpt3.5. the free perplexity internet search is bounds better than that bing thing. i thought the file upload would be worth it, but no, not worth that much money per month.
Plugins are in theory good, but the hurdle to developing and deploying them combined with only being able to use them with a subscription was kind of a killer.
GPTs are also pretty good, and being able to invoke them in regular chat is also handy, but the lack of monetization and the ability to easily surface them outside of chatgpt is also kind of a problem. These problems are more fixable than the plugin issue IMO since I think the architecture of plugins is a limiting factor.
Perhaps just me, but responses are way worse than it was few months ago. Now the system just makes shit up and says "Yes you are right" when you catch it on BS.
It is practically unusable and I'll likely cancel paid plan soon.
It was always like this ("Now the system just makes shit up and says 'Yes you are right' when you catch it on BS."). The scales are just falling from your eyes as the novelty fades.
I could understand the sentiment when you think that OpenAI is really doubling down just on LLMs recently, and forgoing a ton of research in other fronts.
They’re rapidly iterating though, and it’s refreshing to see them try a bunch of new things so quickly while every other company is comparatively slow to release anything.
To be honest, I hate takes like this. ChatGPT, which basically revolutionized the whole AI industry and the public's imagination about what AI can do, was released not even 15 months ago, and since then they have consistently released huge upgrades (GPT 4 just a couple months later) and numerous products since then. I still haven't used another model that comes close to GPT 4. But since it's been, say, all of 23 hours since OpenAI released a new product (memory) they're "losing their luster".
The same nonsense happened with Apple, where like a month after they first released Apple Watch people were yelling "What's next???!!!! Apple is dying without Steve Jobs!"
> Frankly, OpenAI seems to be losing its luster, and fast.
I don't think it's hugely surprising given the massive hype. No doubt OpenAI are doing impressive things, but it's normal for the market to over value it initially as everyone tries to get onboard, and then for it to fall back to a more sensible level.
I mean they just happened to train the biggest, most fine tuned model on the most data out of everyone I guess.
Transformers were invented with the support of Google (by the researchers, not by Google).
Open community has been creating better and better models with a group effort; like how ML works itself, it's way easier to try 100,000 ideas on a small scale than it is to try a couple of ideas on a large scale.
Unpopular opinion… but IMO almost all of Karpathy’s fame an influence come from being an incredible educator and communicator.
Relative to his level of fame, his actual level of contribution as far as pushing forward AI, I’m not so sure about.
I deeply appreciate his educational content and I’m glad that it has led to a way for him to gain influence and sustain a career. Hopefully he’s rich enough from that that he can focus 100% on educational stuff!
I read that post recently and it felt prescient to someone who has not been deeply involved in ML
Even the HN discussion around this had comments like "this feels my baby learning to speak.." which are the same comparisons people were saying when LLMs hit mainstream in 2022
Disagreeing here! I think we often overlook the value of excellent educational materials. Karpathy has truly revitalized the AI field, which is often cluttered with overly complex and dense mathematical descriptions.
Take CS 231, for example, which stands as one of Stanford's most popular AI/ML courses. Think about the number of students who have taken this class from around 2015 to 2017 and have since advanced in AI. It's fair to say a good chunk of credit goes back to that course.
Instructors who break it down, showing you how straightforward it can be, guiding you through each step, are invaluable. They play a crucial role in lowering the entry barriers into the field. In the long haul, it's these newcomers, brought into AI by resources like those created by Karpathy, who will drive some of the most significant breakthroughs. For instance, his "Hacker's Guide to Neural Networks," now almost a decade old, provided me with one of the clearest 'aha' moments in understanding back-propagation.
I don’t think we disagree. Education is crucial and the value is enormous, but this hasn’t been what he was paid for in the past. I am hopeful that he finds a way to make this his job more directly than at Tesla or OpenAI as the whole world will benefit.
Education and communication is important. It brings new people into the field, and helps grow those that are already part of the field, both of which are essential to long term growth and progress. Using phrases like “actual contribution” to refer to non-educational acts is entirely dismissive to the role that great educators play to in the march for progress. Where would you be today if such education was unavailable?
He contributed to pushing forward AI, no “actual” about it. The loss of a great educator should be viewed with just as much sadness as the loss of a great engineer.
His job at Tesla or OpenAI wasn’t as an educator though. I think a clearer version of my point is that most of his impact has come from activities he has done “on the side” and hasn’t gotten paid for from his job. I’m hopeful it can be his main gig now given that YouTube creators seem to be making more money.
his actual level of contribution as far as pushing forward AI
He did pioneering research in image captioning - aligning visual and textual semantic spaces - the conceptual foundation of modern image generators. He also did an excellent analysis of RNNs - one of the first and best explanations of what happens under the hood of a language model.
That's incredibly impolite and totally without foundation. You make him look like a peasant :-)
What do you know about his work?
He's been leading the vision team at Tesla, implementing in the field all the papers that were available in the subject of autonomous driving and vision (he explicitly wrote that). He has not published about it surely due to obligations with Tesla.
ImageNet was very influential, but this just shows he was eighth author on a twelve author paper from almost a decade ago. Is there better evidence of sustained contributions to the field?
>Relative to his level of fame, his actual level of contribution as far as pushing forward AI, I’m not so sure about.
He lead a team of one of the most common uses of DNNs, if that isn't 'pushing AI forward', I think you're confused. It's certainly pushing it forward quite a bit more than the publishing game where 99% of the papers are ignored by the people actually building real applications of AI.
Its IMHO too. His contribution to educational content is incredible, and very few individuals have the ability to explain things the way he does. However, I am also unsure about his contribution to the field itself. It is a side effect of working in the industry on the product side. You don't have a chance to publish papers, and you don't want to reveal your secrets or bugs to everyone.
I think the ability to teach is a direct outcome of the ability to think and articulate ideas clearly. This is a meta skill that will make a person effective in any area of work.
I'd say that that his work on AI has been significant and his ability to teach has contributed to that greatly.
Would have to agree. Looking at Karpathys research career it’s hard to pin point something and say he’s the inventor of so and so. There are plenty of other researchers for whol you can easily say he’s the inventor of so and so and they have much lesser fame than Karpathy, for example Kaiming He for ResNet, John Schulman for PPO etc.
I don’t see that as an issue though, just a natural consequence of his great work in teaching neural networks!
>his actual level of contribution as far as pushing forward AI, I’m not so sure about.
I mean, I don't know why people still try to devalue educating the masses. Anyone who's had to knowledge share know how hard it is to make a concise but approachable explanation for someone who knows relatively little about the field.
In addition, he's still probably in a standing well above the 80% mark in terms of technical prowess. even without influencer fame I'm sure he can get into any studio he wishes.
> Relative to his level of fame, his actual level of contribution as far as pushing forward AI, I’m not so sure about.
I'd agree with that, however I've always wondered how easy it is for folks at that level to get hands on keyboards and not wind up spending their days polishing slide decks for talks instead.
> My immediate plan is to work on my personal projects and see what happens. Those of you who’ve followed me for a while may have a sense for what that might look like
Short on money after being an exec at tesla during a huge rise in its stocks? More likely he has too much money and maybe doesn't really want or need to work and is doing passion projects instead
Readit News
http://badmephisto.com/
https://github.com/karpathy/nanoGPT
And it's accompanying video series:
https://karpathy.ai/zero-to-hero.html
Another example (although I honestly don't remember if he made this one between jobs) is: https://github.com/karpathy/micrograd
https://karpathy.ai/zero-to-hero.html
[1] https://youtube.com/playlist?list=PLkt2uSq6rBVctENoVBg1TpCC7...
https://www.youtube.com/watch?v=NfnWJUyUJYU&list=PLkt2uSq6rB...
Being a popular AI influencer is not necessarily correlated with being a good researcher though. And I would argue there is a strong indication that it is negatively correlated with being a good business leader / founder.
Here's to hoping he chills out and goes back to the sorely needed lost art of explaining complicated things in elegant ways, and doesn't stray too far back into wasting time with all the top sheisters of the valley.
Edit: the more I think about it, the more I realize that it probably screws with a person to have their tweets get b-lined to the front page of hackernews. It makes you a target for offers and opportunities because of your name/influence, but not necessarily because of your underlying "best fit"
if only we compensated that knowledge properly. Youtube seems to come the closest, but Youtube educators also show how much time you have to spend attracting views instead of teaching expertise.
> It makes you a target for offers and opportunities because of your name/influence, but not necessarily because of your underlying "best fit"
That's unfortunately life in a nutshell. The best fits rarely end up getting any given position. May be overqualified, filtered out in the HR steps, or rejected for some ephemeral reason (making them RTO, not accepting their counteroffer, potentially illegal factors behind closed doors, etc).
it's a crappy game so I don't blame anyone for using whatever cards they are dealt.
Actually for all the attention that the top Youtubers get (in terms of revenue), the reality is that it's going to be impossible to replace teaching income with popular Youtube videos alone.
Based on what I've seen, 1 million video views on Youtube gets you something like $5-10K. And that's with a primarily US audience that has the higher CPM / RPM. So your channel(s) would need to get to about 6 million views per year, primarily US driven, in order to get to earning a median US wage.
Something I've been thinking a lot about is the transition into post scarcity and how we need to dramatically alter the incentive structures and payment allocations.
I've been asking this question for about a decade and still have no good solutions: "What do you do when x% of your workforce is unemployable?" (being that x% of jobs are removed without replacement. Imagine sophisticated and cheap robots. Or if needed, magic)
This is a thought experiment, so your answer can't be "there'll be new jobs." Even if you believe that's what'll happen in real life, it's not in bounds of the thought experiment. It is best to consider multiple values of x because it is likely to change and that would more reflect a post scarcity transition. It is not outside the realms of possibility that in the future you can obtain food, shelter, and medical care for free or at practically no cost. "Too cheap to meter" if you will.
I'll give you two answers that I've gotten that I find interesting. I do not think either are great and they each have issues. 1) jobs programs. Have people do unnecessary jobs simply so they create work wherein we can compensate them. 2) Entertainment. People are, on average, far more interested in watching people play chess against one another than computers, despite the computer being better. So reasons that this ,,might,, not go away.
This can be self-fulfilling.
In an organization beyond a certain size, there will be more almost-adequate-fits than there are leadership positions. This could be about like a recognized baseline which seems like it really needs to be scrutinized closely to see exactly who might be slightly above or below the line.
Or in a small company where there is not any almost-fit whatsoever, imagination can result in an ideal that is equally recognizable, but also might not be fully attainable.
Either way it could be OK but not exactly the best-fit.
If good fortune smiles and the rare more-than-adequate-fit appears anywhere on the horizon though, it's so unfamiliar they fly right over the radar.
Here's a gem of educator. Check out his other videos.
https://www.youtube.com/watch?v=dhYqflvJMXc
A teacher can usually adapt the content depending on its audience, I would not teach the research in my field at the same level to professionals, PhDs, master students, bachelor students, amateurs, or even school students.
If what I'm teaching is fairly complex, it requires a lot of background that I could teach, but I would not have the time to do so, because it would be to the detriment of other students. So, while I usually teach 'from scratch', depending on my audience I will obfuscate some details (that I can answer separately if a question is asked) and usually I will dramatically change the speed of the lessons depending on the previous background, because I need to assume that the student has the prerequisite background to understand at that speed fairly complex material.
As an example, I gave some explanations to a student from zero to transformers, it took several hours with lots of questions, the same presentation to a teacher not in the field took me 1h30 and to a PhD in a related field took 25 minutes, the content was exactly the same, and it was from scratch, but the background in the audience was fairly different.
Just a friendly heads-up, it’s “bee-lined.”
I normally wouldn’t point that out, but “b-lined” could be read to suggest the opposite of your intention; a lower priority, a la “B-list celebrity.”
/s
Plugins were a failure. GPTs are a little better, but I still don't see the product market fit. GPT-4 is still king, but not by that much any more. It's not even clear that they're doing great research, because they don't publish.
GPT-5 has to be incredibly good at this point, and I'm not sure that it will be.
As the popularity has exploded, and ethical questions have become increasingly relevant, it is probably worth taking some time to nail certain aspects down before releasing everything to the public for the sake of being first.
Altman saga, allowing military use and other small things step by step tarnish your reputation and pushes you to the mediocrity or worse.
Microsoft has many great development stories (read Raymond Chen's blog to be awed), but what they did at the end to other competitors and how they behave removed their luster, permanently for some people.
My experience is limited. I got it to berate me with a jailbreak. I asked it to do so, so the onus is on me to be able to handle the response.
I'm trying to think of unethical things it can do that are not in the realm of "you asked it for that information, just as you would have searched on Google", but I can only think of things like "how to make a bomb", suicide related instructions, etc which I would place in the "sharp knife" category. One has to be able to handle it before using it.
It's been increasingly giving the canned "As an AI language model ..." response for stuff that's not even unethical, just dicey, for example.
If that's the foundation your luster is built on - then it's not really ridiculous.
GPT popularized LLMs to the world with GPT-3, not too long before GPT-4 came out. They made a lot of big, cool changes shortly after GPT-4 - and everyone in their mother announced LLM projects and integrations in that time.
It's been about 9 months now, and not a whole lot has happened in the space.
It's almost as if the law of diminishing returns has kicked in.
Add to that a company environment that seems to be built on money-crazed stock option piling engineers and a CEO that seems to have gotten power-crazed.. I mean they grew far too fast I guess..
Note well: I haven't actually used it myself, so I'm speculating (guessing) rather than saying that this is how it is.
Idk, I just tried Gemini Ultra and it's so much worse than GPT4 that I am actually quite shocked. Trying to ask it any kind of coding question ends up being this frustrating and honestly bizarre waste of time as it hallucinates a whole new language syntax every time and then asks if you want to continue with non-working, in fact non-existing, option A or the equally non-existent option B until you realise that you've spent an hour trying to make it at least output something that is even in the requested language and finally that it is completely useless.
I'm actually pretty astonished at how far Google is behind and that they released such a bunch of worthless junk at all. And have the chutzpah to ask people to pay for it!
Of course I'm looking forward to gpt-5 but even if it's only a minor step up, they're still way ahead.
Here are some hilarious highlights: https://twitter.com/Suhail/status/1757573182138290284
Google DeepMind is still an AI research powerhouse that is producing a ton of innovation both internal and publicly published.
Good.
I have no idea what's really going on inside that company but the way the staff were acting on twitter when Altman got the push was genuinely scary, major red flags, bad vibes, you name it, it reeked of it.
I was surprised and touched by their loyalty, but maybe I missed something you noticed.
Just imagine what valuation OpenAI would have as a grid monopolist combined with nVidia, ARM, Intel and AMD! Hundreds of trillions of dollars!
Personally, the chat UI is the main limiting factor in my own adoption, because a) it’s not in the tool I’m trying to use, and b) it’s quicker for me to do the work than describe the work I need doing.
I'm not particularly interested in having it outright program for me (other than say to sketch how to do something as inspiration, which I'll rewrite rather than copy) because I think typically I'd want to do it a certain way and it would take far longer to NLP an LLM to write it in whatever syntax than to WhateverSyntaxProgram it myself.
If ChatGPT doesn't have product-market fit, what actually has?
My guess is it isnt, these systems are hard to trust, and the rhetoric "were aiming for AGI" suggests to me that they know this and AGI might be the only surefire way out.
If you tried to replace all of a devs duties with current LLMs it would be a disaster, making sense of all that info requires focus and background thinking processes simulataneously which i dont believe we have yet.
Overall a chatbot like GPT-4 may be useful, but not that useful as it stands.
If you can write well, it's not really going to improve your writing. Granted, you can automate a few tasks, but it does not give you 10X or even 2X improvement as sometimes advertised.
It might be useful here and there for coding, but it's not reliable.
GPTs are also pretty good, and being able to invoke them in regular chat is also handy, but the lack of monetization and the ability to easily surface them outside of chatgpt is also kind of a problem. These problems are more fixable than the plugin issue IMO since I think the architecture of plugins is a limiting factor.
It is practically unusable and I'll likely cancel paid plan soon.
I could understand the sentiment when you think that OpenAI is really doubling down just on LLMs recently, and forgoing a ton of research in other fronts.
They’re rapidly iterating though, and it’s refreshing to see them try a bunch of new things so quickly while every other company is comparatively slow to release anything.
The same nonsense happened with Apple, where like a month after they first released Apple Watch people were yelling "What's next???!!!! Apple is dying without Steve Jobs!"
I don't think it's hugely surprising given the massive hype. No doubt OpenAI are doing impressive things, but it's normal for the market to over value it initially as everyone tries to get onboard, and then for it to fall back to a more sensible level.
Transformers were invented with the support of Google (by the researchers, not by Google).
Open community has been creating better and better models with a group effort; like how ML works itself, it's way easier to try 100,000 ideas on a small scale than it is to try a couple of ideas on a large scale.
I've been anti-Google for a while now so I'm not biased.
I don't think openAI have this sown up.
Relative to his level of fame, his actual level of contribution as far as pushing forward AI, I’m not so sure about.
I deeply appreciate his educational content and I’m glad that it has led to a way for him to gain influence and sustain a career. Hopefully he’s rich enough from that that he can focus 100% on educational stuff!
That blog post inspired Alec Radford at Open AI to do the research that produced the "Unsupervised sentiment neuron": https://openai.com/research/unsupervised-sentiment-neuron
Open AI decided to see what happened if they scaled up that model by leveraging the new Transformer architecture invented at Google, and they created something called GPT: https://cdn.openai.com/research-covers/language-unsupervised...
"In fact, I’d go as far as to say that
when the initial attention paper was less than a year old, and two years before the transformer paper.Even the HN discussion around this had comments like "this feels my baby learning to speak.." which are the same comparisons people were saying when LLMs hit mainstream in 2022
Take CS 231, for example, which stands as one of Stanford's most popular AI/ML courses. Think about the number of students who have taken this class from around 2015 to 2017 and have since advanced in AI. It's fair to say a good chunk of credit goes back to that course.
Instructors who break it down, showing you how straightforward it can be, guiding you through each step, are invaluable. They play a crucial role in lowering the entry barriers into the field. In the long haul, it's these newcomers, brought into AI by resources like those created by Karpathy, who will drive some of the most significant breakthroughs. For instance, his "Hacker's Guide to Neural Networks," now almost a decade old, provided me with one of the clearest 'aha' moments in understanding back-propagation.
He contributed to pushing forward AI, no “actual” about it. The loss of a great educator should be viewed with just as much sadness as the loss of a great engineer.
He did pioneering research in image captioning - aligning visual and textual semantic spaces - the conceptual foundation of modern image generators. He also did an excellent analysis of RNNs - one of the first and best explanations of what happens under the hood of a language model.
What do you know about his work?
He's been leading the vision team at Tesla, implementing in the field all the papers that were available in the subject of autonomous driving and vision (he explicitly wrote that). He has not published about it surely due to obligations with Tesla.
Dead Comment
Are you sure about your perspective?
https://scholar.google.com/citations?view_op=view_citation&h...
He lead a team of one of the most common uses of DNNs, if that isn't 'pushing AI forward', I think you're confused. It's certainly pushing it forward quite a bit more than the publishing game where 99% of the papers are ignored by the people actually building real applications of AI.
I'd say that that his work on AI has been significant and his ability to teach has contributed to that greatly.
Deleted Comment
I don’t see that as an issue though, just a natural consequence of his great work in teaching neural networks!
I mean, I don't know why people still try to devalue educating the masses. Anyone who's had to knowledge share know how hard it is to make a concise but approachable explanation for someone who knows relatively little about the field.
In addition, he's still probably in a standing well above the 80% mark in terms of technical prowess. even without influencer fame I'm sure he can get into any studio he wishes.
Deleted Comment
Deleted Comment
I'd agree with that, however I've always wondered how easy it is for folks at that level to get hands on keyboards and not wind up spending their days polishing slide decks for talks instead.
Does anyone here know?
You can check his GitHub: https://github.com/karpathy