oh my please read TFA it has exactly the answers you seek.
He wants to see a comparison and the article shows no out put comparison. It shows two inputs, says one gives more accurate output than the other without showing us...
Readit Newsjdefr89 commented on Attention Is the New Big-O: A Systems Design Approach to Prompt Engineering alexchesser.medium.com/at... · Posted by u/alexc05
I don't buy the arguments made here. You can't call it attention optimized without opening the LLM brain and assessing what happens in the attention layers. Does the quoted paper did some of that? I know Anthropic is advanced in this area but I haven't seen that many results elsewhere yet. I mean the fact that optimizing for attention makes better prompt is a solid hypothesis, but I don't read a proof here.
Yea they sound like plausible enough arguments.. I often test my vague input against structured ones and for many tasks it didn't seem like hallucinations happened significantly less. Honestly the more structured you have to be, the less "general" your model probably is, and in theory you want your model to be as general as possible. Seems like here, you're simply helping it over-fit.. At least that's what my intuition tells me but I am yet to really check that out either.
jdefr89 commented on Apple and Amazon will miss AI like Intel missed mobile gmays.com/the-biggest-bet... · Posted by u/gmays
Its sad content like this makes it too front page. AI has been developed and used at both companies far before LLM TRANSFORMERS made such a hit. The author clearly doesn't have even a basic understanding of what AI is... AI is a giant umbrella term. The hype is essentially all about Transformer based models and I time will show its a hype bubble they don't need to be part of. I will never understand how people with so little technical knowledge on a subject can make grandiose claims like "Apple is losing out on AI!" and be taken seriously. Every company is high on their own damn supply with this pushing AI nonsense. I don't need language models shoved in my face by yet another company. I respect Apples approach far more than everyone else trying to ride the LLM bandwagon.
hear hear! I've worked with exceptional people from all sorts of backgrounds, but pound for pound, Waterloo CS grads are significantly better on average than grads from any other CS program. They start off way ahead, but also maintain that edge throughout their careers.
Disclosure: I'm a US citizen with multiple CS degree from MIT and my son is studying CS at Waterloo now.
Nice. I an researcher at MIT LL. In order to work at the lab you need to be a US citizen since a lot of it is federally funded. I tried to get my buddy, a Waterloo grad currently at Google, an interview but he is Canadian...
Dude writes like he’s an under-appreciated genius, unfairly unrecognized in his time.
His (entirely not-unique) conclusion that the transformer architecture has plateaued is, for the moment, certainly true, but god damn it’s been a while since I’ve encountered an individual quite so lustfully engaged with his own farts.
How about anyone pushing AI or AGI or anything of the sort? You think they aren’t high on their own supply? The author doesn’t hold a candle to Altman when it comes to sniffing their own farts… Cmon now. If anything it’s the opposite.
He just sounds bitter with a weird grudge against Altman
Gpt5 was an incremental improvement. That’s fine. Was hyped hard but what did you expect? It’s part of the game
That’s exactly the problem the author has. Lying and hype have replaced genuine innovation. It’s sad that lying and pushing nonsense is “part of the game” because it shouldn’t be. The game is only of benefit to a handful of people like Altman, not the humanity or to the field in general. I am sick of it as well… Tech has become grifter central and everyone is high on their own fucking supply…
This is a genre of article I find particularly annoying. Instead of writing an essay on why he personally thinks GPT-5 is bad based on his own analysis, the author just gathers up a bunch of social media reactions and tells us about them, characterizing every criticism as “devastating” or a “slam”, and then hopes that the combined weight of these overtorqued summaries will convince us to see things his way.
It’s both too slanted to be journalism, but not original enough to be analysis.
I think from the authors perspective, LLM hype has been mostly the same exact thing you’re accusing him of doing. People with very little technical background claiming AGI is near or all these CEOs pushing these nonsense narratives are getting old.. People are blindly trusting these people and offloading all their thinking to a sophisticated stochastic machine. It’s useful yes, super cool yes. Some super god like power or something that brings us to AGI? No probably not. I can’t blame him. I am sick of the hype. Grifters are coming out of the woodwork in a field with too many grifters to begin with. All these AI/LLM companies are high of their own supply and it’s getting old.
jdefr89 commented on Hopfield Networks Is All You Need (2020) arxiv.org/abs/2008.02217... · Posted by u/liamdgray
So far I’ve needed hundreds of “all you needs” beginning to think papers with that title are not all I need… Let’s try and get a bit more creative than stealing the same name scheme used when Transformers were introduced.
Large language models have their place, but they’re quite limited and no real substitute for human intelligence or judgement at its best.
That said, abusive corporate environments (and they’re pretty much all abusive) turn people into automatons. Execs can thus wave LLMs around and say, “Look, you’re so useless a machine can do your job.”
And even then it does a shitty job. It misses special cases and causes messes. But it’s cheap in the short term, which is all that matters to the boss’s career. Things will go to shit in a few years, but if you’re good at executive social climbing, you are three promotions away from your bad decisions by the time anyone figures out that’s what they were.
It's ironic because the executive aspects of the job are easily the ones almost anyone can do. I say that as someone who has created and sold startup with a couple friends. We had to take on the role of both engineers and executives. Guess which one was far far easier and which one actually mattered?
Good ones really do and they work hard to enhance the productivity of their workers. I always say to my people that my job is to make their jobs easier, and to a large degree that's true.
There are other types who prefer the power trip, or that if you're a bad manager then your job is pretty easy, etc. But GOOD execs actually do care quite a lot. I tend to tangle with HR over salaries because I'd rather hire a handful of really good people for 15% above market and get double the productivity than a lot of people at or below market.
This article is confusing the corporate raider mindset rampant in big public companies with the genuine growth mindset most companies actually have. Everyone focuses on productivity and costs, enshittification only happens when you're near enough of a monopoly you can afford to squeeze your customers. Most companies can't do that and most companies try to just keep improving. Don't look at MS or Exxon and think that's who most companies actually work. That's just the beginning of the end stage of a megacorporation's life.
Unfortunately It seems most companies only purport to have "growth mindset" In my experience, at least now, the author nails the issue right on the head. You have executives with a dogma that ultimately means controlling others... I see little proof the opposite is true.