How was Keller able to pull off so much impactful work over so many organizations? i.e. what proportion of his impact was attributable to novel ideas, solo engineering implementation ability, engineering leadership, or just darn good timing?
I was at Intel when Jim Keller came. He was one of the most inspiring technical leaders I have ever encountered. He obviously had very strong engineering knowledge as well. When he came in, he took some time to really understanding our process and challenges. Then he created 5 areas that Intel needed to improve. He said “we are going to fix these, or burn down the place trying”. And these weren’t vague goals like “execute more effectively” or “be customer obsessed”, but really concrete and measurable things like “reduce empty silicon area by 50%”. He had a task force of principle engineers on each task, and every meeting he would spend almost the entire time going into detail of the progress and challenges on them. Anytime someone would complain and say the goal couldn’t be done, his response was “AMD did it, why can’t you?”. He was always trying to simplify things and cut through complexity. His vision was that if we hit those 5 targets, we would win the market. As an engineer, his clear vision and strong technical understanding made him the best leader I have ever been under.
it’s a pity he left Intel so quickly. After he was gone, his 5 goals basically fell by the wayside. I don’t think people like being told they’re doing it wrong. I still try to listen to any podcast or interview with him, because I think he has many deep insights.
Moar. Maybe you have some more stories or insights from him. I'm a shameless fanboy. I also listened to I think every podcast (not many of them unfortunately, Lex ones seem the best as expected). This guy oozes wisdom.
You should see his interviews with Dr. Ian Cutress. The man basically knows everything in the industry in and out and knows how to create great teams. I want to interview for Tenstorrent at some point and would love to work for him. I feel being part of great teams is what he has been fortunate with. People very easily forget the rest of the team in chip design and make it about one person. I feel he's a true leader in chip design teams, but I also think he has worked with many great people to achieve that.
People like this are extremely rare and talented, AND they are able to build amazing teams.
Other super talented people want to work with a Jim Keller. So a company hires a Jim Keller not just because you want his brain, but because you can then go to the next 20 amazing people in the industry and say "come work on Jim Keller's team" – now you have 20 min Jim Kellers on your team and that is enough to move mountains in an extremely specialized field such as chip design.
The cult of personality is always an enticing idea, it's pretty much certain that Jim Keller had a stellar team to work with in all of those places.
That being said, even a strong team is still usually only as good as its leadership. You can have all the great ideas in the world, but if the organisation does not allow you to pursue them (or on the other extreme, allows everyone to work on what they think is best) you will still fail to execute.
At the end of the day you need someone to make the correct decisions every step of the way.
To address the article, this will be very interesting because in the market Tensortorrent is targetting the real challenge is software support for PyTorch, TensorFlow, distributed training, etc... and less about the hardware itself. It's not like AMD GPUs can't do matrix multiplications at competitive TFLOPS but instead that if I wanted to use them to run a model I'd have to work through several workarounds to get similar performance to an Nvidia GPU that is mostly plug-and-play.
We will see if Jim Keller's many talents include managing software.
And they're blurring the line by adding more AI-focused acceleration instructions to their GPUs (maybe focused on their CDNA line rather than the consumer RDNA, but they're expanding their capabilities on both).
There seems to be a bit of a philosophy difference between NVidia and AMD acceleration - NVidia add new specialised "cores", while AMD adds those similar capabilities as accelerated instructions to their shader units.
Though this might just be marketing - I don't think we've got an open reference for the NVidia shader ISA like we do with AMD so difficult to compare, but the capabilities and integration in shaders of many of these acceleration paths seems to imply they might also be implemented as shader instructions, which stretches the definition of a separate "core" to me, and so might just be a marketing difference rather than anything technical. Same with things like the BVH acceleration used in ray tracing - AMD added shader instructions, while NVidia talks loudly about "RT Cores".
Yes that's my point. AMD has the hardware, but using that hardware with the existing DL ecosystem is much more painful and they haven't been able to bridge that gap.
> Erm, Itanium never had any kind of market leadership. It was a failure.
That's true. But part of the reason Itanium as a failure is because AMD came out with x86-64, which was so successful that Intel ended up being forced to adopt it (it didn't help that the first generation of Itanium chips was... underwhelming).
I think the general consensus is that AMD could only swoop in as it did because Itanium was such a disaster of epic proportions, both technically and on a management level. There was no way Itanium was ever going to be successful. x86-64 was just the final nail in the coffin.
I found that Lex interview enlightening for two reasons. Firstly, Keller is a profoundly talented and interesting guy. And secondly, Lex is a truly painful and incompetent interviewer; virtually every time Lex spoke I begged my screen for him to stay quiet and let Keller speak.
His guests are often so interesting that it's worth persevering through Lex's un-profound musings. (Every time he uses the word 'beautiful' or 'love' it reminds me of a pretentious teenager who lacks life experience)
I had the same feeling with a lot of interviews from the past, but somehow, about a year ago, something changed. Not sure if it was a conscious decision or just the result of so much practice, but for quite something I haven’t screamed at my screen when listening to Lex.
That's exactly what I thought. Jim Keller had a lot of deep insights that he was boiling down to simple terms. It was obvious he had thought about these ideas for a long time and that what he was saying came from a lot of experience. He would talk about things like switching lanes in driverless cars being a matter of ballistics.
Lex Friedman just interrupted him in a whiny voice to say "it's not thaaat simple". He basically just got in Jim Keller's way. It would be like someone interrupting a john carmack presentation every few minutes to say "I don't know about all that". It was a shame.
Fridman's interviews are an unusual combination of being well researched and incompetently conducted. When the interviewees are so interesting and well-chosen, and otherwise not the sort of person who would get a lot of media attention (Jim Keller is one example), it is indeed painful when he interrupts the guest or... sigh... interacts with them verbally in any way, really.
Unrelated, but Fridman's proclivity for injecting Elon Musk as a discussion topic into the vast majority of his interviews is genuinely weird.
Keller was great. He was actually great in all public appearances I saw him in. And I find it hard to find something I disagree with him. Usually with people I look up to, there are _some_ things I'm not on board with, but I don't seem to have that with Keller.
I understand that until recently he kind of stayed out of the public spotlight, and I wonder why, because he has such a great presence.
Some design decisions are also really not obvious. I've seen engineering decisions of the kind... We'll put another microcontroller because we would need less wiring between these 2 boards and the wiring would be more expensive.
A clever one I saw was where they milled a PCB thinner in a section to make it bendy and did not need 2 boards because of this.
Can confirm. Most people probably wouldn't believe the effort that is spent on improving the physical engine models in the ECUs in order to save a few bucks on sensors. Precisely, as you wrote, because those few bucks multiply while a software effort is one-time.
I remember in the mid '70s looking under the hood of my family's enormous station wagon and being greeted by a solid surface of hoses and valves, all dedicated to meeting the recently-boosted emissions standards. Open an internal-combustion hood now, and it's much, much simpler.
At this kind of scale, finding a "small" saving may pay for tens of engineer years and justify hiring people just for that. Same thing to optimize cloud costs.
Readit News
https://wccftech.com/jim-keller-tenstorrent-wants-to-compete...
it’s a pity he left Intel so quickly. After he was gone, his 5 goals basically fell by the wayside. I don’t think people like being told they’re doing it wrong. I still try to listen to any podcast or interview with him, because I think he has many deep insights.
Other super talented people want to work with a Jim Keller. So a company hires a Jim Keller not just because you want his brain, but because you can then go to the next 20 amazing people in the industry and say "come work on Jim Keller's team" – now you have 20 min Jim Kellers on your team and that is enough to move mountains in an extremely specialized field such as chip design.
That being said, even a strong team is still usually only as good as its leadership. You can have all the great ideas in the world, but if the organisation does not allow you to pursue them (or on the other extreme, allows everyone to work on what they think is best) you will still fail to execute.
At the end of the day you need someone to make the correct decisions every step of the way.
We will see if Jim Keller's many talents include managing software.
And they're blurring the line by adding more AI-focused acceleration instructions to their GPUs (maybe focused on their CDNA line rather than the consumer RDNA, but they're expanding their capabilities on both).
There seems to be a bit of a philosophy difference between NVidia and AMD acceleration - NVidia add new specialised "cores", while AMD adds those similar capabilities as accelerated instructions to their shader units.
Though this might just be marketing - I don't think we've got an open reference for the NVidia shader ISA like we do with AMD so difficult to compare, but the capabilities and integration in shaders of many of these acceleration paths seems to imply they might also be implemented as shader instructions, which stretches the definition of a separate "core" to me, and so might just be a marketing difference rather than anything technical. Same with things like the BVH acceleration used in ray tracing - AMD added shader instructions, while NVidia talks loudly about "RT Cores".
Erm, Itanium never had any kind of market leadership. It was a failure.
That's true. But part of the reason Itanium as a failure is because AMD came out with x86-64, which was so successful that Intel ended up being forced to adopt it (it didn't help that the first generation of Itanium chips was... underwhelming).
Lex Friedman just interrupted him in a whiny voice to say "it's not thaaat simple". He basically just got in Jim Keller's way. It would be like someone interrupting a john carmack presentation every few minutes to say "I don't know about all that". It was a shame.
Unrelated, but Fridman's proclivity for injecting Elon Musk as a discussion topic into the vast majority of his interviews is genuinely weird.
I understand that until recently he kind of stayed out of the public spotlight, and I wonder why, because he has such a great presence.
Dead Comment
I don't think you understand what $8 per vehicle represents in cost savings.
$8 in parts can literally be 10s of million in savings per year.
And then you are basically tied to this supplier forever as switching something so fundamental isn't easy.
In automotive, using less screws to save 50c on each vehicle is considered a big win.
A clever one I saw was where they milled a PCB thinner in a section to make it bendy and did not need 2 boards because of this.