They both have ~0.25TBps memory bandwidth & huge GPUs. They very much are in the same space.
I'm very Team Red (AMD) but I dig & respect the 4x 40Gbps USB ports here. Its not popular but it is a good way to attach pretty fast storage. Also the 200Gbps connect-x7 is just awesome. Users can add what they will to Strix Halo, but it's neat seeing good connectivity on chip.
I hope AMD can get better about USB4 & Thunderbolt 4. It really feels like it should be part of their APUs. Apple shipping 6x TB5 ports on the new M3 Ultra is a beautiful example of what life should be like.
Sort of. Sure you can game on the framework desktop, but I'd expect them to do similar on inference workloads and come with the same max ram and bandwidth.
The coolest part is the thing is powered by a USB C. Granted it is not an intel, but the first powerful mini pc that does not have a separate dc power port is a breath of fresh air .
> The coolest part is the thing is powered by a USB C. [This is] a breath of fresh air.
Serious question, looking for a serious answer from you: Why?
The wafer-style USB-C connector is mechanically inferior to the size of barrel-style connectors that are typically used for 100+W applications. It's far less resistant to torque and shear, and due to its small size is significantly easier to accidentally dislodge. While accidental disconnection isn't a huge problem for a battery-powered machine, this thing is wall-powered.
Making things worse, unless you want to roll the dice on Amazon not sending you something that might burn your house down due to underspecced conductors and/or connectors, you're going to have to pay a significant fraction of the cost of the PSU for the cable to connect it.
There are also the ergonomic factors to consider. On the overwhelming majority of consumer hardware that's powered with a barrel-style connector, there's only one place where that connector fits. However, on much-to-most of the consumer hardware that's powered with a USB-C connector, there are many plugs that will accept the power supply cable but will refuse to use it to power the hardware. Perhaps one day in the distant future, manufacturers will universally pay the money to make all USB-C-shaped plugs capable of powering the USB-C-connector-powered-hardware they're built in to, but that day is not today.
And sure, you can reasonably retort with "Well, you just keep trying plugs until it works.", as well as "Well, just turn on a light and read the printing on the plugs.". These aren't huge hassles. But I remember that one of the big hassles that the USB-C connector saved us from was "You don't have to try to plug it in three times to get it right.". Going from "Well, it just works, first try!" to "Well, you just have to find the right plug to make it work. Keep trying!" is backwards and a little sad.
A use case is portability. I can transport the device between dock equipped workstations. Having to plug and unplug just one port vs 2 is a lot easier.
I am not concerned with durability of the port. My experience with one of many usb port suited for high power delivery has been uneventful. I know my devices well enough.
The only drawback is insufficient power delivery from docks. It’s only necessary to me that it works with less power.
Benchmarks of what? Memory speed matters for some things but not others. It matters a lot for AI training, but less for AI inference. It matters a lot for games too, but nobody would play a game on this or a mac.
AI inference is actually typically bandwidth limited compared to training, which can re-use the weights for all tokens <sequence length> * <batch size>. Inference, specifically decoding, requires you to read all of the weights for each new token, so the flops per byte are much lower during inference!
Looks to be a smart play from NVIDIA to recapture the local LLM market, this looks very competitive with Apple M4 and AMD AI Max, but cheaper(!).
Nvidia and cheap don't go together most of the time so
I think they must have been very worried by developers and enthusiasts buying other hardware.
Consistent with their very developer focused strategy and probably a very smart idea since it's low enough spec to avoid canabalising sales of their existing products.
Too late to edit my post but wanted to apologise for causing confusion.
I suppose "cheapest" can be a very subjective term if the comparison is between things with different capabilities.
NVIDIA is a cheaper option than AMD only if you compare assume you want/need the fast networking NVIDIA are bundling with their system. According to the article the NIC would add $1500-$2000 to the price of another systems. I also failed to account for the extra memory bandwidth offered by M4 max. The apple system costs more but if you want/need that bandwidth then it's the cheapest of the three.
I guess "the system has a niche where it offers very good price/performance" is what I should have said. Not as snappy though.
I'd say the SGX is the "cheapest" only if you're trying to go beyond 96GB.
the AMD system is 96GB max for the GPU. The 128GB allocates a minimum of 32GB to the CPU.
the Nvidia system is designed to be connected to a second if so desired, making it the cheapest way to get 256GB.
If you're just going for something under 96GB, haven't seen something cheaper than the AMD system for anything that can't fit on a traditional GPU. And even then, GPUs are obscene ripoff prices lately. Here's hoping these won't be scalped too.
AMD AI Max+ 395 systems should be around $2k with 128gb DDR5X, the Mac Studio maxed out is just north of $12k. So it's in the middle depending. This will also vary greatly in terms of usability for other things.
That said, a nice OEM option if you need something for AI workloads where the GPU market is completely soaked with scalpers. Been considering a drive to California just so I can get a GPU directly at MicroCenter instead of paying scalper overheads.
Yes, you can spend $12k on a mac studio. However the Mac studio with the m4 max has 128GB ram, double the memory bandwidth, and costs $3,699. LESS than the DGX spark. Granted it doesn't have 200gbe.
If it's the memory bandwidth you are after the Mac Mini with the m4 pro has similar, but max 64GB ram.
> This will also vary greatly in terms of usability for other things.
This is key. Nvidia has a terrible reputation with long term support (as market leaders, they can easily afford that). Apple just now (last November) dropped OS updates for their 2014 boxes. While a Mac Studio 2025 will not be a ridiculous amount of compute power in 10 years, I fully expect Nvidia to completely abandon support and software updates for this in five years tops.
Hopefully, considering the interest it generated, I'd hope the Linux crowd will be able to carry it further, maybe with decent open-source drivers, way past the expiration date Nvidia sets.
Cheaper? The DGX spark is $3,000 with 128GB ram. A framework desktop with the AMD strix halo 395 with 128gb ram is $2,000 and has better memory bandwidth. No price I can see for the identically spec'd (and nearly identical looking) Ascent gx10.
[edit] Oops, the Spark is $4,000, only the Ascent is at $3k now. Strix Halo systems vary from slightly slower (6%) to the same (on systems with LPDDR5x-8533, like the HP laptop).
> NVIDIA ConnectX-7 NIC these days often sells for $1500-2200 in single unit quantities, depending on the features and supply of the parts. At $2999 for a system with this buit-in that is awesome.
Double check your memory bandwidth numbers. AMD's Strix Halo is 256 bits at 8000MT/s for about 238GB/s while NVIDIA's GB10 is 276GB/s (likely 256 bits at the more standard speed of 8533MT/s).
A 100GB model running on one of these would be a disaster, you'd probably run out of space for the KV cache apart from anything else. But if you had two of them performance should be acceptable.
I think they would be canabalising their other product lines if they had more memory bandwidth.
Readit News
- [0] https://frame.work/desktop
Edit: I misread the specs and posted this a bit too hastily. I'd delete this comment but it already has children.
I'm very Team Red (AMD) but I dig & respect the 4x 40Gbps USB ports here. Its not popular but it is a good way to attach pretty fast storage. Also the 200Gbps connect-x7 is just awesome. Users can add what they will to Strix Halo, but it's neat seeing good connectivity on chip.
I hope AMD can get better about USB4 & Thunderbolt 4. It really feels like it should be part of their APUs. Apple shipping 6x TB5 ports on the new M3 Ultra is a beautiful example of what life should be like.
I hope this being more accessible than other Nvidia gear allows it to develop a healthy software ecosystem that doesn't depend so much on Nvidia.
Deleted Comment
Deleted Comment
Serious question, looking for a serious answer from you: Why?
The wafer-style USB-C connector is mechanically inferior to the size of barrel-style connectors that are typically used for 100+W applications. It's far less resistant to torque and shear, and due to its small size is significantly easier to accidentally dislodge. While accidental disconnection isn't a huge problem for a battery-powered machine, this thing is wall-powered.
Making things worse, unless you want to roll the dice on Amazon not sending you something that might burn your house down due to underspecced conductors and/or connectors, you're going to have to pay a significant fraction of the cost of the PSU for the cable to connect it.
There are also the ergonomic factors to consider. On the overwhelming majority of consumer hardware that's powered with a barrel-style connector, there's only one place where that connector fits. However, on much-to-most of the consumer hardware that's powered with a USB-C connector, there are many plugs that will accept the power supply cable but will refuse to use it to power the hardware. Perhaps one day in the distant future, manufacturers will universally pay the money to make all USB-C-shaped plugs capable of powering the USB-C-connector-powered-hardware they're built in to, but that day is not today.
And sure, you can reasonably retort with "Well, you just keep trying plugs until it works.", as well as "Well, just turn on a light and read the printing on the plugs.". These aren't huge hassles. But I remember that one of the big hassles that the USB-C connector saved us from was "You don't have to try to plug it in three times to get it right.". Going from "Well, it just works, first try!" to "Well, you just have to find the right plug to make it work. Keep trying!" is backwards and a little sad.
I am not concerned with durability of the port. My experience with one of many usb port suited for high power delivery has been uneventful. I know my devices well enough.
The only drawback is insufficient power delivery from docks. It’s only necessary to me that it works with less power.
Nvidia and cheap don't go together most of the time so I think they must have been very worried by developers and enthusiasts buying other hardware.
Consistent with their very developer focused strategy and probably a very smart idea since it's low enough spec to avoid canabalising sales of their existing products.
I suppose "cheapest" can be a very subjective term if the comparison is between things with different capabilities.
NVIDIA is a cheaper option than AMD only if you compare assume you want/need the fast networking NVIDIA are bundling with their system. According to the article the NIC would add $1500-$2000 to the price of another systems. I also failed to account for the extra memory bandwidth offered by M4 max. The apple system costs more but if you want/need that bandwidth then it's the cheapest of the three.
I guess "the system has a niche where it offers very good price/performance" is what I should have said. Not as snappy though.
the AMD system is 96GB max for the GPU. The 128GB allocates a minimum of 32GB to the CPU.
the Nvidia system is designed to be connected to a second if so desired, making it the cheapest way to get 256GB.
If you're just going for something under 96GB, haven't seen something cheaper than the AMD system for anything that can't fit on a traditional GPU. And even then, GPUs are obscene ripoff prices lately. Here's hoping these won't be scalped too.
That said, a nice OEM option if you need something for AI workloads where the GPU market is completely soaked with scalpers. Been considering a drive to California just so I can get a GPU directly at MicroCenter instead of paying scalper overheads.
If it's the memory bandwidth you are after the Mac Mini with the m4 pro has similar, but max 64GB ram.
This is key. Nvidia has a terrible reputation with long term support (as market leaders, they can easily afford that). Apple just now (last November) dropped OS updates for their 2014 boxes. While a Mac Studio 2025 will not be a ridiculous amount of compute power in 10 years, I fully expect Nvidia to completely abandon support and software updates for this in five years tops.
Hopefully, considering the interest it generated, I'd hope the Linux crowd will be able to carry it further, maybe with decent open-source drivers, way past the expiration date Nvidia sets.
[edit] Oops, the Spark is $4,000, only the Ascent is at $3k now. Strix Halo systems vary from slightly slower (6%) to the same (on systems with LPDDR5x-8533, like the HP laptop).
> NVIDIA ConnectX-7 NIC these days often sells for $1500-2200 in single unit quantities, depending on the features and supply of the parts. At $2999 for a system with this buit-in that is awesome.
So far, project digits looks disappointing.
I think they would be canabalising their other product lines if they had more memory bandwidth.
Deleted Comment