zcbenz (u/zcbenz) - Readit News

zcbenz commented on Apple's MLX adding CUDA support github.com/ml-explore/mlx... · Posted by u/nsagent

albertzeyer · a month ago

This is exciting. So this is using unified memory of CUDA? I wonder how well that works. Is the behavior of the unified memory in CUDA actually the same as for Apple silicon? For Apple silicon, as I understand, the memory is anyway shared between GPU and CPU. But for CUDA, this is not the case. So when you have some tensor on CPU, how will it end up on GPU then? This needs a copy somehow. Or is this all hidden by CUDA?

zcbenz · a month ago

In the absence of hardware unified memory, CUDA will automatically copy data between CPU/GPU when there are page faults.

zcbenz commented on PyTorch Internals: Ezyang's Blog blog.ezyang.com/2019/05/p... · Posted by u/Anon84

ForceBru · 5 months ago

Why is MLX Apple silicon only? Is there something fundamental that prevents it from working on x86? Are some core features only possible on Apple silicon? Or do the devs specifically refuse to port to x86? (Which is understandable, I guess)

I'm asking because it seems to have nice autodiff functionality. It even supports differentiating array mutation (https://ml-explore.github.io/mlx/build/html/usage/indexing.h...), which is something JAX and Zygote.jl can't do. Instead, both have ugly tricks like `array.at[index].set` and the `Buffer` struct.

So it would be cool to have this functionality on a "regular" CPU.

zcbenz · 5 months ago

Most features are already supported on x86 CPUs, you can pip install mlx on Linux , and you can even use it on Windows (no official binary release yet but it is building and tests are passing).

zcbenz commented on PyTorch Internals: Ezyang's Blog blog.ezyang.com/2019/05/p... · Posted by u/Anon84

zcbenz · 5 months ago

For learning internals of ML frameworks I recommend reading the source code of MLX: https://github.com/ml-explore/mlx .

It is a modern and clean codebase without legacies, and I could understand most things without seeking external articles.

zcbenz commented on Show HN: You need a few lines of JS, not a vector database github.com/frost-beta/not... · Posted by u/zcbenz

minimaxir · a year ago

If you don't need a vector database, how are you storing the embeddings to compare the query against?

zcbenz · a year ago

You can use any serialization that supports numbers, in the repo I used BSER (a binary format of JSON), you can even use raw JSON but it would be very inefficient for float numbers though.

Posted by u/zcbenz a year ago

Show HN: You need a few lines of JS, not a vector database github.com/frost-beta/not...

Posted by u/zcbenz a year ago

Show HN: Chrohime – cross-platform C library wrapping Chromium's native UI code github.com/photoionizatio...

zcbenz commented on Show HN: Sisi – Semantic Image Search CLI tool, locally without third party APIs github.com/frost-beta/sis... · Posted by u/zcbenz

progx · a year ago

Uses only 1 core 100% under linux, can this be changed?

10 images, each ~20 kb size, took more than 10 minutes to index, is that normal without GPU-acceleration?

zcbenz · a year ago

No it is not normal, I only tested x64/arm64 macs, I will try on linux.

zcbenz commented on Show HN: Sisi – Semantic Image Search CLI tool, locally without third party APIs github.com/frost-beta/sis... · Posted by u/zcbenz

ivanjermakov · a year ago