Readit News logoReadit News
justinpombrio commented on Memory access is O(N^[1/3])   vitalik.eth.limo/general/... · Posted by u/jxmorris12
Dylan16807 · 2 months ago
The core aspect of information theory is how much information fits in a sphere. Getting from there to memory access latency in a real computer is several abstractions away and a lot of those abstractions might not hold.
justinpombrio · 2 months ago
Could you name one that seems likely to fail?
justinpombrio commented on Why haven't local-first apps become popular?   marcobambini.substack.com... · Posted by u/marcobambini
quotemstr · 3 months ago
> Difference from CRDTs

The author has made a CRDT. He denies that his algorithm constitutes a CRDT. It's a straightforward merge, not a "fancy algorithm".

What specific aspect of a CRDT does this solution not satisfy? The C? The R? The D? The T?

justinpombrio · 3 months ago
I was going to say that that's not a CRDT because it requires a centralized server (the conflict resolution is "order in which the server received the messages", and clients aren't allowed to share updates with each other, they can only get updates from the server). But now I'm looking at definitions of CRDTs and it's not clear to me whether this is supposed to count or not.

Still, every algorithm that's actually labeled a CRDT shares a magical property: if my replica has some changes, and your replica has some changes, our replicas can share their changes with each other and each converge closer to the final state of the document, even if other people have been editing at the same time, and different subsets of their changes have been shared with you or I. That is, you can apply peoples' changes in any order and still get the same result. I don't think it's useful to call anything without that property a CRDT.

justinpombrio commented on The Expression Problem and its solutions (2016)   eli.thegreenplace.net/201... · Posted by u/andsoitis
Kranar · 3 months ago
Nothing you mention is related to this article and neither Rust or C# solve the expression problem.

The expression problem is about being able to extend both data types (new cases) and operations (new functions) without modifying existing code while preserving static type safety.

C#'s extension methods can't be virtual, so you can't use them to actually add any new operations which can be dispatched on. They're just syntactic sugar for adding static methods which in non-OOP languages can be accomplished by declaring a free function.

Rust traits let you add new types (just impl the Trait), but it doesn't let you add new functions, so it doesn't do anything to solve the problem.

justinpombrio · 3 months ago
Rust's traits _do_ solve the expression problem.

Each data type is a `struct`. Each operation is a trait. You `impl` each trait on each struct.

This works even if you're using a library that has declared `struct A` and `struct B` and `trait F` and `trait G`, and you want to add both a `struct C` and a `trait H`, and fill out the whole 3x3 grid without modifying the library.

The library says:

    struct A { ... }
    struct B { ... }

    trait F { ... }
    impl F for A { ... }
    impl F for B { ... }

    trait G { ... }
    impl G for A { ... }
    impl G for B { ... }

    fn some_function<T: F + G>(data: T) { ... }
Your code says:

    use library::{A, B, F, G};

    struct C { ... }
    impl F for C { ... }
    impl G for C { ... }

    trait H { ... }
    impl H for A { ... }
    impl H for B { ... }
    impl H for C { ... }

    fn another_function<T: F + G + H>(data: T);
Now `library::some_function()` can be called with an A, B, or C, even though C was defined by the user of the library. And `another_function()` can be called with A, B, or C, even though A was defined by the library.

justinpombrio commented on Show HN: I made a tool to port tweets to Bluesky mantaining their original date   bluemigrate.com... · Posted by u/nols05
Kye · 9 months ago
Some people got the idea it has something to do with some song because (not an actual dictionary) Urban Dictionary said so when the actual meaning is "sky tweet."
justinpombrio · 9 months ago
> (not an actual dictionary) Urban Dictionary

<troll> The definition of dictionary is just "Words about words" (source: Urban Dictionary), so I'd say that Urban Dictionary qualifies. </troll>

justinpombrio commented on What can be computed? A practical guide to the theory of computation (2018) [pdf]   softouch.on.ca/kb/data/Wh... · Posted by u/nill0
jltsiren · 10 months ago
This is unnecessary nitpicking. You can easily derive a logical contradiction of your choice by assuming that a system that can be formally modeled can prove something that cannot be proven within a formal system. If a specific theorem doesn't prove that, a simple corollary will.

If you want to escape the limits of computability, you have to assume that the physical Church–Turing thesis is false. That the reality allows mechanisms that cannot be modeled formally or simulated on any currently plausible computer.

justinpombrio · 10 months ago
> by assuming that a system that can be formally modeled can prove something that cannot be proven within a formal system

Something that cannot be proven within which formal system? Every true statement can be proven by some formal system. No formal system can prove all true statements.

(This is all about Godel incompleteness. Turing incomputability does apply though, see the sibling comments! It's important to keep them separate, they're saying different things.)

justinpombrio commented on What can be computed? A practical guide to the theory of computation (2018) [pdf]   softouch.on.ca/kb/data/Wh... · Posted by u/nill0
Tainnor · 10 months ago
I'm confused by this. Why are you talking about Gödel?

LLMs, like any model of computation, are subject to the limits of Turing Machines, so they cannot for example solve the halting problem.

justinpombrio · 10 months ago
Oof, I pattern matched OP to a different argument I've seen a lot. That's embarrassing.

Yes, that's correct, it's provably impossible to construct an LLM that, if you ask it "Will this program halt? <program>", answers correctly for all programs. Likewise humans, under the assumption that you can accurately simulate physics with a computer. (Note that QM isn't an obstacle, as you can emulate QM just fine with a classical computer with a mere exponential slowdown.)

justinpombrio commented on What can be computed? A practical guide to the theory of computation (2018) [pdf]   softouch.on.ca/kb/data/Wh... · Posted by u/nill0
ccvannorman · 10 months ago
What a wonderful text. Easy to read, concise, clear, interesting -- and above all, important.

I would add context for 2025 about the fundamental limits this places on what (modern) AI is in principal capable of. Perhaps some "non-computable" features would need to be hard-coded into AI, so that it could at least better approximate the types of incomputable problems we might ask it to do?

Also, a search of the text for "conscious" does not yield anything, which is probably a good thing. This text also reminds me of the questions like, "What does it mean to be conscious?" and, "How are human brains able to reason about (things like) incomputability, which in some sense, computers as we currently understand them could never do?" and, "What specifically beyond pure mathematics does a brain have or need in order to be conscious enough to reason about such things?"

justinpombrio · 10 months ago
Gödel's Incompleteness Theorem places a limit on what you can prove within a formal system. Neither humans nor LLMs are a formal system, so it says nothing about them.

Someone's going to think that, since you can formally model computer programs (and thus formally model how an LLMs runs), that means that Gödel's Incompleteness Theorem applies to computer programs and to LLMs. But that's irrelevant; being model-able doesn't make something a formal system!

"Formal system" has a very technical meaning: it has a set of consistent axioms, and the ability to enumerate formal proofs using those axioms. For example, Zermelo-Fraenkel set theory is a formal system [1]. It has nine formal axioms, which you can see listed in its Wikipedia article. Utterly different kind of thing than humans or LLMs. Comparing an LLM to ZFC is like comparing a particular watermelon to the number 4.

[1] https://en.wikipedia.org/wiki/Zermelo%E2%80%93Fraenkel_set_t...

justinpombrio commented on Is this the simplest (and most surprising) sorting algorithm ever? (2021)   arxiv.org/abs/2110.01111... · Posted by u/gnabgib
teach · 10 months ago
I used to teach this to beginners all the time. I called it "exchange sort", though I'm not sure where I ever got that name from.

Beginners to sorting are often still somewhat struggling with for loops (especially the three-part C kind), struggling with indexing an array, struggling with swaps. It's overwhelming.

This is kind-of like pseudocode for a simple sort -- it has the same rough outline, but no tricky bits. It allows them to get their brains around the basic algorithm, and only after that you trace through an execution on the board to show them that it wastes a lot of time moving things around.

Then you can show them insertion sort as an optimization, and from that move on to fancier ones.

justinpombrio · 10 months ago
Can you give a reference to your teaching materials? The way this sort works is bizarre enough that I'm curious what you said about it. (Also to verify that this is actually the algorithm you taught, given how many other people thought it was something else.)
justinpombrio commented on Is this the simplest (and most surprising) sorting algorithm ever? (2021)   arxiv.org/abs/2110.01111... · Posted by u/gnabgib
thaumasiotes · 10 months ago
I expect them to have exactly the same number of swaps in every iteration of the loop past the first one, because they are doing exactly the same thing. It's not just the same number of swaps. It's essentially the same set of swaps, in a different order. In the iteration where i = 1, the number of swaps may differ. Was I wrong about that?
justinpombrio · 10 months ago
Yeah, 2410 vs. 2509 swaps not including the first iteration. You should try measuring these things yourself, I'm not sure if it's doing what you think it's doing.
justinpombrio commented on Is this the simplest (and most surprising) sorting algorithm ever? (2021)   arxiv.org/abs/2110.01111... · Posted by u/gnabgib
thaumasiotes · 10 months ago
> The number of swaps is nearly the same between Bubble Sort and Can't Believe Sort. For a random list with 100 elements, it's 2505 swaps vs. 2513 swaps.

Why is there any difference? Does the entire difference come from the iteration where i = 1?

> Bubble Sort always does exactly n(n-1)/2 comparisons

Bubble sort can do less than this. When you're bubbling down into a sorted list, and you find a value lower than you are, you can assume that all values below that one are also lower than you are and terminate the inner loop early.

justinpombrio · 10 months ago
> Why is there any difference? Does the entire difference come from the iteration where i = 1?

Sometimes it's more swaps, and sometimes it's less. The vague pattern looks like Can't Believe Sort does slightly fewer swaps on larger lists, but that could also have been noise.

Why do you expect them to have exactly the same number of swaps?

> Bubble sort can do less than this.

Oh sorry yeah, I implemented bubble sort without any sort of early return, for a more direct comparison.

u/justinpombrio

KarmaCake day4631March 26, 2011
About
"zall" + "ambo" + "@gmail.com"
View Original