Readit News logoReadit News
enjalot commented on Visualize Latent Spaces   github.com/enjalot/latent... · Posted by u/skadamat
benreesman · 2 years ago
First, this is awesome and we need more of this kind of thing.

Second, disclaimer: I am not now and might never be a serious algebraic and/or differential geometer. Just a fan at the moment.

I've been calling the useful transformations in LLM latent manifolds things like "substantially affine", and I think that's probably true enough of the current crop.

I don't think this about `{V, I}-JEPA` (about which there's a lot of information and I plan to look into it a lot more) or Sora (about which there is less information but is still impressive AF). One imagines that `V-JEPA` and Sora have some deep parallels/symmetries.

Either way, I'll wager that serious Riemannian geometry is rapidly on it's way to table stakes. We have extreme high-dimension spaces that result from backprop and gradient descent, some combination of smooth/continuous/differentiable/compact seem pretty likely to fall out? Along with interesting curvature tensors and parallel transport for moving around in them? And TDA for figuring it out numerically/computationally?

I'd love if an expert chimed in, I'm trying to describe an intuition with a fluency that involves pointing and gesturing.

enjalot · 2 years ago
The math is often behind me, I just try to visualize what I can. So I don't know how interesting this would be to you, but these folks are pretty good at math and trying to figure out what's going on in the latent spaces: https://transformer-circuits.pub/
enjalot commented on Visualize Latent Spaces   github.com/enjalot/latent... · Posted by u/skadamat
tudorw · 2 years ago
Did you look at using PHATE for dimension reduction?
enjalot · 2 years ago
I hadn't seen it, looking at the API it seems like it could be pretty straightforward to drop it in and see how the projections look.
enjalot commented on Visualize Latent Spaces   github.com/enjalot/latent... · Posted by u/skadamat
enjalot · 2 years ago
Author of the project here. Definitely appreciating the supportive comments. I'd be happy to answer questions folks have and am very interested in what kind of data folks end up visualizing with it!
enjalot commented on SQLite 3 Fiddle   sqlite.org/fiddle/... · Posted by u/sgbeal
forrestthewoods · 3 years ago
Confession: I have 15 years dev experience and have never written a single line of SQL code.

What’s a good tutorial for someone who knows how to program but doesn’t know the various SQL commands and gotchas?

enjalot commented on Friendlier SQL with DuckDB   duckdb.org/2022/05/04/fri... · Posted by u/hfmuehleisen
tosh · 3 years ago
How does DuckDB compare to SQLite (e.g. which workloads are a good fit for what? Would it be a good idea to use both?)

I found https://duckdb.org/why_duckdb but I'm sure someone here can share some real world lessons learned?

enjalot · 3 years ago
one thing I love about DuckDB is that it supports Parquet files, which means you can get great compression on the data. Here's an examples getting a 1 million row CSV under 50mb and interactive querying in the browser: https://observablehq.com/@observablehq/bandcamp-sales-data?c...

the other big thing is better native data types, especially dates. With SQLite if you want to work with timeseries you need to do your own date/time casting.

enjalot commented on D3 6.0   github.com/d3/d3/blob/mas... · Posted by u/coldsnap427
enjalot · 5 years ago
We are hosting a d3 meetup online Thursday 8/27 (tomorrow) more info here: http://meetu.ps/e/Jh8GY/1kwFr/d

Mike Bostock will be doing an AMA at the end if you have burning d3v6 questions (though you can expect some more documentation on upgrading to come very soon!)

I'll also plug our other two speakers, Mike Freeman and Amelia Wattenberger who are amazing d3 educators & authors.

It's exciting that this is the first big online d3 meetup since the pandemic put a stop to in-person events, and as a consequence of being online anyone can participate!

enjalot commented on The present phase of stagnation in the foundations of physics is not normal   backreaction.blogspot.com... · Posted by u/mathgenius
enjalot · 7 years ago
clearly it's the sophons.

(reference to the Three Body Problem, an awesome book)

enjalot commented on The Trouble with D3   medium.com/@enjalot/the-t... · Posted by u/danso
substack · 7 years ago
That's not really a fair comparison. Just because something is "baked in" doesn't mean you can't get similar functionality from other libraries in a more ala-carte fashion.
enjalot · 7 years ago
it is alacart and baked in at the same time! This article breaks down the API in graleater detail. https://medium.com/@Elijah_Meeks/d3-is-not-a-data-visualizat...

All of the functionality in d3 is split up into smaller modules that can be required independently.

enjalot commented on The Trouble with D3   medium.com/@enjalot/the-t... · Posted by u/danso
BigJono · 7 years ago
I must be missing something, this doesn't answer the question at all. Literally everything you said is true for just building the graph in JS/SVG. What does D3 give on top of that?
enjalot · 7 years ago
I think this article does a great job breaking down the d3 API and answers your question in detail “D3 is not a Data Visualization Library” @Elijah_Meeks https://medium.com/@Elijah_Meeks/d3-is-not-a-data-visualizat...
enjalot commented on Some excerpts from recent Alan Kay emails   worrydream.com/2017-12-30... · Posted by u/dasmoth
roi · 8 years ago
The thing is that Licklider's vision of computers as "interactive intellectual amplifiers for all humans, pervasively networked world-wide" has already come to pass, and created huge economies of scale and exponential pressures for compatibility and conformity that didn't exist before.

In the 1970's a few dozen brilliant people could create a completely new and self-contained computer system because the entire computing world was tiny and fragmented. there wasn't the imperative to be compatible to all-pervasive standards (even IBM's dominance in business was being challenged by the minis).

These day if you want to create a new computer system that people will use you need at the minimum to provide a networking stack and a functional web browser, some emulation or compatibility system to support legacy software that people rely on, device drivers for a huge range of hardware, etc. All this not only takes a huge amount of work, it also punctures the design integrity of your system, making it into a huge mountain of compatibility hacks before you even start on your own new concepts. But the deadliest enemy of innovation is the mental inertia of masses of users with a long history of interacting with computers. They are no longer the blank slates who have never seen a computer you had in the 70's.

Even in the realm of art people realized that the romantic or modernistic model of artistic revolution that Kay invokes is untenable and retreated into postmodernism.

enjalot · 8 years ago
I think this is why it is necessary to fund "searching for problems" research. We don't need to solve the 1970s vision of computing, we need to solve the 2020s vision of computing, whatever that may be.

u/enjalot

KarmaCake day670June 20, 2010
About
http://enja.org enjalot@gmail.com
View Original