Readit News logoReadit News
anacoluthe commented on Making geo joins faster with H3 indexes   floedb.ai/blog/how-we-mad... · Posted by u/matheusalmeida
cullenking · 2 days ago
We do something similar for some limited geospatial search using elastic search. We make a set of h3 indexes for each of the hundreds of millions of gps recordings on our service, and store them in elastic search. Geospatial queries become full text search queries, where a point is on the line if the set of h3 indexes contains the point. You can do queries on how many cells overlap, which lets you match geospatial tracks on the same paths, and with ES coverage queries, you can tune how much overlap you want.

Instead of using integers IDs for the hexes, we created an encoded version of the ID that has the property that removing a character gets you the containing parent of the cell. This means we can do basic containment queries by querying with a low resolution hex (short string) as a prefix query. If a gps track goes through this larger parent cell, the track will have hexes with the same prefix. You don’t get perfect control of distances because hexes have varying diameters (or rather the approximation, since they aren’t circles they are hexes), but in practice and at scale for a product that doesn’t require high precision, it’s very effective.

I think at the end of this year we’ll have about 6tb of these hex sets in a four node 8 process ES cluster. Performance is pretty good. Also acts as our full text search. Half the time we want a geo search we also want keyword / filtering / etc on the metadata of these trips.

Pretty fun system to build, and the concept works with a wide variety of data stores. Felt like a total hack job but it has stood the test of time.

Thanks uber, h3 is a great library!

anacoluthe · a day ago
Beware that the parent hexagon does not contain its children...
anacoluthe commented on Learning Go as a Python Developer: The Good and the Bad   new.pythonforengineers.co... · Posted by u/shantnutiwari
alexanderh · 4 years ago
This is what I came to the comment section to say... You absolutely can pin dependencies.... da fudge?

Sounds like this guy needs to finish learning Python before he learns something else.

From what you suggested, to containerizing things with something like Docker, there are ways to make Python more easily distributable.

anacoluthe · 4 years ago
What if the depedencies you pinned have non-pinned depedencies?

packageA==1.0.0 depends itself on packageB

Therefore, you can find yourself with a different set of deps. Had a bug like this once.

anacoluthe commented on Google to pay $1B in France to settle fiscal fraud probe   reuters.com/article/us-fr... · Posted by u/etchezaldun
jquery · 6 years ago
Is this a shadow trade war in response to America’s dominance in tech? I keep seeing absolutely massive fines of American tech companies in Europe, fines marked up to account for Google’s global revenues. Do they make their homegrown companies pay such outrageous fines? $1B is a staggering amount of money, probably all of Google’s profits from France for years to come.

To be clear I think France has every right to do this and Google can decide to close up shop there if they want, nobody is forcing Google to operate in an unfriendly environment.

anacoluthe · 6 years ago
Maybe it is a response to a more global situation. Please look into corruption charges and fines of US Justice on European companies such as Alcatel, Société Générale, Alstom and you'll that these fines are quite a good instrument to weaken strategic European competitors (and the opposite is true as well) and even take them over. Example: https://www.economist.com/business/2019/01/17/how-the-americ...
anacoluthe commented on Using Flywheel Batteries to Charge Electric Vehicles in Ten Minutes   calcalistech.com/ctech/ar... · Posted by u/orrhirschauge
DubiousPusher · 8 years ago
I gotta say, flywheels are rad. They have all kinds of cool applications including moving satelites, https://en.wikipedia.org/wiki/Reaction_wheel.
anacoluthe · 8 years ago
For example, some wind tunnels are powered using flywheels (c.f. high enthalpy wind tunnel F4 from ONERA in France is using a 15 tons flywheel to get the necessary power).
anacoluthe commented on France’s Prometheus reusable engine becomes ESA project, gets funding boost   spacenews.com/frances-pro... · Posted by u/nolok
frankharv · 8 years ago
What's really a shame is SpaceX is going to have to compete with these massive government programs while SpaceX created their own program from scratch with private investors.

How is this fair? Sure DOD and NASA are customers but SpaceX financed their advancements and had to convince the US government to use their platform with successful launches.

This sounds similar to Airbus's support. Why is this allowed. US intelligence should be sabotaging these free market cheaters. Europe is acting like China, where the government is financially supporting major industries competing with the US. I will call it what it is. Economic Warfare. Maybe more aptly Economic Welfare Warfare.

anacoluthe · 8 years ago
SpaceX was awarded $1.6 billion for 12 cargo missions 10 years ago. Given their current pricing, that sounds a lot like public investment...

The model is simply different in Europe where public money funds most of the development costs: market size is smaller in Europe (compared to US institutional market for example), geographic return makes things more complicated and it is of course all about independent space access.

Finally, all US institutional satellites must be launched on American launchers ("Buy American Act"), which is not the case in Europe. Free market, were you saying?

u/anacoluthe

KarmaCake day18November 12, 2017View Original