What H3 do I belong to if my house is split between three different ones, pretty much equally? Any/all of them?
You take a smaller H3 :-) The maximum area of a resolution 15 H3 is 1 square meter, so unlikely to split a house in two.
Readit Newsmaxmouchet commented on Stop using zip codes for geospatial analysis (2019) carto.com/blog/zip-codes-... · Posted by u/voxadam
maxmouchet commented on Using less memory to look up IP addresses in Mess With DNS jvns.ca/blog/2024/10/27/a... · Posted by u/todsacerdoti
Actually 4294967296 couldn't ever appear as the maximum value you can fit in the protocol field is 1 less than that... my problem here was I couldn't manage to keep the 2 numbers I was comparing (the one in the article and 2^32) straight haha! This was mistake was noted by a commenter here https://news.ycombinator.com/item?id=41963745
That said you're ultimately right that my upstream provider is filtering the 4294901866 value from the article as well anyways for the reasons you stated.
Ah right haha. Thanks for the heads up, I should have checked ^^
maxmouchet commented on Using less memory to look up IP addresses in Mess With DNS jvns.ca/blog/2024/10/27/a... · Posted by u/todsacerdoti
This is the same idea I used for my Cloud IP lookup tool [1], lets it all work in browser with a small file to search against
Oh this is nice, and a cool use of HTTP range requests!
maxmouchet commented on Using less memory to look up IP addresses in Mess With DNS jvns.ca/blog/2024/10/27/a... · Posted by u/todsacerdoti
> As an aside – I’m storing the ASN in a uint32, is that right? I looked in the ip2asn file and the biggest one seems to be 401307, though there are a few lines that say 4294901931 which is much bigger and would not fit in a uint32. I decided to just ignore the gigantic ASNs and assume I could fit everything in a uint32.
Does anyone have a theory to how that 4294967296 might have made its way into the data? I'm getting 0 matches on the actual internet with:
birdc show route protocol peer4 | grep 4294967296 -c
Edit: actually, I'm not seeing it in the linked iptoasn data either?
Bogus announcements are probably filtered by your upstream(s) (see [1] for a common list of filters).
IP-to-ASN mappings are typically built from route collectors [2,3] that peer with various networks and receive their announcements. AFAIK route collectors don't filter anything and it's easy to find bogus announcements (e.g. private ASNs) in the data.
I can't find 4294967296 from a quick glance at the latest RouteViews data but I can find other private ASNs. For example AS7594 - AS2764 - AS4294901866 for 210.10.189.0/24 seen by the route-views.perth collector.
I don't know what kind of filtering iptoasn.com is doing but at work (ipinfo.io) we do filter bogus origins, as well as a bunch of other things like RPKI/IRR-invalid routes and hyper-specific prefixes (> /24 or /48) [4].
[1] https://bgpfilterguide.nlnog.net
[2] https://www.routeviews.org/routeviews/
[3] https://www.ripe.net/analyse/internet-measurements/routing-i...
maxmouchet commented on Using less memory to look up IP addresses in Mess With DNS jvns.ca/blog/2024/10/27/a... · Posted by u/todsacerdoti
Another solution is to use an MMDB (“MaxMind DB”) file [1] which is essentially a binary tree + deduplicated values (same as idea 3.1).
There are several free ASN MMDBs [2,3] but you can also build your own MMDB files from any Prefix->Value mapping with the mmdbwriter library [4] or a CLI tool built on top of it like mmdbctl [5].
Assuming the ASN MMDB is fully loaded in memory, it would use around 60MB.
[1] https://maxmind.github.io/MaxMind-DB/
[2] https://dev.maxmind.com/geoip/docs/databases/asn/
[3] https://ipinfo.io/products/free-ip-data-downloads
[4] https://github.com/maxmind/mmdbwriter
[5] https://github.com/ipinfo/mmdbctl
(I work for IPinfo, but there are lots of other companies offering MMDB files).
maxmouchet commented on AWS data center latencies, visualized benjdd.com/aws/... · Posted by u/orliesaurus
Obviously the biggest contribution to latency is distance. But there's also some close-ish regions with poor latency because there's not fiber running directly between them (for example, over the poles)
Are there an examples of regions which dramatically violate the triangle equality? (That is, where the A--C latency is much worse than the best A--B + B--C latencies)?
Just as a curiosity, could you use that idea to "infer" which data-centers are most likely directly connected by fiber, and show only the likely fiber connections?
I can't speak for AWS specifically, but in my PhD thesis [1] I found a bunch of such examples by using RIPE Atlas probes. Essentially looking for pairs of probes where the RTT between probe A and probe C is larger than probe A-B + B-C.
Now there are some issues with this methodology (all common issues with ICMP/RTT measurements + traffic was not really routed through the "relay" probe), but such pairs do exist.
[1] https://theses.hal.science/tel-03666771/document (see page 84 for an example; if you can read French :-))
maxmouchet commented on Pi calculation world record with over 202T digits storagereview.com/news/st... · Posted by u/radicality
As pi never repeats itself, that also means that every piece of conceivable information (music, movies, texts) is in there, encoded. So as we have so many pieces of pi now, we could create a file sharing system that's not based on sharing the data, but the position of a piece of the file in pi. That would be kinda funny
maxmouchet commented on Pi calculation world record with over 202T digits storagereview.com/news/st... · Posted by u/radicality
Nice achievement but always a bit disappointing that those records are based on throwing more money at the problem, rather than new theoretical grounds, or software improvements (IIRC y-cruncher is not open source).
maxmouchet commented on A transport protocol's view of Starlink potaroo.net/ispcol/2024-0... · Posted by u/fanf2
That is true for ICMP packets directed at the router itself.
Pretty sure that is the reason why he used
> ping across a Starlink connection from the customer side terminal to the first IP end point behind the Starlink earth station
instead of pinging the terminal itself.
(ICMP packets passing _through_ a router are handled just like any other traffic and are therefore OK for an analysis like this)
Additionally, I would not expect such “baseline” changes in the minimum RTT to be due to variation in CPU processing time.
These changes are more typical of a physical path change, as suggested by the author.
CPU/soft processing latency would look more like additive noise.
I give some examples of RTT patterns in this talk: https://ripe77.ripe.net/archives/video/2250/
Also, can this data collection see enough of internal China traffic?
I've also played a bit with an interactive visualization of the graph by using map tiles [4].
[1] https://github.com/TheOpteProject/LGL
[2] https://github.com/maxmouchet/minilgl (cleaned-up version of the code)
[3] https://www.maxmouchet.com/internet-viz/
[4] https://github.com/maxmouchet/internet-visualization