Readit News logoReadit News
edg5000 commented on Disrupting the largest residential proxy network   cloud.google.com/blog/top... · Posted by u/cdrnsf
megous · 9 days ago
I'd still like the ability to just block a crawler by its IP range, but these days nope.

1 Hz is 86400 hits per day, or 600k hits per week. That's just one crawler.

Just checked my access log... 958k hits in a week from 622k unique addresses.

95% is fetching random links from u-boot repository that I host, which is completely random. I blocked all of the GCP/AWS/Alibaba and of course Azure cloud IP ranges.

It's almost all now just comming of a "residential" and "mobile" IP address space from completely random places all around the world. I'm pretty sure my u-boot fork is not that popular. :-D

Every request is a new IP address, and available IP space of the crawler(s) is millions of addresses.

I don't host a popular repo. I host a bot attraction.

edg5000 · 8 days ago
In addition to a rate limit, a page limit per IP is needed; this is specifically for things like source code repos (with massive commit histories), mailing archives, etc.

A whitelist would be needed for sites where getting all the pages make sense. And probably in addition to the 1Hz, an additional limit of 1k/day would be needed.

I can see now why Google has not much solid competition (Yandex/Baidu arguably don't compete due to network segmentation).

Scraping reliably is hard, and the chance of kicking Google off their throne may be even further reduced due to AI crawler abuse.

PS 958k hits is a lot! Even if your pages were a tiny 7.8k each (HN front page minus assets), that would be about 7G of data (about 4.6 Bee Movies in 720p h256).

edg5000 commented on Disrupting the largest residential proxy network   cloud.google.com/blog/top... · Posted by u/cdrnsf
edg5000 · 10 days ago
Residential proxies are the only way to crawl and scrape. It's ironic for this article to come from the biggest scraping company that ever existed!

If you crawl at 1Hz per crawled IP, no reasonable server would suffer from this. It's the few bad apples (impatient people who don't rate limit) who ruin the internet for both users and hosters alike. And then there's Google.

edg5000 commented on Microsoft suspects some PCs might not boot after Windows 11 January 2026 Update   windowslatest.com/2026/01... · Posted by u/nsoonhui
giancarlostoro · 15 days ago
The Creality one runs decent on Mac and Windows, sadly on Linux its a nightmare, and technically why I ditched Ubuntu / popOS for Arch Linux, but I can't help but still feel it runs a little weirder + its out of date compared to Mac and Windows versions. My buddy used to use Orca slicer on my printer, that one iirc should run on Mac too, but I havent tried it.
edg5000 · 14 days ago
Does Creality have special changes made to the slicer? If it's just the profilem, then running the PrusaSlicer app image might be the easiest. PrusaSlicer appimage has always worked perfectly on Ubuntu 22 LTS.
edg5000 commented on Microsoft suspects some PCs might not boot after Windows 11 January 2026 Update   windowslatest.com/2026/01... · Posted by u/nsoonhui
bdcravens · 15 days ago
A number of the Autodesk tools and Solidworks, for modeling. Slicers can use APIs native to Windows to perform model repairs. Bambu Lab's farm manager only runs on Windows.
edg5000 · 15 days ago
Not sure about Autodesk, but have you tried FreeCAD? I own a perpetual SolidWorks license but haven't even activated it. Used it quite lot on another license but I just prefer FreeCAD so much. It does choke on high primitive counts though. Probably has worse FEA (invokes external simulation tools) but that is an assumption, never did FEA. Mostly did parametric CAD, not much technical drawings either, can't say much about that.

For slicers I use PrusaSlicer on Linux (don't have a Prusa; it's really good for generic slicing). But I can see how Bambu stuff could be an issue if it's Win only and not Wineable.

edg5000 commented on Iran's internet blackout may become permanent, with access for elites only   restofworld.org/2026/iran... · Posted by u/siev
cryptoegorophy · 15 days ago
Spacex satellites blockage was the surprise. How did they do it? I thought it would be the best dooms day kind of insurance. Turns out not.
edg5000 · 15 days ago
My wild guess is that jamming is local. Major cities may be fully jammed. To get an idea about GNSS jamming range (different signal of course, probably much easier to jam), there are maps online where you can see which parts of Europe are currently GNSS-jammed. But I have the same question as you.
edg5000 commented on Microsoft suspects some PCs might not boot after Windows 11 January 2026 Update   windowslatest.com/2026/01... · Posted by u/nsoonhui
bdcravens · 15 days ago
I switched to Macs almost completely for personal and devlopment use about 13 or 14 years ago. However, last year I started a 3d printing side hustle, and got an HP laptop for running the print studio since the amount of hardware I could get for less than $1000 was hard to ignore. However, things like this, and other weird issues (my fonts have gone all wonky a couple of times after random updates) make me want to switch it over to a Linux distro (even though the software support for what I need is much better in the Windows world, and in some cases, better than even on the Mac)
edg5000 · 15 days ago
> the software support for what I need is much better in the Windows world

Please elaborate; can you name a few tools and what you use them for? Just curious.

edg5000 commented on TikTok is officially US-owned for American users, here's what's changing   9to5mac.com/2026/01/23/ti... · Posted by u/WaitWaitWha
pr337h4m · 16 days ago
This (and PAFACA in general) is a massive disgrace from a 1A POV.
edg5000 · 16 days ago
What is 1A and PAFCA?
edg5000 commented on Gas Town's agent patterns, design bottlenecks, and vibecoding at scale   maggieappleton.com/gastow... · Posted by u/pavel_lishin
wordswords2 · 17 days ago
There is nothing professional, analytical or scientific about Gas Town at all.

He is just making up a fantasy world where his elves run in specific patterns to please him.

There is no metrics or statistics on code quality, bugs produced, feature requirements met.. or anything.

Just a gigantic wank session really.

edg5000 · 17 days ago
Are you being sarcastic or serious? Meeting requirements is implicitly part of any task. Quality/quantification will be embedded in the tasks (e.g. X must be Y <unit>); code style and quality guidelines are probably there somewhere in his tasks templates. Implicitly, explicit portions of tasks will be covered by testing.

I do think it's overly complex though; but it's a novel concept.

edg5000 commented on Gas Town's agent patterns, design bottlenecks, and vibecoding at scale   maggieappleton.com/gastow... · Posted by u/pavel_lishin
edg5000 · 17 days ago
First time I'm seeing this on HN. Maybe it was posted earlier.

Have been doing manual orchestration where I write a big spec which contains phases (each done by an agent) and instructions for the top level agent on how to interact with the sub agent. Works well but it's hard utilize effectively. No doubt this is the future. This approach is bottlenecked by limitations of the CC client; mainly that I cannot see inter-agent interactions fully, only the tool calls. Using a hacked client or compatible reimplementation of CC may be the answer. Unless the API was priced attractively, or other models could do the work. Gemini 3 may be able to handle it better than Opus 4.5. The Gemini 3 pricing model is complex to say the least though (really).

edg5000 commented on Eigent: An open source Claude Cowork alternative   github.com/eigent-ai/eige... · Posted by u/WorldPeas
a7m-1st · 21 days ago
You can have a try; almost all sota models are supported all powered thanks to https://github.com/camel-ai/camel
edg5000 · 20 days ago
Wow, CAMEL looks very interesting, newer heard of that. Will look into it.

u/edg5000

KarmaCake day291July 30, 2022View Original