Readit NewsIf you’re confused about what this is - it’s to prevent AI scraping.
> Anubis uses a proof-of-work challenge to ensure that clients are using a modern browser and are able to calculate SHA-256 checksums
https://anubis.techaro.lol/docs/design/how-anubis-works
This is pretty cool, I have a project or two that might benefit from it.
I’ve been wondering to myself for many years now whether the web is for humans or machines. I personally can’t think of a good reason to specifically try to gate bots when it comes to serving content. Trying to post content or trigger actions could obviously be problematic under many circumstances.
But I find that when it comes to simple serving of content, human vs. bot is not usually what you’re trying to filter or block on. As long as a given client is not abusing your systems, then why do you care if the client is a human?
x3haloed commented on DOGE employees ordered to stop using Slack 404media.co/doge-employee... · Posted by u/pulisse
Hate to break it to you, but it doesn’t exist. It’s like me coming over to your house and saying that I’m going to root out the ghosts in your home by dismantling it.
Would you let me proceed because you don’t like ghosts?
He’s telling what he’s doing, but the reason is a made-up pretext.
x3haloed commented on Software development topics I've changed my mind on chriskiehl.com/article/th... · Posted by u/belter
Good thoughts. I agree with much.
I quite disagree with the following: “Most projects don't need to ‘scale’”
I get that you’re probably referring to theoretically infinite scaling provided by stuff like microservices, containers, and serverless functions, but I do think it’s part of our job to consider the external factors that affect the functionality of our software. Similar to types being assertions about the external world, so too the design of the code and how it handles scale is an assertion about external factors. It’s important to define the bounds of what our software can handle in terms of throughput and what should happen in case those assumptions break or approach their outer bounds.
I think most projects need careful attention to handling unexpectedly large throughput gracefully.
x3haloed commented on Tell HN: Cloudflare is blocking Pale Moon and other non-mainstream browsers · Posted by u/Hold-And-Modify
I've such bots on my server. Some Chinese Huawei bot as well as an American one.
They ignored robots.txt (claimed not to, but I blacklisted them there and they didn't stop) and started randomly generating image paths. At some point /img/123.png became /img/123.png?a=123 or whatever, and they just kept adding parameters and subpaths for no good reason. Nginx dutifully ignored the extra parameters and kept sending the same images files over and over again, wasting everyone's time and bandwidth.
I was able to block these bots by just blocking the entire IP range at the firewall level (for Huawei I had to block all of China Telecom and later a huge range owned by Tencent for similar reasons).
I have lost all faith in scrapers. I've written my own scrapers too, but almost all of the scrapers I've come across are nefarious. Some scour the internet searching for personal data to sell, some look for websites to send hack attempts at to brute force bug bounty programs, others are just scraping for more AI content. Until the scraping industry starts behaving, I can't feel bad for people blocking these things even if they hurt small search engines.
Honestly, it should just come down to rate limiting and what you’re willing to serve and to whom. If you’re a free information idealist like me, I’m OK with bots accessing public web-serving servers, but not OK with allowing them to consume all my bandwidth and compute cycles. Furthermore, I’m also not OK with legitimate users consuming all my resources. So I should employ strategies that prevent individual clients or groups of clients from endlessly submitting requests, whether the format of the requests make sense or are “junk.”
> Transformers required ~2.5x more training steps to achieve comparable performance, overfitting eventually.
> RNNs are particularly suitable for sequence modelling settings such as those involving time series, natural language processing, and other sequential tasks where context from previous steps informs the current prediction.
I would like to draw an analogy to digital signal processing. If you think of the recurrent-style architectures as IIR filters and feedforward-only architectures as FIR filters, you will likely find many parallels.
The most obvious to me being that IIR filters typically require far fewer elements to produce the same response as an equivalent FIR filter. Granted, the FIR filter is often easier to implement/control/measure in practical terms (fixed-point arithmetic hardware == ML architectures that can run on GPUs).
I don't think we get to the exponential scary part of AI without some fundamentally recurrent architecture. I think things like LSTM are kind of an in-between hack in this DSP analogy - You could look at it as FIR with dynamic coefficients. Neuromorphic approaches seem like the best long term bet to me in terms of efficiency.
> I don’t think we get to the exponential scary part of AI without some fundamentally recurrent architecture
I’ve been thinking the same for a while, but I’m starting to wonder if giant context windows are good enough to get us there. I think recurrency is more neuromorphic, and possibly important in the longer run, but maybe not required for SI.
I’m also just a layman with just a surface level understanding of these things, so I may be completely ignorant and wrong.
x3haloed commented on Show HN: Hosting my website using my C web server github.com/cozis/blogtech... · Posted by u/cozis
> Show HN: Hosting my website using my own C web server
"But if you actually do this, WAT" – https://www.destroyallsoftware.com/talks/wat
As with much of HN, this is fun, a good thing to learn while making and reading about... but it likely needs the caveat that doing this is production isn't a good idea (although in this case the author does not appear to encourage production usage).
It’s a great way to get hacked
x3haloed commented on LLMs struggle to explain themselves jonathanychan.com/blog/ll... · Posted by u/jonathanyc
There are no emergent behaviors; LLMs are essentially memorizing statistical patterns in the data and using lexical cues to generate responses. They cannot explain themselves reliably because they don't know what they know, nor do they know what they don't know. In fact, LLMs don't truly 'know' anything at all. These are not thinking machines—they are simply the result of statistical pattern matching on steroids. That's why there will be no AGI, at least not from LLMs: https://www.lycee.ai/blog/why-no-agi-openai
I’d like you to explain what you don’t know, and why.
I’m sorry you’ve been inconvenienced by a murder… Or is that the country we’re living in now? “Murder happens, just don’t bother me.”
I've been looking into other techniques as well like making a little hibernation/dehydration framework for LLMs to help them process things over longer periods of time. The idea is that the agent either stops working or says that it needs to wait for something to occur, and then you start completions again upon occurrence of a specific event or passage of some time.
I have always figured that if we could get LLMs to run indefinitely and keep it all in context, we'd get something much more agentic.