HN has a hair trigger about banning IPs that request too fast (sorry about that; we don't have a lot of spare performance), so I wrote something people can use to get their IP unbanned once if it gets banned by accident.
http://news.ycombinator.com/unban?ip=<ip address>
Obviously you have to use it from another IP address, like your phone.
Since there are so few images on HN, there is no reason to have more than a couple connections per IP on port 80.
It will radically reduce your server load and there will be no blacklists/whitelists to maintain.
Unless there are dozens of people from the exact same IP and not an IP pool, it won't be a problem.
The worst case is they will see a longer delay for an initial page load in their browser by half a second. But it helps the server tremendously, especially since HN seems to use Apache.
Actually looks like I am wrong.
Static objects are coming from Amazon while dynamic are coming from another server @theplanet.com
So you are right, it's FreeBSB, but it's still Apache which really needs connection throttling. But there might be a reverse proxy in place. You can also IP throttle with a module in nginx.I would expect it to solve most of your performance problems for the foreseeable future (at the very least, by letting you scale horizontally and move the DB, frontends, and memcaches to separate boxes - plus ending memory leaks/etc by moving most of the data off the MzScheme heap).
The obvious downside is that it would use your (or someone at YC's) time. First to merge the changes I make to http://ycombinator.com/arc/arc3.tar into the production code, then to buy/setup some extra boxes and do the migration. We're probably talking, roughly, a day. It also has the unfortunate side effect of costing HN's src some of its pedagogical value, since it adds external dependencies and loses 'purity'.
Been looking for an excuse to learn arc for a while now ...
The site is very much hacked together, but works... In a lot of ways, this reflects the hacker ethos of getting something up and running quickly at low cost while still producing value.
A revamp might have negative impact too by attracting a wider, more mainstream audience which could possibly dilute the purity of the community here.
Careful now :) It's not like there's anything stopping HN attracting a wider audience anyway; there's no restriction on who can register. Anyone can come and join in, which (in my opinion) is as it should be.
Any engineer that has live code has made this mistake before.
My hope is that it will only take a day or so to deploy it, once it's ready.
I would agree that there is also little to no desire to make Hacker News "the news place" - where it supports thousands of posts a second and is extremely popular. In general Hacker News is used (and the hope is to stay that way) by startups and people interested in startups - it's slowly growing out to include more types of people - marketing, companies, blog posts who just want a lot of hits, etc - and not many people want to purposely support that.
http://www.hnsearch.com/api
Quote: "HNSearch was built by the team at ThriftDB to give back to the community and to test the capabilities of the ThriftDB flexible datastore with search built-in."
Interesting API all the same though.
edit: oh, official API is above. Disregard this one :-)
Deleted Comment
It speeds up browser startup dramatically. Especially when you leave lots of tabs open as your "to read" list.
Firefox has a better solution for this but then again, I don't use firefox.
I only loaded so many pages because I love HN. :)
The ban was lifted a few days later, not sure if automatically or thanks to my (unresponded) email request.
It is triggered very quickly and it seems to last forever (maybe 15min would be better?).
I ask pg to kindly consider making it a bit more lenient.
I doubt HN goes under deliberate/malicious attacks, etc...
I'm making a HN extension that preloads some data such as the comments and the links on the next page (it's still with reasonable delays).
But at the moment it's impossible for it to function without risking the user getting banned.
Pretty much any site with decent traffic is under constant attack, and the high profile of HN means it'll be under far more scrutiny than others.
[−]sunstone1 10 hours ago | link [dead]
Well I never had my IP banned but I did have my account hell banned after about a dozen posts as you can see. Oh, actually no, you can't see, because it's banned. No, I never bothered to get another account, now I'm just a taker not a giver.
Most of the time it's clear why a user was banned, but looking at sunstone's history I don't really see a reason. While the algorithm will never be perfect, it would be nice if there was a clearer solution for misfires.