Warning: long somewhat related story that is basically humblebragging, but the summary is that bypassing Twitter ratelimits is not very hard.
I didn't feel like playing around with Twitter's annoying certificate pinning so I just uploaded the Twitter APK to Corellium, turned on what they call the "network monitor", opened the Twitter app since it lets you use Twitter without signing in. I clicked around, searched and viewed tweets. Then I looked at the requests in the log and saw it has a similar guest token process to the website but with a few differences. Anyways, if you recreate these requests, with one IP address you can generate a few OAuth tokens with no expiry per day. These tokens are for unauthenticated users so obviously they have no write privileges but that's not what was needed here. So if you have a proxy provider with a large pool of IPs where you can buy like 1GB of bandwidth you can use a very small percent of your bandwidth allowance and get thousands of tokens/secrets easily, all with their own separate rate limits. It doesn't even matter what IP you end up using the tokens on. Then I followed https://docs.google.com/document/d/1xVrPoNutyqTdQ04DXBEZW4ZW... and the fact that /statuses/lookup.json still allows you to return 100 (!) tweets at once to reconstruct something close to what the 50% Twitter firehose would look like. And Twitter doesn't even block datacenter IP addresses! Was going to display the data at https://firehose.lol but the fact that it required a few hundred requests a second made me feel bad so I didn't end up running the program for more than a few minutes at a time and shut it down.
Looking at (a fraction of) the Firehose for a few minutes was interesting, originally I accidentally forgot to not display tweets labelled possibly_sensitive so I saw some pretty salacious material for a few seconds. Lots of Chinese gambling ads even though Twitter is blocked there, dubious investment promoters, accounts with usernames like FirstnameLastname3781264872 who would tweet three random words at each other every couple of seconds, and a handful of funny tweets.
Nice. This isn't nearly as efficient, but a simpler way to bypass the ratelimit is to use archive.md, which is immune to the ratelimit. It's useful if you don't have an account and just want to see a few tweets here and there.
This is the most impressive project in Nim I’ve seen yet. Rewriting any major front-end, complete with working authentication and handling idiosyncrasies of the private API is a herculean task. For context, Twitter would have a team of two dozen or more supporting what this does, effectively. Kudos to the author for accomplishing this feat!
The frontend uses Karax, which is my favorite frontend/SPA library in any language. It is an absolute joy to use, even if it's a bit rough around the edges.
And the flagship instance of Nitter is unusable more often than not - the fact that it's working today is likely an artifact of people having given up given the week long total outage.
I feel the same way. Don't know if it's because of the algorithm or something else but Twitter these days seems to be 80% bots shilling crypto and/or simping for Musk.
tinfoil hat time: what if that was the point all along? twitter was a vital space for organizing protests - think arab spring, occupy wall street.
one of the richest guys in the world (who's also very anti-union, btw!) buys it up in a time where inequality is getting worse and worse and social fabrics are starting to tear and makes it unusable.
no more space for organizing. one fewer threat to capital.
He owns the site. If this is truly his intention then all he has to do is turn off the main switch. There's really no reason to give him the benefit of doubt and twist the narrative into "Musk is a genius and everything is going according to plan". The simpler theory is likely to be the correct one – he has never run a social network before and has no idea how to stop bleeding users and cash so is desperately throwing ideas at the wall hoping something sticks.
He would be really stupid if that is his thinking, because as we just saw these users would just move to a competitor as long as there is one. Twitter doesn't have a monopoly on short conversation-based social media.
Given the dude paid $44B for a site clearly worth significantly less than that, and then promptly ran it into the ground, your "tinfoil hat time" answer honestly seems like the only rational answer. It checks out on more levels than any of his actions have.
I swear, the bots is only increasing ever since Elon bought the site. I've been getting more and more DM from spam accounts, somehow from Japan too (Whaaat?).
Hey there, just wanted to thank you because you also fixed my Twitter Spaces downloader app[0]! After the API changes the default bearer token I was using (same as yours) stopped working, but after changing the same way you all's back to normal :D
Is this really permanent? I'd love to know more about this bearer token.
Because in the other github issue thread it seemed like every time they found a way around Twitter's safeguards, it was shutdown.
It seems like they've literally hard coded a token into the source code. Meaning thousands of nitter-instances, thousands of users, around the world, will use the same token.
I'm curious that an "unofficial" API has been allowed to continue working, however intermittently, at all. I appreciate using Nitter, but something about it doesn't add up to me.
NewPipe has been working for me for years with occasional breaking changes on YouTube's side that required an urgent update. I watch pretty much everything at 2x speed 1080p and have no issues with buffering.
Readit News
I didn't feel like playing around with Twitter's annoying certificate pinning so I just uploaded the Twitter APK to Corellium, turned on what they call the "network monitor", opened the Twitter app since it lets you use Twitter without signing in. I clicked around, searched and viewed tweets. Then I looked at the requests in the log and saw it has a similar guest token process to the website but with a few differences. Anyways, if you recreate these requests, with one IP address you can generate a few OAuth tokens with no expiry per day. These tokens are for unauthenticated users so obviously they have no write privileges but that's not what was needed here. So if you have a proxy provider with a large pool of IPs where you can buy like 1GB of bandwidth you can use a very small percent of your bandwidth allowance and get thousands of tokens/secrets easily, all with their own separate rate limits. It doesn't even matter what IP you end up using the tokens on. Then I followed https://docs.google.com/document/d/1xVrPoNutyqTdQ04DXBEZW4ZW... and the fact that /statuses/lookup.json still allows you to return 100 (!) tweets at once to reconstruct something close to what the 50% Twitter firehose would look like. And Twitter doesn't even block datacenter IP addresses! Was going to display the data at https://firehose.lol but the fact that it required a few hundred requests a second made me feel bad so I didn't end up running the program for more than a few minutes at a time and shut it down.
Looking at (a fraction of) the Firehose for a few minutes was interesting, originally I accidentally forgot to not display tweets labelled possibly_sensitive so I saw some pretty salacious material for a few seconds. Lots of Chinese gambling ads even though Twitter is blocked there, dubious investment promoters, accounts with usernames like FirstnameLastname3781264872 who would tweet three random words at each other every couple of seconds, and a handful of funny tweets.
you can also use the Googlebot user agent to see the page, despite it being a different format
https://github.com/karaxnim/karax
Of course, because if Nitter goes down nobody bats an eye.
one of the richest guys in the world (who's also very anti-union, btw!) buys it up in a time where inequality is getting worse and worse and social fabrics are starting to tear and makes it unusable.
no more space for organizing. one fewer threat to capital.
Adding almost any restriction only hurts casual users, and attackers are rarely casual users.
Whatever Elmo is doing, it's not working
0: https://github.com/Chiplis/moonbird
I’ve been a nitter user for 4 years now and will be as long as it works.
Because in the other github issue thread it seemed like every time they found a way around Twitter's safeguards, it was shutdown.
It seems like they've literally hard coded a token into the source code. Meaning thousands of nitter-instances, thousands of users, around the world, will use the same token.
And potentially so will the AI companies.
So I just don't see how this can work.
https://www.reddit.com/r/fossdroid/comments/10b0krt/comment/...
Or if it does work, its absurdly slow.
Twitter should provide a noscript/basic (x)html interop www portal.