I was thinking, if I was to build a new micro-blogging aka Twitter alternative, what technological choices would I make to get it quickly off the ground but allow for scale?
What database technology or approach would you go for?
Would you build a (mobile first) web application first or would you start straight away with a native iOS/Android app (maybe Flutter)?
Would you go for a centralised or decentralized approach? If the latter, how would you decentralize it without sacrificing the "public town square" effect that Twitter currently has but is clearly lacking with the fragmentation of Mastodon?
To answer my own question, I would probably build a centralized platform like Twitter is now, probably opt for a fast NoSQL database like Google's Firestore in Datastore mode and to keep things simple I would probably even make sure that tweets would get automatically deleted after some years as I don't think it's needed to build a forever growing database of people's thoughts in that moment that persists for decades to come. Micro-blogging always felt to me as a thing right now, a thought in this moment but that thought could be different in a few days, months and definitely a few years, so why store it forever. Feels like I could save a huge operational cost and prevent abuse by not keeping tweets for beyond their relevancy.
What are your thoughts?
Content moderation (incl comments) doesn't scale so don't build something with public town squares. That's only a feature platform builders want in order to sell advertisements. If the thought of not having an ad-driven platform leads you to "users won't pay for it" then maybe think of a platform users would pay for or some other way to have it be sustainable.
>“You know — “ said Eliot, “Kilgore Trout once wrote a whole book about a country that was devoted to fighting odors. That was the national purpose. There wasn’t any disease, and there wasn’t any crime, and there wasn’t any war, so they went after odors.”
“This country,” said Eliot, “had tremendous research projects devoted to fighting odors. They were supported by individual contributions given to mothers who marched on Sundays from door to door. The ideal of the research was to find a specific chemical deodorant for every odor. But then the hero, who was also the country’s dictator, made a wonderful scientific breakthrough, even though he wasn’t a scientist, and they didn’t need the projects any more. He went right to the root of the problem.”
“Uh huh,” said the Senator. He couldn’t stand stories by Kilgore Trout, was embarassed by his son. “He found one chemical that would eliminate all odors?”
“No. As I say, the hero was dictator, and he simply eliminated noses.”
=========
Your solution is somewhat similar. You want to solve the problem of people saying racist things, so you decide to destroy the public square. Going for the root of the problem, I suppose.
"Public Square" online is a fallacy. A public square is typically thought of where someone grabs the attention of all passerby and is voicing their thoughts. Those thoughts voiced are short lived and can only be broadcast as far as the sound wave will carry them. Social media like twitter allows millions of people to spread their thoughts, commentary and keep them semi-permanently online for a long time, decade or more.
I never stated I was solving the problem of people saying racist things. I'm challenging the assumption that designing a new social media platform should repeat existing designs that have public content with public comments driven by advertisements. In my opinion if someone designs with that premise it will end up the same as the existing social media platforms we have today (with a side effect of new JS frameworks being borne).
Is this the only product design Tech has: public content and sell advertisements?
Maybe global public squares don't work for the humans we have. That's ok.
In fact, I'd even go so far as to say that moderation is absolutely required for any "public square". You can't have society without some agreed-upon rules.
The real question is where are the lines, and how do you police them (and who polices the police, etc etc). It's not like we haven't solved those. It's just that the solutions themselves are imperfect like the world we live in.
You might get racist boards, but then its easy to get rid of all of them at once.
150 being https://en.wikipedia.org/wiki/Dunbar%27s_number of course.
I have no way to distribute anything. I tried to do my own annotations board on literature but no one joined. I just think it sounds cool to be in a personable board like that.
There was something similar in the Weatherford's book on Genghis Khan [1][2]. This system was described to be very effective for communicating and coordinating the huge military.
> In Genghis Khan's military system, a tumen was recursively built from units of 10 (aravt), 100 (zuut) and 1,000 (mingghan), each with a leader reporting to the next higher level.
Note: I am not aware of how good the Weatherford book is, it felt one-sided to me. So I am not sure how good the civic system that depended on the Tumen was in the mongol era.
[1] https://en.wikipedia.org/wiki/Mingghan
[2] https://en.wikipedia.org/wiki/Tumen_(unit)#Genghis_Khan's_or...
You don't have to shut them down, you know. The British Government did this all throughout the 1970s to 1990s, where pubs (and later online services) where Republican terrorists hung out were very much left alone. They could have swooped in and scooped the lot up, but they didn't.
Because if they ever did want to scoop them all up, they knew exactly where to look, and why would you disturb that?
Anyway, from a technical point of view, this is what Mastodon instances already can offer.
Sorry for being a prick; but the royal 'we' is the exact opposite of the 'we' in all of humanity. Royal 'we' is when the speaker refers to him-or-herself with the first-person plural pronoun, as was common among sovereigns ("we the Emperor of the French"). These days most commonly seen in academic articles, where single authors refer to themselves as "we" for some reason.
They mean "we" as in "you (the reader) and I (the author)". "We can see that", etc. It's as if the author is a guide on a journey.
I am convinced that the presence of authorities on social media has to be regulated, and if I were to launch an alternative to Twitter I would consider [and look for ways to] build ethical safeguards [concerning public officials] into the system.
[1] https://data.iana.org/TLD/tlds-alpha-by-domain.txt
Making it even easier to register domain names. Rebuilding your friend network between the DNS names of friends, family and other associates. Sharing photos and videos and birthday reminders.
I feel like it's marketing all of the technologies of the early web that enabled these things, but with great UX and an order of magnitude less complexity for getting all of it setup.
A public square is "an open public area in a city or town where people gather". DNS is far too abstract to even remotely resemble a public square--just like "capitalism" or "zoning laws" are too abstract to resemble a public square.
Nor is foo.com a digital public square (just like your house isn't a physical public square). Public squares (digital and physical) are places where people already gather.
"But, but, what about filter bubbles / echo chambers???" one says. To which I say "it's not a problem I'm trying to solve, or particularly interested in. I'm not even sure that's actually a problem per-se to begin with." Seriously, the world is too big, there are too many people and too many ideas, for any one of us to accept being exposed to everything that's "out there".
Filters are a good thing, in fact a necessary thing. Of course we'd like to think that in a well educated, civilized, rational society, people would choose to make some effort to gain exposure to new and contradictory ideas for the purpose of expanding their minds and adapting and learning and what-not. But be that as it may, you still have to accept that some people will choose to filter out things to a lesser or greater degree than some others, and that some people will choose to filter things differently than you will. And that's OK.
- Does LinkedIn show us that people have less hostile posts if their real world professional connections are watching? If Twitter tried to show your posts to people it thinks you know, whether or not you follow each other, would we all be more civil?
- Does the higher bar of needing to generate an image or video to post (and a 2nd class presentation of comments) on some platforms, and lack of explicit 'reply' functionality stop arguments or bickering? The bar of typing a post in Twitter is just too low?
- Does TikTok's recommendation based on behavior rather than "interests" create less polarized bubbles? Showing you a funny dance video just because it knows you'll watch it may disrupt you from seeking out The Enemy just to disagree with them.
- Does removing an asymmetric "follow" relationship in favor of symmetrical "connections" disarm people whose hobby is having incendiary positions? If you can only have more "audience" by opening yourself up to see stuff from more people, do you then choose to connect with people that add value rather than valuing followers who will amplify you?
It’s a much harder problem to fix. There are some partial solutions, IMO. Although none are all that practical to implement.
The answer to this question seems almost implicit for you, but it's far from obvious to me.
Why would advertisers prefer public town squares? They just want people to see their ads.
(I also like "public town square" places, as a user)
I don't have a problem with the local public town square, but I do agree that giving equal voice to all the bad ideas, while also making it easier for those with bad ideas to find each other, is dangerous. It's made more dangerous in an era where our elected leaders also say bad ideas out loud, which emboldens others to do the same. Even in a public square, you can't just yell "fire" without consequence.
Deleted Comment
Twitter is part of that fragmentation, but it also accomplished something decent that competitors like Reddit don't: while there are individual communities, the follow interface made them somewhat porous. People can slide into conversations or get boosted by just about anyone. Making this work without local "subreddit-style moderators" was really technically remarkable, just from a spam perspective. Making moderation work to the point where a company could actually run advertising was extremely impressive.
I don't know what you're calling for exactly, but it sounds a whole lot more fragmented than what Twitter accomplished. I don't know that this will be a better outcome.
But no one blinks an eye. So HatefulRacistUncle might be “HorriblyMisguidedAndRacistYoungCollegeStudent” and that’s also not ok to some of us.
So who polices this? Maybe we just go back to not reading things we dislike. You can block people on Twitter ya know.
Why do we have to do that? Isn't the idea of having easy access for all enough? What people are so marginalized that they can't mute hateful speech and get on with their day?
I'm not sure if all of Twitter's problems come from mere scale; it suffers from what analyst Ben Thompson has called the Pollyannaish Assumption: we focus on the upsides of the "global public square" without accurately evaluating the downsides, and bluntly, I see this a lot in discussions on HN. It's the subtext of "why can't people just block and move on." Arguably, focusing on the downsides without accurately evaluating the upsides, which I also increasingly see a lot on HN, isn't really an improvement -- but if we're going to have platforms like Twitter and Facebook, the question of "what do you do about harassment, hate speech, and stochastic terrorism on those platforms" needs a better answer than "suck it up, buttercup."
to add to your point: 1/ tech stack/architecture isn't #1 or even #10 issue. think everyone (all) agree on this 2/ larger issue is how to handle open discourse. given how we are wired as species, is there a social network that is truly open that doesn't descend into vitriol? if so - what behaviors are rewarded, what are the policies etc. - this is #1 question
What's interesting is what value Twitter gives and what could a new platform target? For me, there's a slice of Twitter users that provide information, entertainment, and context that I can't get elsewhere. This could be something from a mid-level analyst re: market volatility or a college professor at a small Canadian university on Chinese politics. These people aren't consistent enough or polished enough to have established means of amplifying their opinions (a blog, newsletter, podcast, etc.) though they have so much domain expertise. Where do you find them? On Twitter. The low friction posting encourages them to share.
If I could get rid of the celebrities, the thought leaders, the self marketers, the low content hate, and the bots, and only be left with the above in a moderated environment I'd welcome this global town square.
Sadly many of those people come for the self-marketers and thought leaders though.
I've noticed this in Reddit too. A lot of posters in places like AskHistorians and AskScience have left, questions just don't get answered as much anymore and the quality of responses has gone way down. I expect those people came for the memes and interesting articles (and maybe /r/GoneWild) and participated in this value additive stuff while they happened to be around. But as Reddit got less interesting, more focused on memes and political fighting, and generally became less of a fun place to hang out they seem to spend less time on it and post less.
My real life experience is very different - I know a ton of gay Republicans, for example. Most people in a group are not the median, and all the Reddit approach does is alienate anyone that _was_ open to new ideas.
If you really want to build a twitter alternative think about how you would DISTRIBUTE it. How would you get a critical mass on the platform so that is viable, i.e. interesting enough for users to stick around.
Which existing social networks (not media ones, real ones) would you try to capture first? Facebook and Tinder both did the Uni campus strategy quite successfully. Slack did the Bay Area startups strategy. What’s your strategy?
The OP specifically asked about this, and reading the comments a lot of people here evidently do care.
The better question is how to reach people and get them off their insert favourite platform , what makes you unique? What can you do to make sure people start buying into it? What’s your value proposition?
It’s far more a social question than a technology question (still with respect to OP who is interested in the tech)
Please don't leave us hanging. How had you designed it?
I would like to know even more: Did you have integration with other networks via ActivityPub or other protocols?
There is a huge risk of wasting time if I cannot migrate my social network. New services are prone to being discontinued, why should I invest in a new network that locks me in?
I don't understand why not all new networks offer ActivityPub migration by default. Distribution should be much easier.
what was it called? now I'm curious.
Deleted Comment
Communicating via short untargeted messages was always going to be a shitshow, why not let Elon Musk run it if he so desperately wants to?
Database: I’d use whatever you know and helps you get to market, but abstract it away from the rest of your app so you can swap out later. Lots of good choices here; I’d pick mysql or postgres running on rds. I would also design for sharding by user on day 1 because that can be extremely painful to add later, however, care would have to be taken to ensure it doesn’t slow you down too much early. I’d also just use an rdbms for caching unless it becomes a problem, only then would I reach for redis. I would also avoid a fanout-on-write approach in favor of fanout-on-read. It will save you a lot of headaches later.
Firestore is nice but can get expensive quickly so I’d plan for it to be a temporary thing.
2. Would go for a react native app first but there are lots of options to choose from. Go with tech you know but keep in mind that it’s basically impossible to get a AAA mobile experience on web, and in this space polish really matters.
3. Decentralized social networks never took off because they are too hard to build. Avoid at all costs unless you have some angle (ie you have some novel tech discovered during your PhD). A middle ground could be a centralized network that is open source and makes data portability front and center.
I’ll also say that growth matters a lot here. As a founder I think 20% of the teams headspace should be thinking about engineering. The rest on growth and retention.
Probably a bigger benefit though, is being able to use a different DB in production (probably big scale) vs on your personal machine while developing.
So we've just switched underlying connection and fixed a dozen of (mostly reports) places with plain SQL queries with MySQL-specific syntax. ORM library handled rest just perfectly.
So I agree with you with this one important exception.
But I think the point is still valid. You probably don't want your early design to focus on an edge case.
That said, a clear API between your DB and app is worthwhile for a lot of reasons (in general).
React is a decent choice also here because you can port it wherever. One code base for web/iOS/Android is probably critical given how few dev resources you'd want to spend on it (you've got the right idea there too).
Also agree w/r/t decentralization. I think this is a fantasy for nerds but most people really do not care about privacy no matter how much people try to explain it to them.
I guess one thing I would add here is I'd work very hard to minimize operational costs for as long as I can, until you can show the explosive growth that'll interest investors. There are a ton of scrappy ways to save money on a tech stack like this early on, and you can probably find a way to get that AWS/GCP/Azure credit money to basically make this free to operate until your F&F round.
One you have a userbase, just... idk, iterate with them. Everyone and their mother probably has ideas for what they'd like Not-Twitter to be, but finding what the GCD is for that would be important to sustain growth I bet.
Every social web I imagine branches out from me (if I am the user in question) and then goes out to gather information from there. So on and so forth. Unless I'm thinking about the wrong information gathering system.
However this is not something that I've ever done IRL and I rather doubt that most of those who invoke this idea do this regularly either.
The kinds of discussions that you might imagine having in public with strangers who might have different values and backgrounds would be wildly different from the same discussions online. The metaphor is quite a broken one and at this point about as useful as pretending that the screen I'm looking at is an analog to my actual desktop (which mostly one exists to hold the screen).
While this may come across as mere pedantry I think it is actually quite important to ensure that when we discuss platforms that are going to be nothing at all like a town square that we are more realistic about what kinds of conversation and interactions we might want to encourage or discourage.
The point of the public square proponents is that it would be nice if one of public platforms would step up to perform the function in a way suitable for it.
On the internet, enforcing the preconditions needed for the town square to work well is difficult at best.
Besides that I'm unconvinced that this kind of platform for spreading new ideas is what most users on Twitter want from Twitter. The metaphor has become repeated so often that it almost sounds like it is Twitter's reason to be but I rather doubt that it was designed with that in mind.
Some thoughts:
- Start with thinking about how you're going to maintain engagement without pushing people into algorithmically-generated echo chambers, or into attack/defence behaviour that will just turn your New Twitter into the same cesspit as Old Twitter.
- Also think about how you're going to moderate content - because you're going to have to do it whether you like it or not, and its going to have to be at various scales from post-level to policy-level. There's a very good argument to be made that Legacy Twitter's product is content moderation [1], not software. Human nature being what it is, you may find that eventually it's your product too.
- Also, consider how you are going to deal with state-level and semi-state-level actors who will attempt to infiltrate your platform to use it as an amplifier.
The tech platform will help with the above, of course, but above all I think you need to consciously design the thing that you're building. That thing isn't a software platform or a social network or a community. It's something else. Figure that out first.
And if you want to build something that makes the world better (as opposed to worse, as in Old Twitter), you might consider how people can use your new platform to get stuff done in the world, rather than just shouting and meme-ing. I wish I knew how to do that though.
If you do build something like Twitter then please try to build it so that it doesn't cause harm.
[1] https://www.theverge.com/2022/10/28/23428132/elon-musk-twitt...
Not just state actors but anyone with an agenda and budget really. companies, ngos, political parties etc. Manipulating twitter to push an agenda or narrative is big business and not limited to nation states.
When a social media site runs on ads, it will inevitably begin to shift to being advertiser friendly. When it's a paid site, it will struggle to get users. Might be worth investigating a Patreon or Reddit type funding, where paid users get benefits, awards that can be handed out, message amplification and ad blocking, but where anyone can create an account.
It might be useful to also integrate the Brave browser format where the users get credit for viewing ads that can be traded to other people or spent in an online store for fabulous toys and prizes.
The reason I'm saying all of this is that what an alternative to Twitter needs in order to serve the people is a funding method that does not rely on stocks and shareholders, but that also has a method to manage invasive business practice while remaining profitable. The system should be technically owned by the people and therefore be a place that people would want to gather together without the worry of money having to change hands in order to use it or have a painless experience.
Aside from that, it should offer frictionless open APIs to developers (<- very useful for getting people invested in the platform), it should have a topic filter (for instance, I am not interested in Sports at all, so being able to filter out Sports related topics would vastly improve the experience for me), and in addition to that, there should be a free-for-all town-hall section that removes any filter other than your personal blocklist and does not allow ad/business accounts to post but could have non-targeted generic ad billboards.
That should help minimize echo chambers or at least provide an easy way to step out of your echo chambers without abandoning them.
People need a place to speak and think and breathe without ads and agendas being shoved on them, where ordinary people can speak freely to other people and be evaluated on the quality of their thoughts and posts and not on artificial amplification of weaponized memes and polarizing agendas crafted to serve the financial desires of the wealthy elite.
Also, you can't trust one person (even yourself) to hold the line against the riches and power of the world, you can't trust even an organization founded on the purest of intentions (like Mozilla) to be able to resist both the peaceful and hostile machinations of such people should you reach market saturation.
The company charter would need to be built on principals designed to prevent the consolidation of power and control over the platform into the hands of the few, with some system in place to enable the people to eject other people from the platform entirely. I'm not sure what that would be, maybe every 3 months the top 10-1000 accounts get put to a vote (from verified human account users) and if there are enough votes then those accounts get memorialized and the user has to take 3 months off from the platform and then create a new account.
Sorry for the brain dump, but these are things that might have either improved Twitter or prevented it from becoming a billionaire's plaything and are therefore things I would like to see in whatever rises from its Phoenix Ashes.
The "how" is the absolute last question, the "what" is the first.
Can you imagine the following conversation:
"Hey you should sign up to Facebook!"
"Oh, why?"
"It's got a great tech stack!".
Me nether.
To design an alternative Twitter you must first understand what made Twitter popular in the first place. And then at some point, in the far distant future, let the name of a tech into your head, or a line of code onto your screen.
Said in good faith. I know so many engineers who wasted so much time because they couldn't understand that outside of the constraints of a job their skills had no monetary value by default.
The real question Elon should have asked is what's beyond Twitter, i.e. what should he have built instead of trying to pivot Twitter towards, with all its baggage?
Twitter is Serious Internet Business now. Which is why most of the noise comes from a very few individuals and it is riddled with bots. The value of your opinion is based entirely on your number of followers.
Building an alternative is difficult. Google has tried and failed, multiple times. I think Elon would prefer for it to return to the water cooler days.
People put out RSS feeds. People subscribe to RSS feeds.
No server side agorithm and api just returns feeds sorted by users preference.
If I follow too many people to view chronological, then I can have client side algorithms to sort.
Aggregate queries across all the feeds on the server identify hashtag trends and create useful metrics (retweets, likes, etc).
RSS might replace Twitter for your particular use case. However, it doesn't replace Twitter in general case for the public because RSS is one-directional.
The phenomenon of the "Twittersphere" includes bi-directional activity like replies and retweets.
As an analogy, this Hacker News site has users taking part in reading and writing activities. A few users like to spread the word that they consume HN via RSS just fine (e.g. maybe get feeds from https://hnrss.github.io/).
But users (who are not just pure lurkers) can't use RSS to upvote/downvote comments or post their own replies. Therefore, RSS can't replace HN's website for general usage.
Likewise, RSS can be a way of consuming NYTimes newspaper, but RSS can't replace the NYTimes itself.
RSS is an undeniable convenience for readers but its limited scope does not provide viral mechanics and feedback loops for writers publishers.
RSS works at the abstraction level of "protocol for data download". Sites like Twitter and HN, etc work at abstraction level of "virtual marketplace of ideas" -- and that function is out of scope for RSS.
So whatever can replace Twitter will look like something closer to Twitter than RSS.
EDIT reply to: >That's a client side question, IMHO. If you want your client to show you replies,
I was talking about the RSS-user-themselves wanting the capability to _write_ the replies and not reading others' replies. RSS is not a read+WRITE protocol. It's a pull-based reading protocol.
Deleted Comment
Also you could add friends and view what they subscribe to and like.
And just a basic text search is useful to find material related to a topic of interest.
I think it’s important to remember that this isn’t meant to consume infinite time but just a way to see what you’ve noted as interesting. I think it’s like a newspaper where you spend 15-60 minutes reading. Not something that consumes every waking moment with infinite discovery and content.