Hi! Birdwatch ML Engineer here-- these context notes are from Birdwatch. They are written and rated by users, and the notes are only added as context to Tweets if they are rated highly enough by multiple raters who have tended to disagree in the past. The core algorithm is open source as well as all of the data, and there is lots of public documentation about it too:
I honestly felt nothing but contempt for the oligarchic structure of fact checking. Facebook hired organisations like the Washington Post to do their fact checking. I have not forgotten how many people there lies killed in Iraq. I don't forget how the journalists at the time were willing to compromise their ethics in exchange for White House access and exclusive stories from the front lines, simply because journalistic interest is not the public interest. We also saw more recently during the pandemic incompetent journalists without adequate medical training censoring BMJ articles from Facebook: https://www.bmj.com/content/376/bmj.o95.
You are making one of the only tools I can see that can possibly make "fact-checking" a reality in a nominally unbias way that can be broadly accepted by the public. A democratic fact-checking system for a largely democratic website. Even if birdwatch will get things wrong, I appreciate the effort to reform this institution of "fact-checking" with something righteous and moral. No need to touch on my spicy political views, but I think this is a transparent, scalable, fair tool that can truly be embraced by the entire society rather than simply leading to a partisan divide.
Your system understands I'm on social media because I want to hear what the average person has to say, and I would go to a news website if I wanted to get a news websites take on what the facts were. To me, if a news website wants people on twitter or Facebook to pay attention to their "fact check" editorial, they should do it the same way everybody else has to. By creating compelling writing that gets attention and that people agree with.
Do you think there is a risk to "fact checking" gives the public a false sense of knowledge, and reduces certain important skills?
It seems apparent to me that, in the process of a manual substation of truth, through whatever means of research an individual chooses to check the validity of a subject, they accidentally learn more during the task of that research verification both directly through the claim that was made, and as well in that field in a more general sense.
For example: if two users are arguing over a Bible verse, and one side wishes to verify the claim the other side made, the traditional method would mean anything from reading a Wikipedia entry and a few related sources, to a book on the subject, to some other means. During the search for the specific thing they were after, they would also learn about: the general historical context of that period, other related verses, events of a similar nature throughout history, cultural peculiarities, linguistical differences, etc.
On the other hand, if "fact checking" is simply a thing done on a persons behalf, not only is the accuracy of the check a concern, but the act will not encourage learning about that related context. The recipient will not have a reason to learn of those other things, and might never learn or maintain that valuable skill of research and verification.
>They are written and rated by users, and the notes are only added as context to Tweets if they are rated highly enough by multiple raters who have tended to disagree in the past.
Interesting, I remember someone on /r/slatestarcodex having the idea to rate/sort reddit comments by that metric, which I dub "sort by should-be-uncontroversial" (because normally-disagreeing people all think it's correct).
(Then again, it almost might incentivize politician-style vacuous/feel-good statements.)
Each US presidential administration I've been under has always cited cherry-picked, sound-bite US statistics -- and has the gall to take credit for them. It's like someone's constantly watching the metrics, and every positive blip becomes a candidate for being a talking point.
They take cruel advantage of how opaque the connections are between the macroeconomic, political, and social inputs, and the outputs as experienced by individuals.
Even worse, talking points like that prime people to expect the government to be constantly manipulating variables in the economy, which causes inefficiency and necessarily leads to more unelected officials running more of the country -- since constant manipulation is inaccessible to legislators.
Of course the administration uses cherry-picked statistics to get credit. And of course the opponents of the administration use cherry-picked statistics to criticize the administration.
Beyond obvious short-term political goals, the reason for this is simple: in a machine as large and complicated as the government, it's really really hard to accurately ascribe meaningful credit (or guilt) for any change, and especially to do so in a way that is understandable by voters.
Each administration makes changes (positive and negative) that won't be felt by voters for decades, if at all; and under each administration, voters endure the consequences (positive and negative) of previous administrations. But that doesn't get people to the polls, so why talk about it.
> it's really really hard to accurately ascribe meaningful credit (or guilt) for any change
It's a bitter pill to swallow that I not only have to give these people my money to do this work, but they're wasting time trying to ascribe credit or blame for it.
> and especially to do so in a way that is understandable by voters.
This is the old "voters are actually stupid" idea. I don't buy it. Voters seem to have no problem deciding for themselves who deserves what credit, what politicians want is to market their "success" and to pass off their "failure."
Again.. using our own funds to do it. I find the whole thing inappropriate. If we decide to credit you, you will be re-elected, if we decide to blame you, we will invoke the courts.. and I doubt they will have a "really really hard time ascribing guilt" to the appropriate party.
This isn't common just in politics, but in your average big corp, even tech. You need to justify your accomplishments today even if the real benefit won't be seen a few months or few years from now.
It's pretty wild how the @whitehouse twitter account is used. If you looked at only the ones about the price of gas you'd get the impression its been a great ride.
Are you implying that the whitehouse twitter only got stupid starting in 2021? Granted, that account was not so loud in 2016-2020 as perhaps now (maybe... I haven't counted tweets) because the president of that time was tweeting 10+ times per day (some really unmatched crazy shit too).
>It's like someone's constantly watching the metrics, and every positive blip becomes a candidate for being a talking point.
I would add "or negative", and I wouldn't qualify it with "it's like". As far as I can tell, this is the one-sentence description of how the public face of politics works, everywhere, always.
It's not 'cruel' for any administration to indicate basic realities without some arbitrary historical and legal context.
While we should be eternally vigilant and skeptical, the lack of very specific context in this case is nowhere near a blatant manipulation.
In fact, I would say the 'problem' is maybe the opposite - I am somewhat more skeptical that this is a 'Musk led personal intervention' to draw arbitrary cynicism towards a political entity he does not like - playing 'moral equivalence' games with people who say "The economy is doing good!" (without nuanced context) and "I won the election!" (without the obvious 'context' that the statement is literally false, or blatantly misleading).
That said, it's just skepticism, I really can't say one way or the other obviously.
There's clearly a grey threshold in what we can tolerate from government and political statements, and it's very hard to fathom where that line is - but this one is not near that line.
If any administration wants to claim "Lowest unemployment ever!" in a Tweet, well then that's fine. They can say that as long as it's true, a history lesson is not needed in this case.
In any case, if they are going to do this, they need a set of publicly stated criteria for it, and they need to apply the criteria objectively and consistently.
This is a different kind of statement when it's not hedged in any way as compared to the tweet as quoted in the article.
> “Seniors are getting the biggest increase in their Social Security checks in 10 years through President Biden’s leadership” (emphasis mine)
Of course it's easy to assume that they said "the biggest increase" to make it sound as good as possible and it's easy to assume that they said "in 10 years" because they didn't want to just lie. But then, why did they need to include "through President Biden's leadership"? Isn't that technically false if it happened "through 50-year-old legislation"?
That being said, why do you think that statement isn't intentionally misleading? Or if you do think it's intentionally misleading, why do you think the context shouldn't be added? I don't buy that it's arbitrary; it's clearly relevant to the statement that was made.
Certainly I'd agree that this policy is absolutely ripe for abuse (through arbitrary application) but so far I haven't seen a reason for people to think that it will be or has been abused, barring the assumptions they choose to make.
> the lack of very specific context in this case is nowhere near a blatant manipulation
I disagree. First of all the increase wasn't Biden's administration merit and that's one blatant lie.
Then if the social security benefits have increased only to keep up with inflation, to celebrate this as positive is clearly misinformation. Those receiving the benefits are as poor as they were before, while society as a whole becomes poorer. Nothing to celebrate here.
My question then is, would/will the same fact-checking apply to a different government that Musk does support? He’s been pushing for this equal fact checking and equal platform for both parties, but as far as I can tell we’re not seeing these banners anywhere on politics Twitter besides official government accounts.
Yeah, it's ironic that their claiming responsibility for "inflation-based increase" in social security is correct -- because of their loose monetary policy that caused massive inflation.
> They take cruel advantage of how opaque the connections are between the macroeconomic, political, and social inputs, and the outputs as experienced by individuals.
That's why the more transparency and readily-available information there is, the better : )
First, think about the people who work in your marketing department. Now think about them with the reach to influence the entire world. That's politics, and it's terrifying.
"In the fall of 1972, President Nixon announced that the rate of increase of inflation was decreasing. This was the first time a sitting president used the third derivative to advance his case for reelection." - Hugo Rossi
> President Nixon announced that the rate of increase of inflation was decreasing. This was the first time a sitting president used the third derivative to advance his case for reelection.
I can't think of a more fitting jerk.
(though arguably Nixon was actually talking about a fourth derivative (of price)...)
- price is 0th derivative
- inflation is 1st derivative (change in average price)
- rate of increase in inflation is 2nd derivative
- that rate of increase is decreasing — 3rd derivative
So I think Nixon is talking about a negative 3rd derivative of price. Am I missing one?
In my country we have a lowered VAT under certain circumstances. We had a new government tell us they were "reducing the VAT discount to half", just to avoid the word "raise", since they had promised that no taxes would be raised. I was reminded of that quote then -- in this case they were using the multiplication of two negatives rule.
When? He bragged in July when the MoM rate was zero and the YoY rate fell by half a percent. I can't find him bragging anywhere that the rate is holding steady though.
Inflation is the first derivative. The rate of increase of inflation is the 2nd derivative. The idea that the rate of increase of deflation is itself decreasing would then be the 3rd derivative (think "the rate of the rate of increase of inflation is negative")
I'm way more in favor a of a system that employs the userbase to "fact check" than whatever busybody connected to a partisan "fact checker" at twitter doing it manually and adding their own particular bias.
Why do you think the userbase would be superior? Twitter, like most social media platforms, prioritizes controversy. Pot-stirrers are the ones who get amplified. Why should we trust an incredibly polarized userbase to somehow lack bias?
The problem is that special knowledge is also prone to bias.
Some correct, but unpopular expert opinion can be also voted to be wrong, when the level of required knowledge is substantially high.
I mean the team being laid off doesn't mean birdwatch won't be used unless it literally just breaks down. Birdwatch is user sourced.
If Elon Musk gets rid of birdwatch I will scream bloody murder because that's probably the best approach I've seen to fact checking since the invention of wikipedia summaries on google search results.
I think there’s a massive difference between correcting something that is untrue, and just deciding you want to have your own say because the truth isn’t representative of what you feel is the broader issue.
Sure, we can say it’s due to an old law passed by republicans. But then the dems can argue that the modern GOP no longer support that… so really this just turns into a mess. Was the fact true? Yes. Was it representative of the broader argument? Depends who to you ask.
I think there are prices of fact check you could’ve put in there - for example nominal vs real terms, but this wasn’t that.
I think if the White House knows that in the days leading up to the mid terms anything they post on Twitter will get a right wing talking point attached to it they’ll quite rightly stop posting at all whether the “context” is true or not.
The context makes it clear that the increase wasn't due to good leadership, but instead was due to high inflation. I find that helpful even though I support the current administration.
With context, it sounds like they were trying to take credit for high inflation. That probably wasn't the intention, so I think it's clear that the original statement was misleading.
It didn't claim it was passed by Republicans, it explicitly stated it was signed by President Nixon and made no claims about which party controlled Congress nor did it mention Nixon's party affiliation. Not even a (R) after his name.
> I think there’s a massive difference between correcting something that is untrue, and just deciding you want to have your own say because the truth isn’t representative of what you feel is the broader issue.
Well stated. I think the job of the platform should be limited to calling out actual falsehoods, which this is not.
Additionally, a great platform could surface this kind of additional context effectively, but only as provided by the users of the platform. That's something I would love to see.
Malinformation is info that’s true but presented in a disingenuous way. Misinformation is false info that’s stemming from a person who truly believes what they’re saying. Disinformation is false info spread purposefully.
I thought during the last admin someone decided it was illegal for the President to delete tweets or block users because they become Presidential records as soon as they are created.
Hypothetical scenario: if the Ministry of Truth erases all the records of an inconvenient event in the past, would it be "misinfornation" to mention that fact?
I wonder how deep the economic analysis will go to decide that a tweet needs "context"?
For example, if an administration (cough) touts that this year's economic growth has been among the highest in a decade and that an increase in spending is "paid for", but that's just because a last year's index point was in the toilet because of a pandemic and we're getting back to where we were before...
Twitter could take the same approach that TV took, and has abandoned, long ago: each side gets equal air time to present their opinion. If a twit comes from a blue politician, the red team gets to respond, and vice versa.
Readit News
https://twitter.github.io/birdwatch/
https://twitter.com/birdwatch/status/1585794012052611076?s=2...
https://arxiv.org/pdf/2210.15723.pdf
I honestly felt nothing but contempt for the oligarchic structure of fact checking. Facebook hired organisations like the Washington Post to do their fact checking. I have not forgotten how many people there lies killed in Iraq. I don't forget how the journalists at the time were willing to compromise their ethics in exchange for White House access and exclusive stories from the front lines, simply because journalistic interest is not the public interest. We also saw more recently during the pandemic incompetent journalists without adequate medical training censoring BMJ articles from Facebook: https://www.bmj.com/content/376/bmj.o95.
You are making one of the only tools I can see that can possibly make "fact-checking" a reality in a nominally unbias way that can be broadly accepted by the public. A democratic fact-checking system for a largely democratic website. Even if birdwatch will get things wrong, I appreciate the effort to reform this institution of "fact-checking" with something righteous and moral. No need to touch on my spicy political views, but I think this is a transparent, scalable, fair tool that can truly be embraced by the entire society rather than simply leading to a partisan divide.
Your system understands I'm on social media because I want to hear what the average person has to say, and I would go to a news website if I wanted to get a news websites take on what the facts were. To me, if a news website wants people on twitter or Facebook to pay attention to their "fact check" editorial, they should do it the same way everybody else has to. By creating compelling writing that gets attention and that people agree with.
Do you think there is a risk to "fact checking" gives the public a false sense of knowledge, and reduces certain important skills?
It seems apparent to me that, in the process of a manual substation of truth, through whatever means of research an individual chooses to check the validity of a subject, they accidentally learn more during the task of that research verification both directly through the claim that was made, and as well in that field in a more general sense.
For example: if two users are arguing over a Bible verse, and one side wishes to verify the claim the other side made, the traditional method would mean anything from reading a Wikipedia entry and a few related sources, to a book on the subject, to some other means. During the search for the specific thing they were after, they would also learn about: the general historical context of that period, other related verses, events of a similar nature throughout history, cultural peculiarities, linguistical differences, etc.
On the other hand, if "fact checking" is simply a thing done on a persons behalf, not only is the accuracy of the check a concern, but the act will not encourage learning about that related context. The recipient will not have a reason to learn of those other things, and might never learn or maintain that valuable skill of research and verification.
Now if only if the feed would stop disappearing tweets in the middle of reading them, that would be great.
I guess this can't happen with the Nitter front-end. Just use LibRedirect.
https://libredirect.github.io/
Interesting, I remember someone on /r/slatestarcodex having the idea to rate/sort reddit comments by that metric, which I dub "sort by should-be-uncontroversial" (because normally-disagreeing people all think it's correct).
(Then again, it almost might incentivize politician-style vacuous/feel-good statements.)
They for sure would since there's a word for that: demagoguery.
Deleted Comment
They take cruel advantage of how opaque the connections are between the macroeconomic, political, and social inputs, and the outputs as experienced by individuals.
Even worse, talking points like that prime people to expect the government to be constantly manipulating variables in the economy, which causes inefficiency and necessarily leads to more unelected officials running more of the country -- since constant manipulation is inaccessible to legislators.
Beyond obvious short-term political goals, the reason for this is simple: in a machine as large and complicated as the government, it's really really hard to accurately ascribe meaningful credit (or guilt) for any change, and especially to do so in a way that is understandable by voters.
Each administration makes changes (positive and negative) that won't be felt by voters for decades, if at all; and under each administration, voters endure the consequences (positive and negative) of previous administrations. But that doesn't get people to the polls, so why talk about it.
It's a bitter pill to swallow that I not only have to give these people my money to do this work, but they're wasting time trying to ascribe credit or blame for it.
> and especially to do so in a way that is understandable by voters.
This is the old "voters are actually stupid" idea. I don't buy it. Voters seem to have no problem deciding for themselves who deserves what credit, what politicians want is to market their "success" and to pass off their "failure."
Again.. using our own funds to do it. I find the whole thing inappropriate. If we decide to credit you, you will be re-elected, if we decide to blame you, we will invoke the courts.. and I doubt they will have a "really really hard time ascribing guilt" to the appropriate party.
Dead Comment
May 2021 (with forecast!) https://twitter.com/WhiteHouse/status/1398685824934363138
Dec 2021 https://twitter.com/whitehouse/status/1470870982588088325
August 2022 https://twitter.com/whitehouse/status/1554497308024311809
https://en.wikipedia.org/wiki/Presidential_Records_Act
I would add "or negative", and I wouldn't qualify it with "it's like". As far as I can tell, this is the one-sentence description of how the public face of politics works, everywhere, always.
It's not 'cruel' for any administration to indicate basic realities without some arbitrary historical and legal context.
While we should be eternally vigilant and skeptical, the lack of very specific context in this case is nowhere near a blatant manipulation.
In fact, I would say the 'problem' is maybe the opposite - I am somewhat more skeptical that this is a 'Musk led personal intervention' to draw arbitrary cynicism towards a political entity he does not like - playing 'moral equivalence' games with people who say "The economy is doing good!" (without nuanced context) and "I won the election!" (without the obvious 'context' that the statement is literally false, or blatantly misleading).
That said, it's just skepticism, I really can't say one way or the other obviously.
There's clearly a grey threshold in what we can tolerate from government and political statements, and it's very hard to fathom where that line is - but this one is not near that line.
If any administration wants to claim "Lowest unemployment ever!" in a Tweet, well then that's fine. They can say that as long as it's true, a history lesson is not needed in this case.
In any case, if they are going to do this, they need a set of publicly stated criteria for it, and they need to apply the criteria objectively and consistently.
Yes, it is. A leader's first responsibility is to tell the truth -- and citing misleading statistics is worse than an outright lie.
It is cruel to lie to people in a way that causes them harm.
This is a different kind of statement when it's not hedged in any way as compared to the tweet as quoted in the article.
> “Seniors are getting the biggest increase in their Social Security checks in 10 years through President Biden’s leadership” (emphasis mine)
Of course it's easy to assume that they said "the biggest increase" to make it sound as good as possible and it's easy to assume that they said "in 10 years" because they didn't want to just lie. But then, why did they need to include "through President Biden's leadership"? Isn't that technically false if it happened "through 50-year-old legislation"?
That being said, why do you think that statement isn't intentionally misleading? Or if you do think it's intentionally misleading, why do you think the context shouldn't be added? I don't buy that it's arbitrary; it's clearly relevant to the statement that was made.
Certainly I'd agree that this policy is absolutely ripe for abuse (through arbitrary application) but so far I haven't seen a reason for people to think that it will be or has been abused, barring the assumptions they choose to make.
I disagree. First of all the increase wasn't Biden's administration merit and that's one blatant lie.
Then if the social security benefits have increased only to keep up with inflation, to celebrate this as positive is clearly misinformation. Those receiving the benefits are as poor as they were before, while society as a whole becomes poorer. Nothing to celebrate here.
That's why the more transparency and readily-available information there is, the better : )
That pretty universal, no?
It's like when Canada's job growth is 70% due to government hiring and the gov't comes out and says "See how well the economy is doing?".
Or companies "spin" some negative stat into a positive one.
"In the fall of 1972, President Nixon announced that the rate of increase of inflation was decreasing. This was the first time a sitting president used the third derivative to advance his case for reelection." - Hugo Rossi
I can't think of a more fitting jerk.
(though arguably Nixon was actually talking about a fourth derivative (of price)...)
- price is 0th derivative - inflation is 1st derivative (change in average price) - rate of increase in inflation is 2nd derivative - that rate of increase is decreasing — 3rd derivative
So I think Nixon is talking about a negative 3rd derivative of price. Am I missing one?
Easy brag: Inflation is low.
Hard brag: Inflation is high but not getting worse. <- Biden was here.
Desperate brag: Inflation is getting worse every month, but the amount it gets worse is dropping. <- Nixon was here.
inflation = d money / dt
rate of increase of inflation = d^2 money / dt^2
rate of change in rate of increase of inflation = d^3 money / dt^3
rate of increase of inflation was decreasing = sign(d^3 money / dt^3)
According to rumor, virtually the entire team responsible for birdwatch just got laid off.
I'm curious what the future of this kind of context will be.
> The team is on a mission that has had the support of 3 CEOs. We're cranking.
> to all team members, contributors and advisors past and present — an amazing, mission-driven set of people.
https://twitter.com/kcoleman/status/1588611638655320064
Also, Elon has endorsed it: https://twitter.com/elonmusk/status/1587798343622737925
Hoping it stays, I love the bottom up nature of it.
[1] https://blog.twitter.com/en_us/topics/product/2021/introduci...
If Elon Musk gets rid of birdwatch I will scream bloody murder because that's probably the best approach I've seen to fact checking since the invention of wikipedia summaries on google search results.
Dead Comment
Sure, we can say it’s due to an old law passed by republicans. But then the dems can argue that the modern GOP no longer support that… so really this just turns into a mess. Was the fact true? Yes. Was it representative of the broader argument? Depends who to you ask.
I think there are prices of fact check you could’ve put in there - for example nominal vs real terms, but this wasn’t that.
With context, it sounds like they were trying to take credit for high inflation. That probably wasn't the intention, so I think it's clear that the original statement was misleading.
Well stated. I think the job of the platform should be limited to calling out actual falsehoods, which this is not.
Additionally, a great platform could surface this kind of additional context effectively, but only as provided by the users of the platform. That's something I would love to see.
I guess Twitter is a great platform then - FTA:
According to a description under the annotation, “Context is written by people who use Twitter, and appears when rated helpful by others.”
They can mute people but not block them.
https://www.cisa.gov/mdm
https://theintercept.com/2022/10/31/social-media-disinformat... (paragraph 2)
For example, if an administration (cough) touts that this year's economic growth has been among the highest in a decade and that an increase in spending is "paid for", but that's just because a last year's index point was in the toilet because of a pandemic and we're getting back to where we were before...
Who decides whether that's context-worthy?
Here's someone with possible details: https://news.ycombinator.com/item?id=33474196