Maybe different teams are different, but on my previous team within Google AI, we thought the goal of google's pubapproval process was to ensure that internal company IP (eg. details about datasets, details about google compute infra) does not leak to the public, and maybe to shield Google from liability. Nothing more.
In all of my time at Google AI, I never heard of pubapproval being used for peer review or to critique the scientific rigor of the work. It was never used as a journal, it was an afterthought that folks on my team would usually clear only hours before important deadlines. We like to leave peer review to the conferences/journals' existing process to weed out bad papers; why duplicate that work internally?
I'm disappointed that Jeff has chosen to imply that pubapproval is used to enforce rigour. That is a new use case and not how it has been traditionally used. Pubapproval hasn't been used to silence uncomfortable minority viewpoints until now. If this has changed, it's a very, very new change.
And the examples of issues flagged in review that Jeff keeps highlighting—like Timnit’s alleged failure to mention recent work to reduce the environmental impact of large models—are themselves a bit worrisome. Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks.
Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. They couch these in terms of rigour, but the substance of the problem has to do with her criticisms against AI.
Ultimately it makes the whole Ethical AI department look more like a rubber stamp for Google.
> Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks.
I get the impression that she wrote a hit piece on Google and published with Google's name. For me, it's correct they demand a retraction. It's simply unprofessional to critque your company for something while not mentioning the work they're doing to combat that.
Having said that, if Jeff were to make public the paper, criticisms of the paper, and improvements made to address the problems described in the paper, that could go a long way towards clearing the air.
The communication doesn't give that impression; instead it says that the paper makes claims that ignore significant and credible challenges to those claims. Dean said that these factors would need to be addressed, not agreed with.
Publishing a transparently one-sided paper in Google's name would be a problem, not because of the side it picks, but because it suggests the researchers are too ideologically motivated to see the see the problem clearly.
Ironically, it indicates systemic bias on the part of the researchers who are explicitly trying to eliminate systemic bias. That's just a bit too relevant to ignore.
"Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks."
Representing a more truthful reality is not 'softening'.
It's only 'softening' for those who have an already accepted, extremist view, and for whom any evidence to the contrary doesn't help their arguments.
While initially sympathetic to the author - the more I read - the more I have completely the opposite view.
Google isnt publicly funded academic institution. Whatever they are doing, in particular publishing, is part of the business/PR. So if the management sees something not good for business it is a reasonable that they decided to not do it. If i were a shareholder i can see how i may have questioned why a person being paid $1M+/year (my understanding this is minimum what a manager in AI at Google would be making) for publicly disparaging Google.
Even more, it sounds like Google didn't ask originally for retraction, they just asked to take into account the newer research contradicting the paper - the thing that any researcher valuing integrity over agenda wouldn't refuse.
If somebody wants to do that research and publishing they just have to find another source of funding, i guess.
Anyway, the firing wasn't over the paper, the firing was over the unacceptably unprofessional reaction to it.
At least when I was there, my papers were getting thoroughly reviewed and often had to make some adjustments before getting approval. Never occurred to me to make any demands from the reviewers or threaten to resign if my paper doesn't get immediately and unconditionally approved. Seems like she's asking for preferential treatment.
Do you want to know whats interesting? I read alot of computer science research, particularly what comes out of Google. Its clear to me that details are left out of specific papers, especially how things are done in sub systems. But, like a jig saw puzzle, I discovered that many papers are actually descriptions of computing systems and algorithms that interact. If you read between the lines and squint your eyes, you can get a much bigger picture of internal google AI systems than you guys think you can.
This response really seems like gaslighting. He doesn't address her concerns and glosses over whether she was held to a different standard than others at GR.
He was also extremely vague, perhaps intentionally, about what the issue actually was. His sentence about when the paper was submitted and approved and all that is impossible to parse and make sense of who did what and when.
Right -- he once again talks about "accepting" her resignation, when it reeeeeally just looks like they fired her. At the very least, she certainly feels like she was fired; why is that not mentioned at all? Even just, I don't know, "sorry we were abrupt?"
Every paper we submitted went through a technical review as well as legal and IP reviews. They were along the lines of cite this, cite that, run these experiments etc.
What's different in her case is that you don't see the names of the people reviewing. Being the devil's advocate, she MIGHT have a pattern of aggressively attacking people who reviewed their work before. So they might have made the reviewers anonymous this time.
They might have enough of a PR budget to make the Google version of the story stick. But, its concerning that, if what you say is true, they are hoping to make that work by leveraging the public's ignorance of how the Google specific process works. Its also not the smartest move, since Google is important enough and public goodwill towards tech is low enough that journalists will have a field day looking for evidence of double standards/a cover up. And they're not making it too difficult for the journalists if that evidence is found in the top comment on a hacker news post.
This really seems like whistling past the graveyard on google's part here. There's too much meat to the story for them to really do much more than obfuscate. The intersection of race and gender, ethical implications of big tech, the capitalistic pursuit of innovation at the expense of individual freedom. All of these look bad for google
We know large language models are super important to google, and there are lots of competitors.
If they approved the paper the message would be "google thinks language models are a waste of resources and racist". There would be no academic debate on this topic as its been framed as woke and published by a militant activist, so any disagreement would be racist (see prior interactions between this researcher and other researchers [1]).
Thats why the standard process of publishing, peer reviewing, academic critique etc would not work.
Why would their researchers working on language models stay? when they can go to Facebook, OpenAI etc. Why would new researchers join?
Academic debate is, in fact, done through conferences and journals. You saying there can be no debate is a strawman position with no basis in reality. The idea that standard rigor cannot be applied to ethics research is absurd, and seems to insinuate that the entire field is absent discipline.
The proper response to her position would be to publish a response or critique. Attacking her entire field does nothing to further the conversation.
I have to assume both sides here are adults that can deal with criticism of their chosen discipline without immediately resigning, or not joining a specific company over it.
But in this scenario, shielding Google from liability is actually a primary concern given that Timnit is discussing ethics/bias. A paper on say novel transformer architecture, the lottery ticket hypothesis in a new setting, a new RL benchmark suite, etc is not going to expose Google to legal risk the way ethical AI research often can.
This. I have been arguing the unpopular opinion that most AI ethics work in corporate settings is not designed to empower real research. It was a matter of time before an actual researcher with an ethical compass was removed unceremoniously. Anyone in an AI ethics team at a large company — you need to know exactly what your job means to the company, because it isn’t safe.
Then what's the point of hiring her and people like her to work for Google in the first place? So that Google could claim that they have Ethical AI researchers and Google's AI research is indeed ethical?
How would publishing a paper open a 3rd party up to legal risk? Research papers aren't laws, and it is chronologically impossible for a research paper to influence already on the book laws.
That confused me as well -- where I work we have a legal dept. approval for IP issues, and that's it. Academic review doesn't make sense in that context or time frame.
"Pubapproval hasn't been used to silence uncomfortable minority viewpoints until now."
This is sad gaslighting of a reasonable concern the team had.
Having to endure some external review for what could otherwise be sensitive material.
The inability for the SJW crowd to work reasonably within very reasonable terms, to then resort to aggressive tactics such as 'demand the names and opinions of everyone on the board' and then publicly misrepresenting the situation is going to lose you a lot of favour.
Every time I read one of these stories I immediately feel sympathetic to the individual, but then upon learning more, I feel duped and maligned for having been effectively lied to.
The doors are wide open for progress, those who take it to micro-totalitarian lengths are not doing anyone any favours.
I don't think the approval process is being used to enforce rigor in general, the (claimed) problem is the paper lacks rigor specifically in regards to claims about google's behavior.
Publishing a paper with a lack of rigor about some obscure mathematical technique isn't a problem for google (beyond some possible but unlikely mild reputation damage). Publishing a paper with a lack of rigor that says google is doing unethical things, when those things are questionably accurate, that is something google would (and should) have a problem with.
Whether the paper actually lacks rigor in a relevant way is not something I can comment on.
Why wouldn't you want to weed out bad papers as close to the source as possible to save company embarrassment and external people's time? If you see something wrong during a review why not push it back to the author before it does rounds outside the company? That would seem like a very bad practice to me.
Yes it is disingenuous for Dean to pretend that this was a normal process applied to a normal situation. Clearly whatever happened on that team, this latest round was not the beginning or even most important part. Gebru's letter mentions her threatening to sue Google previously, for instance. [1] The discussion about rigour in a conference paper or internal review is obviously a pretense.
When I was in academia, it was not unusual for the referees to reject a paper for this reason. Of course, you are informed of that and always have the option to rewrite the paper to include that information.
In some cases, though, it's not simply a matter of listing it as other work in the Intro - you may need to incorporate it into your models, etc.
beyond reviewers or editors asking you to make changes like this (for example, "so-and-so just published <blah> which means that your sentence about <blah> is obsolete"), we're talking about research coming from a corporation. If one part of the company is trashing another part of the company, and it's based on stale results, asking to have the paper updated to include the latest results is reasonable.
If you are actually an academic researchers in an academia institution, and are exposed to a large scope of the dealings of the community, then you might find that professional academia is as political as corporations, if not more so...
This reads to me like Google felt that the paper painted some of their other technologies in a poor light, and wanted to insert language that made them look better. The way he describes their objections, they strike me as the sort of thing that is routinely addressed in the camera-ready version of papers by adding a few lines to the related work section. Not the sort of thing that a conference reviewer or an internal reviewer would reject a paper over.
Previously, we only had one side of this story. But if this is Dean's best spin on Google's side of the story, I'm very tempted to conclude they're in the wrong here. Obviously I don't have all the information, but the information I do have feels consistent with the idea that someone important at Google didn't like Gebru's paper for corporate-political (meaning making Google look good, as opposed to political-political) reasons, they tried to get Gebru to play ball, she refused, and now they have to back-project a justification in the name of "scientific integrity".
Unfortunately, I think this is a story where most people's opinions about who's in the right will be more informed by their previous opinions about Gebru and Dean than the narrower question of what happened with this particular paper. I'm probably even guilty of that to some extent myself, given that I'm a fan of some of Gebru's previous work.
> they strike me as the sort of thing that is routinely addressed in the camera-ready version of papers by adding a few lines to the related work section
What I don't understand is why in the discussion nobody proposed amending the paper rather than withdrawing it. If Dean's issue was it didn't cite papers X,Y and Z, rather than demand a withdrawal, why didn't he just demand "I want you to amend the paper to add cites to X,Y,Z". And then, if Gebru and her coauthors were willing to add those cites, that would resolve it.
Indeed, from what I understand, "you should add a cite to X" is common peer review feedback, and a lot of papers get citations added due to requests from peer reviewers. So this isn't hugely different from that scenario.
It would seem that withdrawal over this issue would only make sense if Gebru and her coauthors refused to amend the paper to add the requested citations, but I haven't heard anything saying that she did refuse to do so. It isn't clear if the alternative solution of amending rather than withdrawing was ever brought up in the discussion by either party.
Not that I'm a researcher or anything, but if I was, and a superior told me "we need you to withdraw your paper because it doesn't cite X,Y,Z", my immediate response would be "How about I add the citations you are requesting and resubmit it with those additions?"
Jeff's document says she submitted without asking for approval, so the request was to withdraw that unapproved submission. It is reasonable to interpret that as permission to revise and resubmit. She seems to have had her heart set on this particular conference and submission deadline.
> What I don't understand is why in the discussion nobody proposed amending the paper rather than withdrawing it. If Dean's issue was it didn't cite papers X,Y and Z, rather than demand a withdrawal, why didn't he just demand "I want you to amend the paper to add cites to X,Y,Z". And then, if Gebru and her coauthors were willing to add those cites, that would resolve it.
Unless corporate tells you to kill the paper and you need something that resembles legal cover.
In Timnit's account, she said she asked for specific feedback so it could be addressed and the paper published; this was denied to her and is part of her complaint over which she resigned.
I've seen these review comments before and I don't know if we can say from our position whether it's corrections that can be made for camera ready or not.
If a paper does not analyze the improvements from recent work, and just older work that has been surpassed (deemed inefficient by Dean), the new reanalysis might not be as favorable to the results as the paper proposes which means the paper is moot.
Or the second point is about bias in language models, but it sounds like these issues are mitigated in recent research, which means people are already aware and have solved a bulk of the issue being described in the paper.
But certainly it's possible that the paper's contributions stands strong even after accounting for the recent work that Dean mentions. In that case, it could be corrected for camera ready. But my point is that we can't tell right now without seeing the paper, and the relevant research that was omitted.
But this review process had never been used for introducing comments of this sort to a paper ever before. Plus, there was no suggestion of change, just retraction.
Actually, the opposite. Speaking personally, I have been a fan boy of Dean's work on a number of fronts for a decade. I had awareness of her work but more superficially, and saw it as largely pissing into the wind. Reading hers more closely now, her papers and her communication, she has my complete respect. I better understand the way in which she was scaling this incredibly difficult mountain. Deeply, deeply impressive.
And Jeff has lost mine. Even if his comments are not just bad faith, his blindness borders on incompetence for the role he has. Yet another clever, blind white man.
I hope a position in the Biden administration can be found for her, and her vision can have a larger societal impact.
I don't know Jeff Dean. I have read some of his work, watched some of his presentations. He seems a credible bloke.
This, though, looks and feels like thinly-veiled retroactive and pretty unconvincing PR. It's short on detail and appears somewhat at odds with several points from Timnit Gebru's resignation note [0]:
- Dean says the paper was reviewed by a "cross-functional team". Gebru says she received the feedback through a "privileged and confidential document to HR"
- Dean says the paper was submitted for review on the day it was due to be published; Gebru says they had notified "PR & Policy 2 months before".
- Dean suggests the feedback was due to the paper not highlighting mitigating work for some of the limitations the paper was exposing. That seems like a very normal part of the research process. Why, then, does Gebru claim that she was told that a "manager can read you a privileged and confidential document" and that no other recourse or exploration of the feedback was permissible?
The only thing we know from the outside is that reality will be far more nuanced and complicated than the tidbits that leak out. Even allowing for that though - and reading some of the related comments here - Google isn't coming out of this well at the moment.
Dean says the paper was submitted for review on the day it was due to be published; Gebru says they had notified "PR & Policy 2 months before".
Wait, really? That’s an important detail if true. The one-day timeline was a central part of the narrative surrounding this story. Notifying them two months ahead of time makes this a completely different situation.
I’m a bit skeptical. Could this claim be verified somehow? Since it’s very public news at this point, we may as well try to be rigorous.
From what I read it sounds like Timnit cleared the general idea of the paper 2 months before. The paper itself seems to have been submitted for approval 1 day prior to the deadline. Another HN commenter says that submitting papers for approval "hours" before the deadline is common at Google.
> Gebru says she received the feedback through a "privileged and confidential document to HR"
I agree that Google isn't looking good at the moment. But if I had a colleague who seemed intent on finding ways to place the company in legal jeopardy, then I too would avoid direct communication whenever possible.
>Dean says the paper was reviewed by a "cross-functional team". Gebru says she received the feedback through a "privileged and confidential document to HR"
I don't think these two takes are at odds with each other at all. It sounds like a cross-functional team reviewed the paper and produced that "privileged and confidential document" but Gebru didn't find that document sufficiently detailed.
There are a lot of people commenting that she didn't actually resign. I agree, but it sounds like the conversation went like this:
employee: I'm not happy about x, y and z. If you don't do those, I'm going to quit.
manager: well we are not going to do those, so thank you for your time. We accept your resignation and would like it to start immediately (i.e. you're fired).
If you are gonna tell your manager that you plan to resign if a condition isn't met, then what do you expect them to say if they don't plan to fulfill that condition? It sounds like she was expecting them to say "Hey, well we don't want to meet your demands, but sure, we're happy to have a disgruntled employee around here, so feel free to stick around, or you could just quit on your own timeline, no sweat".
I suspect that many people would be fired on the spot for threatening to resign, so don't threaten it if you aren't okay with that consequence.
There are three different discussions happening all at the same time, which is muddying things.
1. Was the treatment of her research in internal review reasonable?
2. Was terminating her employment reasonable after she sent the (now public) email to the women-and-allies brain listserv?
3. Was the end of her employment at Google a resignation or a firing?
To me, (3) is by a wide margin the least interesting part of the story and all of the discussion here is missing the point entirely. Whether she was fired or resigned has zero bearing on whether (1) and (2) show reasonable or unreasonable actions.
For what it's worth, I have read the abstract of the paper and discussions around it and I have seen more than one person rate it as not very interesting. Why is all this fuss about a paper restating common knowledge? For example, they say datasets need to be filtered for bias, and that large models consume ... duh ... a lot. We already know that, where's the new shiny architecture for fair modelling?
As a manager I would only do this if I really wanted to fire the person already. For employees I care about I would give them an out.
Part of being a good manager is understanding your employees and helping them succeed. If someone makes a statement like this in the heat of frustration it doesn't necessarily mean they will actually quit. If they're a valuable member of the team you should present them with an opportunity to save face and remain.
To me this seems like taking an opportunity to fire someone they already wanted to get rid of. Either that or a bad manager who wanted to flex their power as a threat to the rest of the team.
> As a manager I would only do this if I really wanted to fire the person already. For employees I care about I would give them an out.
One of the more difficult lessons I learned as a manager: Once a team member starts giving ultimatums in order to get their way or override decisions, it's not in your best interests to give in for the sake of keeping that employee. The obvious exception is if you realize you were actually wrong from the start, but reversing decisions for the sake of caving to someone's demands is a problem.
If someone is so ready to quit that they'll flaunt it to the company, it's doubtful that reversing a single decision is going to suddenly make them happy again with their employer. Worse yet, it sends a message that threatening to quit is the way to get what you want from the company. Once you validate the strategy, you will get a lot more of it.
Unfortunately, if someone already has one foot out the door and has been complaining openly (even on Twitter, in this case) about their employer, it's best for everyone to go their separate ways. From there, focus on identifying and fixing any underlying problems to minimize the chance of this happening again in the future.
Given the employee's general attitude, I find it easy to believe her manager was already not happy with her.
See her previous (potential) legal troubles with Google that she even acknowledges herself.
> I was in the middle of a potential lawsuit for which Kat Herller and I hired feminist lawyers who threatened to sue Google (which is when they backed off--before that Google lawyers were prepared to throw us under the bus and our leaders were following as instructed) and the next day I get some random “impact award.”
Yes. I probably didn't use the word "resign" as such but there was a time in my career when I went to my manager's manager or maybe another level up the chain and basically said I could not work for my direct manager. I got my way even though it involved working in a bit of a backwater for a time. But I was prepared to leave if I didn't get my way. (Didn't hurt that I knew said manager was a bit on the outs with the exec who I knew quite well.)
It's more nicely worded to be sure, and definitely less aggressive, but it still says the same thing, which is "I'm gonna be an unhappy employee if I can't find out who reviewed my work". If they aren't willing to tell her who reviewed her work (which they may or may not have valid reasons for doing, but clearly they don't want to do), then they are dealing with someone who is going to be an unhappy employee since their conditions won't be met. Sure, not a resignation, but if your employer doesn't think you'll be a happy employee, why keep you around?
In any case, she has been very vocal on twitter and has not seemed to deny that she gave some sort of ultimatum. If she didn't give an ultimatum, it would only make Google look worse, so why not mention that on Twitter (given that she has tweeted probably 100 things about this incident in the last two days)? Given the absence of a denial, I'm going to assume that it was worded as an ultimatum.
> manager: well we are not going to do those, so thank you for your time. We accept your resignation and would like it to start immediately (i.e. you're fired).
The thing is, there's a big difference between resigning and being fired for cause, even if both end with you not working at the company anymore.
IANAL, but my best guess is that she was just let go without cause. Your employer can fire you at any time and technically doesn't need a reason. Typically, "terminated with cause" is a specific thing where they fire you and give a specific reason (e.g. stealing) that might have bearing on whether you receive unemployment benefits, accrued vacation, etc. It's hard to imagine that they fired her in that way and that she was just plain-old-fired (there's a reason for it, but not legal cause).
This debate over whether she was fired or resigned is distracting from the real issue. Why isn’t Google willing to listen and work with Timnit? What are the issues she’s raising? These are more important questions, fired/resigned is just an easy thing to be outraged or give leeway over.
> Why isn’t Google willing to listen and work with Timnit?
Maybe she isn't that important for the company? Why bother working with someone making demands if they aren't worth your time. Google might have actually thought she was detrimental to the company and wanted an excuse to fire her and she gave it.
I suspect that they were looking for a way to separate themselves from her. I was very upset when I read her attack on Yann LeCun. At least in my eyes, she was hurting Google brand. I read her tweets and she seems to see racism in every turn. In my opinion, that was a behavior of a political activist, not an AI researcher.
After reading Jeff Dean's response, I can only come to the conclusion that Timnit Gebru acted like a primadonna. It is completely normal to have to obtain prepublication signoff on material before it is submitted to a conference (and manager signoff even before abstract submission). Given the breadth of experience at Google, it seems strange not to avail yourself of this. Demanding to know the identity of reviewers is absurd (no journal would tolerate that) and deeply unprofessional. Making it an explicit ultimatum was her decision. Denigrating the entire area of research at Google on a large mailing list is the action of someone who wants to be terminated.
People claiming that the deficiencies in the paper are minor and wouldn't be blockers obviously have little experience submitting to academic journals. Other parts of Google doing deeply technical work probably don't have the same level of review as the Ethical AI group -- for obvious reasons.
There is usually a long back and forth -- there are even memes about the infuriating comments from "Reviewer 2" [1][2]. Omitting to mention argument-obsoleting developments in the field (from your own lab!) is more than enough to send you back to extensive redrafting.
To be clear on terminology -- a retraction is an academic black mark, and occurs to a paper after publication, usually for reasons of research misconduct. This is not an instance of that.
I would agree with this assessment. It seems people think this is like academia -- it's not. There are no guarantees of tenure at a private company, even one with a research culture. If she wants that, then give up the paycheck and go back to academia. Google doesn't owe her anything. (I bet she'll get a decent settlement.)
AI ethics are important and I'm willing to bet that the Google execs want to know about biases and other problems in their products and probably don't care that much what is published. They probably don't want high maintenance researchers that cause them headaches and are obsessed with microaggressions, etc. I expect that this stuff has been a distraction for too long and they are happy to be rid of this and happy for the signal it sends across the company.
I have little sympathy or concern for Ivy educated elite in-fighting. There is probably a huge number of qualified people just dying to have the opportunity to work there, have access to those resources and do research.
I think you should be more open about what potentially actually happened because your first sentence seems very accusatory without proper information, the very bias that Gebru speaks of. According to her email that Dean references in his, her "feedback" for the paper was given in the form of a confidential HR document. That seems highly nonstandard for a review process supposedly only concerned with scientific rigor and is not addressed at all by Dean's statement. Further, his statement clearly, likely intentionally, muddies the water about what the actual sequence of events were that led to the paper being created, internal reviewers originally being notified, when the submission happened, who approved it, who submitted it externally, who asked for this to be retracted, and when and in what order all this happened. The fact that his statement, that was apparently meant to clarify these exact proceedings, makes it even more vague about what actually happened seems pretty damning to Dean's case, in my opinion.
Both Gebru's and Dean's statements have very little overlap in flavor and in facts, so it's pretty apparent something is going on here that is nontrivial and abnormal.
To be sure, Google has other objectives. But the people in this area are essentially academics, and I'd expect them to have a similar process given their backgrounds. The papers from Google are routinely of high calibre.
The non-research motive is arguably to control the narrative about using ML (I admit I still choke at calling it AI) and big data techniques. I fail to see how this is advanced by having a very public spat.
I've no doubt some senior exec decided she had to go, but I don't think it is because of any intersectional reason, or to cover up any particular publication, but that she wasn't seen as an asset to the company. Strategies for negotiating exercise of power are very different at Google to social sciences academia (or twitter, which is engorged with righteousness over this), which she seems not to have grasped. There are few enfants terribles in corporate tech that don't have controlling stock.
That's a very snide comment. I've read her tweets and other Brain tweets (eg, from mmitchell_ai, though she deleted them, which seems wise as they were angry). I don't take any of them at their surface as being truthful. I think we're only seeing the 10% of the iceberg. But I don't find Gebru convincing.
> It is completely normal to have to obtain prepublication signoff on material before it is submitted to a conference (and manager signoff even before abstract submission)
To be clear, both of these had been received for the paper in question.
That isn't the impression I had from reading the principal's statements, to wit:
> Unfortunately, this particular paper was only shared with a day’s notice before its deadline — we require two weeks for this sort of review — and then instead of awaiting reviewer feedback, it was approved for submission and submitted.
It seems like someone short-circuited the review process and submitted without review to meet a deadline, with a post-submission review. When this occurred, it was not all green lights. The expectation is that you then pull the submission and, post review, submit elsewhere. After all, conferences are all virtual now, it is not a hardship to submit elsewhere.
If you have other information as to the facts or the internal process do tell.
Timnit responded with an email requiring that a number of conditions be met in order for her to continue working at Google, including revealing the identities of every person who Megan and I had spoken to and consulted as part of the review of the paper and the exact feedback. Timnit wrote that if we didn’t meet these demands, she would leave Google and work on an end date. We accept and respect her decision to resign from Google.
According to her Twitter feed, Timnit was asking for the same transparency that normally goes with Google pub reviews -- where all the reviewers and their feedback are known to the reviewee. This red-stamp "review" did not follow the normal review process. Not a Googler, just going on what she says:
Considering her previous attacks to anyone who disagrees with her and her previous public feuds, I'd say people may be afraid of being stamped as a racist/sexist because they had to reject her paper. She was a disruptive employee who threatened her employer and the employer didn't accept it. Why is Google obliged to keep her employed?
Yeah the way I see it, once someone makes an ultimatum, the relationship is destroyed, there's no trust. You can ignore the quality of her paper or the critiques from Google, they probably just decided to end the relationship because of this.
Not really. People bluff in negotiations all the time. A normal response would have been, “I can’t do that. Are you still committed to leaving?” Instead Google went all scorched earth and it’s still unclear why. It also indicates that they don’t particularly value her in progress work in the lab since normally you want the 2 weeks or whatever to squeeze some wrap up work out of folks.
People bluff but you generally have to be prepared to walk if your bluff is called. If I walk into a manager's office and demand 2x salary or I walk and the response is "we believe you are fairly compensated," you're now in a rather uncomfortable situation.
Which is of course not to say you can't have a more measured negotiation. But it can be hard to walk back from give me X or I do Y, especially if there isn't a lot of middle ground between giving you X or not giving you X.
"Do X or I'll quit" is a childish way to negotiate, its akin to a child throwing a tantrum if they don't get their way. It's extremely unprofessional and not something a team player would do. I think its clear why they took her up on her offer.
If an employee is adversary toward the company, the trade off needs to be made no matter how much valuable the employee's work is.
This is a non-issue tbh. She wants to leave. Company wants her to leave. They both agree to part way because the premises are fulfilled (i.e. company can't meet her requirements).
If you want to get technical, I'd bet her employee's status is still in tact for 2 weeks; she just doesn't have access to laptop and etc.
> People bluff in negotiations all the time. A normal response would have been, “I can’t do that. Are you still committed to leaving?” Instead Google went all scorched earth and it’s still unclear why. It also indicates that they don’t particularly value her in progress work in the lab since normally you want the 2 weeks or whatever to squeeze some wrap up work out of folks.
I strongly encourage people to read some books on negotiations - as well as read up on legal ramifications to some negotiations.
Pretty much all books/courses on negotiations say: Ultimatums have their place, but are a minefield (i.e. they can blow up on you), and should be used as a last resort. From a negotiations standpoint, the response was adequate - which is why they all caution against using such an approach.
As for the 2 week thing, this is a convention, but not a requirement. In my company, it's not unusual for someone to be shown the door the same day they announce they plan to leave to another company (it's not the norm, but not at all unusual). The manager/company always ponders whether there are risks in keeping the employee for a few more weeks vs the gains, and this is the question Jeff pondered - that he did this is quite normal. Will the employee provide anything useful to us in those two weeks (e.g. handoff work to others, etc)? Could he/she cause problems (bad mouth people to fellow employees, steal IP, etc). If it's a disgruntled employee, they are usually shown the door the same day. In Timnit's case, it's unlikely there was any value in letting her stay for 2 more weeks.
I once intended to leave the company I was working for. The night before, I took out everything of (personal) value from my cubicle, as well as from my work machine. Only then did I have the discussion with my manager.
Having seen how she communicates and handles difficult situations, I think she really should read those kinds of books. Sometimes her behaviors are textbook examples of what not to do.
(Hint: If you're trying to influence someone, or a whole industry, you are negotiating, whether you choose to think of it that way or not).
> Google went all scorched earth and it’s still unclear why
Possibly because she made a personal problem into a team/department problem by asking their colleagues to stop working ("stop writing your documents because it doesn’t make a difference"). I couldn't imagine a company where such a call for work refusal wouldn't immediately lose you a ton of goodwill.
Some people don't have time for an bluff game, person choose the tone in negations with in power opponent without considering full extent of bluffing consequences.
People are consistently leaving out the fact that she wrote an internal email encouraging her coworkers to reach out to congress about Google's behavior, at a time when big tech companies are being dragged in front of congressional hearings nearly every month
My researcher acquaintances at industry labs at IBM, Microsoft, HP, Xerox, ATT, Bell, DEC, Compaq, etc. never have had to have their papers reviewed internally before submitting them to conferences or journals. What's up with Google?
Technically you can submit it without pubapprove unless somebody rats you out and you might face some repercussions. Otherwise every paper is supposed to go through AI Ethics committee review along with other types of reviews.
I wonder how consistently these standards are applied, even zeroing in on Google. If it turns out that most people within the company don’t face a barrier from this process in the same way then that sounds like a really problematic symptom.
Our aim is to rival peer-reviewed journals in terms of the rigor and thoughtfulness in how we review research before publication.
But Gebru writes that HR and her management chain delivered her feedback in a surprise meeting where she was not allowed to read the actual feedback, understand the process which generated it, or engage in a dialogue about it:
Have you ever heard of someone getting “feedback” on a paper through a privileged and confidential document to HR?
A week before you go out on vacation, you see a meeting pop up at 4:30pm PST on your calendar (this popped up at around 2pm). No one would tell you what the meeting was about in advance. Then in that meeting your manager’s manager tells you “it has been decided” that you need to retract this paper by next week...
And you are told after a while, that your manager can read you a privileged and confidential document and you’re not supposed to even know who contributed to this document, who wrote this feedback, what process was followed or anything. You write a detailed document discussing whatever pieces of feedback you can find, asking for questions and clarifications, and it is completely ignored.
I've been through the peer review process at Physical Review Letters, SIGMOD, and VLDB. You get a document containing all the reviewer's comments, plus a metareviewer's take on the overall decision and what has to change. You can engage in dialogue with the metareviewer, including a detailed response letter justifying your choices, highlighting things the reviewers may have missed, and explaining where you plan to make changes. You get additional rounds of comments from the reviewers in light of that letter on later drafts.
I'm not a Googler, and I have no idea what the standard review process looks like there, but what Gebru describes does not sound at all like peer review. I also note that Dean does not contradict Gebru's account of the meeting or feedback process. If I had a paper rejected in this fashion, I would also demand to know what the hell was going on and who was responsible.
I think the person who submitted feedback in privileged and confidential way made a great choice. I expect more people taking this route in the future actually, people are scared in the current political athmosphere. Look at what Jeff Dean is getting, even though he didn’t do anything bad.
A manager in ethics shouldn’t ask Google to break the law by not providing confidientality that was requested.
I would certainly be fearful of providing an honest review of a paper championed by such a powerful figure at Google who could get me fired and unemployable with a tweet. Revealing the reviewers and throwing them under the bus would have been the end of any honest reviewing at Google. Dean made the right move here.
> A manager in ethics shouldn’t ask Google to break the law by not providing confidientality that was requested.
It is not implied anywhere that the reviewers requested confidentiality. It is certainly not implied that Google would be violating the law to rescind that confidentiality (and, to be clear: they almost certainly would not be).
It does feel off, and it’s easy to read as Google trying to suppress her findings. But there’s a simple charitable reading as well. Gebru has recently and very publicly developed a reputation for behaving with hostility toward colleagues. Google is a company that prizes (or at least claims to prize) psychological safety of employees. I can see all of this being the chain reaction caused by a number of her coworkers expressing that they were unwilling to give an honest opinion about her work if she would be able to tie it back to them. If an ordinary employee caused their coworkers so much concern they would probably already have been dismissed, but Gebru is especially talented and was high-ranking.
I imagine some reviewers extracted agreement from management before giving negative feedback on this paper that the “anonymous” in “anonymous feedback” was a promise. This explains the unusualness of the situation, why the feedback flowed though a special HR channel, why specifically management was unwilling to let her have a written copy of the feedback (that could be closely analyzed to de-blind the reviewers), and why management accepted her resignation and the resulting fallout rather than agree to de-blind the reviewers.
Readit News
In all of my time at Google AI, I never heard of pubapproval being used for peer review or to critique the scientific rigor of the work. It was never used as a journal, it was an afterthought that folks on my team would usually clear only hours before important deadlines. We like to leave peer review to the conferences/journals' existing process to weed out bad papers; why duplicate that work internally?
I'm disappointed that Jeff has chosen to imply that pubapproval is used to enforce rigour. That is a new use case and not how it has been traditionally used. Pubapproval hasn't been used to silence uncomfortable minority viewpoints until now. If this has changed, it's a very, very new change.
Ultimately it makes the whole Ethical AI department look more like a rubber stamp for Google.
I get the impression that she wrote a hit piece on Google and published with Google's name. For me, it's correct they demand a retraction. It's simply unprofessional to critque your company for something while not mentioning the work they're doing to combat that.
Having said that, if Jeff were to make public the paper, criticisms of the paper, and improvements made to address the problems described in the paper, that could go a long way towards clearing the air.
Publishing a transparently one-sided paper in Google's name would be a problem, not because of the side it picks, but because it suggests the researchers are too ideologically motivated to see the see the problem clearly.
Ironically, it indicates systemic bias on the part of the researchers who are explicitly trying to eliminate systemic bias. That's just a bit too relevant to ignore.
In paper reviews you can often see reviewers asking the authors to rewrite, clarify, add extra experiments, add missing citations. It's all normal.
Representing a more truthful reality is not 'softening'.
It's only 'softening' for those who have an already accepted, extremist view, and for whom any evidence to the contrary doesn't help their arguments.
While initially sympathetic to the author - the more I read - the more I have completely the opposite view.
Even more, it sounds like Google didn't ask originally for retraction, they just asked to take into account the newer research contradicting the paper - the thing that any researcher valuing integrity over agenda wouldn't refuse.
If somebody wants to do that research and publishing they just have to find another source of funding, i guess.
Anyway, the firing wasn't over the paper, the firing was over the unacceptably unprofessional reaction to it.
What's different in her case is that you don't see the names of the people reviewing. Being the devil's advocate, she MIGHT have a pattern of aggressively attacking people who reviewed their work before. So they might have made the reviewers anonymous this time.
If they approved the paper the message would be "google thinks language models are a waste of resources and racist". There would be no academic debate on this topic as its been framed as woke and published by a militant activist, so any disagreement would be racist (see prior interactions between this researcher and other researchers [1]).
Thats why the standard process of publishing, peer reviewing, academic critique etc would not work.
Why would their researchers working on language models stay? when they can go to Facebook, OpenAI etc. Why would new researchers join?
[1] https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...
The proper response to her position would be to publish a response or critique. Attacking her entire field does nothing to further the conversation.
Demand the best from your multi-billion dollar corporations.
Submitting conference papers last minute is... normal.
You only bring in HR protections to protect the company from a legal standpoint.
This is sad gaslighting of a reasonable concern the team had.
Having to endure some external review for what could otherwise be sensitive material.
The inability for the SJW crowd to work reasonably within very reasonable terms, to then resort to aggressive tactics such as 'demand the names and opinions of everyone on the board' and then publicly misrepresenting the situation is going to lose you a lot of favour.
Every time I read one of these stories I immediately feel sympathetic to the individual, but then upon learning more, I feel duped and maligned for having been effectively lied to.
The doors are wide open for progress, those who take it to micro-totalitarian lengths are not doing anyone any favours.
https://melwy.com/blog/black-female-scientist-timnit-gebru-f...
Publishing a paper with a lack of rigor about some obscure mathematical technique isn't a problem for google (beyond some possible but unlikely mild reputation damage). Publishing a paper with a lack of rigor that says google is doing unethical things, when those things are questionably accurate, that is something google would (and should) have a problem with.
Whether the paper actually lacks rigor in a relevant way is not something I can comment on.
[1] https://www.platformer.news/p/the-withering-email-that-got-a...
Deleted Comment
In some cases, though, it's not simply a matter of listing it as other work in the Intro - you may need to incorporate it into your models, etc.
Deleted Comment
Previously, we only had one side of this story. But if this is Dean's best spin on Google's side of the story, I'm very tempted to conclude they're in the wrong here. Obviously I don't have all the information, but the information I do have feels consistent with the idea that someone important at Google didn't like Gebru's paper for corporate-political (meaning making Google look good, as opposed to political-political) reasons, they tried to get Gebru to play ball, she refused, and now they have to back-project a justification in the name of "scientific integrity".
Unfortunately, I think this is a story where most people's opinions about who's in the right will be more informed by their previous opinions about Gebru and Dean than the narrower question of what happened with this particular paper. I'm probably even guilty of that to some extent myself, given that I'm a fan of some of Gebru's previous work.
What I don't understand is why in the discussion nobody proposed amending the paper rather than withdrawing it. If Dean's issue was it didn't cite papers X,Y and Z, rather than demand a withdrawal, why didn't he just demand "I want you to amend the paper to add cites to X,Y,Z". And then, if Gebru and her coauthors were willing to add those cites, that would resolve it.
Indeed, from what I understand, "you should add a cite to X" is common peer review feedback, and a lot of papers get citations added due to requests from peer reviewers. So this isn't hugely different from that scenario.
It would seem that withdrawal over this issue would only make sense if Gebru and her coauthors refused to amend the paper to add the requested citations, but I haven't heard anything saying that she did refuse to do so. It isn't clear if the alternative solution of amending rather than withdrawing was ever brought up in the discussion by either party.
Not that I'm a researcher or anything, but if I was, and a superior told me "we need you to withdraw your paper because it doesn't cite X,Y,Z", my immediate response would be "How about I add the citations you are requesting and resubmit it with those additions?"
Unless corporate tells you to kill the paper and you need something that resembles legal cover.
Deleted Comment
If a paper does not analyze the improvements from recent work, and just older work that has been surpassed (deemed inefficient by Dean), the new reanalysis might not be as favorable to the results as the paper proposes which means the paper is moot.
Or the second point is about bias in language models, but it sounds like these issues are mitigated in recent research, which means people are already aware and have solved a bulk of the issue being described in the paper.
But certainly it's possible that the paper's contributions stands strong even after accounting for the recent work that Dean mentions. In that case, it could be corrected for camera ready. But my point is that we can't tell right now without seeing the paper, and the relevant research that was omitted.
And Jeff has lost mine. Even if his comments are not just bad faith, his blindness borders on incompetence for the role he has. Yet another clever, blind white man.
I hope a position in the Biden administration can be found for her, and her vision can have a larger societal impact.
This, though, looks and feels like thinly-veiled retroactive and pretty unconvincing PR. It's short on detail and appears somewhat at odds with several points from Timnit Gebru's resignation note [0]:
- Dean says the paper was reviewed by a "cross-functional team". Gebru says she received the feedback through a "privileged and confidential document to HR"
- Dean says the paper was submitted for review on the day it was due to be published; Gebru says they had notified "PR & Policy 2 months before".
- Dean suggests the feedback was due to the paper not highlighting mitigating work for some of the limitations the paper was exposing. That seems like a very normal part of the research process. Why, then, does Gebru claim that she was told that a "manager can read you a privileged and confidential document" and that no other recourse or exploration of the feedback was permissible?
The only thing we know from the outside is that reality will be far more nuanced and complicated than the tidbits that leak out. Even allowing for that though - and reading some of the related comments here - Google isn't coming out of this well at the moment.
[0]: https://www.platformer.news/p/the-withering-email-that-got-a...
EDIT: Fixed spelling of Timnit Gebru's name.
Wait, really? That’s an important detail if true. The one-day timeline was a central part of the narrative surrounding this story. Notifying them two months ahead of time makes this a completely different situation.
I’m a bit skeptical. Could this claim be verified somehow? Since it’s very public news at this point, we may as well try to be rigorous.
I agree that Google isn't looking good at the moment. But if I had a colleague who seemed intent on finding ways to place the company in legal jeopardy, then I too would avoid direct communication whenever possible.
I don't think these two takes are at odds with each other at all. It sounds like a cross-functional team reviewed the paper and produced that "privileged and confidential document" but Gebru didn't find that document sufficiently detailed.
employee: I'm not happy about x, y and z. If you don't do those, I'm going to quit.
manager: well we are not going to do those, so thank you for your time. We accept your resignation and would like it to start immediately (i.e. you're fired).
If you are gonna tell your manager that you plan to resign if a condition isn't met, then what do you expect them to say if they don't plan to fulfill that condition? It sounds like she was expecting them to say "Hey, well we don't want to meet your demands, but sure, we're happy to have a disgruntled employee around here, so feel free to stick around, or you could just quit on your own timeline, no sweat".
I suspect that many people would be fired on the spot for threatening to resign, so don't threaten it if you aren't okay with that consequence.
1. Was the treatment of her research in internal review reasonable?
2. Was terminating her employment reasonable after she sent the (now public) email to the women-and-allies brain listserv?
3. Was the end of her employment at Google a resignation or a firing?
To me, (3) is by a wide margin the least interesting part of the story and all of the discussion here is missing the point entirely. Whether she was fired or resigned has zero bearing on whether (1) and (2) show reasonable or unreasonable actions.
link to abstract and discussion - https://old.reddit.com/r/MachineLearning/comments/k69eq0/n_t...
Part of being a good manager is understanding your employees and helping them succeed. If someone makes a statement like this in the heat of frustration it doesn't necessarily mean they will actually quit. If they're a valuable member of the team you should present them with an opportunity to save face and remain.
To me this seems like taking an opportunity to fire someone they already wanted to get rid of. Either that or a bad manager who wanted to flex their power as a threat to the rest of the team.
One of the more difficult lessons I learned as a manager: Once a team member starts giving ultimatums in order to get their way or override decisions, it's not in your best interests to give in for the sake of keeping that employee. The obvious exception is if you realize you were actually wrong from the start, but reversing decisions for the sake of caving to someone's demands is a problem.
If someone is so ready to quit that they'll flaunt it to the company, it's doubtful that reversing a single decision is going to suddenly make them happy again with their employer. Worse yet, it sends a message that threatening to quit is the way to get what you want from the company. Once you validate the strategy, you will get a lot more of it.
Unfortunately, if someone already has one foot out the door and has been complaining openly (even on Twitter, in this case) about their employer, it's best for everyone to go their separate ways. From there, focus on identifying and fixing any underlying problems to minimize the chance of this happening again in the future.
See her previous (potential) legal troubles with Google that she even acknowledges herself.
> I was in the middle of a potential lawsuit for which Kat Herller and I hired feminist lawyers who threatened to sue Google (which is when they backed off--before that Google lawyers were prepared to throw us under the bus and our leaders were following as instructed) and the next day I get some random “impact award.”
https://www.platformer.news/p/the-withering-email-that-got-a...
Deleted Comment
Deleted Comment
> If I can’t know who is providing reviews to my work, it’s difficult for me to imagine continuing to work as a part of this group.
Many people would still describe this (erroneously) as an ultimatum even though it could be bleakly summarized in the same vein as a true ultimatum.
In any case, she has been very vocal on twitter and has not seemed to deny that she gave some sort of ultimatum. If she didn't give an ultimatum, it would only make Google look worse, so why not mention that on Twitter (given that she has tweeted probably 100 things about this incident in the last two days)? Given the absence of a denial, I'm going to assume that it was worded as an ultimatum.
The thing is, there's a big difference between resigning and being fired for cause, even if both end with you not working at the company anymore.
Maybe she isn't that important for the company? Why bother working with someone making demands if they aren't worth your time. Google might have actually thought she was detrimental to the company and wanted an excuse to fire her and she gave it.
People claiming that the deficiencies in the paper are minor and wouldn't be blockers obviously have little experience submitting to academic journals. Other parts of Google doing deeply technical work probably don't have the same level of review as the Ethical AI group -- for obvious reasons.
There is usually a long back and forth -- there are even memes about the infuriating comments from "Reviewer 2" [1][2]. Omitting to mention argument-obsoleting developments in the field (from your own lab!) is more than enough to send you back to extensive redrafting.
To be clear on terminology -- a retraction is an academic black mark, and occurs to a paper after publication, usually for reasons of research misconduct. This is not an instance of that.
[1] https://twitter.com/redpenblackpen/status/113344056990719590... [2] http://jasonya.com/wp/wp-content/uploads/2016/01/PowerRespon...
AI ethics are important and I'm willing to bet that the Google execs want to know about biases and other problems in their products and probably don't care that much what is published. They probably don't want high maintenance researchers that cause them headaches and are obsessed with microaggressions, etc. I expect that this stuff has been a distraction for too long and they are happy to be rid of this and happy for the signal it sends across the company.
I have little sympathy or concern for Ivy educated elite in-fighting. There is probably a huge number of qualified people just dying to have the opportunity to work there, have access to those resources and do research.
Tweet explaining how she "resigned": https://twitter.com/timnitGebru/status/1334343577044979712?s...
An external reviewer of the paper: https://www.reddit.com/r/MachineLearning/comments/k69eq0/n_t...
I think you should be more open about what potentially actually happened because your first sentence seems very accusatory without proper information, the very bias that Gebru speaks of. According to her email that Dean references in his, her "feedback" for the paper was given in the form of a confidential HR document. That seems highly nonstandard for a review process supposedly only concerned with scientific rigor and is not addressed at all by Dean's statement. Further, his statement clearly, likely intentionally, muddies the water about what the actual sequence of events were that led to the paper being created, internal reviewers originally being notified, when the submission happened, who approved it, who submitted it externally, who asked for this to be retracted, and when and in what order all this happened. The fact that his statement, that was apparently meant to clarify these exact proceedings, makes it even more vague about what actually happened seems pretty damning to Dean's case, in my opinion.
Both Gebru's and Dean's statements have very little overlap in flavor and in facts, so it's pretty apparent something is going on here that is nontrivial and abnormal.
The non-research motive is arguably to control the narrative about using ML (I admit I still choke at calling it AI) and big data techniques. I fail to see how this is advanced by having a very public spat.
I've no doubt some senior exec decided she had to go, but I don't think it is because of any intersectional reason, or to cover up any particular publication, but that she wasn't seen as an asset to the company. Strategies for negotiating exercise of power are very different at Google to social sciences academia (or twitter, which is engorged with righteousness over this), which she seems not to have grasped. There are few enfants terribles in corporate tech that don't have controlling stock.
There's a classic documentary about this phenomenon: https://www.youtube.com/watch?v=9UiF7BsC1Ig
To be clear, both of these had been received for the paper in question.
> Unfortunately, this particular paper was only shared with a day’s notice before its deadline — we require two weeks for this sort of review — and then instead of awaiting reviewer feedback, it was approved for submission and submitted.
It seems like someone short-circuited the review process and submitted without review to meet a deadline, with a post-submission review. When this occurred, it was not all green lights. The expectation is that you then pull the submission and, post review, submit elsewhere. After all, conferences are all virtual now, it is not a hardship to submit elsewhere.
If you have other information as to the facts or the internal process do tell.
This sheds some new light...
https://twitter.com/timnitGebru/status/1334881120920498180
She has a Twitter army ready to set their aim on her reviewers and a track record, they can't reveal the names.
Which is of course not to say you can't have a more measured negotiation. But it can be hard to walk back from give me X or I do Y, especially if there isn't a lot of middle ground between giving you X or not giving you X.
If an employee is adversary toward the company, the trade off needs to be made no matter how much valuable the employee's work is.
This is a non-issue tbh. She wants to leave. Company wants her to leave. They both agree to part way because the premises are fulfilled (i.e. company can't meet her requirements).
If you want to get technical, I'd bet her employee's status is still in tact for 2 weeks; she just doesn't have access to laptop and etc.
I strongly encourage people to read some books on negotiations - as well as read up on legal ramifications to some negotiations.
Pretty much all books/courses on negotiations say: Ultimatums have their place, but are a minefield (i.e. they can blow up on you), and should be used as a last resort. From a negotiations standpoint, the response was adequate - which is why they all caution against using such an approach.
As for the 2 week thing, this is a convention, but not a requirement. In my company, it's not unusual for someone to be shown the door the same day they announce they plan to leave to another company (it's not the norm, but not at all unusual). The manager/company always ponders whether there are risks in keeping the employee for a few more weeks vs the gains, and this is the question Jeff pondered - that he did this is quite normal. Will the employee provide anything useful to us in those two weeks (e.g. handoff work to others, etc)? Could he/she cause problems (bad mouth people to fellow employees, steal IP, etc). If it's a disgruntled employee, they are usually shown the door the same day. In Timnit's case, it's unlikely there was any value in letting her stay for 2 more weeks.
I once intended to leave the company I was working for. The night before, I took out everything of (personal) value from my cubicle, as well as from my work machine. Only then did I have the discussion with my manager.
Having seen how she communicates and handles difficult situations, I think she really should read those kinds of books. Sometimes her behaviors are textbook examples of what not to do.
(Hint: If you're trying to influence someone, or a whole industry, you are negotiating, whether you choose to think of it that way or not).
Possibly because she made a personal problem into a team/department problem by asking their colleagues to stop working ("stop writing your documents because it doesn’t make a difference"). I couldn't imagine a company where such a call for work refusal wouldn't immediately lose you a ton of goodwill.
> ...if we didn’t meet these demands, she would leave Google and work on an end date.
Google's response to this was not accepting of her terms, it was to force an end-date for her —immediately— and lock her out of their systems ASAP.
Deleted Comment
Deleted Comment
(We have a publication approval process, but its purpose is to prevent IP leaks and maybe shield liability, not uphold scientific rigour standards)
They know they have a bubbling ethics problem and they've screwed over their best chance to do anything other than completely fuck it up.
In history, this will be a wtf double facepalm moment.
https://twitter.com/le_roux_nicolas/status/13346019609729064...
Deleted Comment
Our aim is to rival peer-reviewed journals in terms of the rigor and thoughtfulness in how we review research before publication.
But Gebru writes that HR and her management chain delivered her feedback in a surprise meeting where she was not allowed to read the actual feedback, understand the process which generated it, or engage in a dialogue about it:
Have you ever heard of someone getting “feedback” on a paper through a privileged and confidential document to HR?
A week before you go out on vacation, you see a meeting pop up at 4:30pm PST on your calendar (this popped up at around 2pm). No one would tell you what the meeting was about in advance. Then in that meeting your manager’s manager tells you “it has been decided” that you need to retract this paper by next week...
And you are told after a while, that your manager can read you a privileged and confidential document and you’re not supposed to even know who contributed to this document, who wrote this feedback, what process was followed or anything. You write a detailed document discussing whatever pieces of feedback you can find, asking for questions and clarifications, and it is completely ignored.
(from https://www.platformer.news/p/the-withering-email-that-got-a...)
I've been through the peer review process at Physical Review Letters, SIGMOD, and VLDB. You get a document containing all the reviewer's comments, plus a metareviewer's take on the overall decision and what has to change. You can engage in dialogue with the metareviewer, including a detailed response letter justifying your choices, highlighting things the reviewers may have missed, and explaining where you plan to make changes. You get additional rounds of comments from the reviewers in light of that letter on later drafts.
I'm not a Googler, and I have no idea what the standard review process looks like there, but what Gebru describes does not sound at all like peer review. I also note that Dean does not contradict Gebru's account of the meeting or feedback process. If I had a paper rejected in this fashion, I would also demand to know what the hell was going on and who was responsible.
This feels off.
A manager in ethics shouldn’t ask Google to break the law by not providing confidientality that was requested.
It is not implied anywhere that the reviewers requested confidentiality. It is certainly not implied that Google would be violating the law to rescind that confidentiality (and, to be clear: they almost certainly would not be).
I imagine some reviewers extracted agreement from management before giving negative feedback on this paper that the “anonymous” in “anonymous feedback” was a promise. This explains the unusualness of the situation, why the feedback flowed though a special HR channel, why specifically management was unwilling to let her have a written copy of the feedback (that could be closely analyzed to de-blind the reviewers), and why management accepted her resignation and the resulting fallout rather than agree to de-blind the reviewers.