Am an admirer of Mudge & co. from back in the day, though my initial reaction was doesn't he seem a bit over powered for Twitter? Their recent security issues with that hack were devops management and governance problems, no new science involved, they just dropped the ball on some solved problems.
Speculating on what kind of problems Twitter would need that calibre of person for, maybe detecting inevitable AI driven botnets and deepfakes?
Could also see how they need leadership with that tangible cred as a decider function, e.g. the personality cult around Jack might make delegation and scaling their management team a challenge if everyone is trying to find a way to compete for his attention, so they need someone with enough weight that he can legitimately defer to on security issues.
For some reason, high level managers need very expensive consultants to tell them the obvious truths.
As a very expensive consultant I am frequently appalled at how management teams completely ignore their own employees.
Forget about wasteful, these companies can afford consultants. It is just disrespectful to their own staff. Developers and operations guys already know the solution but can't get through layers upon layers of management. And so it takes a guy with salary high enough that he can be allowed to talk to the a high level manager directly, to get the message.
I was once in a uni class where the prof was a very expensive consultant. He said that a large % of his jobs for companies like GM or Ford was to grab a notepad and walk the floor. He would ask line employees why they did things the way they did and what they would change. He got reams of suggestions, typed them up, submitted them and collected a giant paycheck.
Sometimes they were blindingly obvious things. One employee would drive two bolts, quickly change heads, then drive a third bolt. While the carcass advanced to the next station, the emloyee would change back to the original head and be ready for the next carcass. That employee's suggestion: Make all three bolts the same size so I don't have to change heads.
Of course, that is just a fun anecdote. All such efficiencies have been implemented long ago... for auto assembly. But the point is still valid. Your employees know a ton about your business. Even the ones who do the most mundane tasks. Listen to them. Or... pay a very expensive consultant to listen to them.
Back in the mid-1990s, I did a lot of web work for traditional media. That often meant figuring out what the client was already doing on the web, and how it was going, so I’d find the techies in the company, and ask them what they were doing, and how it was going. Then I’d tell management what I’d learned. This always struck me as a waste of my time and their money; I was like an overpaid bike messenger, moving information from one part of the firm to another. I didn’t understand the job I was doing until one meeting at a magazine company.
The thing that made this meeting unusual was that one of their programmers had been invited to attend, so management could outline their web strategy to him. After the executives thanked me for explaining what I’d learned from log files given me by their own employees just days before, the programmer leaned forward and said “You know, we have all that information downstairs, but nobody’s ever asked us for it.”
I remember thinking “Oh, finally!” I figured the executives would be relieved this information was in-house, delighted that their own people were on it, maybe even mad at me for charging an exorbitant markup on local knowledge. Then I saw the look on their faces as they considered the programmer’s offer. The look wasn’t delight, or even relief, but contempt. The situation suddenly came clear: I was getting paid to save management from the distasteful act of listening to their own employees.
In the early days of print, you had to understand the tech to run the organization. (Ben Franklin, the man who made America a media hothouse, called himself Printer.) But in the 19th century, the printing press became domesticated. Printers were no longer senior figures—they became blue-collar workers. And the executive suite no longer interacted with them much, except during contract negotiations.
This might have been nothing more than a previously hard job becoming easier, Hallelujah. But most print companies took it further. Talking to the people who understood the technology became demeaning, something to be avoided. Information was to move from management to workers, not vice-versa (a pattern that later came to other kinds of media businesses as well.) By the time the web came around and understanding the technology mattered again, many media executives hadn’t just lost the habit of talking with their own technically adept employees, they’d actively suppressed it.
I think your perception of what a high level person like Mudge does is a bit off. At Stripe he did amazing work, but it was mostly organizational. He built a world class team, defined a set of short, medium and long term goals that enabled the company to reduce risks over time, had his team build tools to measure that risk and held them accountable for moving the needle.
That is to say, he did executive level work. I imagine he’ll do similarly great work at Twitter, setting them up for long term success, just as he did at Stripe
Lot of consultants commenting seem to think what they do is simple, it is really not.
They have a long experience of asking the right questions to the right people , filtering out useless inputs, creating the right abstractions strategically from tactical inputs, there are very niche skills, it may be easy for us, but for the non technical manager it is black magic
> For some reason, high level managers need very expensive consultants to tell them the obvious truths.
As a very expensive consultant I'm surprised the reason is not more clear to you. Most managers like to pay consultancy firms so that they can cover their asses.
If they just followed the guidelines of what the McKinsey team said then surely they weren't in the wrong. Whilst if they followed the advice of a subordinate or god forbid made a decision it's on them.
The problem is usually in-fighting between middle managers protecting their turf. Why can't the incentives for middle managers be fixed? Seems like if a company took anonymous performance reviews of Directors and VPs from lower-level employees it would prevent a lot of the bullshit.
Leaders that make decisions that have ramifications that are trivially worth 9+ figures are willing to pay the extra 500k to take their trust in the word of a consultant from 95% to 99%.
> but can't get through layers upon layers of management
i think you hit the nail on the head here. the problem is that the upper management usually arent clueless, they are just better leaders than they are doers...
> As a very expensive consultant I am frequently appalled at how management teams completely ignore their own employees.
Tell me about it. On one of my previous jobs I would constantly speak out about what we were doing would cause people to lose hundreds of thousands of USD, and every time I was laughed out of the meeting for "overthinking" things (I guess that's what wanting to do things right is called these days). When one of our customers lost US$200k and nobody had any idea why or how or when I knew that company would be the death of me, and coincidentally I was on the same meeting I planned to use to quit my job.
If anything it taught me that working in finance is not for me :p
I would imagine nobody takes Twitter security seriously (quite evidently including themselves) which probably creates recruiting impediment. If Jack feels he needs some gravitas to attract a good team, hiring Mudge isn't a bad idea. (Might also provide some legitimacy to an off-menu Twitter threat intel product as well.)
There's probably a little bit of housekeeping for infrastructure and operations security. I imagine all of the fun stuff is customer-facing.
A watering-hole as large as Twitter will be under constant attacks of every sort and every source, from individual white/grey/black-hats trying to poke holes in the service itself, spammers who want to figure out how to create as many automated accounts and tweets as possible without getting blocked, government-sponsored hackers trying to get personal information on dissidents or defectors, etc etc.
> Their recent security issues with that hack were devops management and governance problems
If those are the hard business/technology management problems (and they are), then try solving those while also securing them.
Security by itself is relatively straightforward: 0days, red teaming, scanning for known vulns, etc. Just like every other aspect of the business, you hire someone to do one specific thing and you've got lots of options.
But then try to infuse security into every single aspect of a business and product, in a way that increases both velocity & quality of product dev, without sacrificing efficient organizational management. There are already a ton of inscrutable complexities between all facets of the organization and its products. Trying to add security to all that is like teaching a juggling elephant to ice skate.
So you need someone who's very good at managing security in the context of all those other problems. That's hard to find. It helps to have people who've seen problems in the same general space from a lot of different angles.
> Their recent security issues with that hack were devops management and governance problems, no new science involved, they just dropped the ball on some solved problems.
You don't hire security personnel to address previous breaches.
When it comes to security, better to err on the side of too much than too little. I'm sure there's enough work to keep him busy, question might be whether it's intellectually stimulating enough for him.
Hypothesis: large numbers of state actors run influence networks on Twitter and have no clue where to start now that the actors have had years to adapt to the weak response from Twitter.
Twitter has global geo-political implications. Literally, a war could have broken out with Trump's account being hijacked. He just left Stripe, where he was responsible for a PCI environment. If anything, his last job he was "overpowered" for relative to Twitter.
Twitter hired Mudge because Twitter needs a security engineer with a friendly ear in the US Government. Mudge has been building goodwill with Congress since the 90s and Twitter is seeking to leverage that.
They have more money than they know what to do with. Let's face it, this is a company that runs what's essentially a very simple web site, yet they have a 34B market cap.
The fact that you don't often see announcements for big tech head-of-security appointments on the front page of hackernews says a lot about why they hired him.
Mudge name handle does rings a bell and the main reason is that he was featured and interviewed in the arguably the most famous 49th issue of Phrack magazine containing the popular article "Smashing The Stack For Fun and Profit"[1]. In term of popularity this article is at least as popular in computer security circle in term of its profound effect as Dijkstra's seminal paper on "Go To Statement Is Considered Harmful" in computer science community.
I wouldn't call that a "seminal paper". However, "Smashing The Stack For Fun and Profit" joins a couple of classic papers such as "The Geometry of Innocent Flesh on the Bone: Return-into-libc without Function Calls (on x86)" (ROP) and the Spectre/Meltdown papers as being fundamental to application security.
Possibly of interest to the HN crowd, just before this Mudge was (and I think still is) involved in a non-profit named CITL that focuses on performing static and dynamic analysis (eg fuzzing) to evaluate hardening measures in consumer software. For example, their work on browsers: https://cyber-itl.org/comparisons/comparison.html?comparison...
It's very interesting work but I haven't seen wide discussion on it so far.
Very progressive move by Twitter. Meanwhile I tried to warn a car parts shop for 6 months of their massive problem [1] (remote js execution on their site that hadled payments) and even after it was in the media they didn't answer me or fix the issue.
Wrote them on Facebook one day and they tried to sue me
As an Austrian myself, I'm not at all surprised at their reaction. To them, their website is "just IT stuff" and they simply don't have a notion that it would involve any security.
To them you'll likely seem like an overzealous geek that shouldn't mess with their business website. I've experienced this before myself and it's not particularly a good position to be in.
Their site has most likely been technically abandoned, i.e. no one capable is in charge anymore.
It'd be best to talk to the owner, show them your "hack" (change it to cats on your phone and let them verify in their browser) and offer them to fix it for free.
That's how one does these things in our small country. ;-)
that's actually what I tried to do. In my mind it would go like "hey your site has included a script from a domain I own" - "haha right, that's some legacy stuff thanks for noticing us" - "no worries
but obviously that's not what happened
1. Telling them in person (they didn't understand)
2. Asking for the IT persons Phone number (they didn't give it to me)
3. Leaving my phone number and email (they never contacted me)
4. Notifying the austrian CERT (they never got an answer from the owner)
5. Notifying the press (standard.at posted an article about it, they didn't respond)
6. Writing them on Facebook (ob boy did they respond :D)
But since my first police raid I don't publish anything before letting my lawyer read it. He said if they do press charges they haven't got a chance since I have a paper trail of everything I did and didn't harm them or their site in any way
Delete all profiles with five or more digits in the username. Joking aside it will be interesting to see what they can come up with for solutions to platform integrity. I find myself using Twitter more and more and the only SoMe I really "follow".
“ Zatko said he appreciated Twitter’s openness to unconventional security approaches, such as his proposal for confusing bad actors by manipulating the data they receive from Twitter about how people interact with their posts.”
Would this be like offensive cyber security? Or active security?
I figured it out. Beto O’Rourke was in Cult of the Dead Cow hahaha. He also used to be in a post-hardcore band with a dude from At the Drive-In. How awesome can one person get?!
Readit News
Speculating on what kind of problems Twitter would need that calibre of person for, maybe detecting inevitable AI driven botnets and deepfakes?
Could also see how they need leadership with that tangible cred as a decider function, e.g. the personality cult around Jack might make delegation and scaling their management team a challenge if everyone is trying to find a way to compete for his attention, so they need someone with enough weight that he can legitimately defer to on security issues.
Congrats to all involved. Interesting times.
As a very expensive consultant I am frequently appalled at how management teams completely ignore their own employees.
Forget about wasteful, these companies can afford consultants. It is just disrespectful to their own staff. Developers and operations guys already know the solution but can't get through layers upon layers of management. And so it takes a guy with salary high enough that he can be allowed to talk to the a high level manager directly, to get the message.
Sometimes they were blindingly obvious things. One employee would drive two bolts, quickly change heads, then drive a third bolt. While the carcass advanced to the next station, the emloyee would change back to the original head and be ready for the next carcass. That employee's suggestion: Make all three bolts the same size so I don't have to change heads.
Of course, that is just a fun anecdote. All such efficiencies have been implemented long ago... for auto assembly. But the point is still valid. Your employees know a ton about your business. Even the ones who do the most mundane tasks. Listen to them. Or... pay a very expensive consultant to listen to them.
http://www.shirky.com/weblog/2013/11/healthcare-gov-and-the-...
Back in the mid-1990s, I did a lot of web work for traditional media. That often meant figuring out what the client was already doing on the web, and how it was going, so I’d find the techies in the company, and ask them what they were doing, and how it was going. Then I’d tell management what I’d learned. This always struck me as a waste of my time and their money; I was like an overpaid bike messenger, moving information from one part of the firm to another. I didn’t understand the job I was doing until one meeting at a magazine company.
The thing that made this meeting unusual was that one of their programmers had been invited to attend, so management could outline their web strategy to him. After the executives thanked me for explaining what I’d learned from log files given me by their own employees just days before, the programmer leaned forward and said “You know, we have all that information downstairs, but nobody’s ever asked us for it.”
I remember thinking “Oh, finally!” I figured the executives would be relieved this information was in-house, delighted that their own people were on it, maybe even mad at me for charging an exorbitant markup on local knowledge. Then I saw the look on their faces as they considered the programmer’s offer. The look wasn’t delight, or even relief, but contempt. The situation suddenly came clear: I was getting paid to save management from the distasteful act of listening to their own employees.
In the early days of print, you had to understand the tech to run the organization. (Ben Franklin, the man who made America a media hothouse, called himself Printer.) But in the 19th century, the printing press became domesticated. Printers were no longer senior figures—they became blue-collar workers. And the executive suite no longer interacted with them much, except during contract negotiations.
This might have been nothing more than a previously hard job becoming easier, Hallelujah. But most print companies took it further. Talking to the people who understood the technology became demeaning, something to be avoided. Information was to move from management to workers, not vice-versa (a pattern that later came to other kinds of media businesses as well.) By the time the web came around and understanding the technology mattered again, many media executives hadn’t just lost the habit of talking with their own technically adept employees, they’d actively suppressed it.
That is to say, he did executive level work. I imagine he’ll do similarly great work at Twitter, setting them up for long term success, just as he did at Stripe
They have a long experience of asking the right questions to the right people , filtering out useless inputs, creating the right abstractions strategically from tactical inputs, there are very niche skills, it may be easy for us, but for the non technical manager it is black magic
As a very expensive consultant I'm surprised the reason is not more clear to you. Most managers like to pay consultancy firms so that they can cover their asses.
If they just followed the guidelines of what the McKinsey team said then surely they weren't in the wrong. Whilst if they followed the advice of a subordinate or god forbid made a decision it's on them.
i think you hit the nail on the head here. the problem is that the upper management usually arent clueless, they are just better leaders than they are doers...
You need tools for your group.
IT wants to use tool MSFT.
Everybody has been using tool X for a while. They like it, but there is issue Y.
How does upper management resolve this?
Tell me about it. On one of my previous jobs I would constantly speak out about what we were doing would cause people to lose hundreds of thousands of USD, and every time I was laughed out of the meeting for "overthinking" things (I guess that's what wanting to do things right is called these days). When one of our customers lost US$200k and nobody had any idea why or how or when I knew that company would be the death of me, and coincidentally I was on the same meeting I planned to use to quit my job.
If anything it taught me that working in finance is not for me :p
There's probably a little bit of housekeeping for infrastructure and operations security. I imagine all of the fun stuff is customer-facing.
And for Mudge, for why it might be worth taking on relatively 'simple' problems - money. Money is nice.
Deleted Comment
If those are the hard business/technology management problems (and they are), then try solving those while also securing them.
Security by itself is relatively straightforward: 0days, red teaming, scanning for known vulns, etc. Just like every other aspect of the business, you hire someone to do one specific thing and you've got lots of options.
But then try to infuse security into every single aspect of a business and product, in a way that increases both velocity & quality of product dev, without sacrificing efficient organizational management. There are already a ton of inscrutable complexities between all facets of the organization and its products. Trying to add security to all that is like teaching a juggling elephant to ice skate.
So you need someone who's very good at managing security in the context of all those other problems. That's hard to find. It helps to have people who've seen problems in the same general space from a lot of different angles.
You don't hire security personnel to address previous breaches.
See also:
https://www.kcl.ac.uk/csss/assets/10957%E2%80%A2twitterconfl...
https://www.getrevue.co/profile/caseynewton/issues/a-catastr...
https://www.youtube.com/watch?v=VVJldn_MmMY
Good for him.
[1]http://phrack.org/issues/49/4.html
It's very interesting work but I haven't seen wide discussion on it so far.
Wrote them on Facebook one day and they tried to sue me
[1] https://blog.haschek.at/2019/threat-vector-legacy-static-web...
To them you'll likely seem like an overzealous geek that shouldn't mess with their business website. I've experienced this before myself and it's not particularly a good position to be in.
Their site has most likely been technically abandoned, i.e. no one capable is in charge anymore.
It'd be best to talk to the owner, show them your "hack" (change it to cats on your phone and let them verify in their browser) and offer them to fix it for free.
That's how one does these things in our small country. ;-)
but obviously that's not what happened
1. Telling them in person (they didn't understand)
2. Asking for the IT persons Phone number (they didn't give it to me)
3. Leaving my phone number and email (they never contacted me)
4. Notifying the austrian CERT (they never got an answer from the owner)
5. Notifying the press (standard.at posted an article about it, they didn't respond)
6. Writing them on Facebook (ob boy did they respond :D)
But since my first police raid I don't publish anything before letting my lawyer read it. He said if they do press charges they haven't got a chance since I have a paper trail of everything I did and didn't harm them or their site in any way
What does "tried to sue" mean?
Did they file a suit? Did they threaten to sue but not follow through?
Would this be like offensive cyber security? Or active security?
If youre manipulating the engagement statistics, then you're dealing with a particular type of bad actor...
This is taking me down memory lane.
Beto O’Rourke... What???