Congratulations Katie! It's beautiful to see something that I hoped would be real, especially after seeing Interstellar's gorgeous rendition. And may you inspire hundreds of thousands of girls to enter the fields of science and technology.
> Very long baseline interferometry (VLBI) is a technique for imaging celestial radio emissions by simultaneously observing a source from telescopes distributed across Earth. The challenges in reconstructing images from fine angular resolution VLBI data are immense. The data is extremely sparse and noisy, thus requiring statistical image models such as those designed in the computer vision community. In this paper we present a novel Bayesian approach for VLBI image reconstruction. While other methods often require careful tuning and parameter selection for different types of data, our method (CHIRP) produces good results under different settings such as low SNR or extended emission. The success of our method is demonstrated on realistic synthetic experiments as well as publicly available real data. We present this problem in a way that is accessible to members of the community, and provide a dataset website (vlbiimaging.csail.mit.edu) that facilitates controlled comparisons across algorithms.
What strikes me as really amazing is the cross functional nature of these modern achievements. I did not realize that this image was created with statistical image models and a Bayesian approach.
Also, this link included -> http://vlbiimaging.csail.mit.edu/ introduces the field and offers a good explanation for those interested in learning more:
> Imaging distant celestial sources with high resolving power requires telescopes with prohibitively large diameters due to the inverse relationship between angular resolution and telescope diameter. However, by simultaneously collecting data from an array of telescopes located around the Earth, it is possible to emulate samples from a single telescope with a diameter equal to the maximum distance between telescopes in the array. Using multiple telescopes in this manner is referred to as very long baseline interferometry (VLBI).
No. Angular resolution is essentially the angular distance between two points that are still resolved as separate points. So if your resolution increases, angular resolution decreases, because you can resolve two points that are closer together.
I'm not sure, but from a class I'm taking right now, I have a faint inkling that the lesser light you let in, the more resolution you have i.e the more you're able to distinguish between two close together objects.
I see a lot of mention on various forums about the storage they used (5pb) but am just wondering if anyone know what kind of backend they used to house this? From what I saw there were too many disks - in the wrong type of enclosure - to be running on a single server, which suggests multiple physical servers. I've seen a prior CERN research paper on gluster and ceph (iirc) and am just wondering if anyone in the know could enlighten me?
The WaPo article also references a few of the interesting issues they had:
"Then they spent the two years parsing literal truckloads of data, some of which had to be shipped on hard drives from the South Pole and defrosted outside a supercomputer facility at MIT."
It is an interesting presentation, but I do NOT understand Katie's explanation about how they were going to minimize the bias [to "see" already predicted black hole visualization] while creatively interpreting inputs from sparsely placed telescopes around the earth.
Disclaimer: this is based on watching the talk and some basic machine learning knowledge. Im no expert.
They have a sparse set of data that is part of an image. They have trained a model to look at the sparse set and make an educated guess about what the full image looks like. They do this by feeding it full images.
The full images you feed into the model thus have an effect on the final image generated. In order to see how large that effect is, they trained different versions of the model with different sets of complete images. Some were images of what we thought a black hole looked like. There is potential that this heavily influences the model and ensures that the output looks like what we expect it to, even if that isnt actually true.
They also trained the model with non-blackhole images. Since the output of the model was approximately the same, this indicates that the resulting output picture doesnt look like what we think a black hole looks like just because it was trained with black hole images. It likely really looks like that.
The model doesn't need to be told what a black hole looks like. The sparse measurements combined with knowledge of how sparse data can be combined to form a generic image is enough. The model learned that the sparse data is not likely pure noise, instead there are shapes and lines and gradients that relate the sparse data points to each other.
Her analogy of sketch artists is good. If you have a functionally complete description and give it to 3 sketch artists from different cultures who are used to different looking people, they will still draw the same person. However if your description isnt actually detailed enough, their sketches will significantly differ as they use their existing knowledge and bias to fill in the gaps with what they think is likely.
If you want to understand that, do not listen to TED talks. They have a terrible format and are designed to make people feel smart rather than impart knowledge.
I had the same issue. It seemed like with many plausible solutions, there is some bias in the image. I agree there are some reasonable constraints, like it should be energetic stuff in a sphere around a dark sphere, but how many solutions would have fit that criteria?
It's a TEDx talk. In this case it's an interesting one by an expert in her field, but TEDx have a lot lower demands for participating, so try to keep the distinction clear.
If Katie was a man do you think people would be going through git histories and their published papers trying to determine if she is being over-credited for her achievements?
People are already doing this on Reddit[0][1], and it's pretty silly because they obviously have no idea how Github or scientific research works. There's an effort underway to undermine Katie Bouman's contributions and it's absolutely ridiculous.
Edit: I just checked Twitter, apparently there are thousands of idiots who believe this "850,000/900,000 lines written by Andrew, therefore he wrote the algorithm" narrative. It's amazing how willing people are to eat up a low-hanging narrative as long as it confirms their world-view. All it takes is a very crude understanding of how software development works to see through this narrative.
This[0] comment seems to be another in that vein, though it seems to have more details, even if it repeats the 850k lines stat which doesn't really hold up.
At the same time, it's totally common for professors to ride on the efforts of research students under their direction, to the point of being ethically questionable.
You might as well credit the Linux operating system to only a single man, whose effort is certainly largely responsible, but for who also does not in any way represent the whole of effort.
So how do you explain all the famous male scientists and inventors of history? The most famous of them all are because they discovered cool things, not necessarily the "hardest" or "most significant" thing. Everyone knows e=mc2, but most people do not know about von neuman architecture, even though the latter has had a significantly greater impact on peoples lives. Taking the first picture of a black hole is as sexy as it gets in science. Katie would be famous no matter what she was.
I am not a female, but I am a bootcamp grad with no college degree working as a sw engineer. At almost every place I've worked at so far I've had co-workers go through my GitHub and then make comments about how I shouldn't be working there.
I have no idea why people are doing this to Dr. Bouman, if it's gender related or not. Just stating my experiences.
Also, this is a few articles and a Facebook meme. There's no Nobel prize or anything. Why begrudge someone their 15 minutes of science fame for leading a cool effort, making cool science happen?
I think people get consistently annoyed when blanket credit is given to eg Elon Musk regarding SpaceX (as though he's an aerospace god that did it all by himself), and when far too much credit was given to Steve Jobs regarding the iPhone. SpaceX is a case where Gwynne Shotwell deserves a chunk of the credit that is typically thrown at Musk by the media (because his stardom sells).
It's not that they don't deserve their fair share of credit, to be clear. It's that they do not deserve the level of overwhelming credit the media intentionally tries to bestow upon them, to create an idol that sells / generates clicks.
You pretty well see it in every thread regarding those two people. The hype train tries to give them credit, whether the media or fans, and other people get annoyed by it and call it out because it's obviously ridiculous to so overly credit such vast accomplishments requiring thousands of contributors to a given individual.
On the other hand, looking at git histories is basically how the social parts of engineering (e.g., money and power) at a place like Google works, at its fundamental level.
This has persisted for a very, very long time. I still remember when people would comment things like, "I worked with so-and-so unorthodox former Google employee, and he didn't commit code."
There are a lot of Googlers on HN. There are a lot of people who work at places that culturally align themselves with how that company runs.
It probably has something to do with why some women feel underpaid or unwelcome at these places.
It definitely has something to do with people commenting things like, "So is this the case of the product manager taking credit..." The tension between the product manager who "didn't do anything" and the engineer who "did all the work" and how the "org" sees that and measures "performance" are all swimming in the back of HN people's heads when they snipe some random academic.
Settling the score in a way so reductive is extremely appealing. But at least in duels, the other person gets to fire back.
But why would she be assumed to be the product manager (or its equivalent in the academic realm)? She has a doctorate in Computer Science from MIT, so she clearly has the technical chops. And she's in the early stages of her academic career, so she hasn't reached the point where she would have the ability to claim grad students' work as her own (which would also be a huge ethical lapse, though apparently it does happen [often with women as the victims]).
In my experience, people don't start looking into these things without some other suspicion. In a work setting, that would be things like impressions of poor productivity, claimed output not matching perceptions of competency, etc. But those involve a ton of data points, based on direct interactions with the person. In this case, the article gives us the following demographic data points:
- 29 years old
- Woman
- Computer Science doctorate from MIT
- Assistant professor of computing and mathematical sciences at the California Institute of Technology
Which of those data points suggests that her work output should be questioned?
Bullshit, this discovery was front page news worldwide. It's ridiculous on its face to assert that recognition isn't given to male scientists, given how many famous male scientists there are.
People do the analog of this all the time in cases where they believe an individual is receiving disproportionate accolade for a team effort. It's extremely common in entrepreneurial situations. As a typical example, take Elon Musk. Whenever anybody directly attributed e.g. SpaceX's rockets to him, they'd often be quickly "corrected" by a rather large number of people. Elon himself would also go out of his way to emphasize that the things "he" had achieved were largely a product of the great people that made up his team.
I think people let their own personal biases destroy their impartiality. Replace her name with Musk, algorithm with science/engineering, and 'image of a black-hole' with reusable rocket. The article would (and does) read like something posted by a sycophantic fanboy. It wouldn't be doing him any favors, and certainly isn't doing her any favors. However, I also do not think this article is representative of her in any way, shape, or form.
For instance it tries to frame things in the most narcissistic way possible. They found one image posted where the developer stated, "Watching in disbelief as the first image I ever made of a black hole was in the process of being reconstructed." So she made that image. Not a team, not the project of a coordinate global effort, no - she made that image. Even the image framing itself is indicative. It's a tiny out of focus image of a laptop and a giant in focus image of her with an artificial pose. The article itself continues with a similar narrative in all the eye-catching spots such as the headline and image captions: "Katie Bouman designed an algorithm that made the image possible" "Katie Bouman: The woman behind the first black hole image", and so on.
But as mentioned, I doubt this is indicative of the developer herself. She's probably just being used by the media. She's attractive, young, and has the right genitalia = stories that'll get a million clicks and shares = $$$. When you actually read the very small number of quotes from her, they seem much more realistic and in stark contrast to the media sensationalism:
- "No one of us could've done it alone. It came together because of lots of different people from many different backgrounds."
- "We're a melting pot of astronomers, physicists, mathematicians and engineers, and that's what it took to achieve something once thought impossible".
Of course they would. At least some feminists would. Even now some Nobel prizes (e.g. Watson for DNA structure) and the credit for some discoveries attributed to some men is disputed by some feminist activists. I hate all this identity politics though.
Also if she was a man her story and contribution wouldn't be as sensationalized as has been done.
Outside of the serious tech community, no one knows who Linus Torvald is. They do know that Linux exists.
Well, I take part of that back. He did have some personal pieces about "he's the guy that's a bully of the project" (when he took a personal hiatus from the project)
I mean if she was a man I doubt two members of congress would've have bothered to comment or that there'd be article after article focusing on them specifically.
Do you think a man in a similar position would be elevated on a pedestal, to the exclusion of the algorithm designer (Mareki Honma) and the primary software author (Andrew Chael)?
Who gets elevated on a pedestal and who actually did the work appear to be so badly correlated across the entire breadth of human cultures that you would assume that they only share the most tenuous of causal linkages.
Readit News
> Very long baseline interferometry (VLBI) is a technique for imaging celestial radio emissions by simultaneously observing a source from telescopes distributed across Earth. The challenges in reconstructing images from fine angular resolution VLBI data are immense. The data is extremely sparse and noisy, thus requiring statistical image models such as those designed in the computer vision community. In this paper we present a novel Bayesian approach for VLBI image reconstruction. While other methods often require careful tuning and parameter selection for different types of data, our method (CHIRP) produces good results under different settings such as low SNR or extended emission. The success of our method is demonstrated on realistic synthetic experiments as well as publicly available real data. We present this problem in a way that is accessible to members of the community, and provide a dataset website (vlbiimaging.csail.mit.edu) that facilitates controlled comparisons across algorithms.
What strikes me as really amazing is the cross functional nature of these modern achievements. I did not realize that this image was created with statistical image models and a Bayesian approach.
Also, this link included -> http://vlbiimaging.csail.mit.edu/ introduces the field and offers a good explanation for those interested in learning more:
> Imaging distant celestial sources with high resolving power requires telescopes with prohibitively large diameters due to the inverse relationship between angular resolution and telescope diameter. However, by simultaneously collecting data from an array of telescopes located around the Earth, it is possible to emulate samples from a single telescope with a diameter equal to the maximum distance between telescopes in the array. Using multiple telescopes in this manner is referred to as very long baseline interferometry (VLBI).
[0]: https://en.wikipedia.org/wiki/Synthetic-aperture_radar
Not trained in this field, but this reads like a certain mistype. Shouldn't resolution increase with telescope diameter?
Reconstructing Video from Interferometric Measurements of Time-Varying Sources https://arxiv.org/abs/1711.01357
So maybe we will also see a video of a black hole, soon.
"Then they spent the two years parsing literal truckloads of data, some of which had to be shipped on hard drives from the South Pole and defrosted outside a supercomputer facility at MIT."
https://www.washingtonpost.com/science/2019/04/10/see-black-...
I'd love to read more about this if anyone has an article with more details.
Found it through this pdf: https://fskbhe1.puk.ac.za/people/mboett/Texas2017/Doeleman.p...
https://youtu.be/BIvezCVcsYs
Do you understand Katie's explanation?
They have a sparse set of data that is part of an image. They have trained a model to look at the sparse set and make an educated guess about what the full image looks like. They do this by feeding it full images.
The full images you feed into the model thus have an effect on the final image generated. In order to see how large that effect is, they trained different versions of the model with different sets of complete images. Some were images of what we thought a black hole looked like. There is potential that this heavily influences the model and ensures that the output looks like what we expect it to, even if that isnt actually true.
They also trained the model with non-blackhole images. Since the output of the model was approximately the same, this indicates that the resulting output picture doesnt look like what we think a black hole looks like just because it was trained with black hole images. It likely really looks like that.
The model doesn't need to be told what a black hole looks like. The sparse measurements combined with knowledge of how sparse data can be combined to form a generic image is enough. The model learned that the sparse data is not likely pure noise, instead there are shapes and lines and gradients that relate the sparse data points to each other.
Her analogy of sketch artists is good. If you have a functionally complete description and give it to 3 sketch artists from different cultures who are used to different looking people, they will still draw the same person. However if your description isnt actually detailed enough, their sketches will significantly differ as they use their existing knowledge and bias to fill in the gaps with what they think is likely.
If Katie was a man do you think people would be going through git histories and their published papers trying to determine if she is being over-credited for her achievements?
Edit: I just checked Twitter, apparently there are thousands of idiots who believe this "850,000/900,000 lines written by Andrew, therefore he wrote the algorithm" narrative. It's amazing how willing people are to eat up a low-hanging narrative as long as it confirms their world-view. All it takes is a very crude understanding of how software development works to see through this narrative.
[0] https://www.reddit.com/r/unpopularopinion/comments/bbykvf/ka...
[1] https://www.reddit.com/r/pics/comments/bbuvff/this_is_andrew...
[0] https://www.reddit.com/r/pics/comments/bbql1i/this_is_dr_kat...
You might as well credit the Linux operating system to only a single man, whose effort is certainly largely responsible, but for who also does not in any way represent the whole of effort.
It's the ship of Theseus all over again.
That said, if Katie was a man, her story would not be as groundbreaking in a social context, and thus she would not be as celebrated.
Dead Comment
I have no idea why people are doing this to Dr. Bouman, if it's gender related or not. Just stating my experiences.
Just so I'm fair, my GitHub does suck.
Dead Comment
It's not that they don't deserve their fair share of credit, to be clear. It's that they do not deserve the level of overwhelming credit the media intentionally tries to bestow upon them, to create an idol that sells / generates clicks.
You pretty well see it in every thread regarding those two people. The hype train tries to give them credit, whether the media or fans, and other people get annoyed by it and call it out because it's obviously ridiculous to so overly credit such vast accomplishments requiring thousands of contributors to a given individual.
On the other hand, looking at git histories is basically how the social parts of engineering (e.g., money and power) at a place like Google works, at its fundamental level.
This has persisted for a very, very long time. I still remember when people would comment things like, "I worked with so-and-so unorthodox former Google employee, and he didn't commit code."
There are a lot of Googlers on HN. There are a lot of people who work at places that culturally align themselves with how that company runs.
It probably has something to do with why some women feel underpaid or unwelcome at these places.
It definitely has something to do with people commenting things like, "So is this the case of the product manager taking credit..." The tension between the product manager who "didn't do anything" and the engineer who "did all the work" and how the "org" sees that and measures "performance" are all swimming in the back of HN people's heads when they snipe some random academic.
Settling the score in a way so reductive is extremely appealing. But at least in duels, the other person gets to fire back.
In my experience, people don't start looking into these things without some other suspicion. In a work setting, that would be things like impressions of poor productivity, claimed output not matching perceptions of competency, etc. But those involve a ton of data points, based on direct interactions with the person. In this case, the article gives us the following demographic data points:
- 29 years old
- Woman
- Computer Science doctorate from MIT
- Assistant professor of computing and mathematical sciences at the California Institute of Technology
Which of those data points suggests that her work output should be questioned?
I think people let their own personal biases destroy their impartiality. Replace her name with Musk, algorithm with science/engineering, and 'image of a black-hole' with reusable rocket. The article would (and does) read like something posted by a sycophantic fanboy. It wouldn't be doing him any favors, and certainly isn't doing her any favors. However, I also do not think this article is representative of her in any way, shape, or form.
For instance it tries to frame things in the most narcissistic way possible. They found one image posted where the developer stated, "Watching in disbelief as the first image I ever made of a black hole was in the process of being reconstructed." So she made that image. Not a team, not the project of a coordinate global effort, no - she made that image. Even the image framing itself is indicative. It's a tiny out of focus image of a laptop and a giant in focus image of her with an artificial pose. The article itself continues with a similar narrative in all the eye-catching spots such as the headline and image captions: "Katie Bouman designed an algorithm that made the image possible" "Katie Bouman: The woman behind the first black hole image", and so on.
But as mentioned, I doubt this is indicative of the developer herself. She's probably just being used by the media. She's attractive, young, and has the right genitalia = stories that'll get a million clicks and shares = $$$. When you actually read the very small number of quotes from her, they seem much more realistic and in stark contrast to the media sensationalism:
- "No one of us could've done it alone. It came together because of lots of different people from many different backgrounds."
- "We're a melting pot of astronomers, physicists, mathematicians and engineers, and that's what it took to achieve something once thought impossible".
Also if she was a man her story and contribution wouldn't be as sensationalized as has been done.
Well, I take part of that back. He did have some personal pieces about "he's the guy that's a bully of the project" (when he took a personal hiatus from the project)
Dead Comment
Dead Comment
So yes, that happens a lot.
>to the exclusion of the algorithm designer and the primary software author?
How can you possibly infer that from a git history?