Please forgive me for asking a controversial question (particularly so early in the morning), but if there are all of these biological correlations with race, what does it mean that “race is a social construct”? Is the idea that black people have greater bone mineral density (per TFA) due to social or environmental causes (e.g., diet)? For what it’s worth, I’m a staunch egalitarian and I don’t see that changing either way.
EDIT: Really pleased with the largely constructive conversation in this thread. Was worried that this was going to be coopted as an ideological flame thread. Thanks for the insightful answers and good faith engagement. Keep up the good work!
From what I understand, the definition of a social construct is a bit nuanced, and it's possible to both believe race is a social construct and that it correlates with things.
As a metaphor, let's say we create two categories in the world for all people - tallers (people above six feet in height) and shorters (people below six feet in height). Human height objectively exists, but these categories are social constructs. Likewise, human variations in genes based on ancestry clearly exists, but the discrete racial categories we define (black, white, asian, etc.) are social constructs since we could create other discrete categories (Irish, Slavic, etc.).
So saying race is a social construct does not mean your genetic make up does not matter or correlate with anything, but that grouping people into the set of commonly agreed upon races is not the inherent way it has to be. At the same time, these groupings do represent distinct genetic make up and so correlate with physical attributes. It's just that different groupings with different correlations are also possible.
So if you were going to create a test for "Is this a social construct?" What would it be?
It seems like the definition from your comment would be "Could we take this labeling system, and define different labels or the labels differently?"
I watched some of the youtube, and in the thought experiment she proposes she take a continuous trait(height) and arbitrarily splits it into two buckets. And talks about how this is kinda silly. But this is something we do all the time, with hypertension, diabetes, disabled, Alzheimer's, the 1%, capitalism, Canadian, etc...
And many of these categories are far more continuous than something like sex and gender which as far as categories go are pretty discrete.
Maybe these things are social constructs, but if they are then we surely must come to the conclusion that almost everything we care about in the world is a social construct.
But with the taller/shorter, you would commonly have two tallers producing a shorter, or two shorters producing a taller. If we were talking about husbandry, we would say that tallers and shorters don't "breed true".
But with race, you never have two white people produce a black person, or vice versa (and no, a black person with albinism is not a white person).
IMO it's because it's very dubious the AI is doing what the paper is maintaining it's doing.
People are not 'race x' or 'race y' biologically, as if race is some discrete set of features common to a whole population. Every individual has a set of biological features inherited from their ancestors which, theoretically, could include any or all so-called 'races'. Human beings have a continuum of features that is heavily interlaced amongst all the 'races'.
Putting it another way, if we were alien visitors, and had in front of us a representative sample of dead bodies from the entire world, we would be hard pressed to sort those bodies into 'races' based on biological features.
For example, we currently use melanin levels as a key indicator of 'race' today, but an alien, lacking the social context of the significance of say, high levels of melanin, may well consider it a secondary feature since its shared with otherwise unrelated people
>> People are not 'race x' or 'race y' biologically, as if race is some discrete set of features common to a whole population.
Clearly people are biologically different based on race and the AI here is picking up on that. My kids orthodontist even told me they align teeth in part based on race. The Asian arch is flatter across the front for example. I asked about this because an engineer I worked with had a father in dentistry and told me my kid had "German teeth in an Irish mouth" which matched her ancestry, which he didn't know - just said that in response to my description of the crowding.
So YES, races have biological differences. If not, we wouldn't be able to tell where people are from. I get that it's not cool to discriminate based on race, but it's not OK or even practical to deny that it exists (see dentistry example above).
Aliens may as well sort us by the colour of our pants as soon as the colour of our skin, but let's not pretend that such an obvious visual variation as skin tone would be overlooked.
White cat, tabby cat, grey cat, etc? We don't try to say one sort of cat is better than other, but we can tell them apart very well.
Maybe what you're saying is that the aliens would not have the same prejudices associated with that marker as we have.
> People are not 'race x' or 'race y' biologically, as if race is some discrete set of features common to a whole population. Every individual has a set of biological features inherited from their ancestors which, theoretically, could include any or all so-called 'races'. Human beings have a continuum of features that is heavily interlaced amongst all the 'races'.
Exactly. One of the people I'll always point to as evidence that race is a social construct is Barack Obama. He was the "first black president". In reality he is, genetically, as "white" as he is "black". We still call him black because of the color of his skin.
If people insist for long enough that racial categories are inherently biological you'll eventually end up in one-drop judgement territory. Not a great place to have a discussion.
That race is mostly a social construct, does not abnegate the fact that it can also exist physiologically.
'Aliens' visiting Earth would immediately categorize us into groups very crudely resembling the groupings we use today, because our visible characteristics are the most immediately obvious artifacts of our existence.
(Edit: when I say 'immediately' I'm indicating this would be an obvious, first order thing to do from the first pictures they have of us, not an 'Enlightened Alien' scientific form of categorization).
If all they had were 'pictures' of us, the race categories we use would be the obvious grouping, or something resembling that.
They would see that most of the people in Sub Saharan Africa looked quite different from those in East Asia. (And difference between Sub-Saharan Africans and East Asians is more than 'melanin').
There's a 'continuum' between every biological grouping, that doesn't mean those categories don't exist. It just means we're going to argue a lot about where and how to draw the lines.
Race as a 'Social Construct' relates to all of the other attributes that we associate with race, and individual lived experiences due to how they are perceived etc..
To your point, Aliens wouldn't immediately pick up on the 'Social Construct' bit, at least not right away and so they wouldn't have the prejudices that we do, but if they could only observe from afar, they would see exactly what we see, and visual distinctions would be the 'first order of separation' even if it was, after further understanding (i.e. genetics) a less important distinction as you hint.
Edit: someone provided this like I'd like to also include it [1] which illustrates some of the current debate over the notion of race, and that it's clearly politicized.
Race in the context of physical differences does not map to the social concepts of race.
For example, US documents usually include Latino as a possible race, even though Latin Americans are white, black, indigenous Americans, or a mix - and Spaniards are what usually would be classified as white. If you check older forms you'd see Italians and Irish people categorized as a different (non white) category, etc.
Race is distinct from Nationality. "Latino" is not a synonym for citizens of Latin American countries, but the name of a racial group which originated from that region of the world.
It gets confusing with countries like China, Japan, and India, which are more racially homogeneous, and where the country name is the same as the (common) name of the predominate racial groups.
You've apparently asked a very interesting question, judging by the volume of replies!
The other replies here are mostly good, but I'd also like to note that "race is a social construct" refers to how "races" aren't really objective categories (What defines if somebody is "white"?) and more of a subjective thing, particularly at the margins. We can build classifiers that can match most people's (in our current cultural context) perceptions most of the time, but that doesn't make it a rigid natural phenomenon.
For example, I could build a classifier that looks at household finances and decides if people are lower, middle, or upper-class. I'd bet that I could get it good enough that most people off the street would agree with the results most of the time. However, that doesn't make "social class" some sort of objective, unchanging, universal truth. Somebody from 100 years ago would probably find us all to be upper-class. Somebody from the far-flung future would (hopefully) find us mostly to be near-destitute.
> You've apparently asked a very interesting question, judging by the volume of replies!
Agreed, and upvotes too! It seems like I've struck upon something people have been interested in talking about.
> The other replies here are mostly good, but I'd also like to note that "race is a social construct" refers to how "races" aren't really objective categories (What defines if somebody is "white"?) and more of a subjective thing, particularly at the margins. We can build classifiers that can match most people's (in our current cultural context) perceptions most of the time, but that doesn't make it a rigid natural phenomenon.
I certainly believe this to be the case, but when I hear "race is a social construct" it's almost always in the context of denying biological differences between the races in the same way that some extreme (though mainstream and influential) people take "gender is a social construct" to mean that literally all differences between the sexes are socially constructed including height, weight, strength, etc (otherwise known as "blank slatism").
That said, unlike biological sex, there are fewer valid social implications that we can draw from race (e.g., there are a bunch of social implications which fall out from women's unique ability to bear children, but no analogues which fall out from race) and we have drawn many false implications from race which have been tremendously harmful to individuals of different races, so if we have to reduce everything to a slogan or a binary (as our simplistic society increasingly demands), then "race is a social construct" isn't a bad one.
It can be both real physiological differences and a social construct, as "what level of bone density tips you from x to y" becomes a question, as does "is an x person with calcium deficiency actually a y person" sort of things that are obviously not the case.
As someone with a scientific bent who is of the left, I always find it incredibly frustrating when people say “x is a social construct”, because it’s technically true, but also utterly elides the dual nature of the category. Race is a social construct that can be used to infer true things (probabilities) about the real world! Other social constructs that have this property: sociology, economics, physics…
This isn’t to say a lot of people who are into race science don’t wildly overstate their claims, but there isn’t literally nothing to it.
How race is a social construct: it was imposed as an idea for reasons that were not connected to physiological fact (although often claiming the contrary at the time, the claims were the same kind of made up BS as phrenology, another idea from the same time period).
There are biological correlations with inherited genetic lineage. That only has weak correlation with assigned race. Lots of different lineages of people are "black", Africa is big and diverse. Lots of different lineages of people are "white". But for example, if your lineage is from a malarial region, your chances of being a genetic sickle cell carrier are higher. Most people from those areas are dark skinned, so it correlates with being "black".
Also there's the impact of racism, which affects everything from nutrition to poverty to pollution exposure, and does so on an individual, a regional, and a national scale. And this has biological consequences.
The problem is the question itself, both the one you asked and the one the researchers asked. Is it recognizing ethnicity or race? Clumping ethnicities as "race" is the social construct. It's not just skin that is different between people. If a certain culture prizes certain features in people, people with those traits will breed more, thus amplifying biological features within a socio-cultural group.
My question is, using medical imaging can it particularly identify say people of african origin but it cannot tell apart east vs west africans? Can it uniquely identify asians but can it not tell apart an indian person from a Korean? Or given proper training can it discern between north and south koreans or between a french person and a greek person?
Race is a social construct not because groups of people are all identical but because both science and major-religion have concluded that humans share a common human(homosapien) ancestry, therefore there is one human race and multiple ethnicities and geographical super-ethnicities (south east asian and north european for example).
Edit: This is also why race on id cards is silly. Not because you don't want to identify people based on appearance but because ethnicity is more granular and leads to less confusion. Would it be more identifying to say indian or asian? African or north-african?
I strongly believe the modern black/white/asian "race" is a darwinian invention to try and understand and classify nature better, based on intuition instead of science.
Nature does not create "bins" to sort people or things. Some traits are clustered and we latch on the most visible clusters to define "races".
In reality, there is broad overlap and if you look up close, the whole concept becomes hairy. Someone with a father of Scandinavian descent and a mother with African lineage, what is that person, black, white, 50/50?
It's the same with gender/sex. While the biological substrate is clearly variable, the categories are social.
Not quite: Sex is very clearly split into two distinct mechanisms. There's definitely sex-linked distributions of different traits, like male and female typical ranges of height, or agreeable personality, risk-seeking/risk aversion and so on, where both sexes operate the same mechanism.
Races are mostly clusters in variation within one mechanism - eg. skin color is largely a gradient of more or less melanin, and what gets selected for depends on the environment. It's not intrinsically linked to the whole cluster of race-typical traits, those just are in the same genetic bundle that gets inherited from generation to generation, and most of those traits can be mixed.
Sex itself has a sharp divide from the mechanisms themselves being completely different.
“Race is a social construct” does not mean that inheritable biological differences between groups people does not exist. It means that the categories we use to classify people into races are cultural. For example the "black" and "white" categories in the US.
"race" as we use it colloquially is a lot less fine grained than biological differences between humans. For instance there's not a monolithic genomic signature around "black" or "white" and yet those are categories we use for "race".
As someone pointed out in another reply, "races" are buckets people are distributed into, frequently based on physical aspects. The point of saying "race is a social construct" is the nature of those buckets.
Northern Europeans, Southern Europeans, and Eastern Europeans (or rather people descending from those areas) differ in a number of physical aspects. Are they different races? For those who think race matters today, it seems not---they are all "white". Back in the heyday of scientific racism in the first half of the 20th century, they absolutely were---that is why the US had different immigration limits for different parts of Europe. Are native Australian people the same race as Africans?---there's no especially close genetic relationship as far as I know.
Physical differences exist. How you use those differences to divide people into groups and, more importantly, how you treat people of those groups is a social construct.
If the software is told of the existence of race via an attribute in its ML training, then it will see race in additional data. Otherwise, it won't, IMHO.
Also, due to omission of many other variables (such as culture), those variables are being conflated with race. I personally think a lot of what is commonly accused of being racism, sexism etc. is really just "culturalism" or "preferentialism"... let me think of an example... Given two bars, one filled with rap music and the other filled with techno music, and I pick the techno one every time... am I racist (assuming the predominant race in these 2 bars differs)? Or just preferentialist, culturalist or (frankly) "techno-ist" (if that were a thing)?
Because I think it's much harder to get angry about preferences than it is to get angry about racism, I think that given a choice, we need to consider the less-triggery inputs to a perceived problem
if there are all of these biological correlations with race, what does it mean that “race is a social construct”?
I could divide people into 'tall', 'medium' and 'short' buckets. Someone else could categorize them all into just two categories (eg: 'tall' and 'short') instead, and someone else might argue that I've chosen bad cutoffs, and tell me I incorrectly put a bunch of 'medium' people in the 'short' bucket. People have differences, but the criteria by which we choose to group them is subjective.
That is only true if the distribution of things within the continuum is not "lumpy". For height it isn't, but for other things it is. For example gender is highly bimodal, yet still a continuum. Would you say that "male" and "female" are subjective socially constructed categories? Of course not.
Race is probably somewhere in-between. There are people all over the spectrum but there are pretty clear large groups with sparsely populated gaps in-between them.
From what I understand, race is a social construct doesn't mean that there can be no physical differences. After all, humans have been able to tell* what race someone is with their stupid human eyes. It's making a claim the important aspects of race are a social construct.
If the defining physical differences between races is both melanin and sternum width, that doesn't seem to be more relevant than just skin melanin.
* In many cases. It's more error prone than most people want to admit.
> humans have been able to tell* what race someone is with their stupid human eyes.
It's more that humans have been able to define, with significant disagreements, what race someone is with their stupid human eyes. Race is nothing but a collection of these judgments. I'm not brown because I'm black, I'm black because I'm brown.
Race, of course, is based on physical traits. But the groupings and boundaries are culturally determined and very arbitrary. Minor nose and cheek shape differences are obvious to the Hutu and Tutsi (enough they massacred each on the premise they are different racial groups) but to the average American they're both just black. Similarly, a century ago, it was entirely obvious to the British and the Italians that they were different races of people, although closely related. Today that idea is completely alien in America, where they're both white.
Perhaps the best example is the American views on what defines blackness. Because I grew up in a community where mixed race white/black was seen as a distinct race from white or black, I have a very hard time interpreting race the way Americans do sometimes -- Barrack Obama and Kamala Harris's mixed parentage is obvious to me, which makes them obviously not black to me (think like in the same way Obama is obviously not white) and I have a hard time wrapping my head around Americans seeing them as such, but apparently they do, since even a small amount of physical traits that suggest recent African descent categorizes you as black there.
Yes, some anatomical differences are certainly affected by environment, but the quoted phrase is typically used in the context of treating races differently based on behavior ("X people are immoral", "Y people are stupid"). It's meant to encourage people to treat each other with equity and consider that such observed differences may be distorted by news, your like-minded cohorts, sociological conditions, etc, which are all theoretically addressable. But maybe I'm misunderstanding your question, since clearly the idea that races have anatomical differences is not contested by anybody - after all, anatomical difference is the very thing we base the word "race" on, colloquially. You don't need non-obvious differences to see that, since we already have obvious differences.
> since clearly the idea that races have anatomical differences is not contested by anybody
This is not true. I had a college professor explain the "race is a social construct" idea, and her position was staunchly that there was _no_ biological basis for race. See also this article by the scientific american[1]:
> Today, the mainstream belief among scientists is that race is a social construct without biological meaning.
This is the idea that GP is responding to - clearly there must be some biological basis for race, if an AI can determine race from an x-ray.
The genetic differences between different people in Africa are far greater than the genetic differences between black and white people in North America. The term "race" as used to describe belonging to a different groups which generally experience their lives vastly differently to other groups is thus inherently social. The hypothesis goes as follows - if black and white people are so similar to each other biologically, why are their life experiences so different? It would seem that there's more to it than just the biochemical composition of our bodies that dictates what kind of a life you'll have.
Differences between African groups being bigger than between blacks and whites in NA don't make the rough categorization no longer work. For example, there's rather little genetic variety within dogs, or between dogs and wolves, but that doesn't say much about how impactful the variation is: Chihuahuas and Huskies are genetically really similar to each other, but no one in their right minds would disagree that you can comfortably put them in different, very real, very genetics-based buckets.
In general, arguments of variation within a group are not arguments against considering between-group differences, or those between-group differences being real. The variation within a category cannot be dismissed, though, it's hugely important for understanding the world properly and for guiding personal conduct.
> The genetic differences between different people in Africa are far greater than the genetic differences between black and white people in North America.
This is often repeated, but the point of the question is that the OP calls this into question.
If you consider it from the perspective that race and culture are typically conflated into just race, then "race is a social construct" is fairly logical.
The differences exist on a spectrum and there’s no hard line between one or the other. Yes, as you mention, there are biological correlations. But at the edge cases, it’s quite subjective. How many races are there? Ask a different person and they’ll have a different list. Is Asian a race or is that ridiculously general? etc etc
Biologically, forget about bone density etc. Skin color, facial features, face shape, etc are discernable directly.
Race as a social construct is the idea (originally rooted in imperialism and colonialism) that certain races are inferior mentally and societal development wise. A few Brits saw Africans residing in huts and living on farming using primitive tools and concluded that they could not develop any further than that and that their brain development was limited. (Of course, this was perpetrated to enable guilt free enslaving and "civilizing" them and exploiting them for labour. I am sure that if African societies were simply introduced to western civilization and allowed to trade and travel, the ideas from west would have been adopted and assimilated much quickly)
So, biologically, races are distinct, identifiable and have evolved to meet the needs of their local environment. But socially, races as inferior or superior was perpetrated with ulterior motives and have been shown as false time and again.
Basically groups may have physical characteristics. European nations starting from at least the 1500s began using these physical/religious groupings to mark certain groups for predatory expropriation and premature death.
To quote Ruth Wilson Gilmore: “The racial in racial capitalism isn’t secondary, nor did it originate in color or intercontinental conflict, but rather always group-differentiation to premature death. Capitalism requires inequality and racism enshrines it.”[1]
Cedric J. Robinson (among others) have discussed how capitalism and racialization are continually co-created.
1. Abolition Geography and the problem of innocence, in Futures of Black Radicalism.
Racism - judging people by the colour of their skin - has been on life support for a long time [1] in most liberal republics/democracies but nefarious actors are attempting to revive it in the name of identity politics. It is this type of race which can be detected using X-rays. The same group tends to support the doctrine of cultural relativism [2]. Claiming all cultures are equally valid they call criticism of behaviour related to specific cultural traits "racism", mangling the definition of that term in the process. Cultures, by definition, are social constructs.
Cultural relativism is not the norm and should not become such since there are differences between cultural traits where it is possible to state that some are objectively better than others. As an example, the cultural trait of genital mutilation is objectively worse than that of leaving girls' bits alone - and I'm open to stating the same about boys even though that would raise up a storm of protest. The cultural trait of parents marrying off their offspring without said offspring having a say in the matter is objectively worse than than that of having the offspring decide for themselves who they want to share their life with. The cultural trait of having people who achieved success within the bounds of the law - whether those be inventors, writers, athletes, successful farmers, builders or architects or anything else - is objectively better than that of having successful criminals and hoodlums as role models - yes, "street culture" with gang bangers as role models is objectively worse than whatever name can be given to cultures which have/had those inventors (etc) as role models.
X-rays can not be used to detect whether you might mutilate your newborn's genitals, marry off your 5yo daughter to your 20yo nephew or leave your children to be raised by the local street gang leaders since these traits do not depend on the colour of your skin even though there is often a correlation; correlation does not imply causation [3]. Take for example Michael Skråmo [4], a Swedish-Norwegian man who very much looked the part of such but ended up as a recruiter for islamic state in the Nordic countries. Contrast him to e.g. Luai Ahmed, a Yemeni refugee who lives in Sweden and is a vocal critic of everything Skråmo stood for. It was not Skråmo's white skin and blonde hair which made him ready to pick up a Kalashnikov, it is not Ahmed's brown skin and black hair which made him averse to the negative cultural traits related to islam.
MLK was right when he longed for a society where people would be judged on the content of their character and we were well on our way of achieving that goal. Unfortunately there are those who derive their identity - and income - from their purported position as fighters against racism (without scare quotes), a fight which was nearing its conclusion. While most old soldiers fade away [5] some have taken it upon themselves to revive their old enemy so as to keep their purpose - and income - alive. Their culture is not mine and I consider it to be objectively worse than, e.g. MLK's. If you then consider that MLK was a "black" man while I am of north-west European descent and as such have "white" skin the truth becomes clear, it is not the colour of our skin which makes us alike - it is the content of our character.
Race is not a social construct. Culture is. Nature is not a social constrict, Nurture is.
I initially had this same belief, but I realized I had such a limited understanding of race after reading the wikipedia article on it [0]. That we divide not based on actual genetics but visible markers makes it a social construct. And the way we've chosen to look at race politically has changed over time.
"Because the variation of physical traits is clinal and nonconcordant, anthropologists of the late 19th and early 20th centuries discovered that the more traits and the more human groups they measured, the fewer discrete differences they observed among races and the more categories they had to create to classify human beings. The number of races observed expanded to the 1930s and 1950s, and eventually anthropologists concluded that there were no discrete races.[93] Twentieth and 21st century biomedical researchers have discovered this same feature when evaluating human variation at the level of alleles and allele frequencies. Nature has not created four or five distinct, nonoverlapping genetic groups of people."
I think everyone would agree that there are some pretty obvious biological racial markers that differentiate racial groups. Skin color comes to mind :)
The social construct argument just says that the specific categories and lines we draw are fairly arbitrary. Why is an afghani middle eastern (white?) but a Pakistani south-Asian? Is a Russian from Vladivostok really more closely related to a brit than a Mongolian? Idk - but, to me, the social construct argument just says “who cares? The specific groupings are pretty arbitrary anyway”
I think we can agree that thanks to our minds we have leaped over our physical differences encoded in our DNA. Evolutionary differences have little influence on our survival and reproduction, socio-economic constructs and technology has pretty much taken over that role.
The funny thing is that it isn't really arbitrary at all.
It's location based. Humans only recently gained the ability to travel vast distances, and in the past lived (and bred) within a small, localized region. The "arbitrary" location based ethnicities actually do reflect that genetic ancestry
Of course, really wide ones like black/white/asian lose some of their meanings.
It's not a surprise that race effects how someone looks. It doesn't take a genius to see things like different colored skin, different nose, or height differences...
The question is what about the looks of a chest X-ray are connected to race. I agree with the research here, it's non obvious what is being extracted by the AI.
If I had to guess, maybe something about the quality of the scan itself. Perhaps one race was scanned at one particular hospital, vs a different hospital scanning a different race. Then it's just picking out the different scanner.
Or: Race is a rough proxy for breeding populations that have been separated for long time, and subject to different environments' selection pressure. Over time, selection and blind luck build up all manner of small differences, which both observably exist at the surface (so why not inside the body) and that you'd expect to exist from basic evolutionary principles. You'd expect a Norwegian forest cat, a Saharan desert cat and a Burmese to be different in innumerable small ways because they grow up in totally different environments. You'd also expect there to be a lot of overlapping, well, catness to them all. Lions purr, after all. There's nothing complicated about any of it, humans just tie themselves into knots when it comes to humans in a way they don't when it comes to cats.
With that said, the simple explanation is that the AI picks up on these small patterns in a way humans don't. The brain and neural networks are fundamentally pattern-recognition engines. The AI is just seeing something we don't either notice or can't see.
The way we interpret race is largely a social construct due to 'apparent' physical differentiation, less so underlying biological or genetic factors.
We end up segregating ourselves for a variety of reasons, in which case groups that are physically very distinguished end up forming almost an ethnic basis.
For example, two groups with varying genetic makeup and maybe a number of non-obvious biological differences, but who otherwise looked identical - would have the similar life experience in terms of their social treatment by other groups.
But irrespective of how a person is socialized - if you're Black, people are going to treat you one way, and if you're White, people are going to treat you a little different. That 'lived experience' differential is a somewhat unavoidable.
The degree of that variability is obviously debatable, but surely it exists to some degree.
I suppose you could make a parallel in ethnicity: a century ago, the difference between a Scottish-American and an English-American would have been apparent by lineage, accent, Church affiliation, and that might have affected relationships, status etc..
Whereas after a few generations of integration, there is definitely 'no' (or not much) difference between those two groups, and no vector for differentiation/discrimination. The historical ethnic situation was a 'social construct'.
That said, some of the argumentation used to promote the idea that there is no genetic basis for race is a little odd, the 'Africans have more genetic variation than other groups combined' is often used, but frankly I do not understand how that doesn't mean there are material differences between them and other groups.
And of course there is no 'hard line' between groups, but there is also no 'hard line' between the Scottish and English, there are many people who have attributes of both cultures, but that doesn't negate the existence of either group.
I think we're a bit oversensitive these days to these issues. Systematic racism exists and we should think about it, but that doesn't mean there's a boogeyman behind every door.
I think in this case it's also worth examining what exactly the AI is finding out, because it may not be just 'bone marrow'.
I see a lot of "oh it's probably just picking up on x y z" when x, y, and z are things they explicitly checked for:
1) "It's probably just the names or other metadata" – they only gave it pixel data to train on. To control for things like metadata overlaid on the image (e.g., a name written on the image) they divided the images into 3x3 sections and trained classifiers on each section separately.
2) "It's probably some artifact of how the hospital marked up the images" – they used something like 7 different datasets from different hospitals and different modalities (X-Ray and CT).
If it is cheating somehow, it's not doing it in an obvious way that you can think of in a minute or two. Also note that they had more than just medical folks working on the paper; the author list includes plenty of computer scientists. It's unlikely they're making an elementary ML mistake here.
One major risk source I see is that the size of the training data for the races isn't the same. For white vs. black patient data, there's between a 2:1 and 3:1 ratio bias in both the training and test data (and a much higher ratio bias for Asian... as high as 20:1 in some of these categories).
This gives the CNN more information on one race than another, which can create a classifier that performs very well on the training and test data it has access to but then flakes spectacularly on data outside the training set (because the source isn't representative of the total variance in the global population).
I don't see why this is necessarily bad. An ML model is picking up on subtle anatomical or physiological differences between races. So what, that doesn't automatically mean the AI is racist or biased...
It's not necessarily bad, if it's actually working.
The fact that it works on an 8x8 massively pixelated version of the x-ray points to the possibility that it's not actually working, which would be bad if you based patient treatment decisions on an training set that was actually teaching the AI something else entirely.
What do you mean, not working? That the AI was randomly choosing the correct race 82% of the time by luck?
I'm confused by what your implying because it would seem to me that the authors went through many steps to try to pinpoint how the AI was doing this identification and how baffling it was to everyone that even with a lot of x-ray information removed (8x8 pixels compared to say 4k), it somehow was still correctly picking the race.
What would this "something else entirely" that you are implying actually be?
Since it's effectively the same anatomical structure, presented in a grayscale image, I think bone density would change the average color of the image regardless of what level of pixelation was applied. The chance of detection through some other method isn't ruled out though. It could be hitting off of something mundane like the margins on the edge of an image, how centered the torso is in frame, foreign objects in the body from procedures, any random thing that could bias the data can also lead to false positives. The only way to verify would be to use a new data set, ideally from new hospitals, and see if it has similar results.
But it does mean that AI models trained on data might end up just perpetuating racial correlations even when you don't think it's aware of race. It's another example of why interpretability is important: the model might end up depending on correlations you don't mean it to.
We know that various features visible in medical images correlate with race, eg breast density, bone density, etc. Most likely the network is just learning a classifier on top of these features.
This is trivially verifiable but conspicuously absent from the paper.
> In some early results, the agent was doing well — suspiciously well. What tipped the team off was that, when the agent reconstructed aerial photographs from its street maps, there were lots of details that didn’t seem to be on the latter at all. For instance, skylights on a roof that were eliminated in the process of creating the street map would magically reappear when they asked the agent to do the reverse process...
I suspect there must be some confounder here - like the positioning used for the CXRs correlating with race, based on the methodology used in a particular region / hospital.
Seems the most likely explanation for it still working even when pixellated as 8x8?
That was my initial reaction as well but they validate on separate datasets from training which makes this unlikely.
The performance despite degradation may be the same phenomenon that results in adversarial examples that are indistinguishable to human eyes, ie we know that neural nets are highly sensitive to visually imperceptible differences.
It's not "conspicuously absent from the paper". They have a whole group of experiments on this: "Experiments on anatomic and phenotype confounders" and conclude "Race detection is not due to obvious anatomic and phenotype confounder variables."
I wonder if it has anything to do with the machines that are being used to take the images. Maybe some groups have access to one type of imaging machine where other groups have access to some other type of imaging machine.
Or some scans are ordered for a diagnostic that’s more common in one group over another. I hope that the details show that treatment and diagnostic quality are indistinguishable between race classifications.
We were marvelling at a surface shaded render made on a new Siemens MR from an T1 MPRage on a very still and compliant patient. It basically looked like a black and white photo (though with the tools at hand we could cut the image in half and look at the brain). You could see the facial hair and you could identify the patient if you knew them.
Medical imaging is moving along at pace and it would be interesting to see what could be inferred from a dataset of images of this quality.
AIs do not have magical abilities, I do not trust this result. AI can pick up though easily on technical artifacts. Something like a cofactor: Since they used different databases, maybe one dataset had a high number of people of one self declared race and the other the other self declared race; and each using a different intensity maximum or so.
They accounted for specifically intensity maximum, but your overall concern is solid; I don't see anything in the paper suggesting they accounted for a full spectrum of risk factors (broad-image noise, rotation, individual "stuck pixels" that could create a hard-to-spot thumbprint in the image, for example).
Simply testing in multiple external populations already rejects this hypothesis, unless you think they all had the same scanners with the same stuck pixels.
Readit News
EDIT: Really pleased with the largely constructive conversation in this thread. Was worried that this was going to be coopted as an ideological flame thread. Thanks for the insightful answers and good faith engagement. Keep up the good work!
As a metaphor, let's say we create two categories in the world for all people - tallers (people above six feet in height) and shorters (people below six feet in height). Human height objectively exists, but these categories are social constructs. Likewise, human variations in genes based on ancestry clearly exists, but the discrete racial categories we define (black, white, asian, etc.) are social constructs since we could create other discrete categories (Irish, Slavic, etc.).
So saying race is a social construct does not mean your genetic make up does not matter or correlate with anything, but that grouping people into the set of commonly agreed upon races is not the inherent way it has to be. At the same time, these groupings do represent distinct genetic make up and so correlate with physical attributes. It's just that different groupings with different correlations are also possible.
This video explains it pretty well IMO: https://youtu.be/koud7hgGyQ8
It seems like the definition from your comment would be "Could we take this labeling system, and define different labels or the labels differently?"
I watched some of the youtube, and in the thought experiment she proposes she take a continuous trait(height) and arbitrarily splits it into two buckets. And talks about how this is kinda silly. But this is something we do all the time, with hypertension, diabetes, disabled, Alzheimer's, the 1%, capitalism, Canadian, etc...
And many of these categories are far more continuous than something like sex and gender which as far as categories go are pretty discrete.
Maybe these things are social constructs, but if they are then we surely must come to the conclusion that almost everything we care about in the world is a social construct.
Dead Comment
But with race, you never have two white people produce a black person, or vice versa (and no, a black person with albinism is not a white person).
People are not 'race x' or 'race y' biologically, as if race is some discrete set of features common to a whole population. Every individual has a set of biological features inherited from their ancestors which, theoretically, could include any or all so-called 'races'. Human beings have a continuum of features that is heavily interlaced amongst all the 'races'.
Putting it another way, if we were alien visitors, and had in front of us a representative sample of dead bodies from the entire world, we would be hard pressed to sort those bodies into 'races' based on biological features.
For example, we currently use melanin levels as a key indicator of 'race' today, but an alien, lacking the social context of the significance of say, high levels of melanin, may well consider it a secondary feature since its shared with otherwise unrelated people
Clearly people are biologically different based on race and the AI here is picking up on that. My kids orthodontist even told me they align teeth in part based on race. The Asian arch is flatter across the front for example. I asked about this because an engineer I worked with had a father in dentistry and told me my kid had "German teeth in an Irish mouth" which matched her ancestry, which he didn't know - just said that in response to my description of the crowding.
So YES, races have biological differences. If not, we wouldn't be able to tell where people are from. I get that it's not cool to discriminate based on race, but it's not OK or even practical to deny that it exists (see dentistry example above).
White cat, tabby cat, grey cat, etc? We don't try to say one sort of cat is better than other, but we can tell them apart very well.
Maybe what you're saying is that the aliens would not have the same prejudices associated with that marker as we have.
Exactly. One of the people I'll always point to as evidence that race is a social construct is Barack Obama. He was the "first black president". In reality he is, genetically, as "white" as he is "black". We still call him black because of the color of his skin.
If people insist for long enough that racial categories are inherently biological you'll eventually end up in one-drop judgement territory. Not a great place to have a discussion.
'Aliens' visiting Earth would immediately categorize us into groups very crudely resembling the groupings we use today, because our visible characteristics are the most immediately obvious artifacts of our existence.
(Edit: when I say 'immediately' I'm indicating this would be an obvious, first order thing to do from the first pictures they have of us, not an 'Enlightened Alien' scientific form of categorization).
If all they had were 'pictures' of us, the race categories we use would be the obvious grouping, or something resembling that.
They would see that most of the people in Sub Saharan Africa looked quite different from those in East Asia. (And difference between Sub-Saharan Africans and East Asians is more than 'melanin').
There's a 'continuum' between every biological grouping, that doesn't mean those categories don't exist. It just means we're going to argue a lot about where and how to draw the lines.
Race as a 'Social Construct' relates to all of the other attributes that we associate with race, and individual lived experiences due to how they are perceived etc..
To your point, Aliens wouldn't immediately pick up on the 'Social Construct' bit, at least not right away and so they wouldn't have the prejudices that we do, but if they could only observe from afar, they would see exactly what we see, and visual distinctions would be the 'first order of separation' even if it was, after further understanding (i.e. genetics) a less important distinction as you hint.
Edit: someone provided this like I'd like to also include it [1] which illustrates some of the current debate over the notion of race, and that it's clearly politicized.
[1] https://en.wikipedia.org/wiki/Race_(human_categorization)
Deleted Comment
That doesn't mean race doesn't exist any more than the fact that height is a continuum doesn't mean that short people and tall people don't exist.
For example, US documents usually include Latino as a possible race, even though Latin Americans are white, black, indigenous Americans, or a mix - and Spaniards are what usually would be classified as white. If you check older forms you'd see Italians and Irish people categorized as a different (non white) category, etc.
It gets confusing with countries like China, Japan, and India, which are more racially homogeneous, and where the country name is the same as the (common) name of the predominate racial groups.
The other replies here are mostly good, but I'd also like to note that "race is a social construct" refers to how "races" aren't really objective categories (What defines if somebody is "white"?) and more of a subjective thing, particularly at the margins. We can build classifiers that can match most people's (in our current cultural context) perceptions most of the time, but that doesn't make it a rigid natural phenomenon.
For example, I could build a classifier that looks at household finances and decides if people are lower, middle, or upper-class. I'd bet that I could get it good enough that most people off the street would agree with the results most of the time. However, that doesn't make "social class" some sort of objective, unchanging, universal truth. Somebody from 100 years ago would probably find us all to be upper-class. Somebody from the far-flung future would (hopefully) find us mostly to be near-destitute.
Agreed, and upvotes too! It seems like I've struck upon something people have been interested in talking about.
> The other replies here are mostly good, but I'd also like to note that "race is a social construct" refers to how "races" aren't really objective categories (What defines if somebody is "white"?) and more of a subjective thing, particularly at the margins. We can build classifiers that can match most people's (in our current cultural context) perceptions most of the time, but that doesn't make it a rigid natural phenomenon.
I certainly believe this to be the case, but when I hear "race is a social construct" it's almost always in the context of denying biological differences between the races in the same way that some extreme (though mainstream and influential) people take "gender is a social construct" to mean that literally all differences between the sexes are socially constructed including height, weight, strength, etc (otherwise known as "blank slatism").
That said, unlike biological sex, there are fewer valid social implications that we can draw from race (e.g., there are a bunch of social implications which fall out from women's unique ability to bear children, but no analogues which fall out from race) and we have drawn many false implications from race which have been tremendously harmful to individuals of different races, so if we have to reduce everything to a slogan or a binary (as our simplistic society increasingly demands), then "race is a social construct" isn't a bad one.
I love thinking about future historians view of today, I think it's better and more useful than futurism.
This isn’t to say a lot of people who are into race science don’t wildly overstate their claims, but there isn’t literally nothing to it.
There are biological correlations with inherited genetic lineage. That only has weak correlation with assigned race. Lots of different lineages of people are "black", Africa is big and diverse. Lots of different lineages of people are "white". But for example, if your lineage is from a malarial region, your chances of being a genetic sickle cell carrier are higher. Most people from those areas are dark skinned, so it correlates with being "black".
Also there's the impact of racism, which affects everything from nutrition to poverty to pollution exposure, and does so on an individual, a regional, and a national scale. And this has biological consequences.
Deleted Comment
My question is, using medical imaging can it particularly identify say people of african origin but it cannot tell apart east vs west africans? Can it uniquely identify asians but can it not tell apart an indian person from a Korean? Or given proper training can it discern between north and south koreans or between a french person and a greek person?
Race is a social construct not because groups of people are all identical but because both science and major-religion have concluded that humans share a common human(homosapien) ancestry, therefore there is one human race and multiple ethnicities and geographical super-ethnicities (south east asian and north european for example).
Edit: This is also why race on id cards is silly. Not because you don't want to identify people based on appearance but because ethnicity is more granular and leads to less confusion. Would it be more identifying to say indian or asian? African or north-african?
I strongly believe the modern black/white/asian "race" is a darwinian invention to try and understand and classify nature better, based on intuition instead of science.
In reality, there is broad overlap and if you look up close, the whole concept becomes hairy. Someone with a father of Scandinavian descent and a mother with African lineage, what is that person, black, white, 50/50?
It's the same with gender/sex. While the biological substrate is clearly variable, the categories are social.
Races are mostly clusters in variation within one mechanism - eg. skin color is largely a gradient of more or less melanin, and what gets selected for depends on the environment. It's not intrinsically linked to the whole cluster of race-typical traits, those just are in the same genetic bundle that gets inherited from generation to generation, and most of those traits can be mixed.
Sex itself has a sharp divide from the mechanisms themselves being completely different.
Deleted Comment
Northern Europeans, Southern Europeans, and Eastern Europeans (or rather people descending from those areas) differ in a number of physical aspects. Are they different races? For those who think race matters today, it seems not---they are all "white". Back in the heyday of scientific racism in the first half of the 20th century, they absolutely were---that is why the US had different immigration limits for different parts of Europe. Are native Australian people the same race as Africans?---there's no especially close genetic relationship as far as I know.
Physical differences exist. How you use those differences to divide people into groups and, more importantly, how you treat people of those groups is a social construct.
Dead Comment
Also, due to omission of many other variables (such as culture), those variables are being conflated with race. I personally think a lot of what is commonly accused of being racism, sexism etc. is really just "culturalism" or "preferentialism"... let me think of an example... Given two bars, one filled with rap music and the other filled with techno music, and I pick the techno one every time... am I racist (assuming the predominant race in these 2 bars differs)? Or just preferentialist, culturalist or (frankly) "techno-ist" (if that were a thing)?
Because I think it's much harder to get angry about preferences than it is to get angry about racism, I think that given a choice, we need to consider the less-triggery inputs to a perceived problem
Dead Comment
Race is probably somewhere in-between. There are people all over the spectrum but there are pretty clear large groups with sparsely populated gaps in-between them.
What race is Obama? And how is your answer to that question not a social construct?
If the defining physical differences between races is both melanin and sternum width, that doesn't seem to be more relevant than just skin melanin.
* In many cases. It's more error prone than most people want to admit.
It's more that humans have been able to define, with significant disagreements, what race someone is with their stupid human eyes. Race is nothing but a collection of these judgments. I'm not brown because I'm black, I'm black because I'm brown.
Perhaps the best example is the American views on what defines blackness. Because I grew up in a community where mixed race white/black was seen as a distinct race from white or black, I have a very hard time interpreting race the way Americans do sometimes -- Barrack Obama and Kamala Harris's mixed parentage is obvious to me, which makes them obviously not black to me (think like in the same way Obama is obviously not white) and I have a hard time wrapping my head around Americans seeing them as such, but apparently they do, since even a small amount of physical traits that suggest recent African descent categorizes you as black there.
This is not true. I had a college professor explain the "race is a social construct" idea, and her position was staunchly that there was _no_ biological basis for race. See also this article by the scientific american[1]:
> Today, the mainstream belief among scientists is that race is a social construct without biological meaning.
This is the idea that GP is responding to - clearly there must be some biological basis for race, if an AI can determine race from an x-ray.
[1] https://www.scientificamerican.com/article/race-is-a-social-...
In general, arguments of variation within a group are not arguments against considering between-group differences, or those between-group differences being real. The variation within a category cannot be dismissed, though, it's hugely important for understanding the world properly and for guiding personal conduct.
This is often repeated, but the point of the question is that the OP calls this into question.
Slavery, Race and Ideology in the United States of America: https://u.pcloud.link/publink/show?code=XZ3bwqXZT2m8MI2egSRA...
Side note: If you've seen Ken Burn's docs then you've probably seen her before: https://en.wikipedia.org/wiki/Barbara_J._Fields
I'd say that that only occurs in the odd case of 'Latino'. For some reason Spanish-speaking got amalgamated into a thing it didn't belong.
Deleted Comment
Deleted Comment
Biologically, forget about bone density etc. Skin color, facial features, face shape, etc are discernable directly.
Race as a social construct is the idea (originally rooted in imperialism and colonialism) that certain races are inferior mentally and societal development wise. A few Brits saw Africans residing in huts and living on farming using primitive tools and concluded that they could not develop any further than that and that their brain development was limited. (Of course, this was perpetrated to enable guilt free enslaving and "civilizing" them and exploiting them for labour. I am sure that if African societies were simply introduced to western civilization and allowed to trade and travel, the ideas from west would have been adopted and assimilated much quickly)
So, biologically, races are distinct, identifiable and have evolved to meet the needs of their local environment. But socially, races as inferior or superior was perpetrated with ulterior motives and have been shown as false time and again.
or genes
I saw a visualization of the clustering on twitter.
https://twitter.com/rokomijic/status/1426614501856751620
To quote Ruth Wilson Gilmore: “The racial in racial capitalism isn’t secondary, nor did it originate in color or intercontinental conflict, but rather always group-differentiation to premature death. Capitalism requires inequality and racism enshrines it.”[1]
Cedric J. Robinson (among others) have discussed how capitalism and racialization are continually co-created.
1. Abolition Geography and the problem of innocence, in Futures of Black Radicalism.
Cultural relativism is not the norm and should not become such since there are differences between cultural traits where it is possible to state that some are objectively better than others. As an example, the cultural trait of genital mutilation is objectively worse than that of leaving girls' bits alone - and I'm open to stating the same about boys even though that would raise up a storm of protest. The cultural trait of parents marrying off their offspring without said offspring having a say in the matter is objectively worse than than that of having the offspring decide for themselves who they want to share their life with. The cultural trait of having people who achieved success within the bounds of the law - whether those be inventors, writers, athletes, successful farmers, builders or architects or anything else - is objectively better than that of having successful criminals and hoodlums as role models - yes, "street culture" with gang bangers as role models is objectively worse than whatever name can be given to cultures which have/had those inventors (etc) as role models.
X-rays can not be used to detect whether you might mutilate your newborn's genitals, marry off your 5yo daughter to your 20yo nephew or leave your children to be raised by the local street gang leaders since these traits do not depend on the colour of your skin even though there is often a correlation; correlation does not imply causation [3]. Take for example Michael Skråmo [4], a Swedish-Norwegian man who very much looked the part of such but ended up as a recruiter for islamic state in the Nordic countries. Contrast him to e.g. Luai Ahmed, a Yemeni refugee who lives in Sweden and is a vocal critic of everything Skråmo stood for. It was not Skråmo's white skin and blonde hair which made him ready to pick up a Kalashnikov, it is not Ahmed's brown skin and black hair which made him averse to the negative cultural traits related to islam.
MLK was right when he longed for a society where people would be judged on the content of their character and we were well on our way of achieving that goal. Unfortunately there are those who derive their identity - and income - from their purported position as fighters against racism (without scare quotes), a fight which was nearing its conclusion. While most old soldiers fade away [5] some have taken it upon themselves to revive their old enemy so as to keep their purpose - and income - alive. Their culture is not mine and I consider it to be objectively worse than, e.g. MLK's. If you then consider that MLK was a "black" man while I am of north-west European descent and as such have "white" skin the truth becomes clear, it is not the colour of our skin which makes us alike - it is the content of our character.
Race is not a social construct. Culture is. Nature is not a social constrict, Nurture is.
[1] https://www.nationalreview.com/2015/11/racism-america-histor...
[2] https://en.wikipedia.org/wiki/Cultural_relativism
[3] https://en.wikipedia.org/wiki/Correlation_does_not_imply_cau...
[4] https://en.wikipedia.org/wiki/Michael_Skr%C3%A5mo
[5] https://en.wikipedia.org/wiki/Old_soldiers_never_die
You would think so, but the proliferation of affirmative action among tech companies and prestigious universities says otherwise.
Dead Comment
Dead Comment
"Because the variation of physical traits is clinal and nonconcordant, anthropologists of the late 19th and early 20th centuries discovered that the more traits and the more human groups they measured, the fewer discrete differences they observed among races and the more categories they had to create to classify human beings. The number of races observed expanded to the 1930s and 1950s, and eventually anthropologists concluded that there were no discrete races.[93] Twentieth and 21st century biomedical researchers have discovered this same feature when evaluating human variation at the level of alleles and allele frequencies. Nature has not created four or five distinct, nonoverlapping genetic groups of people."
0: https://en.wikipedia.org/wiki/Race_(human_categorization)
The social construct argument just says that the specific categories and lines we draw are fairly arbitrary. Why is an afghani middle eastern (white?) but a Pakistani south-Asian? Is a Russian from Vladivostok really more closely related to a brit than a Mongolian? Idk - but, to me, the social construct argument just says “who cares? The specific groupings are pretty arbitrary anyway”
It's location based. Humans only recently gained the ability to travel vast distances, and in the past lived (and bred) within a small, localized region. The "arbitrary" location based ethnicities actually do reflect that genetic ancestry
Of course, really wide ones like black/white/asian lose some of their meanings.
The question is what about the looks of a chest X-ray are connected to race. I agree with the research here, it's non obvious what is being extracted by the AI.
If I had to guess, maybe something about the quality of the scan itself. Perhaps one race was scanned at one particular hospital, vs a different hospital scanning a different race. Then it's just picking out the different scanner.
With that said, the simple explanation is that the AI picks up on these small patterns in a way humans don't. The brain and neural networks are fundamentally pattern-recognition engines. The AI is just seeing something we don't either notice or can't see.
We end up segregating ourselves for a variety of reasons, in which case groups that are physically very distinguished end up forming almost an ethnic basis.
For example, two groups with varying genetic makeup and maybe a number of non-obvious biological differences, but who otherwise looked identical - would have the similar life experience in terms of their social treatment by other groups.
But irrespective of how a person is socialized - if you're Black, people are going to treat you one way, and if you're White, people are going to treat you a little different. That 'lived experience' differential is a somewhat unavoidable.
The degree of that variability is obviously debatable, but surely it exists to some degree.
I suppose you could make a parallel in ethnicity: a century ago, the difference between a Scottish-American and an English-American would have been apparent by lineage, accent, Church affiliation, and that might have affected relationships, status etc..
Whereas after a few generations of integration, there is definitely 'no' (or not much) difference between those two groups, and no vector for differentiation/discrimination. The historical ethnic situation was a 'social construct'.
That said, some of the argumentation used to promote the idea that there is no genetic basis for race is a little odd, the 'Africans have more genetic variation than other groups combined' is often used, but frankly I do not understand how that doesn't mean there are material differences between them and other groups.
And of course there is no 'hard line' between groups, but there is also no 'hard line' between the Scottish and English, there are many people who have attributes of both cultures, but that doesn't negate the existence of either group.
I think we're a bit oversensitive these days to these issues. Systematic racism exists and we should think about it, but that doesn't mean there's a boogeyman behind every door.
I think in this case it's also worth examining what exactly the AI is finding out, because it may not be just 'bone marrow'.
https://lukeoakdenrayner.wordpress.com/2021/08/02/ai-has-the...
or the paper itself
https://arxiv.org/pdf/2107.10356.pdf
I see a lot of "oh it's probably just picking up on x y z" when x, y, and z are things they explicitly checked for:
1) "It's probably just the names or other metadata" – they only gave it pixel data to train on. To control for things like metadata overlaid on the image (e.g., a name written on the image) they divided the images into 3x3 sections and trained classifiers on each section separately.
2) "It's probably some artifact of how the hospital marked up the images" – they used something like 7 different datasets from different hospitals and different modalities (X-Ray and CT).
If it is cheating somehow, it's not doing it in an obvious way that you can think of in a minute or two. Also note that they had more than just medical folks working on the paper; the author list includes plenty of computer scientists. It's unlikely they're making an elementary ML mistake here.
This gives the CNN more information on one race than another, which can create a classifier that performs very well on the training and test data it has access to but then flakes spectacularly on data outside the training set (because the source isn't representative of the total variance in the global population).
The fact that it works on an 8x8 massively pixelated version of the x-ray points to the possibility that it's not actually working, which would be bad if you based patient treatment decisions on an training set that was actually teaching the AI something else entirely.
What do you mean, not working? That the AI was randomly choosing the correct race 82% of the time by luck?
I'm confused by what your implying because it would seem to me that the authors went through many steps to try to pinpoint how the AI was doing this identification and how baffling it was to everyone that even with a lot of x-ray information removed (8x8 pixels compared to say 4k), it somehow was still correctly picking the race.
What would this "something else entirely" that you are implying actually be?
We know that various features visible in medical images correlate with race, eg breast density, bone density, etc. Most likely the network is just learning a classifier on top of these features.
This is trivially verifiable but conspicuously absent from the paper.
That can still be picked out when pixellated down to 8x8 as illustrated in the article? That seems unlikely.
I wonder if the AI is just cheating, as they sometimes inadvertently do. https://techcrunch.com/2018/12/31/this-clever-ai-hid-data-fr...
> In some early results, the agent was doing well — suspiciously well. What tipped the team off was that, when the agent reconstructed aerial photographs from its street maps, there were lots of details that didn’t seem to be on the latter at all. For instance, skylights on a roof that were eliminated in the process of creating the street map would magically reappear when they asked the agent to do the reverse process...
Seems the most likely explanation for it still working even when pixellated as 8x8?
The performance despite degradation may be the same phenomenon that results in adversarial examples that are indistinguishable to human eyes, ie we know that neural nets are highly sensitive to visually imperceptible differences.
They also tested several variants of noise.