I find it really cool that this whole process didn't involve scientific publishers at all. Paper was submitted to Arxiv, there was Turmoil. Now a national lab is reviewing it, all without a Journal submission.
As an academic, I have not so secretly been enjoying the shit storm of revelations lately. Science got along fine for thousands of years without Journals and Conferences deciding what's "correct" and not. It's been a failed experiment. Journals started as a way to help distribute works to other researchers to improve upon the old method of just mailing it to one another. Arxiv is great for peer review. Your work is actually fucking available and not behind some pay wall, where your peers can... review it. Only thing would be better is if we were using OpenReview so we could track discussions, but I'll admit that could get messy real quick as anyone that's open sourced their research work will tell you (lots of questions like "I trained my model, how do I test it?" and "I'm getting a cuda out of memory error, how do I fix this?").
The thing that established journals do better than arXiv is clarity
You can upload whatever trash methods you want, but a normal journal will have at least one guy who tells you to wipe your ass and make your bullshit presentable in public, if only because they're expected to be gatekeepers for a minimum standard of supposed reproducibility.
· By one side the Journal reclaim a high amount of money for being able TO TRY to publish on them (APCs fee), more if one want to be published for to allow free access, as to read the publication in such Journals require subscription, are pay walled.
· By other side, the reviewers of such papers, academics and researchers from other universities, don't receive economical compensation for doing the review, while the taken APCs fee its supposed for doing it? only in few cases the review is compensated by some programs of the reviewer's university or other 3rd parties. Then, what is for all those pay-walls and publish-fees?
Sounds like a big filter... for what? for who? Because can be seen it's not working just as a mere "quality" filter.
Being a paper published in one of those popular Journals, or being done directly in medias like arXiv, both sources needs the same amount of "grain of salt" about what is being read, until others replicate experiments, or contrast theories.
So IMHO, outsider point of view, from time to time I have to use Sci-Hub in fact, I understand if they decide to don't publish in such popular Journals. I consider absolutely legitimate when they do it in distribution channels like arXiv.
What I really would like it were exists an open research platform without pay walled papers, like happens in arXiv, and in addition, peer reviews through the platform were possible, as to read such reviews, and also, equally important, to have available debates about the paper, like if it were hacker news format.
science in the modern sense is far less than 1000 years old, and emerged in tandem with systems of publication which no did not use the contemporary review system but did have editors who decided whether or not to publish submissions. not defending the peer review process as it currently exists but it’s not a bizarre outlier in the history of science
Also, I believe we should focus more on rigorous replication process rather than publishing new things from scientific researchers. Peer review is not enough to check the claims as it merely checks on reasoning of the science instead of experimental confirmation unless it is a very simple and short experiment. In other words, instead of directing funds to exploratory research, more money should be spent on multiple experimental verifications by independent researchers as I believe there are so many papers that need retraction due to unsubstantiated claims.
You can find dozens of proofs on the arxiv that P=NP and that P!=NP. What does peer “review” look like here? Most people won’t bother to write an article specifically rebutting one random incorrect proof.
Now somebody tomorrow posts a proof that P!=NP. Will people pay attention? If it’s a “big name” or somebody at a university with a good PR office then yes; otherwise no. Doesn’t seem ideal to me.
But as an experimental chemist who got a PhD and works at a national lab, if the twitch/Twitter streamers don't produce a semiconductor, it says nothing
Experiments and processes are hard and particular. You're expecting to produce a nanoscale material through macroacale processes. Chemistry can be like producing a microchip with a ballpeen hammer. "I hit it seventy times in thirty seconds in 3.8mL acetone and it just forms a single crystal structure. Do not hit it seventy-one or sixty-nine times."
Chemists are experts at clearly explaining their procedure while leaving out the meaningless keystone detail that will only take a year or two to suss out if you're a clever experimentalist.
And this isn't a clearly outlined procedure.
But, again, I think the best case scenario is to watch an ambitious amateur do materials synthesis and see it work less perfectly than he intended. Maybe he and they stick around for the other 5 or 10 or 150 weeks as he does the same thing again to make sure the problem is nature rather than technique.
If bet-placing is the fashion, shall we place bets about tenacity and thoroughness?
I wonder if there's a language barrier. I can't judge for myself--neither the substantive content nor the biographies--but hypothetically the principle investigators might not have strong (or any) English language proficiency, relying on a younger, less experienced peer to write the paper.
As someone who sucks at languages, I can appreciate that learning new languages does not come easy to everybody, or even to most people, especially in older age. If it seems like all reputable scientists have some English proficiency, that's a selection effect--within the English-speaking world in particular you're not going to bump into or even hear much about those who cannot speak or write English. AFAIU there's a substantial amount of high quality research in, e.g., Japan that is invisible to non-Japanese speakers. Japan's scientific establishment long predates English as a global language, unlike China; and unlike many European countries but similar to the United States there was never a widespread norm of learning foreign languages even among the professional classes, AFAIU. Though Korea is a younger industrialized country, I wouldn't take for granted that the principal discoverers have any English proficiency, especially considering that they would be late middle-age or older if they were working in the lab in 1999.
> if the twitch/Twitter streamers don't produce a semiconductor, it says nothing
If you're referring to Andrew McCalip, he's been very clear that he is not claiming to be doing a serious replication attempt, but he also does zero-g/orbital chemical synthesis for a living so its not quite fair to characterize him as a random twitch streamer.
Something I've also been thinking about is that we don't appear to have any data on the success rate. Maybe the samples of LK99 are the best that the team has been able to produce, but the process requires further refinement and experimentation.
Why should experiments wait to be submitted to some arbitrary academic body before trying to to reproduce them? This is what’s wrong with modern science, everything is academic.
The only thing valuable that academic journals and conferences bring to the table is the peer-review process. Which, to be fair, is needed.
In this case, the peer review is being conducted without the need for publishing venues. However, I don't think that is usually the case. Take a look at the countless number of ArXiv papers that lack any sort of peer review.
Somewhere I read that this is how things used to be in Einstein era (or maybe earlier than that). There was little bureaucracy in getting the idea out.
Richard Rhodes' monumental work on the A and H bomb mentions that from Thompson onward, "being heard" Was a problem. Juniors often struggled to be taken seriously, sometimes seniors ostentatiously "spoke for them" to have new ideas given credence and women were doubly disabled on the "taken seriously" front.
Right up until Murray Gell-Mann and beyond, speaking outside the current limitations to knowledge was hard. I don't want this to descent into AGW and antivaxx denialism, this is inside classic science but considering radical theory/paradigm shifts which were still testable propositions. New models are hard on early stage career.
Blinded peer review was partly designed to help some of that. In a narrow enough field it's impossible for reviewer and submittor not to know each other. There may only be 3-4 people who understand your niche fully. Rueben Hersh discusses that a bit in "what is mathematics really" (I think, could be another book of his, "the mathematical experience")
Rhodes discusses Michael Polyani's theory of science as old fashioned apprenticeship. Journeyman scientists publish reproducible, testable work. Theoreticians.. harder to test sometimes.
While I am very skeptical, when working with what seems to be a fluke, it is more important to test the claim then to test a reproduction.
Figure out how to send an independent lab's equipment and personal there or a sample of the substance to an independent lab that verify and if true. Also ensure that they can also do material analysis such as X-Ray spectroscopy & diffraction , and a battery of other stuff if a miracle did occur.
For all we know, this could be one of those accidents of sloppiness that introduces a particular containment that makes everything line up as a super conductor that nobody else will easily reproduce.
> For all we know, this could be one of those accidents of sloppiness that introduces a particular containment that makes everything line up as a super conductor that nobody else will easily reproduce.
Maybe we should call this the Hyde Phenomenon.
> Jekyll's involuntary transformations increased in frequency and required ever larger doses of the serum to reverse....Eventually, the supply of salt used in the serum ran low, and subsequent batches prepared from new stocks failed to work. Jekyll speculated that the original ingredient had some impurity that made it work.
I think all they have to do is send a sample to another lab to analyze.
Apparently, people feel it's pretty easy to make, but in any case, if they have a sample they can see if it has the claimed properties and analyze what it's made of.
> For all we know, this could be one of those accidents of sloppiness that introduces a particular containment that makes everything line up as a super conductor that nobody else will easily reproduce.
I get that terrible sense too - either the original studies were in error, or some one-off fluke makes it difficult to reproduce.
On the upside however, and I'd caveat this as not being fully in the know about the art of the possible in solid state labs, it seems that the material isn't that awfully difficult to produce given an appropriate lab and equipment. Hopefully we'll know one way or the other shortly.
why don't they just repro their own experiment? If they can do it twice, then it isn't a fluke. at worst it's something specific to their lab. if they can't do it a second time, then the issue is settled.
I'm sure it will take a lot of time and money to run everything again. but all of earth seems willing to give them whatever resources they need.
It doesn't particularly matter if it is reproducible, as long as they have 1 working superconductor that they can hold in their hand (Room temperature/pressure), its a nobel prize. Other people can figure out how to reproduce it since it is clearly possible.
The challenge is proving they have even that 1 sample.
Considering the magnitude of the discovery and the relatively easy steps to reproduce the process, they would have to be outrageously lazy to have put the paper out without doing it a second time. In absence of a statement that they only tried it once, I think it's fair to assume they at least verified they can reproduce it locally.
> On the other hand, he says, researchers at Argonne and elsewhere are already trying to replicate the experiment. “People here are taking it seriously and trying to grow this stuff.”
The submitted title has been heavily editorialized. That’s the only relevant part of the article, and that’s far from implying that there’s a concerted effort at Argonne.
> The submitted title has been heavily editorialized. That’s the only relevant part of the article, and that’s far from implying that there’s a concerted effort at Argonne.
What's wrong with the submitted title (other than being very narrow)? It just says they're attempting to replicate, which they are.
If anything I'd say that "taking it seriously" is stronger language, and the submitted title is slightly underselling it.
Is the original title better? "A spectacular superconductor claim is making news. Here’s why experts are doubtful." Standard nothing title, "a thing happened." I zeroed in on the pithy part of the article, which is SOP on HN. The subtitle is just "skepticism abounds" which we already know. What some, including me, didn't know is that legit USG labs are studying it, and that's what makes it news.
Norman's complaints that lead atoms are too heavy do not seem consistent with the composition of other known superconductors. As far as I know, the most widely used cuprate superconductor is bismuth strontium calcium copper oxide (BSCCO), which was used for the world's first superconducting power line:
>What’s more, lead and copper atoms have similar electronic structures, so substituting copper atoms for some of the lead atoms shouldn’t greatly affect the electrical properties of the material
The lead (II) ions claimed have an even number of electrons. Copper (II) has an odd number. Or if copper (I) is present, then the charge itself is different. Again, this is just a very confusing argument to hear from a physicist.
>First, the undoped material, lead apatite, isn’t a metal but rather a nonconducting mineral.
The cuprate and iron superconductors are not metals either. In fact some are Mott insulators (materials with unexpectedly high resistance) under normal conditions.
There is a case for skepticism about LK-99, but it isn't this one.
One of the common threads seems to be that other scientists think that lk-99 is not a super conductor but just strongly diamagnetic.
As a non physicist i wonder if that is useful in and of itself? Skimming wikipedia it doesn't seem like there are that many strongly diamagnetic materials. Would discovering a new one still be a big discovery (just not earth shattering)
That depends on how the diamagnetism comes about. One way in which it could come about which would still make it a big discovery is if the material is locally superconducting but not globally. That would give you diamagnetism without superconductivity on a usable scale and might in turn open the door to modifying the material to increase the size of the superconducting regions. Pure speculation, obviously, the bigger chance is that if the material is 'just' diamagnetic that it is simply diamagnetic in the same way in which other materials are (all particles of which the samples are made are constructed such that all electrons are paired).
My understanding is that taking the paper at face value it seems like they’re saying the diamagnetism was an encouraging piece of supporting evidence beyond their other rationale for what they believe they’ve found. I’m not enough of a physicist (or one at all) to speak to that though so I guess I’ll just sit here wit ma popcorn for the next week with everyone else
Despite some criticism of the original work, they seem to have been very conscientious about making replication easy.
===========
Nadya Mason, a condensed matter physicist at the University of Illinois, Urbana-Champaign says, “I appreciate that the authors took appropriate data and were clear about their fabrication techniques.” Still, she cautions, “The data seems a bit sloppy.”
Yes this is an excellent thread. You have done some of the only on the ground reporting from Seoul. The takedowns you linked of the paper, CMTC giving it an F, are damning. There's one legit professor Kim from William & Mary who had a tiny part of this, but more and more, it looks like it's the output of a few cranks out of a "Q-Centre." The bulk of the work took place before Kim's involvement and he's listed on 1 of 3 papers so far. He's the only thread of legitimacy in this whole thing, and it's a thin one. The 5 other guys are just loony.
The reason the papers look like junk is that they were leaked by a rogue former team member who wants credit. The papers weren't supposed to have been published yet, peer review is still ongoing.
Having skipped to his analysis of the video, I find him pretty convincing.
He explains why the video is bunk using information provided by the lab, and basic physics. And then 'replicates' using his own sheet of metal and a magnet.
Also, he raises the fact that it's a commercial lab.
I've downgraded my level expectations a couple of orders of magnitude. But hey, let's hope for a miracle.
I've been wondering what the "hanging copper plate" demonstration would look like if you were just seeing eddy currents. I kind of assumed you wouldn't see much movement because otherwise that makes the whole demonstration seem dumb.
...and based on that demo, it really does seem like a dumb way to demonstrate your miracle material. Really doesn't inspire confidence.
Readit News
Arxiv/preprints are peer review
You can upload whatever trash methods you want, but a normal journal will have at least one guy who tells you to wipe your ass and make your bullshit presentable in public, if only because they're expected to be gatekeepers for a minimum standard of supposed reproducibility.
· By one side the Journal reclaim a high amount of money for being able TO TRY to publish on them (APCs fee), more if one want to be published for to allow free access, as to read the publication in such Journals require subscription, are pay walled.
· By other side, the reviewers of such papers, academics and researchers from other universities, don't receive economical compensation for doing the review, while the taken APCs fee its supposed for doing it? only in few cases the review is compensated by some programs of the reviewer's university or other 3rd parties. Then, what is for all those pay-walls and publish-fees?
Sounds like a big filter... for what? for who? Because can be seen it's not working just as a mere "quality" filter.
Being a paper published in one of those popular Journals, or being done directly in medias like arXiv, both sources needs the same amount of "grain of salt" about what is being read, until others replicate experiments, or contrast theories.
So IMHO, outsider point of view, from time to time I have to use Sci-Hub in fact, I understand if they decide to don't publish in such popular Journals. I consider absolutely legitimate when they do it in distribution channels like arXiv.
What I really would like it were exists an open research platform without pay walled papers, like happens in arXiv, and in addition, peer reviews through the platform were possible, as to read such reviews, and also, equally important, to have available debates about the paper, like if it were hacker news format.
The 'republic of letters' was much more similar to the journal-style approach than to modern arxiv, IMO.
And was available only to rich bored people who could self teach themselves
You can find dozens of proofs on the arxiv that P=NP and that P!=NP. What does peer “review” look like here? Most people won’t bother to write an article specifically rebutting one random incorrect proof.
Now somebody tomorrow posts a proof that P!=NP. Will people pay attention? If it’s a “big name” or somebody at a university with a good PR office then yes; otherwise no. Doesn’t seem ideal to me.
doesn't upend your point, but just for the record, what we call science has only been around for about 500 years
But as an experimental chemist who got a PhD and works at a national lab, if the twitch/Twitter streamers don't produce a semiconductor, it says nothing
Experiments and processes are hard and particular. You're expecting to produce a nanoscale material through macroacale processes. Chemistry can be like producing a microchip with a ballpeen hammer. "I hit it seventy times in thirty seconds in 3.8mL acetone and it just forms a single crystal structure. Do not hit it seventy-one or sixty-nine times."
Chemists are experts at clearly explaining their procedure while leaving out the meaningless keystone detail that will only take a year or two to suss out if you're a clever experimentalist.
And this isn't a clearly outlined procedure.
But, again, I think the best case scenario is to watch an ambitious amateur do materials synthesis and see it work less perfectly than he intended. Maybe he and they stick around for the other 5 or 10 or 150 weeks as he does the same thing again to make sure the problem is nature rather than technique.
If bet-placing is the fashion, shall we place bets about tenacity and thoroughness?
As someone who sucks at languages, I can appreciate that learning new languages does not come easy to everybody, or even to most people, especially in older age. If it seems like all reputable scientists have some English proficiency, that's a selection effect--within the English-speaking world in particular you're not going to bump into or even hear much about those who cannot speak or write English. AFAIU there's a substantial amount of high quality research in, e.g., Japan that is invisible to non-Japanese speakers. Japan's scientific establishment long predates English as a global language, unlike China; and unlike many European countries but similar to the United States there was never a widespread norm of learning foreign languages even among the professional classes, AFAIU. Though Korea is a younger industrialized country, I wouldn't take for granted that the principal discoverers have any English proficiency, especially considering that they would be late middle-age or older if they were working in the lab in 1999.
If you're referring to Andrew McCalip, he's been very clear that he is not claiming to be doing a serious replication attempt, but he also does zero-g/orbital chemical synthesis for a living so its not quite fair to characterize him as a random twitch streamer.
One of his favorite movies was It Happens Every Spring (1949), about a chemist who produces a non-reproducible compound.
For a humble mechanical engineer its very helpful to make hammer analogies.
IIRC, the 1989 cold fusion fiasco did not involve scientific publishers as well.
Not saying this is a similar fiasco, just that 34 years ago it was done via a press conference and circulating drafts rather than arXiv.
In this case, the peer review is being conducted without the need for publishing venues. However, I don't think that is usually the case. Take a look at the countless number of ArXiv papers that lack any sort of peer review.
Right up until Murray Gell-Mann and beyond, speaking outside the current limitations to knowledge was hard. I don't want this to descent into AGW and antivaxx denialism, this is inside classic science but considering radical theory/paradigm shifts which were still testable propositions. New models are hard on early stage career.
Blinded peer review was partly designed to help some of that. In a narrow enough field it's impossible for reviewer and submittor not to know each other. There may only be 3-4 people who understand your niche fully. Rueben Hersh discusses that a bit in "what is mathematics really" (I think, could be another book of his, "the mathematical experience")
Rhodes discusses Michael Polyani's theory of science as old fashioned apprenticeship. Journeyman scientists publish reproducible, testable work. Theoreticians.. harder to test sometimes.
Dead Comment
Figure out how to send an independent lab's equipment and personal there or a sample of the substance to an independent lab that verify and if true. Also ensure that they can also do material analysis such as X-Ray spectroscopy & diffraction , and a battery of other stuff if a miracle did occur.
For all we know, this could be one of those accidents of sloppiness that introduces a particular containment that makes everything line up as a super conductor that nobody else will easily reproduce.
Maybe we should call this the Hyde Phenomenon.
> Jekyll's involuntary transformations increased in frequency and required ever larger doses of the serum to reverse....Eventually, the supply of salt used in the serum ran low, and subsequent batches prepared from new stocks failed to work. Jekyll speculated that the original ingredient had some impurity that made it work.
https://en.wikipedia.org/wiki/Jekyl_and_Hyde#Plot
Apparently, people feel it's pretty easy to make, but in any case, if they have a sample they can see if it has the claimed properties and analyze what it's made of.
https://twitter.com/condensed_the/status/1684960318718406656
I get that terrible sense too - either the original studies were in error, or some one-off fluke makes it difficult to reproduce.
On the upside however, and I'd caveat this as not being fully in the know about the art of the possible in solid state labs, it seems that the material isn't that awfully difficult to produce given an appropriate lab and equipment. Hopefully we'll know one way or the other shortly.
I'm sure it will take a lot of time and money to run everything again. but all of earth seems willing to give them whatever resources they need.
The challenge is proving they have even that 1 sample.
The submitted title has been heavily editorialized. That’s the only relevant part of the article, and that’s far from implying that there’s a concerted effort at Argonne.
What's wrong with the submitted title (other than being very narrow)? It just says they're attempting to replicate, which they are.
If anything I'd say that "taking it seriously" is stronger language, and the submitted title is slightly underselling it.
The editorialised title here is misleading, as it's not what the article is really about, it's just mentioned in passing.
[1] https://news.ycombinator.com/newsguidelines.html
https://en.wikipedia.org/wiki/Holbrook_Superconductor_Projec...
>What’s more, lead and copper atoms have similar electronic structures, so substituting copper atoms for some of the lead atoms shouldn’t greatly affect the electrical properties of the material
The lead (II) ions claimed have an even number of electrons. Copper (II) has an odd number. Or if copper (I) is present, then the charge itself is different. Again, this is just a very confusing argument to hear from a physicist.
>First, the undoped material, lead apatite, isn’t a metal but rather a nonconducting mineral.
The cuprate and iron superconductors are not metals either. In fact some are Mott insulators (materials with unexpectedly high resistance) under normal conditions.
There is a case for skepticism about LK-99, but it isn't this one.
As a non physicist i wonder if that is useful in and of itself? Skimming wikipedia it doesn't seem like there are that many strongly diamagnetic materials. Would discovering a new one still be a big discovery (just not earth shattering)
===========
Nadya Mason, a condensed matter physicist at the University of Illinois, Urbana-Champaign says, “I appreciate that the authors took appropriate data and were clear about their fabrication techniques.” Still, she cautions, “The data seems a bit sloppy.”
https://twitter.com/sanxiyn/status/1685094029116297216
He explains why the video is bunk using information provided by the lab, and basic physics. And then 'replicates' using his own sheet of metal and a magnet.
Also, he raises the fact that it's a commercial lab.
I've downgraded my level expectations a couple of orders of magnitude. But hey, let's hope for a miracle.
...and based on that demo, it really does seem like a dumb way to demonstrate your miracle material. Really doesn't inspire confidence.