After the 11th nested 'if' statement, I upped the request to a case of beer. I'm not certain he ever got the code working.
To the larger point, scientists are not programmers. They got into their programs to do research. What keeps them going is not the joy of programming, but the thrill of discovery. Programming is nothing but a means to an end. One they will do the bare minimum to get working. Asking hyper stressed out grad students to also become expert coders isn't reasonable.
And yes, that means that the code is suspect at best. If you load the code on to another computer, make sure you can defenestrate that computer with ease, do not use your home device.
I would say most research, to an ever growing degree, is so heavily dependent on software that it's tough to make that claim anymore. It makes no sense to me. It's like saying Zillow doesn't need software engineers because they are in the Real Estate business, not the software business.
What if you want to share data with a wetlab biologist who want to explore their favorite list of genes on their own?
Not that I'm saying using excel is bad either. I use excel plenty to look at data. But scientists need to know how to use the tools that they have.
If you're deciding who gets a large-scale computational biology grant, and you're choosing between a senior researcher with 5000 publications with a broad scope, and a more junior researcher with 500 publications and a more compuationally focused scope, most committees choose the senior researcher. However, the senior researcher might not know anything about computers, or they may have been trained in the 70's or 80's where the problems of computing were fundamentally different.
So you get someone leading a multi-million dollar project who fundamentally knows nothing about the methods of that project. They don't know how to scope things, how to get past roadblocks, who to hire, etc.
The whole world's data ultimately comes from or ends up in an Excel spreadsheet. Sure, we might use other intermediate data storage methods, but in the end it's going to go into some scientist's or some politician's computer, and by golly it's gonna be in Excel. Trying to rally against Excel is like trying to rally against fundamental forces of nature.
This is just an example of that fundamental force winning.
> All it will do is devalue the value of replication studies because only PHD students do replication studies. It's also not in their best interest especially if they dispute findings of established researchers.
Most studies are done by students regardless, so it seems unlikely that replication studies would be devalued merely because they're done by students. Although disputing the findings of established researchers can be risky, they would be publishing jointly with their PI (or, with the above implementation, multiple PIs), not alone with no support. Few students want to stay in academia, so it usually doesn't matter to them if a professor at some other institution gets offended. Most importantly, if everyone is doing replication studies, there will be so many disputations flying around that any particular person is less likely to be singled out for retaliation.
1. Studies can be much more expensive than most people think. In my field, a moderately sized study can easily cost $100,000+ if you're only accounting for up front cost (e.g. use of equipment, compensating participants). Someone would have to foot the costs of this.
2. Studies can be incredibly labor-intensive. PI's can get away with running studies that require thousands of man-hours because they have a captive market of PHD students, Post-docs, and research assistants all willing to work for low wages or for free. PHD students usually don't have the same amount of man-power.
3. For obvious reasons, studies that require high cost, high man-power work tend to get replicated naturally less. In other words, the least practical studies to replicate happen to also be the most necessary to replicate.
A couple of things I would dispute:
> it seems unlikely that replication studies would be devalued merely because they're done by students
I think academics value work in a particularly skewed way. There is "grant work" and there is "grunt work". Grant work is anything that actively contributes to getting grants for one's institution. Grunt work is everything else. PHD's can do grunt work, but that doesn't mean it will be valued on the job market. For example, software development is actively sought after in (biology) grad students, because it's a very useful skill. However, I've also seen it count against applications as professors because it shows they spent too much time on "grunt work". Software development skills don't win grants.
> Few students want to stay in academia
In some fields there aren't any options except to stay in academia or academia adjacent fields.
Sort of, a huge portion of income is from grants, particularly after the first few years from being hired. More importantly, a huge portion of the University income is from grants. When a researcher recieves a grant, there is an "overhead" percentage that goes to the University. Universities hire, in part, to maximize those overheads, which means getting the researchers with the best chance at getting big grants.
Changing the hiring process may affect how PHD students act, but once they're "in the system", they are subject to all the same problematic incentives.
Now, I don't have a good solution either, unfortunately. What might work is that we require replication work for a PhD or have a certain percentage of a journal dedicated to verification. That, combined with some meta-studies to reward people with citations for replication, might work without fully swimming against the current.
It's a hard problem, really.
I don't think this will work. All it will do is devalue the value of replication studies because only PHD students do replication studies. It's also not in their best interest especially if they dispute findings of established researchers.
Also, we have to get away from the idea that the scientist's job is to think and write, and literally all of the other work can be shuffled off onto low wage (or no wage), low status workers. This is one of the biggest reasons that science is going through such a crisis. If you want enough papers to consistently get grants you probably need at least 4/5 PHD students every few years. This causes a massive glut in the job market. It also dissociates scientists from their work. I've met esteemed computational biologists who could barely work a computer. All of their code was written, run, and analyzed by graduate students or post docs. They were competent enough at statistics, but that level of abstraction from the actual work is troubling.
Each of the suggested actions, taken together, would seem to have a very positive improvement on the status quo. Would you care to explain why the suggested actions would be "somewhere between ridiculous and useless?"
Frankly, having worked in academia long enough to see at least a couple shifts in culture, the only thing I can see that comes out of this is a couple more things get added on to the ever growing checklist of publishing a paper/submitting a grant application.
I think we need to get away from the sort of thinking where large structural problems can be solved by tiny incremental improvements. If you really want to solve the problem, one or more of [Granting Agencies|Journals|Universities] has to be completely torn down and built back up.
The political winds shift and blow away any short term cover the operators ever enjoyed.
The ACLU's CCOPS efforts were launched in 2016 [0] and have since resulted in multiple cities across the country wresting oversight of local surveillance technology into the hands of deliberative bodies.
I joined a local effort to put this oversight model in place in my city, and we're on track to receive unanimous approval from my (large) city council this year. It takes work to do it. You will have to get hands on if you want to participate in this change.
ACLU NorCal has put together a nice guide on how to build the movement in your city if one has not already begun. [1]
[0] https://www.aclu.org/issues/privacy-technology/surveillance-...
[1] https://www.aclunc.org/publications/fighting-local-surveilla...
This is absolutely not assuring. All this means is that the wrong people are going to be surveilled, arrested, and punished.