If you're looking for a "menu" of papers from various subfields, I've been collecting a list of all best paper awards from a set of 30 computer science conferences for the past 25 years. This is just a personal interest of mine, and is a long page with no ads or upsell.
These are papers that were deemed "best papers" in that year, though obviously may not have turned out to be as influential in retrospect, i.e. they're likely not the papers we consider "best" when looking back today.
Tbh I always direct people who want to read old important papers to lecture notes or textbooks, at least in distributed algorithms or CS theory.
I've found that the original papers are always super dense and the material has usually evolved to be more explainable, especially when someone has put in the time to compile it as part of a course.
Yeah, I think test-of-time awards are a better indicator of which papers actually had solid impact; there's often very little overlap b/w best paper awards and test-of-time awards.
Wow. I am so glad to see you here. I have been following your webpage for the last decade perhaps. Thank you for putting those paper references. (Also, on a different note how many different Jeff Huangs have contacted you so far?)
Thanks, I guess you're referring to my page from about 12 years ago when I offered to give emails to other people with my name? I think 5-6 other people reached out and I gave them emails that are still running now. I know one other Jeff Huang who I'm pretty friendly with (and we've had random encounters in person), and the others are strangers.
That's a great resource, thanks. Looking at the list the paper titles could be be more readable, perhaps bigger and perhaps not in blue. There's a lot of information but you can always scroll.
Question though, have you got a process for reading individual papers? In college my old CS professors insisted on reading the abstract, then the conclusions, then the references, and then the rest of it.
Here's one way (YMMV) of reading academic papers. It usually takes time, and a couple of passes, and you may try to understand the ideas and impact first, to then move to the design/implementation, and the experiments that demonstrate the paper claims. http://ccr.sigcomm.org/online/files/p83-keshavA.pdf
Caveat: my training is in control system,s not computer science. I assume these types of papers to be similar enough that the following will be useful.
I have heard that advice before, but I generally disagree. Unless you are doing a literature search, reading the references is a waste of time. And most conclusions aren't really worth reading either. They typically read as though the author has completely exhausted themselves by writing the rest of the paper, and simply restart the last paragraph of the introduction in new words.
If you are trying to get up to speed on a new field, a good introduction can really help with you reference search.
Part of "how to read a paper" depends on what you are trying to do. If it's a seminal paper that are are just going to read, that is a very different thing from reading a paper in search of a solution (or trying to figure out of your idea is novel).
In this second case, your main task is to decide if the paper is even worth reading. IMO, this takes a lot practice. Fully reading a paper to the extent you really understand it can be extremely time consuming. It's important to be willing to throw out the paper if at any point (no matter how much time you've put into it) it becomes clear it doesn't work for you.
I will generally skim the abstract and/or the last two paragraphs (or so) of the introduction. If it still sounds promising I look through the next section or two, which are usually some kind of "problem setup" and "proposed solution scheme". I skip things I don't immediately understand. If my interest is still piqued, I look for the results section where the plots are (if it's a practical paper). If I'm still interested, I go back to Section II and start reading more carefully. I'll spend more time with tricky math, but not too much. Save staring at the same four equations for two hours for at least the third pass.
Oh, and if a paper is tricky (for me) and seems worth my time, printing it out single sided and laying the pages out side by side on my desk can be really helpful.
About 13 years ago when I started this, I selected what I considered the most well-known broad conferences in each subfield (I'm trying to avoid using the words "top" or "best"), though the list is notably missing SIGGRAPH which didn't have such an award, and architecture conferences which I considered more in ECE than CS.
ACM archives -- not really, I haven't added any new conferences because it makes each year even more work to update. And I find that nearly all CS papers are accessible through various sources that you can find in Google Scholar or Semantic Scholar (e.g. author homepages, course websites, arXiv, etc.).
I’ve seen this list shared before, and want to thank you for sharing it again! I had forgotten about this resource and am pretty stoked to be seeing it
Unless you’re working on something that has the potential to be directly influenced by a paper, I’ve found trying to stumble through academic papers to be a huge waste of time. Remember that academic papers are by and large written for other academics, not for general purpose or even specialized engineers (obviously it depends how specialized you are). Take something like merge sort - I’d recommend the Khan academy video before recommending the academic paper if you want to understand how it works. I get that you should understand “why” it works from an academic perspective, but I don’t know that you really get much value from that level of understanding.
This is generally true, but some original papers are beautiful and elegant, and give you amazing insight even with little domain knowledge. Some of my favorites in biology include Shinya Yamanakas original iPS paper (1), Sydney Brenners paper on C. elegans (2). I also like the original BLaST paper.
Anyone with the money and general scientific interest should consider subscribing to Nature or Science. It’s a fun browse every week and you never know what fields interesting finding might fancy you until you look at the articles.
Reading the news on both nature and science websites could be fun. And it's kind of more accurate than those reported by news agencies and often lack the hyperbole from those nonsense PR
I'm a PhD student. Maybe I'm not a good PhD student. But honestly, when I read most papers I usually just end up confused.
The concepts are complicated enough, but then the writing style in papers is just really strange and unnecessarily verbose and over-complicates even simple concepts.
I end up getting most of my knowledge through blog posts and slides which cover the papers vs. the papers themselves.
My perspective: Most people are terrible writers. And that also applies to many scientists. And in addition, there are cargo cults of writing styles that people follow, just so that their writings are perceived as part of the scientific culture.
When regular people publish books, they get proofread and you actually get feedback from people who are trained in proofreading. In academia this not something you can expect from a publisher. And that's why we have lower standards for published academic writings.
Some journals have a proofreading stage before publication. But proofreading can't save bad writing. I blame the overemphasis on "critical thinking" in modern liberal arts education at the expense of fundamental writing skills and thinking in clear terms.
The typical CS paper is written by a grad student who speaks English as a foreign language. They can't express their ideas as well in English as their native language, and they probably don't have that much experience in writing in any language. When they write, they often end up copying phrases and sentence structures from other research papers.
Most papers about new results are published in early stages of research. The authors barely understand the results enough to be convinced that they are correct. They can't explain the results well, because they don't understand them yet. If they continue working on the same topic, they will probably come up with a good explanation after a few years and two or three complete rewrites.
> the writing style in papers is just really strange and unnecessarily verbose and over-complicates even simple concepts
Specialized terminology and complete descriptions can sometimes help to make writing more precise and correct. They can also make writing less clear - intentionally or unintentionally.
Why would the writer of a paper want it to be less clear? Perhaps to artificially inflate the importance of the work (and/or the length of the paper) or to make it seem non-obvious and "novel." Or perhaps to "fit in" with a certain style of writing or language ("academese" or "paperish") commonly used in the discipline.
My advice is: be as clear (and as simple) as possible without sacrificing precision or correctness. Clear papers with good and significant work are incredibly valuable and are likely to be more influential than unclear papers, even if the underlying work is good.
Since so many papers are unclear and poorly written, you may also find that clarity and good writing can help to differentiate your work.
Academese is often overly verbose; with sentences that run on forever. Good writing, readable writing, requires effort. I've found this especially difficult for non native English speakers, myself included.
You're not wrong. Most authors of academic papers are not good writers. Perversely, regardless of where they are on the spectrum spanning from good to mediocre to terrible, writers' output is usually made worse when they know they're writing for an academic context—this is why most of the blog posts you're thinking of, with their informal language, make better papers than the papers themselves.
I wish there were a culture of (a) always writing a blog post to accompany any paper, and (b) rewriting historical* papers to make them more accessible. On the latter point, undergrads should probably be doing this for at least one paper per semester, despite the general expectation that they ordinarily do not deal with "research".
* Doesn't have to be distinguished ones; just any paper you come across and feel is too hard for you/your peers to follow without more effort than should be necessary, or any paper you thought was interesting but didn't get its due in public, or ones that did but have fallen out of cross-generational memory.
>You're not wrong. Most authors of academic papers are not good writers. Perversely, regardless of where they are on the spectrum spanning from good to mediocre to terrible, writers' output is usually made worse when they know they're writing for an academic context—this is why most of the blog posts you're thinking of, with their informal language, make better papers than the papers themselves.
This isn't necessarily the writer's fault at all. I have had reviewers complain that the language in my paper was too informal. I wasn't using slang or something like that. But in so many words, the reviewer wanted _more_ academese. It was the last paper of my grad school career and I was sick of academese. In so many words, I told them to pound sand. That was my only paper to never get published.
That's normal, most papers are written unnecessarily complex, likely to make them appear more impressive than they actually are. Many blog posts that contain just as much information as papers, but presented in a much more intelligible way.
Had a similar experience when working in a research lab.
I think the issue is that I have a hard time learning things without doing myself. And generally trying to reproduce stuff from a paper is really hard.
Much depends on the subfield, but here are two shortcuts to understanding a dense paper:
1. Many CS papers are presented at conferences, and many of these talks are recorded and available.
2. Look for a paper citing the one you are interested in. The “related works” section often contains brief summaries, which are written with the benefit of hindsight.
Somewhat related, an earlier comment of mine on how to acquire copies of a paper without resorting to unauthorised copying [1]
Motivation is to get more people into it, and also give enough of a flavor of the paper for those who aren't used to reading academic papers, or don't have the time.
For someone who might be new to CS I don't think reading papers will be the optimum path towards gaining knowledge.
If the concepts are new and bleeding edge, the reading papers is the only way to become familiar to them. Otherwise, time is better spend reading books and courses, where the said concepts are explained in relation with the others, forming a bigger picture.
Some say you should and some say you shouldn't. But it would be nice to have a resource that explained new CS research for practicing programmers. Something like "Popular Mechanics" or "Scientific American" for programmers. Is there such a thing?
Haha, though for accuracy the website is named after the Y-combinator which is a combinator in Haskell Curry's system of combinatory logic (not to be confused with combinatorial [digital] logic).
Readit News
https://jeffhuang.com/best_paper_awards/
These are papers that were deemed "best papers" in that year, though obviously may not have turned out to be as influential in retrospect, i.e. they're likely not the papers we consider "best" when looking back today.
Links to the papers: https://docs.google.com/spreadsheets/d/1wS6O7-ZoFL7Cfjgt-kdh...
I've been slowly reading through them - some easier than others. (Some I've pretty much skipped - way too long for my interests).
I've found that the original papers are always super dense and the material has usually evolved to be more explainable, especially when someone has put in the time to compile it as part of a course.
Question though, have you got a process for reading individual papers? In college my old CS professors insisted on reading the abstract, then the conclusions, then the references, and then the rest of it.
Summary at - https://derekchia.com/how-to-read-a-research-paper-3-pass-ap...
More Advice from S.Keshav - https://svr-sk818-web.cl.cam.ac.uk/keshav/wiki/index.php/Adv...
I have heard that advice before, but I generally disagree. Unless you are doing a literature search, reading the references is a waste of time. And most conclusions aren't really worth reading either. They typically read as though the author has completely exhausted themselves by writing the rest of the paper, and simply restart the last paragraph of the introduction in new words.
If you are trying to get up to speed on a new field, a good introduction can really help with you reference search.
Part of "how to read a paper" depends on what you are trying to do. If it's a seminal paper that are are just going to read, that is a very different thing from reading a paper in search of a solution (or trying to figure out of your idea is novel).
In this second case, your main task is to decide if the paper is even worth reading. IMO, this takes a lot practice. Fully reading a paper to the extent you really understand it can be extremely time consuming. It's important to be willing to throw out the paper if at any point (no matter how much time you've put into it) it becomes clear it doesn't work for you.
I will generally skim the abstract and/or the last two paragraphs (or so) of the introduction. If it still sounds promising I look through the next section or two, which are usually some kind of "problem setup" and "proposed solution scheme". I skip things I don't immediately understand. If my interest is still piqued, I look for the results section where the plots are (if it's a practical paper). If I'm still interested, I go back to Section II and start reading more carefully. I'll spend more time with tricky math, but not too much. Save staring at the same four equations for two hours for at least the third pass.
Oh, and if a paper is tricky (for me) and seems worth my time, printing it out single sided and laying the pages out side by side on my desk can be really helpful.
Also, do you think ACM opening their archives will have a high impact on which ones you recommend in the future?
ACM archives -- not really, I haven't added any new conferences because it makes each year even more work to update. And I find that nearly all CS papers are accessible through various sources that you can find in Google Scholar or Semantic Scholar (e.g. author homepages, course websites, arXiv, etc.).
This collection is great - thanks so much for your effort!
Anyone with the money and general scientific interest should consider subscribing to Nature or Science. It’s a fun browse every week and you never know what fields interesting finding might fancy you until you look at the articles.
1. https://pubmed.ncbi.nlm.nih.gov/16904174/ 2. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1213120/?report...
The concepts are complicated enough, but then the writing style in papers is just really strange and unnecessarily verbose and over-complicates even simple concepts.
I end up getting most of my knowledge through blog posts and slides which cover the papers vs. the papers themselves.
When regular people publish books, they get proofread and you actually get feedback from people who are trained in proofreading. In academia this not something you can expect from a publisher. And that's why we have lower standards for published academic writings.
Most papers about new results are published in early stages of research. The authors barely understand the results enough to be convinced that they are correct. They can't explain the results well, because they don't understand them yet. If they continue working on the same topic, they will probably come up with a good explanation after a few years and two or three complete rewrites.
Specialized terminology and complete descriptions can sometimes help to make writing more precise and correct. They can also make writing less clear - intentionally or unintentionally.
Why would the writer of a paper want it to be less clear? Perhaps to artificially inflate the importance of the work (and/or the length of the paper) or to make it seem non-obvious and "novel." Or perhaps to "fit in" with a certain style of writing or language ("academese" or "paperish") commonly used in the discipline.
My advice is: be as clear (and as simple) as possible without sacrificing precision or correctness. Clear papers with good and significant work are incredibly valuable and are likely to be more influential than unclear papers, even if the underlying work is good.
Since so many papers are unclear and poorly written, you may also find that clarity and good writing can help to differentiate your work.
I wish there were a culture of (a) always writing a blog post to accompany any paper, and (b) rewriting historical* papers to make them more accessible. On the latter point, undergrads should probably be doing this for at least one paper per semester, despite the general expectation that they ordinarily do not deal with "research".
* Doesn't have to be distinguished ones; just any paper you come across and feel is too hard for you/your peers to follow without more effort than should be necessary, or any paper you thought was interesting but didn't get its due in public, or ones that did but have fallen out of cross-generational memory.
This isn't necessarily the writer's fault at all. I have had reviewers complain that the language in my paper was too informal. I wasn't using slang or something like that. But in so many words, the reviewer wanted _more_ academese. It was the last paper of my grad school career and I was sick of academese. In so many words, I told them to pound sand. That was my only paper to never get published.
I think the issue is that I have a hard time learning things without doing myself. And generally trying to reproduce stuff from a paper is really hard.
1. Many CS papers are presented at conferences, and many of these talks are recorded and available.
2. Look for a paper citing the one you are interested in. The “related works” section often contains brief summaries, which are written with the benefit of hindsight.
Somewhat related, an earlier comment of mine on how to acquire copies of a paper without resorting to unauthorised copying [1]
[1] https://news.ycombinator.com/item?id=23711206
Motivation is to get more people into it, and also give enough of a flavor of the paper for those who aren't used to reading academic papers, or don't have the time.
If the concepts are new and bleeding edge, the reading papers is the only way to become familiar to them. Otherwise, time is better spend reading books and courses, where the said concepts are explained in relation with the others, forming a bigger picture.
See https://mitpress.mit.edu/books/ideas-created-future