Ever wonder if a research paper is destined to be a big deal long before it actually becomes one? Researchers at Cornell University think they have figured out how to tell, and it comes down to something as simple as downloads.
The team published new datasets pulled from arXiv, the go-to repository for unreviewed research papers, and GitHub, the platform coders use to store and share their projects. The goal is to test “lead-lag forecasting,” a method that uses early engagement, including views, downloads, and likes, to predict which papers or projects will matter years down the line.
Why does this even work?
Right now, a paper’s importance is judged by how often other researchers cite it. The catch is that citations can take years to show up. Instead of waiting around for citations, the team focused on how quickly people start reading a paper after it drops. Co-author Yian Yin compares it to a prediction market, except scientists are wagering their time instead of money.
Senior author Sarah Dean notes that every platform today is already collecting this kind of engagement data, and that data turns out to be a surprisingly strong predictor of what comes next.
The researchers pulled anonymized download data from 2.3 million arXiv papers and compared early downloads to citation counts five years later. With just a month’s worth of download data, they were already able to forecast a paper’s five-year popularity.
Does the same trick work for code?
Apparently, yes. The team also analyzed pushes, stars, and forks from nearly 3 million GitHub repositories, and found the same pattern held up. Early stars and pushes ended up translating into more forks five years down the road.
The researchers hope these datasets help others build even sharper forecasting tools. There’s also early talk of using this approach to flag low-quality, AI-generated papers flooding arXiv before they clutter up the platform any further.