Building - A Networked Research Paper Aggregator

Jun 2, 2017

A couple of weeks ago, I decided that I’d like to keep up with cutting-edge research, at least in a very small way, and in a few specific areas. I thought it would be a good thing to read a paper each day, to give me something to think about, and give me ideas which might inspire me to find new solutions for work problems.

So, I went online to find a site which could keep me up to date. But there wasn’t anything. Sure, I could subscribe to keyword-based emails containing papers from one place or another, but there was nothing which would give me an idea of the papers which had just come out and were generating a buzz.

So I decided to build it myself. The result of pinched hours from evenings and weekends over the last fortnight is, a social link aggregator and discussion forum for research of all sorts.

Latest Papers

How does it work? The site is hierarchical, so there’s a main category (called “Top”) in which you can see all the papers on the site, ranked in order of current interest. Underneath this level, there are “subtopics” like “Biological Sciences”. This was inspired by Reddit’s “subreddits”. However, research is a lot more complicated than that, so each of these subtopics can have subtopics, to an infinite depth. If I post a paper in “Glial Neurobiology”, for instance, it’ll appear in “Neurobiology”, “Neuroscience”, “Biological Sciences”, and “Top”. I could also choose to subscribe to the “Neurobiology” topic, and then my personal feed will include items in this topic, and all of its “children”.

Each paper has a comment thread, and users can upvote or downvote both papers themselves and comments written by other users. The comments are threaded. Topics can have more than one parent, too. For instance, Cognitive Neuroscience is a child topic of both Neuroscience and Cognitive Science.

The structure of the site means that you can build a highly personalised reading list based on your exact interests, which will be updated on a minute-by-minute basis, and hopefully surrounded by helpful and insightful comments from the site’s readership. Well, I can hope.

What’s the tech? The site is built on Django, with a MySQL backend. The places where I departed from a normal Django setup were related to scalability. I don’t know how many people are likely to end up using the service, but if it’s a lot, I want to put the site behind a CDN. But you can’t do that if all the views are customised on a per-user basis on the server side. So the main customising actions, such as showing a user’s votes, are created using AJAX calls to an API layer. The same thing goes for on-site actions, such as voting for a paper or comment.

What’s next? I don’t know! is currently in open beta, and I’m not completely sure about what direction to take it in. Should I add pure discussion forums or just focus on the papers? Should I auto-import papers from popular portals, or leave it to users? At the moment, I’m just keen to spread the word that it’s out there, and hopefully attract people who are interested in joining in. If you’re a researcher, and have published papers, please link to them on the site. If you’re just interested in finding interesting things to read, please join up. If there are subject areas which we don’t already cover, you can suggest a subtopic from the site menu on the left hand side.

Got any ideas for the site? Please leave a comment below, I’d be interested to hear everyone’s ideas.