Logo
Forums
Forums
UniSearch
UniSearch
UniSuggest
UniSuggest
AdmitTrend
AdmitTrend
CourseLiX
CourseLiX
RoommateFinder
RoommateFinder
EduTravel
EduTravel
EdulixExpress
EdulixExpress
Chat
Chat
Blog
Grad Studies
About Us
About Edulix

Post Reply  Post Thread 
 
Thread Rating:
  • 7 Votes - 4.14 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Some thoughts on reading research papers
Author Message
DesiLinguist Offline
Edulix MVP
*********
Edulix MVP

DesiLinguist Offline
Edulix MVP
*********
Edulix MVP


Posts: 934
Likes Given: 0
Likes Received: 12 in 8 posts
Joined: Aug 2005
Unisearch: Link
Reputation: 26
Post: #1
Some thoughts on reading research papers
0
0
[I wrote the following post in response to a question about how to read research papers. Taro suggested that I expand on it and post it as a stand-alone thread for the sake of posterity.]

Reading research papers is more an art than a science. Like most data gathering processes, I think it all depends on the eventual goal of reading the paper and the things you can get out of it is more than just knowledge. The basic question is: why are you reading this particular paper?

Scenario A: You are reading the paper as the first ever introduction to the field. This is unlikely because you should most likely read a book chapter or a survey journal paper for that purpose (assuming this paper is a research conference/journal paper).

Scenario B: This is the first research paper you are reading about the field after you have understood the basic terminology and concepts of the field from a lecture, book chapter or a survey paper. If this is the case, you should read the paper so that at the end you can answer the following questions easily: (a) What is the basic idea in the paper that is claimed to work better than the basic (baseline) approach that you read about in the book or learned about in the introductory lecture? Did they approach the problem from a different angle altogether? Or did they just make incremental improvements to the baseline approach? (b) Were their claims borne out by the results? Do you believe their results? (d) If you were to asked to describe why you believe (or don't believe) that the approach in this paper was better than the baseline in 1 or 2 sentences, can you do it? Once you have answered these questions, write down the 1-2 sentences and run them by your advisor or a senior graduate student. If you were right, write the sentences down in your bibliography manager in the notes section. Those turn out to be pretty valuable when you are working on your dissertation a few years later. BTW, for scenrio A, I have found that reading the journal version of the same paper (if the authors have published one) one tends to be much more illuminating than the conference version (at least in AI) since the authors have room to put in more detail.

Scenario C: You are well-versed with the field and have read research papers previously. In that case, think of yourself as a reviewer. In this scenario, you need to be able answer the following questions: (a) Were the authors technically correct in their assumptions? (b) How original was the idea? By this time, you should have a rough clustering in your head of the different types of angles people generally take when solving the problem. Can you figure out which cluster this paper belongs to? It would be even better if you can draw a link in your head between this paper and a previous paper that it is most similar to (many a times, the authors themselves will make this connection for you). TIP: Sometimes, you can even draw a conceptual/theoretical/programmatic link between this paper (which is trying to solve problem A) and another paper which was trying to solve problem B. These links are even more valuable and make the big picture clearer which is what you need when writing Chapter 1 of your dissertation. Is this an impactful paper? Did the authors just blow the field wide open with their angle? If so, you should make an attempt to read the other papers by the same author and also make sure to attend his talks/introduce yourself whenever you get a chance. (d) Did the authors do a good job of citing all the relevant papers? Perhaps they missed an entire set of papers that they should have cited but they failed to make the connection. If you have found something like this, run it by your advisor and send a friendly, informal note to the first author of the paper that they should take a look at these other papers. (e) Did the authors choose the right baseline to compare to? I have come across many a papers where the authors pick and choose which previous approach to compare against rather than choosing the real state-of-the-art. If this is the case, again run it by your advisor and drop a note to the authors.

Scenario D: You are very well versed with the field and are reading the paper to get an idea as to figure out whether you can implement the approach yourself. Taro already alluded to this scenario in his post above. I have found that most of the time, the details in a conference paper are woefully inadequate for the approach to be replicated. Again find a journal version of the paper. Since the goal here is to be intimately familiar with the approach taken by the authors, you need to expend more grey cells here than anywhere else. In addition to being able to answer all the questions from Scenario C, you now also have to pay attention to other things:

1. Data Type and Size: Did the authors use a large enough sample size for the results to be reliable? How does the data that you are planning to use compare to their data? Is it comparable in size? Is their approach predicated on using data of a particular type? If so, do you have that type of data?

2. Algorithmic efficiency: Did the authors use an efficient algorithm? Perhaps they didn't care since they used smaller data sizes in their experiments but if you are planning to inputs of a larger scale, you'll need to figure out how to make their code more efficient. Perhaps they are using a technique that is immediately improvable in terms of efficiency based on something you have read in the past?

3. Technical assumptions about inputs, outputs and the algorithm itself: Did the authors make any assumptions that may not hold in your case? For example, when using statistical models, one often makes independence assumptions that may not actually hold in the data. Make sure you understand the assumptions. If you don't think they will hold for your case, you might need to adapt their model mathematically and programmatically.

4. Evaluation: How did they evaluate their approach? First off, it's best to evaluate your implementation of their approach on the same data (and using the same metrics) so that you can get a direct sense of whether you were actually able to replicate their approach. Also, think about statistical significance. Many a times you can't replicate the authors' results exactly but as long as they are within a 95% confidence interval, you might be okay. A different issue might be the case that the data they evaluated on is not publicly available (which is not something one should do, IMO but sometimes one has no choice). In that case, you might want to email the authors to ask if they can run their system on a publicly available data (or even your data) and share the results with you privately.

In short, the process should be goal-driven rather than an open-ended process. Make sure to ask yourself the appropriate questions after reading the paper and also make sure that you can answer them. Also, use a bibliography manager like JabRef, Bibdesk (for Mac) etc. and store your answers to these questions there in the notes field like I said. It's also good to think about the larger context of the paper-reading beyond the primary goal of knowledge acquisition; paper-reading can also provide opportunities for professional networking, making friends and getting a higher-level sense of the field and the people in it.

I am a linguist. A desi linguist.
http://www.desilinguist.org
09-15-2010 04:53 AM
Visit this user's website Find all posts by this user Like Post Quote this message in a reply
The_Observer Away
Airport Layover Consultant
*********

The_Observer Away
Airport Layover Consultant
*********


Posts: 17,840
Likes Given: 118
Likes Received: 716 in 423 posts
Joined: Sep 2007
Unisearch: Link
Reputation: 384
Post: #2
RE: Some thoughts on reading research papers
0
0
Excellent post! Smile Applause

1) Please mention as many details as possible about your academics, research and work-ex, in terms of projects worked on, technologies used, roles/responsibilities handled, papers published/presented, awards/honors obtained, etc. All those have to be put in the misc details section of your UniSearch profile if you expect any help on Edulix. Also, before requesting for profile evaluations, please go through this excellent post.

2) Please take some time out to read these three threads in their entirety. PLEASE do that before asking questions about universities from a comparison perspective, or jobs, or coursework, or H-1B visas, or "placements", irrespective of which univ/department/program/major you're applying to.

I love to read, and then I regurgitate. I write - a lot - as my posts here and on other outlets would show. I do not make apologies for what I write (and how long it is). I do not sugarcoat things either. I don't tolerate vague questions, for any reasons - ignorance arising out of an inability to locate information when the necessary tools to do so (i.e. the internet, libraries, university websites, other forums/bulletin boards, etc.) are readily available, is not any kind of an acceptable excuse.

"Don't be daft." - Ancalagon The Black

"With the exchange rate where it is now, it should be a strong deterrent against picking a slightly better program for a lot more tuition fees." - coolguru
09-15-2010 10:02 AM
Visit this user's website Find all posts by this user Like Post Quote this message in a reply
a7x Offline
Edulix Senior Member
******

a7x Offline
Edulix Senior Member
******


Posts: 911
Likes Given: 0
Likes Received: 46 in 32 posts
Joined: Apr 2008
Unisearch: Link
Reputation: 43
Post: #3
RE: Some thoughts on reading research papers
0
0
Awesome Applause

Rarely on Edulix. If you need AI/ML help, someone will help you reach out.

Good luck with your applications, and if you are ever in Vancouver, let me know.
09-15-2010 11:22 AM
Find all posts by this user Like Post Quote this message in a reply
MadWiFi Offline
Edulix Maven
*****

MadWiFi Offline
Edulix Maven
*****


Posts: 889
Likes Given: 0
Likes Received: 0 in 0 posts
Joined: Mar 2009
Unisearch: Link
Reputation: 33
Post: #4
RE: Some thoughts on reading research papers
0
0
Fantastic ApplauseApplauseApplause

Not active in edulix now-a-days. Please shoot me an email iff its extremely urgent. I am not doing any profile evaluation till Fall 12. Thanks
Profile
My VI
09-15-2010 12:34 PM
Find all posts by this user Like Post Quote this message in a reply
iwish Offline
Edulix Super Senior
*******

iwish Offline
Edulix Super Senior
*******


Posts: 274
Likes Given: 44
Likes Received: 9 in 7 posts
Joined: May 2010
Unisearch: Link
Reputation: 4
Post: #5
RE: Some thoughts on reading research papers
0
0
awesome sirApplause

09-15-2010 03:42 PM
Find all posts by this user Like Post Quote this message in a reply
darkdroid Offline
New Edulixian
*

darkdroid Offline
New Edulixian
*


Posts: 2
Likes Given: 0
Likes Received: 0 in 0 posts
Joined: Sep 2012
Unisearch: Link
Reputation: 0
Post: #6
RE: Some thoughts on reading research papers
0
0
Excellent post !!!

Don't bother just to be better than your contemporaries or predecessors. Try to be better than yourself

My InfoBank
01-04-2013 06:14 PM
Find all posts by this user Like Post Quote this message in a reply
« Next Oldest | Next Newest »
Post Reply  Post Thread 


View a Printable Version
Send this Thread to a Friend
Subscribe to this thread