This is a review of a blog post by Amardeep Singh blogged about Idea for Discussion: An Academic Blog Review.
In theory, like others, I think the idea of a peer reviewed system for blogs is a good idea. Peer review has several advantages over the status quo; it offers authors feedback on posts they have written, it offers guildance to readers on where to find good posts, and blogs would possibly be able to be used in academic pursuits.
Blogs already have a feedback mechanism for authors. It’s called comments, and to a lesser extent trackbacks. However, these comments are almost always made anonymously or pseudo-anonymously such that nobody knows who actually wrote them. I always sign all my comments as N. Johnson with a link to my blog. It’s not hard to figure out who I am, I just like making people look a little for it. When the author of a comment is not know, his or her expertise can not be accurately judged. But when the identity of a reviewer is known, they sometimes are not as truthful if they think the article is not as good as it should be, perhaps fearing retaliation. I believe any peer reviewed for blogs would need to be an anonymous system such that the reviewers were previously vetted and were found to be experts in their field.
However, as Cognitive Daily: Peer review for blogs points out, the current academic peer review system doesn’t work all that great. It’s hard to get academics to review papers. Not because they don’t think it’s important, but because it requires a large amount of time which could be spent on other things, such as their own research. He quotes Singh’s answer to this problem.
My idea is to have a system of academic blog reviewing, where people self-select individual blog posts they’ve written for review by others, perhaps using a combination of Technorati tags and emailed links. The reviewers could consist of fellow bloggers (credentials no bar) as well as non-blogging academics in a given discipline, who would publish their reviews on a central site. The reviewers could choose to be “onymous” or pseudonymous (as long as it is a consistent pseudonym, and contact information is available to site admins), and be asked to write a significant evaluation to the post in question (say, 250 words). Other reviewers and readers of the reviews could also evaluate the reviewers’ comments, as a way of maintaining standards for reviewers. Troll-like, unfair reviews would be deleted, and their authors denied reviewing privileges.
As I said above, I don’t think having reviews open to everyone is a good idea. We already have that in comments. Reviews must be done by someone knowledgable about the topic. For instance, I would feel vastly unqualified to comment on blog posts at Cognitive Daily, but I may feel qualified (depending on the specific subtopic) to comment on posts at Fermi Paradox. Some people naturally only comment where they know they know they have enough specific knowledge to add something to the discussion, others just like to comment to comment. So besides that issue with his proposal, I think it sounds good; we need a way to evaluate posts and even entire blogs. I hope someone soon implements a system of peer reviews for blogs and it gets used.
