Journals, submission costs, and the benefits of inefficiency
In defense of suckiness
Note: this piece is currently under revision based on a response from an author of the paper discussed here.
I.
Academic publishing usually makes very little sense to people. How little sense this makes to them usually depends on how close to their revision deadline they are.
As funny as academic whining is, we should take this seriously. Academics are important, and while their utility depends on their contribution to human knowledge, their worth is usually judged by how often they publish and where they publish. If which academics get read depends at least a little bit on which journal they get published in, we should probably pay attention to everything about these journals. If Nature, the Harvard Law Review, or Synthese suddenly decided that all submissions must be made in a very pretty Notion page that must autoplay K-pop when you open it, we would be reading academic work only from people who have knowledge of both Notion and K-Pop. The fact that these rules are arcane and ridiculous is maddening enough, but we generally put up with all of it because peer reviewers are great and we’d all be writing nonsense™ without them. Also, we have no choice.
II.
All of this is an arcane ritual, and if you are one of the uninitiated or one of the long-suffering, you may safely escape to Part III without reading further. If you wish to find out, note that I am not responsible for the rage you will feel beyond this point. If you are an academic, this will either be cathartic or triggering.
For one, most journals require exclusive submission, which means that you can’t submit to more than one journal at the same time. Sometimes, you will get lucky and find the one™ on this first attempt. If you don’t, you try with someone else after you’ve spent six months in a back-and-forth. Authors are, in this sense, required to be serial monogamists. It is a very serious offense if you submit to more than one place at a time.
Some scholars attempt to game the system by submitting to many places at once. These two-timing rascals try to lead on multiple journals, take critical peer-review from all of them, and finally decide on the best one that accepts their paper. These people, like most two-timing rascals, often get caught. Both journals then decide that they are the scum of the earth, and refuse to speak to them after a while.
In addition to these rules, Journals also have other annoying requirements. Authors have to format their manuscripts in a particular way. Authors also have to footnote and reference their writing according to citation manuals that determine their worth based, among other sensible rules, on whether periods are italicized. The period at the end of this sentence is italicized. The one at the end of this sentence isn’t. Can you tell the difference? I can, and the fact that I can makes me a thoroughly reprehensible person.
If this wasn’t bad enough, different journals use different citation manuals and style guides which you must follow when submitting. For example, most law reviews use the Bluebook. Some others use OSCOLA, the Chicago Manual of Style, or something else so idiosyncratically unique and pointless that only an academic could have created it.
Now, a lot of people are annoyed, disheartened, or just plain tired about this state of affairs. Some of these people dislike resubmitting to new journals and having to reformat the whole damn thing. Some of these people are rather annoyed with how these rules are written, and how unclear they are. Some of them are just frustrated at how stupidly exploitative this system is.
III.
Why are journals so anal? Why do they make people do these things when they initially submit instead of after they accept? Why, oh why, can we not have nice things? There is a maxim that’s useful to formulate here, and it goes like this:
If X sucks for you, and X has sucked for a long time, there is a good reason why X has continued to suck.
Enter, this excellent new paper, which tells a completely different story, where suckiness can be good. This is the claim:
Counterintuitively, our analysis implies that inefficiencies in academic publishing (e.g., arbitrary formatting requirements, long review times) can serve a function by disincentivizing scientists from submitting low-quality work to high-ranking journals. Our models provide simple, powerful tools for understanding how to promote honest paper submission in academic publishing.
Upon reading this, I reacted much like a person suffering through racism and poverty who is told racism and poverty are actually useful for character development.1 It says that there are good reasons why academic publishing is sucky and that the suckiness actually has some useful effects. But immediate revulsion aside, I like this paper a lot, mostly because it tries to really explain why we can’t have nice things. It has some math, which usually puts me off, but I tried to read it anyway.
They present the general problem something like this.
Researchers sometimes try to make their research look better than it actually is. They sometimes do this by lying outright, but mostly by exaggerating its importance or hiding some aspects of their research protocols. The relationship between researchers and journals is therefore characterized by deception. Researchers also overestimate their own work, because of confirmation bias, as well as other reasons, and submit it to high-ranking journals. And so both low-quality and high-quality researchers benefit from sending articles to high ranking journals.
Journals, including unpaid peer-reviewers, must now spend a lot of time and effort trying to find out which articles contribute to the state of the art and which are falsified ramblings from the unknown. And this doesn’t always work, because sometimes these reviewers mess up, and let in a low-quality paper. This reduces the correlation between journal rank and paper quality. And if bad articles end up in good journals, people read more bad articles and think they’re good, and hire bad researchers thinking that they’re good.
When researchers are acting dishonestly and journals are efficient and there are no costs to submission (like doing some very annoying formatting), there really isn’t a reason not to submit disguised low-quality work to a high-ranked journal. Sure, you might not get accepted, but the possibility that you might get accepted is worth it because the costs are so low. The world slowly descends into chaos.
This is where submission costs come in. As the authors note:
Introducing submission costs creates a range of parameters in which honest submission is possible. The key insight is that imposing costs can promote honesty. This occurs because scientists who have low- and high-quality papers pay the same submission cost but have different expected benefits when submitting to high-ranking journals, as low-quality papers are less likely to be accepted.
Aside from this interesting tidbit, I am a legal academic who is unlikely to understand the rest of the game theory analysis, and I will try to get in touch with the authors in the hope that they will help me understand them better. But I’ll probably commit to understanding this kind of analysis even if they don’t because it seems far more interesting to me than the “oh here’s another flawed approach to justice” paper that every graduate student seems to write.
But the central implications here are clear. Making submission difficult, making resubmission difficult, and overall worsening the lives of researchers seems to have useful effects for the academy as a whole.
I think this is a necessary rejection of an idea that seems to have taken root, which is that a field is improved by improving the lives of people within that field. Of course, we should try to make things easier for researchers, but it’s not necessary that we do so in every way possible. But this paper is interesting because counterintuitive in a very commonsensical (counter-counterintuitive???) way.
Some questions. How much does any of this apply outside science? In analytical legal research, there is some aspect of rigorousness, but that happens to be mainly a matter of clear writing and reasoning, and therefore I’m not sure distinguishing between great law review papers and good-to-average law review papers is so important.
Could it be the case that these results make sense only for the sciences and that the rest of academia shouldn’t pay too much attention to it? Instead, we should probably just try to make the lives of our grad students and other researchers as bearable as possible. How much can it possibly matter, when the rest of us are making everything up anyway?
