13 Aug 2023 - Writer
If there is one thing that can be said about written communication in the academic world, it is its high degree of codification. Communication can take various forms, and while its type largely depends on the domain of research, it usually falls under one of the following generic options:
Short papers (that are often accompanied by a poster) are good vectors for presenting initial results or exposing a problem; they tend to be not definitive piece of work.
Conference papers typically focus on addressing a well-defined and specific problem through the presentation of a solution to that problem and an assessment of its benefits.
Journal papers are denser, deeper, and more mature pieces of work. They can be the result of several months/years of work and present an in-depth vision that significantly contributes to advancing the state-of-the art.
Magazine papers are in general less technical, more accessible (in terms of content) and designed to be read more rapidly. They aim at presenting an overview of a problem space or a vision.
Besides their format, written communications also vary in terms of the nature of the reported content: to shed the light on an emergent/important/pressing issue in order to encourage further discussion and future research, to detail pros and cons of a new solution, to justify novel design choices and demonstrate their benefits (qualitative and quantitative), to give an overview of a domain and highlight its past, present and future, etc. The latter is often referred to as survey. In practice, while the type of communication should be driven by the nature of the work to report, it is not rare for one to shape the actual work to meet the requirements of a specific format. In fact, and that can be very afflicting, the selected format is also very often driven by the prestige provided by a specific type of communication. Every type of paper has its inherent difficulty. Writing a (good) scientific paper is a pretty hard task. It is far from just being a compilation of subject-verb-complement sentences. It needs a structure. It needs a depth. It needs style. And it needs to coherently convey complicated stuff. There is however something that I find quite symptomatic about the case of survey papers in that these papers tend to be – very often – bad papers. It would definitely deserve a whole article to really explain what a good vs. a bad paper is. But in a nutshell, my definition of a good paper is a paper that teaches me something and / or opens up my perspective. In contrast, a bad paper is dull, boring, and contributes very little to enriching one’s knowledge. Please note that I am not talking about style here but about structure and content. I don’t want to discourage anyone who has been engaging in the development of a survey. I think however that it needs to be clear for everyone that writing a good survey paper is going to be tough. It might be because the task is so demanding (if properly carried out) that in 99% of the cases produced survey papers are literally no more than nice keyword-based filters implemented on Google Scholar (or Scopus or Web of Science). Yes, to survey does imply filtering and categorising information, but specifically in the case of research surveys, it is only one part. What makes a survey truly interesting is the depiction of how a research question or a domain has been evolving through produced knowledge. In that sense, survey work is by nature subjective. It reflects one’s vision of this evolution. A reason why I often get upset about survey papers, and their review, and their publication, is because these papers can attract a lot of citations. And when (unfortunately) the number of citations ends up being used as an indicator of the importance of a research, it becomes an issue. A good survey provides a point of view that is informed by experience (browsing through a paper and writing its summary does not qualify). It necessitates distance and general knowledge (taking for granted what three academics say is not sufficient). A large number of survey papers consists in groupings of sequentially enumerated references. In the best case, the referenced work comes with a vague (and often inaccurate) one-sentence description of its content. In the majority of the cases, however, the work is simply numbered. In my opinion, the added-value of these types of survey papers comes from nothing more than the static implementation of a filter function on Google Scholar. “Relevant” material is searched by keywords for the sole purpose of the survey. The paper sometimes explains how these keywords were selected, and provides some statistics on the obtained results, which I assume is to convey a form of principled approach in how the survey was compiled. I am not saying that these results are completely useless. In fact they could be very useful as guidelines for developing open-source research paper filter tools, for instance. But for the sake of presenting a vision, these contributions are pretty much void. This is a very personal opinion here but I believe that the survey paper market is today driven by major forces. Because these types of papers tend to attract a lot of citations, they are regarded as fantastic leverage by some academics. And the sad thing is that in many cases, these academics are likely not to be largely involved in the preparation of the survey. These papers are very often written by PhD researchers, who can also be very junior in a field. In fact, I would also argue that we may also see in this pattern a manifestation of an endemic problem in academia: there are countless PhD researchers left without any proper supervision and who, in the absence of guidance, can find comfort in accumulating readings. Far from me the idea of blaming PhD candidates, I went through that, with almost absent supervision from my official supervisor (I was saved by my unofficial second-supervisor, a person to whom I forever owed what I became as an academic, and who saved me from writing a survey). There is nothing worse than uncertainty to destroy self-confidence. After spending 9-10 months of their first year reading mountains of papers, and by then starting to grasp the mechanics of a self-contented research, PhD researchers can easily be confirmed in their own conviction that there is a value in that reading and that it would be good to share it with the rest of the community. The problem is that this reading is often not backed by any clear understanding of a specific problem, which very much reflects in how the survey is constructed: except a long list of references that the PhD researcher tried to categorise for his/her own good, there is nothing very interesting said in the sruvey. So I know, I may sound harsh. There are some good surveys out there, and there are also PhD researchers who can write very good surveys. My point is that they are rare, and that things really deserve to be improved. But who to blame for the lack of good surveys? Certainly not the PhD researchers. In fact I can often perceive exploitation and lack of concern from the academics listed on the top right of the paper through these papers. The supervisor? Certainly to some extent. As supervisor of less experienced researchers, our moral duty is to help them build confidence, help them develop a vision. But are only the supervisors the ones to blame? What about journals and magazines that accept these types of work for publication? Of course for them, it is good business since it increases their visibility and influence (see the impact factor). And what about reviewers? Can we as reviewers honestly claim that we learn so much from the listing of 167 references? So is the conclusion not to write any survey? I am convinced that good surveys are very valuable to research, to the community and to knowledge. A good survey provides a unique vision to a long term problem. It contributes to the intelligence of the domain: to a work of thinking and structuration. I will explore in detail in a subsequent article what I believe is needed to write a good survey. For the time being, let’s be honest, and let’s just not publish useless filter-type papers.