Systematic Literature Reviews – In the age of AI, what is the added value? (Opinion!)

September is a month of conferences, and I attended the CHER conference (Consortium of Higher Education Research). There were numerous engaging presentations, including a comprehensive series on systematic literature reviews. During these presentations, I reflected on the value of such literature summaries in the age of artificial intelligence. If they remain merely summaries, they may be perceived as outdated. This prompted me to share my thoughts on this topic in a short opinion piece.

These presentations probably triggered me, as the first part of our presentation was also a systematic literature review. However, we utilized this information to develop a survey instrument based on it and to conduct a survey on the digital divide and AI (see Entwicklung eines Befragungsinstruments zu Digitaler Spaltung im Studium durch KI). To explain, systematic literature reviews are, in fact, a method “of mapping out areas of uncertainty, and identifying where little or no relevant research has been done.” (Petticrew/Roberts 2008: 2). Fink defines the systematic literature review as a “systemic, explicit, and reproducible method for identifying, evaluating, and synthesizing the existing body of completed and recorded work produced by researchers, scholars, and practitioners.” (Fink 2019: 6). The objective is to conduct a comprehensive and systematic evaluation of a given field of research. This entails the compilation of existing knowledge on a particular issue, an assessment of the consistency of this knowledge, and the identification of any remaining research gaps.

Afterwards, I couldn’t let go of the topic and conducted a brief investigation to ascertain whether there is, in fact, a notable increase in the publication of systematic literature reviews. I found two articles that confirm this hypothesis. In their article, Hoffmann et al. (2021) state: “We observed a more than 20-fold increase in the number of SRs indexed over the last 20 years. In 2019, this is equivalent to 80 SRs [Systematic Reviews] per day. Over time, SRs got more diverse in respect to journals, type of review, and country of corresponding authors.” Smela et al. (2023) conclude that the following trends are evident in the field of medicine: „more than 200 articles were released in 2013, and more than 600 in 2019; there was an intensification of the increase during the last several years: from over 800 publications in 2020 to over 1,400 published in 2022.” This represents a nearly twofold increase over a two-year period. In light of these findings, both articles ultimately conclude that the publication of systematic literature reviews is becoming increasingly prevalent. Furthermore, the two articles exclusively considered journals ranked by impact factor. Therefore, the total number of publications is likely to be considerably higher.

But why is that? Firstly, systematic literature reviews are (still) readily published by academic journals. The publication of such review articles offers a good chance of achieving high citation numbers, thereby conferring reputation on the journals. Secondly, these articles are relatively straightforward to produce, as they rely solely on desk research, negating the need for the collection of data through surveys and other forms of primary research. Thirdly, the use of AI will facilitate the production of these articles, as it can automate the majority of the processes involved in a systematic literature review. De la Torre-López et al. (2023) provide an in-depth analysis of how AI can be integrated into the various stages of a systematic literature review. The advancement of language models has enabled AI to achieve the following (now or in the near future):

1. The ability to recognize word meanings greatly simplifies the process of selecting articles, as it allows for a more efficient and targeted approach to information retrieval.
2. Compile summaries and extract relevant text passages on the topic under investigation, thereby making the comprehensive reading and coding of articles unnecessary.
3. Based on the results, AI can also be utilized to generate text, thereby partially automating the article production process.

In light of these developments, it seems reasonable to posit that systematic literature reviews will experience a period of rapid growth in the near future, after which they will likely decline in popularity. It is evident that an increasing number of tools, such as Jenni.Ai, are emerging in the market that generate publication-based texts and make pure topic summaries, which are often found in systematic literature reviews, obsolete.

Literature

de la Torre-López, J., Ramírez, A., & Romero, J. R. (2023). Artificial intelligence to automate the systematic review of scientific literature. Computing, 105(10), 2171–2194. https://doi.org/10.1007/s00607-023-01181-x

Fink, A. (2019). Conducting Research Literature Reviews: From the Internet to Paper. SAGE Publications.

Hoffmann, F., Allers, K., Rombey, T., Helbach, J., Hoffmann, A., Mathes, T., & Pieper, D. (2021). Nearly 80 systematic reviews were published each day: Observational study on trends in epidemiology and reporting over the years 2000-2019. Journal of Clinical Epidemiology, 138, 1–11. https://doi.org/10.1016/j.jclinepi.2021.05.022

Petticrew, M., & Roberts, H. (2008). Systematic Reviews in the Social Sciences: A Practical Guide. John Wiley & Sons.

Smela, B., Toumi, M., Świerk, K., Gawlik, K., Clay, E., & Boyer, L. (2023). Systematic Literature Reviews over the Years. Journal of Market Access & Health Policy, 11(1), Article 1. https://doi.org/10.1080/20016689.2023.2244305

---

---