This presentation examines the transformative impact of AI on academic search, focusing on practical implications for librarians and researchers. In particular, this presentation will focus on two major ways in which significantly improved natural language processing capabilities from transformer-based models are affecting how narrative literature reviews are being done.
First, academic search engines are now achieving much higher-quality relevance by moving beyond lexical search to "understand" the meaning of words (so-called semantic search) using dense embedding matching. On top of that, agent-based searching capabilities from tools like Undermind.ai and PaperQA2, which mimic human iterative searching, show even bigger gains in search quality, trading off speed for better results.
Second, academic search engines are now not content to just show top relevant results from the query but are using the capabilities of modern large language models to extract and summarize data from top results. This is most often achieved using variants of the Retrieval Augmented Generation (RAG) technique, which produces an "answer" with citations. This is a widely seen feature in tools like Scopus AI and Primo Research Assistant, among others. Similar techniques are used to generate a "synthesis matrix" of papers, allowing researchers to quickly summarize and compare papers.
Unfortunately, while these new techniques offer significant benefits, they are not without trade-offs. For instance, replacing traditional lexical search with semantic search means accepting the "black-box" nature of these systems, and these systems also often result in less reproducible queries. Similarly, while generated answers grounded in search results can save time, they introduce challenges such as the potential for hallucinations with a lack of faithfulness to the cited sources. How can we mitigate these issues to ensure reliability and trustworthiness in AI-driven academic search tools?
This presentation will be recorded.
