Why you can’t always rely on AI search results

It is easy to see why students may be tempted to paste an assignment title or project outline into a GenAI chatbot, such as Microsoft Copilot or ChatGPT, and ask for references on a topic. The tool will provide a list of sources and summaries of key studies which can appear to save time.

AI research assistant tools such as Elicit, Consensus and Semantic Scholar are designed specifically to identify and retrieve research literature, drawing on academic sources. While these specialised tools can help researchers discover relevant publications more efficiently, they do not remove the need to read, evaluate and verify the original sources before citing them.

Why is using GenAI chatbots a problem?

GenAI tools can help with discovering information, but they should not replace the process of searching for, evaluating, and reading sources. This means:

• Students may miss opportunities to develop information literacy skills. Effective academic research involves developing search strategies, selecting appropriate sources, and critically evaluating evidence.
• References may be incorrect or fabricated. Some GenAI tools still generate citations that appear genuine but do not exist or contain inaccurate publication details.
• Summaries may be misleading. AI tools may misinterpret findings, omit important context, or oversimplify complex arguments.
• The evidence base may not be fully represented. AI tools do not necessarily retrieve the most comprehensive or balanced set of sources.

Students should treat AI-generated references and summaries as prompts for further investigation rather than substitutes for reading and evaluating original sources.

How we support students behind the scenes 

Understanding and using GenAI appropriately is integrated throughout the University of Southampton Library Research Skills Framework, ensuring that we empower students to: 

• Select and use information tools appropriately, understanding the differences between GenAI tools, academic databases and other information sources, and choosing the most suitable approach for the task. 
• Critically evaluate GenAI outputs, recognising that they may contain inaccuracies, bias, incomplete perspectives, or fabricated information, and verifying claims against authoritative scholarly sources.
• Act with academic integrity and transparency, recognising the implications of GenAI use for plagiarism, attribution, falsification and responsible academic practice, and acknowledging AI use where required. 
• Engage directly with original sources, locating, reading, and critically evaluating publications before citing them.

Your role: signposting the UoS guidance for students on searching 

You can help your students develop the skills to question and verify AI-generated content by encouraging them to look at Library guidance that has been co-created with students on using (and not using) GenAI for academic searching. 

You can also encourage students to question GenAI outputs e.g.  

• Ask “Is this accurate? Where is the evidence? Does the response cite credible sources? Are all of the references genuine?” 
• Think critically about the response: could any voices or perspectives be missing? Remember that any biases in the training data will affect the output. 

Encourage cross-checking with trusted sources, for example checking library subject pages, academic databases, other library resources and authoritative websites (for example, government or NHS pages). 

Consider using GenAI tools to help develop and refine their search strategy, but not to conduct the search. 

Bite-sized task 

In this task you will watch the first 2 short videos from our GenAI and searching resource, and consider how the topics relate to your students. 

Step 1 – learn 

Watch the first two videos (total three minutes) featuring one of our students talking about GenAI and searching: Academic Searching and Generative AI

Step 2 – do 

Use Microsoft Copilot to search for literature on a topic you know well. How do the results compare with what you expect? 

• Are any key journal articles or other resources missing? 
• Are any perspectives missing? 
• Do all of the articles exist and are they correctly referenced? 

Step 3 – reflect 

• How would you summarise the key points from the videos or your own testing? 
• How might your students misunderstand or misuse GenAI tools for academic searching? 
• How could you share the GenAI and searching learning resource with your students? 

Share your thoughts on this week’s GenAI Essentials post in Teams and join in the discussion. 

Further links 

Searching and Generative AI  

Library Research Skills Framework 

© 2025. This work is openly licensed via CC BY-NC-SA