With AI being able to learn and solve problems just as a person would, it’s being utilised by organisations and individuals to write emails, predict customer behaviour and everything in between. The use of generative AI programmes such as Perplexity.ai and ChatGPT being used as search engines, however, has raised some potential issues.
OpenAI has announced it is officially launching a search engine in ChatGPT after a successful prototype was unveiled last July. The tool is based on a special version of OpenAI’s GPT-4o and feeds on the results of other search engines (such as Bing and Google), as well as content from information providers and media with which OpenAI has partnerships. Instead of producing a list of links when a search is typed in, the tool responds to queries in a conversational way and incorporates snippets and sources where users can click to learn more. It is also possible for users to refine their search by conversing with the search engine.
This is not the first tool launched to provide direct answers to users' questions (Google Quick Answers does this already), nor is it the first time users of ChatGPT are asking AI to answer questions they would ask a regular search engine. OpenAI is not the first company to combine a large language model and search engine (Bing/Copilot) nor to cite sources within answers (Perplexity.ai). However, ChatGPT Search is the first to formalise the use of all these elements in one tool using AI.
Conversational Interfaces vs Traditional Search Engines
With its combination of search engine and AI elements, users may think ChatGPT Search combines the best of both worlds. But what changes when you use a conversational interface that mentions sources rather than a traditional search engine?
Classic search engines show a list of results based on a question. They are authoritative with their references, and say to users, “here are the sites where you will find what you are looking for.” With conversational interfaces, however, the search engine is now authoritative over the information itself, saying, "here's the information you're looking for and here's the answer to your question."
This becomes problematic when it’s known that large language models can invent information and that conversational interfaces reinforce user trust. Conversational search engines also have an impact on the diversity of information sources available to users. With classic search engines, links that did not appear on the first page of results appear on the second and third page and so forth (although these are often overlooked by users). With ChatGPT Search or Perplexity.ai, these lower-ranking sources disappear completely, limiting the information the user receives.
User Friendly?
Researchers at the University of Washington published an article exploring the differences between search engines (Situating Search). As part of their research, they explored a variety of users and their reasons for using a search engine. Some users know what they are looking for, some want to explore or learn more about a subject, and some want to select the sources they are most confident they will find their answer.
These wide variety of uses are struggling to be supported by conversational search engines. By synthesising information, new tools do a majority of the heavy lifting. Users no longer have to scan and select the results they want, or reformulate their query. AI search engines, therefore, lead to less work for the user but are also limited in the way they can be used. Researchers concluded that we, “should be looking to build tools that help users find and make sense of information rather than tools that claim to do everything for them.”
Trustworthy Information
AI search engines like Perplexity.ai and the new Chat GPT Search indicate the sources on which their responses are based. For many users, the mention of the source and the possibility of tracing this information is a strong argument in favour of conversational interfaces.
However, a comparative study by Stanford University (Evaluating Verifiability in Generative Search Engines) saw researchers prove otherwise. The researchers looked at various tools such as Bing Chat, NeevaAI, Perplexity.ai and YouChat, and found that only half of the statements in the answers were fully supported by the sources indicated. In addition to this, a quarter of sources did not support the statement the AI search engine gave. This lack of accuracy is concerning as these tools have millions of users, most of whom will believe the signalling of sources is a sign of reliability rather than verifying every statement.
The study also found the statements the AI search engine gave that were less supported by sources, were more useful to users. The researchers' explanation for this was that the most reliable tools tend to copy or paraphrase statements in the sources, without the response being very fluid. Conversely, tools that deviate from the sources, such as conversational interfaces, have more freedom to generate fluid responses that seem important and useful.
The researchers also found that existing AI search engines struggle to process queries that cannot be answered through extracting information from multiple sources, and could not appropriately weight sources with relevance to the question posed by the user. The findings of the study, therefore, bring the reliability and trustworthiness of AI search engines into question.
Should AI be used as a search engine? Tools like ChatGPT Search and Perplexity.ai are certainly helpful if the user understands their benefits and limitations. Caution should be taken and tools such as these should be used alongside traditional search engines and other sources of information to find the answers you’re looking for.