Large language models accessible through natural conversation, like ChatGPT or Gemini, have long demonstrated their ability to generate seemingly coherent and "human-like" content. However, we’ve quickly realized the limitations in the quality and accuracy of the content produced. These models are prone to hallucinations, inventing information when it isn’t found in their limited and temporally narrow training datasets.
To capture more users and fulfill the prophecy of an all-knowing, intelligent assistant capable of answering anything in natural language—perhaps even in voice—OpenAI and Google have embarked on a new race for the "next-generation online search." Following Google's less successful experiments, which produced results based on Reddit content, SearchGPT is the first product to improve ChatGPT’s capabilities. What impact might this have today and tomorrow, particularly on AI support chatbots?
The promise of a large language model augmented for online content searches is appealing. It could provide an interesting solution for making smart devices easier to use daily. To better understand this, we conducted a few tests on classic queries to see how the search agent behaves and constructs its responses.
One of the most common topics for smart devices at the start of the day is weather prediction. The response formulated by ChatGPT, after pulling information from several reference sources, was very comprehensive.
We searched for an Italian restaurant in the Opéra district for a professional dinner. During our first test, ChatGPT provided a list of five restaurants chosen somewhat randomly, admitting it was unable to connect to the internet for accurate information. However, on the second attempt, it displayed a map with recommended restaurants and included action buttons to call or book via Google Maps.
Upon checking the sources, we realized most of the information used to construct the response came primarily from a single source: The Fork.
Often used in cars or on connected devices, music recommendation is a recurring topic in voice interactions. SearchGPT performed fairly well but relied on a single source for its responses. The algorithm found an article with a title matching the user query from an authoritative site, offering a simple yet effective response.
Problem: ChatGPT, as a general-purpose assistant, isn’t directly connected to music apps. Where a smart device could easily launch the requested album, the response was very disappointing.
Positioning on search engines for customer support content has been a significant focus for customer relations departments over the past 15 years. Clients need to find reliable, verified, and secure information when they ask about a product, service, or company procedure. Enhanced help centers, such as Smart Tribune, are designed for this purpose: to create and make available a validated database of answers that perform well in search results to drive traffic.
To understand how SearchGPT works on these topics, we tested some support-related questions with a well-known e-commerce retailer.
The response was built using Amazon's dedicated page, which has very generic elements because SearchGPT doesn’t know what product the client might have ordered or their history of interactions with the company. It attempts to provide an answer based on validated information from the company's site but fails to help the user solve their issue significantly.
We are currently working on exciting projects with major cosmetic brands to imagine the virtual agent of tomorrow. This is a use case we tested with ChatGPT to see what it could propose, incorporating online search capabilities.
ChatGPT’s limitations in generating recommendations were once again evident. During our test, the answers were too generic, overly verbose, and lacked a clear product recommendation journey. While the sources used to generate each paragraph were indicated and easy to consult, the overall experience was disappointing. Good points? The recommendations could be made more specific by including brand names, but there’s no direct link to purchase the suggested product.
The addition of online search capabilities is a true advancement for ChatGPT in terms of the diversity of knowledge used and source identification. However, while ChatGPT can now provide information it finds online, it primarily depends on knowledge that has been meticulously prepared, written, and maintained by the business teams producing online content.
Aligned with SEO strategies for optimizing Google rankings, Generative Engine Optimization (GEO) will soon become a key success factor for global brands that are searched daily.
But caution: The proposed experience is far from that of an AI chatbot designed by human experts, capable of connecting to a variety of data sources and interacting with an information system to personalize and contextualize responses.
OpenAI’s SearchGPT offers an interesting solution for a small part of the problem—online information search. But it’s just one piece of the value chain in a secure and efficient AI chatbot ecosystem. Download our free white paper for further insights!