- On February 14th, reports detailed that ChatGPT on Bing was making many factual errors
- This followed the first demo of Google Bard also returning numerous discrepancies
- Experts have called for generative AI tools to be used only for testing now
After Google’s new AI chatbot, Bard, made factual errors in its first live demonstration, issues with Microsoft’s new ChatGPT-powered Bing search engine and Edge web browser, have risen. According to reports, the ChatGPT-powered search service is making a factual errors, as flagged in a report by The Verge. These include failing to understand the exact details of a product, misreading a company’s financial report, and even misrepresenting the details of a popular restaurant in a country.
ChatGPT making errors
The errors come after Microsoft, on February 7th, hosted an event to unveil the latest version of its search engine and web browser, which is powered by a tweaked version of ChatGPT, called the Prometheus Model. This, however, appears to have caused issues with the system aboard Bing search.
According to a report by The Verge, these errors included misreading the figures in a December quarter financial report, published by clothing brand Gap. The latter returned figures that were erroneous, showing that the AI engine’s ability to read the fine prints of figures in a document, and apply its contextual understanding, remain limited.
These errors were further worsened in searches that included finding recommended restaurants in Mexico, and even comparing vacuum cleaner models — search phrases that make for fairly standard search engine activity. More such details were revealed by independent AI and search engine researcher, Dmitri Brereton, whose trysts with ChatGPT on Bing showed results where the latter reported that Croatia was removed from the European Union in 2022. A full list of errors made by ChatGPT during its live demo post unveiling can be accessed here.
It is important to note that ChatGPT on Bing has showed other issues as well, such as generating insulting remarks towards users, or even generating superiority-toned responses on a threatening note.
Why do AI tools make such errors
The issues come off the back of a warning from the developers themselves, who stated that Bing with ChatGPT is presently at an early stage, and may generate errors as its data set is improved. “It’s important to note that last week we announced a preview of this new experience. We’re expecting that the system may make mistakes during this preview period, and user feedback is critical to help identify where things aren’t working well so we can learn and help the models get better,” a statement by a Microsoft spokesperson, as reported by Mint, said.
The errors with ChatGPT and Bing come after Google unveiled Bard, its own generative AI tool powered by its LaMDA AI platform, a few hours before Microsoft rolled out the AI tool on its search engine. These errors typically occur due to a wide variety of reasons, including lack of accurate data sets to be trained on, or erroneous explanation of keywords, among other issues.
Experts warn against premature investments
Such errors have also added up to experts warning against making investments on AI platforms based on the generative tools bet. A case in point was Google, which saw its shares drop 7.7 percent due to errors made by Bard during its live demos. The drop led to Google losing over $100 billion in market value. The drop, however, did not deter investors from tapping businesses in China and betting on them to grow, leading to a phase of double-digit growth in AI startups and companies in the country.
However, experts have warned that such growth could be premature, and failure of generative AI platforms to create accurate replacements to traditional search platforms.
