Artificial intelligence chatbots experiencing delusions - and necessary alterations suggested by OpenAI scientists
In the realm of artificial intelligence, language models have been making significant strides, but a major concern has arisen: if primary evaluations continue to reward 'lucky guessing', these models may continue to learn to guess rather than provide accurate and reliable responses.
This issue has been highlighted by OpenAI, a leading research organisation in AI, who suggest that the primary evaluations need to be redesigned to prevent rewarding guesswork. However, OpenAI did not respond to a request for comment about the proposed changes.
The main problem lies in the abundance of evaluations that aren't aligned for language models. These models, such as Large Language Models (LLMs), are always in "test mode" and answer questions as if everything is binary. This binary approach can lead to inaccuracies, especially when dealing with issues where uncertainty is common in real life.
Human learning, on the other hand, values expressing uncertainty. Unlike language models, which are primarily evaluated based on tests where uncertainty is penalized, humans are encouraged to admit when they are unsure. This is a crucial aspect that current evaluations for language models seem to overlook.
Recognizing this gap, OpenAI announced that its researchers have redesigned the evaluation metric for large language models. The aim is to encourage uncertainty behaviour and avoid hallucination - the production of implausible, irrelevant, or factually inaccurate information. This change is a step towards ensuring that language models better reflect the complexities and uncertainties of real-world scenarios.
However, it's important to note that this is just one step in the ongoing process of improving language models. Widespread, accuracy-based evaluations need to be updated to prevent guessing and to encourage models to handle uncertainty more effectively. This will help to ensure that language models become more reliable and useful tools in our increasingly digital world.
In conclusion, the redesign of evaluation metrics for language models by OpenAI is a significant step towards addressing the issue of models learning to guess. By encouraging uncertainty and discouraging hallucination, these models can become more reliable and better equipped to handle the realities of life where uncertainty is common.
