OpenAI is trying to stop ‘AI hallucinations’ where ChatGPT just makes stuff up

After some prominent AI hallucinations landed in a court case, OpenAI has come forward saying that the team is trying to figure out a new method of training ChatGPT.

In a new research paper, OpenAI stated that the company would begin a new method of training to avoid hallucinations. The new process would involve “process supervision”.

Essentially, “process supervision” works with the AI’s trail of thought and “rewards the model for following an aligned chain-of-thought”. While it won’t be a professor popping a treat into a machine, it allows OpenAI to train the GPT models to follow similar patterns to a human being.

So far, this has been tested with a “MATH dataset”. By utilizing a series of problems with definite answers, OpenAI can begin to teach the AI model to hopefully land on the correct answer when in a language setting.

Hallucinations have been causing issues amongst those using ChatGPT, OpenAI’s chatbot. A recent case has seen a lawyer depend entirely on the AI, which made up a series of cases that the lawyer was going to reference.

What is an AI hallucination?

AI hallucinations is the term given to an AI utilizing the data it has on file and creating something out of nothing. As the AI doesn’t have any real context or actually able to parse the knowledge it has been fed in a human-like manner, it can often begin to combine things in an attempt to produce an answer.

This happened live on stage when Google announced Bard, its competitor to ChatGPT. The video was quickly pulled down when Google noticed this.

OpenAI and “alignment tax”

Part of the problem OpenAI and other AI labs are having with hallucinations, is something detailed in the paper. “Alignment tax” is a cost that reduces the performance of the AI itself. Despite this and trying to bring more accurate answers to ChatGPT, OpenAI has found that its new method brings a “negative alignment tax” within math problems.

If implemented with a language model, this could provide the next iterations of ChatGPT with a superior method of producing factual answers.