Nvidia wants to limit AI with more guard rails for applications like ChatGPT

A new Github project page from Nvidia has been published, possibly giving some major insight into how the company wants to control AI in the future.

The explosion of AI applications is unprecedented. ChatGPT, Google Bard, and Bing are paving the way for language-based models to grow into whole new otherworldly things, as well as assisting with day-to-day research.

However, as it has been pointed out multiple times, ChatGPT and the roster of AI chatbots aren’t perfect. There are a lot of chances for misinformation, as well as manipulating the AI into giving you answers it really shouldn’t.

A new development from Nvidia, however, aims to reign in those using AI at a software level. The new “guardrails” system, or NeMo, is an open-source framework available from GitHub. It’s intended to allow developers to add safeguards to prevent conversations from steering off course.

In the description, Nvidia is attempting to allow developers to regain some control over the wily AI. It’s built with Python, and easily installed through the “pip” terminal command.

Nvidia project aims to curb malicious AI interaction

The project is split into different aspects that the guardrails can do. This includes “Topical Rails”, which should prevent the bot from going “off-topic”. In the example given, the core workings of the AI within the terminal are shown, including the process it takes to ensure a correct answer is given.

There’s also a jailbreak detection layer, that Nvidia says will force the AI to double-check if it is being taken advantage of.

It also has dubbed misinformation as “hallucinations” in a strange turn of phrase. However, the NeMo system is designed to prevent this from happening and keep the bot in check throughout the interaction.

Nvidia has decided to make this AI guardrails system open-source in an attempt to get the community surrounding artificial intelligence to contribute to it. This should stay the hand of those looking to create nefarious usage from the blunders that occur when using these bots.

It should also be noted that Nvidia and OpenAI, the developers behind ChatGPT, intend to use more than 30,000 GPUs to power the bot.