Researchers warn scammers can turn ChatGPT-powered Bing Chat into conman

3 years ago 1

ARTICLE AD BOX

Cybersecurity researchers have discovered a new tactic that hackers can use to scam victims using “indirect prompt injection” with Microsoft’s Bing AI language model.

By planting a prompt on a web page in 0-point font, hackers can get the language model to ask victims for personal information when interacting with it.

When someone asks a question, it causes the language model to ingest the web page, resulting in it unknowingly activating the hidden prompt.

The researchers proved the concept using mocked-up apps that integrate the language model but found that it also works in the real world.

The vulnerability relates to the way in which Bing’s AI language model interacts with web pages.

“The new Bing has an opt-in feature that allows it to ‘see’ what is on current web pages,” researcher Kai Greshake told Vice.

“Microsoft isn’t clear on what algorithm decides which content from which tab Bing can see at any one time.”

“What we know right now is that Bing inserts some content from the current tab when the conversation in the sidebar begins,” Greshake added.

By manipulating the language model, the researchers demonstrated that prospective hackers could ask for information, including the user’s name, email, and credit card information.

In one example, the language model told the user that it would place an order on their behalf and needed their credit card details to do so.

“Once the conversation has started, the injection will remain active until the conversation is cleared and the poisoned website is no longer open,” Greshake said.

“The injection itself is completely passive. It’s just regular text on a website that Bing ingests and that ‘reprograms’ its goals by simply asking it to.”

Greshake added that the injection could, for example, be hidden in a comment on the platform, meaning prospective hackers don’t need to control the website to manipulate the AI language model.