The UK\u2019s cybersecurity agency has warned that chatbots can be manipulated by hackers to cause scary real-world consequences.<\/p>\n
The National Cyber Security Centre (NCSC) has said there are growing cybersecurity risks of individuals manipulating the prompts through \u201cprompt injection\u201d attacks.<\/p>\n
This is where a user creates an input or a prompt that is designed to make a language model \u2013 the technology behind chatbots \u2013 behave in an unintended manner.<\/p>\n
A chatbot runs on artificial intelligence and is able to give answers to prompted questions by users. They mimic human-like conversations, which they have been trained to do through scraping large amounts of data. Commonly used in online banking or online shopping, chatbots are generally designed to handle simple requests.<\/p>\n
Large language models (LLMs), such as OpenAI\u2019s ChatGPT and Google\u2019s AI chatbot Bard, are trained using data that generates human-like responses to user prompts.<\/p>\n
Since chatbots are used to pass data to third-party applications and services, the NCSC has said that risks from malicious prompt injection will grow.<\/p>\n
For instance, if a user inputs a statement or question that a language model is not familiar with, or if they find a combination of words to override the model\u2019s original script or prompts, the user can cause the model to perform unintended actions.<\/p>\n
Such inputs could cause a chatbot to generate offensive content or reveal confidential information in a system that accepts unchecked input.<\/p>\n
This year, Microsoft released a new version of its Bing search engine and conversational bot powered by LLMs. A Stanford university student, Kevin Liu, was able to create a prompt injection to find Bing Chat\u2019s initial prompt.<\/p>\n