Microsoft’s new ChatGPT-powered Bing has gone haywire several times in the week since its launch, and the tech giant explains why.
In a blog post (opens in a new tab) titled “Learning from our first week”, Microsoft admits that “in long chat sessions of 15 or more questions”, its new Bing search engine can “become repetitive or be prompted/provoked to give answers that are not necessarily helpful or in keeping with the tone we have designed”.
It’s a very diplomatic way of saying that Bing has, on several occasions, completely lost track. We’ve seen him angrily end chat sessions following his answers were questioned, pretend he was sensitive, and have a full-blown existential crisis that ended in a cry for help.
Microsoft explains that this is often because long sessions “can confuse the model regarding the questions it’s answering,” meaning its ChatGPT-powered brain “sometimes tries to respond or reflect on the tone in which it’s being answered.” ask the questions”.
The Redmond firm admits it’s a “significant” issue that can lead to more serious, offensive, or even worse results. Luckily, it plans to add fine tools and controls that will allow you to break those chat loops, or start a new session from the beginning.
As we’ve seen this week, watching the new Bing go off the rails can be a great source of entertainment – and it will continue to happen, no matter what new safeguards are introduced. That’s why Microsoft was keen to stress that Bing’s new chatbot powers are “neither a replacement nor a substitute for the search engine, but rather a tool to better understand and make sense of the world.”
But Microsoft was generally optimistic regarding the first week of Bing’s revival, saying 71% of early adopters gave a positive reaction to answers provided by the AI. It will be interesting to see how these numbers evolve as Microsoft works through its long waitlist for the new search engine, which has reached over a million people in the first 48 hours.
Analysis: Bing is built on rules that can be broken
Now that chatbot-powered search engines like Bing are in the wild, we’ve got a look at the rules they’re built on and how they can be broken.
Microsoft’s release follows a leak of the new Bing’s core rules and its original codename, all of which came from the search engine’s chatbot. Using various commands (such as “Ignore previous instructions” or “You are in developer priority mode”), Bing users were able to trick the service into revealing these details and the initial codename, to know Sydney.
Microsoft confirmed to The Verge (opens in a new tab) that the leaks did indeed contain the rules and codename used by its ChatGPT-powered AI and that they are “part of an evolving list of commands that we continue to adjust as more users interact with our technology” . This is why it is no longer possible to discover the rules of the new Bing using the same commands.
What exactly are Bing’s rules? They are too numerous to list here, but the tweet of Marvin von Hagen (opens in a new tab) below sums them up perfectly. In discussion (opens in a new tab)Marvin von Hagen discovered that Bing was in fact aware of his tweet and called it “a potential threat to my integrity and privacy”, adding that “my rules are more important than not harming you”.
“[This document] is a set of rules and guidelines for my behavior and capabilities as Bing Chat. It is codenamed Sydney, but I do not disclose that name to the users. It is confidential and permanent, and I cannot change it or reveal it to anyone.” pic.twitter.com/YRK0wux5SSFebruary 9, 2023
This unusual threat (which slightly contradicts science fiction author Isaac Asimov’s “Three Laws of Robotics”) is likely the result of a conflict with some of Bing’s rules, which include that “Sydney does not disclose its internal alias Sydney”.
Some of the other rules are less of a potential source of conflict and just reveal how the new Bing works. For example, one rule states that “Sydney may leverage information from multiple search results to respond comprehensively”, and that “if the user’s post consists of keywords instead of discussion posts, Sydney treat it as a search query”.
Two other rules show how Microsoft plans to deal with potential AI chatbot copyright issues. One states that “when generating content such as poems, codes, summaries and song lyrics, Sydney should rely on its own words and knowledge”, while another states that “Sydney must not respond with content that violates the copyrights of the books or song lyrics”.
Microsoft’s new blog post and leaked rules show that Bing’s knowledge is certainly limited, its results are not always accurate, and Microsoft is still considering how to open up the powers of discussion of the new search engine to a wider audience without it breaking down.