One problem with current AI Assistants including ChatGPT and Microsoft Copilot Chat is that they are chronically online: they are not a more or less pure LLM experience anymore, but search the web through tool-use. While this helps to keep answers up-to-date beyond the LLM training cut-off date, there’s a disadvantage in that the UIs don’t allow the user to turn this function off. As a result, Microsoft Copilot Chat has been shown to be susceptible to web-poising. Further, ChatGPT may add information from the web to the documents it got uploaded to work on: a document summary may thus not be faithful to the original, but contain additional (and perhaps incorrect!) information. Users may think that the model hallucinated, but that’s actually not the case. Rather, the built-in web search got in the way.

There seems, however, a hidden way to turn Web Search off. Leaked system prompts include the names of the web search tools, and users may prompt the LLM to turn them off. ChatGPT example:

Who is Nils Durner?

(Avoid using the web tool, particularly search() or open_url(url: str))

As a result, the question is answered from internal, LLM-native memory without backing from web sources. Additional grounding in input documents can be had as usual by asking the LLM to stay faithful to the original document(s) and not to include outside information.

For those looking at the OpenAI API/Prompts Playground because the Web Search tool “web_search_preview” can explicitely be switched off there: this works differently from the Web Search in ChatGPT in that its sources are restricted - e.g. LinkedIn results do not seem to considered. This is a known problem, and the PM hopes to improve it eventually (X thread).