In addition to the newly released OpenAI models, I have added Web Search to my LLM frontend. This allows up-to-date information to be worked with:
Prompt: when is the new German chancellor going to be sworn in?
Response: Friedrich Merz is scheduled to be elected as Germany’s new Chancellor on May 6, 2025. (reuters.com) […]
Source references are given inline ☝️ and at the end (after “Citations”):
[…]
Citations:
Search can, at least roughly, be steered using “site:…”:
Prompt: What’s the EUDI-Wallet? Use only site:namirial.com
Response: The European Digital Identity Wallet (EUDI Wallet) is an initiative by the European Union aimed at providing citizens, residents, and businesses with a secure and interoperable digital identity solution. This digital wallet will enable users to store and manage identification data and official documents in digital formats, such as driving licenses or medical prescriptions, facilitating access to public and private services across Europe. (focus.namirial.com)
[…]
The Search feature needs to be enabled deliberately, otherwise the integrated “LLM native” knowledge will be used - as before. The checkbox to turn it on/off is in the Settings section, below Python Use (Screenshot on X).
Although the search engine backend is provided by OpenAI, this behaves differently from ChatGPT: I noticed that, for instance, LinkedIn is not considered. This is currently a known, intended restriction of the OpenAI API.
Also caution when using non-flagship models like the new GPT-4.1-mini: these can and will mix things up, thereby introducing hallucination that users may have wanted to eliminate in the first place by using Web Search.
The basic working principle of this is old-school database-based web search + LLM to interpret these results. This contrasts with the new “Deep Research” approach, offered by OpenAI (through paid ChatGPT) and Google (currently free): Deep Research crawls the web on request, interpreting page content, adapting strategy, and finally producing a long report.
Also, the newly released o3 model leverages tool-use as part of it’s reasoning process. This appears not to be available through the API as of now.
Another restriction I have noticed with o3 is that streaming responses is not available to API users that have not been verified through the new verification process:
We’re adding the verification process to mitigate unsafe use of AI while continuing to make more advanced AI capabilities available to the broader developer community. Because individual availability of the capabilities doesn’t seem to be reflected in the API, the workaround I have chosen is to try streaming first and falling back to non-streaming - luckily, the non-streamed response can simply be wrapped.