Anthropic have released Claude 2, touting “two times better at giving harmless answers”. Initial reactions on the web seem favorable, particularly regarding the support of larger documents that ChatGPT. What I have seen so far with the regular end-user chat UI:
- A book of 1027 pages didn’t load: “Text extraction failed”. Loading a 62-pages legislative document, as well as a short contract in German did work
- Answers from the chatbot are not fully grounded in the document by default, it adds prior and outside knowledge. When asked, it admits to this (screenshot #2: the data protection URL was not in the document)
- When referring to a specific part within the document, it gives the page and section
- It did not catch editor’s notes in the 62 pages document, despite saying otherwise when asked to check if the document may be draft
So certainly not bad, but still not the entirely harmless AI you’re looking for.
In my early testing, it is really good. Not quite GPT-4, but much closer than any other model […] On the downside, don’t use Claude for data, even though it accepts CSV files. It hallucinates answers, while Code Interpreter is right.
Officially not available outside the US and UK, but “more globally available in the coming months”.