I tried the “Instruct” finetuned variant of Falcon 40B (🥇 on OpenLLM leaderboard) to format a plaintext list of sightseeing recommendations in HTML and add some links. And this prediction engine (🙈), the one that “predicts what words are likely to come next” (🙈🙈), glitched out with:

… OK then, in accordance with the prophecy, I’ll visit Navigli. 😊 Seriously, though: the results as observed through the Huggingface chat demo are not good: when asking for pure JSON as a result, it works for simple inputs. But I found that more complex inputs derail it, resulting in an answer that includes a satisfactory JSON first and then roughly: “As an AI model, I cannot return JSON. But here’s some Python code: …”

Demo is here: https://huggingface.co/spaces/HuggingFaceH4/falcon-chat. They note:

🧪 This is only a first experimental preview: the H4 team intends to provide increasingly capable versions of Falcon Chat in the future, based on improved datasets and RLHF/RLAIF.