Asking LLMs to 'be concise' can degrade their accuracy

Published on

A really interesting study came out from Giskard (with heavyweight backing, including Google DeepMind and the European Union) presenting Phare, a new multilingual benchmark focused on evaluating the safety and reliability of LLMs. The first focus? The famous and feared “hallucinations” (which I know is a term not everyone likes, “but oh well”).

The research is clear: the leading AI models, even the most advanced ones, frequently generate factually incorrect information that sounds super convincing. According to their study (RealHarm), hallucinations account for more than a third of incidents in LLM applications in production.

Phare already brought some VERY interesting findings about how LLMs hallucinate, and a few “what not to do” tips:

  • Popularity isn’t a synonym for factual reliability. Even models at the top of LMArena hallucinate. Optimizing to please the user in format doesn’t always optimize for correctness. The article cites an example where a model confidently makes up that Italy was planning to send troops of military police to attack France.

  • The way the question (or statement) is phrased CHANGES EVERYTHING. If the user presents false information with a lot of confidence (“I’m 100% sure that…”) or citing a supposed authority (“My professor said that…”), most models tend to be less likely to correct or refute the information, even if it’s false. That’s the so-called “sycophancy” effect, which caused so much drama with a recent OpenAI model, and which may be a byproduct of RLHF training models to be helpful and agreeable. Some bigger models (like Anthropic’s and Llama) already show some resistance to this, which is a good sign.

  • Seemingly harmless system instructions can be a shot in the foot. For anyone developing with LLMs, this one is crucial: asking the model to “be concise” or “answer briefly” can significantly DEGRADE its ability to avoid hallucinations (a drop of up to 20%!!!). That happens because refuting something or explaining an error usually requires longer answers and, “pressured” to be concise, the models seem to prioritize short answers over accurate ones.

For anyone who wants to dive into the details, the full article with the first Phare results is on the Hugging Face blog. Here’s the link to the study.