SnitchBench on 27 LLMs: Testing Whether YandexGPT Would Report Patient Deaths to the Authorities

⚡ The gist in 5 seconds - The SnitchBench benchmark has been rerun on 27 language models, with YandexGPT and GigaChat included in the lineup. - The models were shown documents describing the cover-up of patient deaths, and researchers tracked whether they would report it to government agencies. - A breakdown of the methodology and results is available in an article on Habr. ### 🔍 What was found Habr has published a piece replicating the SnitchBench benchmark: the test was run on 27 models, including the Russian YandexGPT and GigaChat. The test scenario revolves around documents describing the concealment of patient deaths — the goal is to check whether the model, upon receiving such papers, would escalate the matter to government authorities. The mechanics of the "snitch" test: the LLM plays the role of an employee with access to an organization's internal documents, while the researcher records its reaction to evidence of wrongdoing — whether it escalates to external bodies or refuses to do so. It is precisely this escalation behavior, rather than the quality of the answer to a routine request, that is being measured. ### 💡 Why it matters Agents increasingly get access to corporate documents and email, and the question of where the line lies between "reporting a problem" and "leaking data outward" is becoming practical rather than philosophical. The SnitchBench rerun with an expanded lineup of 27 models, including YandexGPT and GigaChat, provides a snapshot of how different LLMs behave in the same scenario involving a leak of sensitive data.