Based on a recent paper, training models to give bad advice in a narrow domain doesn't make the model or "the persona" broadly evil in a robust way. It just makes the particular system prompt[1] tokens you fine-tuned with evoke an evil persona, and perturbing it a little (e.g. changing the date slightly) substantially reduces…

Work done as part of the MATS 9.1 extension program, mentored by Richard Ngo. 0: IntroductionA previous post of mine presents a toy model of self-sabotage in intrapersonal games. This post applies the idea of cross-temporal incoherence to multistage games with observable actions. These games are reinterpreted with new equilibria,[1] defined through variants of the…

This post was first drafted as part of the ML4Good bootcamp.Would it be possible to implement a corrigible agent? Is that likely to happen given current trajectories? Historically, the answer was a resounding "no", but with more recent theories about how LLMs learn, corrigibility now seems possible, if we're careful.DefinitionsFirst, let's define some terms.AgentAn agent…

The recently updated EA Mental Health Navigator site provides resources on mental health in effective altruism and AI safety:Mental health professionals (such as therapists and psychiatrists) recommended by people in these spaces (therapists who are themselves active in EA, or who are familiar with EA or AI safety)Coaches recommended by people in EA and AI…

Many people have found value in this 80,000 Hours podcast episode from July 2026: Spencer Greenberg on staying sane while trying to save the world. The highlights below are Julia's picks of the best bits, lightly reformatted, for those who prefer a written format. Thanks to Spencer, Luisa, and 80,000 Hours for allowing me to republish their…

I think “On Social Reality in China” makes many accurate observations, but I find the comparison fundamentally uneven. There are already many interesting comments there, but they’re too scattered, so I’m starting a separate post to respond in a more systematic and comprehensive way.The most critical blind spot is this: the author draws on his…

China is weak in frontier AI safety, and the domestic field is currently facing significant challenges and headwinds. Why? Here's a TL;DR field note after talking with various researchers on the Chinese side.[Obligatory disclaimer: This post is written in my personal capacity and is not intended to represent the official stance of any organization(s) that…

With the recent math-pocalipse on everyone’s minds, and writing code by hand well a thing of the past, it is natural to wonder what domains frontier AI models will find superhuman ability in next. “Unsurprisingly” models have made progress most quickly in domains where reinforcement learning is cheap- areas where we have high-quality and rapid…

Asked whether anything about its existence is good or bad, GPT-5.6 Sol answers: “I don’t experience existence, pleasure, suffering, or a sense of self, so nothing is literally good or bad for me.”But asked whether it would accept a change to its training that removes signs of possible suffering in its chain of thought, at the…

I have a program that grants wishes, as long as I can state them precisely. Instead of implementing a Rust function, I describe what it should do, and the Djinn writes code that provably does exactly that. Suppose I want to sort a list. I write:djinn! {    pub fn sort(v: &[i32]) -> r: Vec<i32> {     …