We create two rigorous evals of LLM introspection and:
- we show introspection emerges with scale
- we *train-in* introspection (even on llama 1B!) by SFT'ing on perturbed forward passes, and performance generalizes! 🧵(1/5)
First, what is introspection (non-hype definition)? Models detecting perturbations to their activations.
Past work asked the model “Do you detect an injected thought?" But for small LLMs, adding a steering vector makes it say "Yes" to any question. This eval is confounded (2/5)
