Researcher u/dasjomsyeet systematically evaluated Jacobian Lens entropy (J-space entropy) as an error predictor on Qwen3-4B across 11,400 examples from seven datasets, finding task-dependent utility but not a universal hallucination detector. This work provides nuanced empirical insights into the practical limits of interpretability tools like Jacobian Lens, showing that internal entropy can complement output confidence for factual retrieval but fails on internalized misconceptions and varies greatly by task. The study used only one model (Qwen3-4B) and found that correct mathematical reasoning (GSM8K) had higher baseline entropy, and multiple-choice formatting weakened the signal (CommonSenseQA). A threshold tuned on TriviaQA failed on GSM8K, highlighting task dependency.
Background
Jacobian Lens is an interpretability technique developed by Anthropic that maps internal activations to verbalizable concepts in a 'global workspace.' Entropy in this J-space was hypothesized to indicate when a model is confidently wrong. This study tests that hypothesis rigorously across diverse tasks.