Skip to yearly menu bar Skip to main content


Workshop Thu, Jul 9, 2026 • 4:00 PM – 1:00 AM PDT HALL C

Workshop on Mechanistic Interpretability

Andrew Lee ⋅ Ivan Arcuschin
[ OpenReview] 

Abstract

Video

Chat is not available.

Schedule

Timezone: America/Los_Angeles
4:00 PM
Socializing
5:00 PM
Coffee break
5:30 PM
Opening remarks
Andrew Lee
6:00 PM
Keynote
8:00 PM
Lunch
9:00 PM
Keynote
11:00 PM
Coffee break
Dissociating the Internal Representations of Sycophancy in LLMs
Anthony Baez ⋅ Sheer Karny ⋅ Pat Pataranutaporn
Towards Measuring and Detecting Unverbalized Evaluation Awareness
Luis Montoya ⋅ Reilly Haskins ⋅ Benjamin Adams
Logit-Contribution Scoring Identifies Non-Literal Retrieval Heads
Aryo Gema ⋅ Beatrice Alex ⋅ Pasquale Minervini
Tracking Training Phases in Compositional Learning with Task-Agnostic Measures
Niclas Dern ⋅ Selma Mazioud ⋅ Jakob Heiss ⋅ Avrajit Ghosh ⋅ Curtis J McDonald ⋅ Gabriel Clara ⋅ Bin Yu
The Connectome of a Large Language Model
Ruixuan Deng ⋅ Zehao Jin
What is Missing? Explaining Neurons Activated by Absent Concepts
Robin Hesse ⋅ Simone Schaub-Meyer ⋅ Janina Hesse ⋅ Bernt Schiele ⋅ Stefan Roth
Large Language Models Generate Harmful Content Using a Distinct, Unified Mechanism
Hadas Orgad ⋅ Boyi Wei ⋅ Kaden Zheng ⋅ Martin Wattenberg ⋅ Peter Henderson ⋅ Seraphina Goldfarb-Tarrant ⋅ Yonatan Belinkov
Dual-Contrastive Sparse Autoencoders Reveal Features of Musical Interpretation
Jan Chen ⋅ Manuel Cherep ⋅ Pattie Maes ⋅ Nikhil Singh
Distill to Detect: Exposing Stealth Biases in LLMs through Cartridge Distillation
Shayan Talaei ⋅ Abhinav Chinta ⋅ Fnu Devvrit ⋅ Amin Karbasi ⋅ Azalia Mirhoseini ⋅ Amin Saberi
LoRAcles: Self-Supervised Weight-Space Interpretability at Scale
Celeste De Schamphelaere ⋅ Jan Bauer ⋅ Neel Nanda ⋅ Euan Ong
Old Habits Die Hard: How Conversational History Geometrically Traps LLMs
Adi Simhi ⋅ Fazl Barez ⋅ Martin Tutek ⋅ Yonatan Belinkov ⋅ Shay Cohen
Kronecker Factorization Improves Efficiency and Interpretability of Sparse Autoencoders
Vadim Kurochkin ⋅ Yaroslav Aksenov ⋅ Daniil Laptev ⋅ Nikita Balagansky ⋅ Daniil Gavrilov
Meltdown: Circuits and Bifurcations in Point-Cloud-Conditioned 3D Diffusion Transformers
Maximilian Plattner ⋅ Fabian Paischer ⋅ Johannes Brandstetter ⋅ Arturs Berzins
Invariant Reasoning Directions in Latent Trajectories of Language Models
Arun Vignesh Malarkkan ⋅ Manan R Choudhury ⋅ Utkarsh Byahut ⋅ Yash Ravindra Charde ⋅ Vivek Gupta ⋅ Yanjie Fu
Tracing Psychometric Inference in Large Language Models
Chih-Hao Hsu ⋅ Feng-Chun B Chou ⋅ Pin-Hao A Chen
SPICE: Simple Polysemantic feature Interpretation via Clustering-based Explanations
Sehyun Lee ⋅ Dahee Kwon ⋅ Damin Lee ⋅ Jaesik Choi
Probing Persona-Dependent Preferences in Language Models
Oscar Gilg ⋅ Pierre Beckmann ⋅ Daniel Paleka ⋅ Patrick Butlin
Beyond Steering Vector: Flow-based Activation Steering for Inference-Time Intervention
Zehao Jin ⋅ Ruixuan Deng ⋅ Junran Wang ⋅ Xinjie Shen ⋅ Chao Zhang
What Does a Chemical Language Model Know About Molecules?
Christian Kenneth ⋅ Etowah Adams ⋅ Liam Bai ⋅ Gerard J van Westen
fmxcoders: Factorized Masked Crosscoders for Cross-Layer Feature Discovery
Andreas Demou ⋅ Panagiotis Koromilas ⋅ James Oldfield ⋅ Yannis Panagakis ⋅ Mihalis Nicolaou
The Information Geometry of Softmax: Probing and Steering
Kiho Park ⋅ Todd Nief ⋅ Yo Joong Choe ⋅ Victor Veitch
Unstable Features, Reproducible Subspaces: Understanding Seed Dependence in Sparse Autoencoders
Gleb Gerasimov ⋅ Timofei Rusalev ⋅ Nikita Balagansky ⋅ Daniil Laptev ⋅ Vadim Kurochkin ⋅ Daniil Gavrilov
Feature Geometry of Language Models Transfer Across Modalities to Time Series
Prateek Humane ⋅ Alexis Roger ⋅ Zhenghan Tai ⋅ Gwen Legate ⋅ Andrei Mircea ⋅ Vasilii Feofanov ⋅ Irina Rish
Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistants
Vincent Huang ⋅ Dami Choi ⋅ Daniel Johnson ⋅ Sarah Schwettmann ⋅ Jacob Steinhardt
Observing and Controlling Features in Vision-Language-Action Models
Hugo Buurmeijer ⋅ Carmen Amo Alonso ⋅ Aiden Swann ⋅ Marco Pavone
Personas Seem to Shape How Models Represent Behaviours
Jacob Davies ⋅ Rhea E Karty ⋅ Girish Gupta ⋅ Nathaniel M Hadida ⋅ David Williams-King ⋅ Andrew Draganov
Seeing to Generalize: How Visual Data Corrects Binding Shortcuts
Nicolas Buzeta ⋅ Felipe del Rio ⋅ Cristian Hinostroza ⋅ Denis Parra ⋅ Hans Lobel ⋅ Rodrigo Toro Icarte
Hidden-State Similarity Predicts Re-Elicitation After Inoculation Prompting
Rose N Kwon ⋅ Samy Mammeri ⋅ Sabyasachi Sahoo ⋅ Christian Gagne ⋅ Thomas Jiralerspong
A Mechanistic View of Authority Hierarchy in LLM Sycophancy
Emil Joswin ⋅ Srujananjali Medicherla ⋅ Priyanka Mary Mammen
Subliminal Learning Is Steering Vector Distillation
Camila Blank ⋅ Agam Bhatia ⋅ Senthooran Rajamanoharan ⋅ Arthur Conmy ⋅ Neel Nanda
Feature-Resolved Attention
Dmitry Manning-Coe ⋅ Nura Aljaafari ⋅ Jamie Stephenson ⋅ Hemang Jain ⋅ Ketan Kapre
A Case Study of How Intuition and Reasoning Interact in Language Models
Simon Henniger ⋅ Arnau Marin-Llobet ⋅ Rhea E Karty ⋅ Nada Amin
The Model Organism Lottery: Model Organism Interpretability Strongly Depends on Training Methodology
Andrzej Szablewski ⋅ Raffaello Fornasiere ⋅ Gabriel Konar-Steenberg ⋅ Nikita Menon ⋅ Stefan Heimersheim
The Unembedding Bottleneck: A Mechanistic Analysis of Single-Digit Counting in LLMs
Satwik Sunnam ⋅ Raghav Magazine ⋅ Vatsalya Singh ⋅ Lavanya Kotha ⋅ Xingjian Li ⋅ Min Xu
Hallucination as Commitment Failure: Larger LLMs Misfire Despite Knowing the Answer
Je Won Yeom ⋅ Jaewon Sok ⋅ Heejun Kim ⋅ Park Seonghyeon ⋅ Jeongjae Park ⋅ Taesup Kim
Step-Wise Refusal Dynamics in Autoregressive and Diffusion Language Models
Eliron Rahimi ⋅ Elad Hirshel ⋅ Rom Himelstein ⋅ Amit LeVi ⋅ Avi Mendelson ⋅ Chaim Baskin
Feature Recovery Requires Structured Event Regimes in Sparse Reconstruction
Daniil Laptev ⋅ Vadim Kurochkin ⋅ Gleb Gerasimov ⋅ Timofei Rusalev ⋅ Nikita Balagansky ⋅ Daniil Gavrilov
Demystifying Variance in Circuit Discovery of LLMs
Frank Zhengqing Wu ⋅ Francesco Tonin ⋅ Volkan Cevher
Mechanistic Capability Probes as a Cheap Screen for Sequence-Mixer Architectures
Kia Ghods ⋅ Tharun Kumar Tiruppali Kalidoss ⋅ Elad Hazan
Assign and Add: A Mechanistic Study of Compositional Arithmetic
Brady Exoo ⋅ Alberto Bietti ⋅ John Sous
Localizing RL-Induced Tool Use to a Single Crosscoder Feature
Andrii Shportko ⋅ Shubham Bhokare ⋅ Ahmed Zeyad A Alzahrani ⋅ Bowen Cheng ⋅ Gustavo Mercier ⋅ Jessica Hullman
On the Role of Mechanistic Interpretability for Vision-Language Prompt Learning
Shivanand Kundargi ⋅ Ziwei Zhang ⋅ Kowshik Thopalli ⋅ Tejas Gokhale
Weight-Space Geometry of Offline Reasoning Training
Aleksandr Nikolich ⋅ Igor Kiselev ⋅ Vladimir Platonov ⋅ Karina Romanova
Surrogate Fidelity: When Can Open LLMs Explain Closed Ones?
Philippe Chlenski ⋅ Zachariah Carmichael ⋅ Ayush Warikoo ⋅ Chia-Tse Shao ⋅ Yingxiao Ye ⋅ Aobo Yang ⋅ Vivek Miglani ⋅ Nehal Bandi
Sparse Autoencoders Can Learn Graded Latents for Relational Composition
Theo Farrell ⋅ Patrick Leask ⋅ Noura Al Moubayed
Rethinking Relation-Specific Neurons in Large Language Models
Lea Hirlimann ⋅ Sebastian Gerstner ⋅ François Yvon ⋅ Hinrich Schuetze
Relational Linear Properties in Language Models: An Empirical Investigation
Giovanni Valer ⋅ Luigi Gresele ⋅ Marco Bronzini ⋅ Emanuele Marconato
Crafting Reversible SFT Behaviors in Large Language Models
Yuping Lin ⋅ Pengfei He ⋅ Yue Xing ⋅ Yingqian Cui ⋅ Jiayuan Ding ⋅ subhabrata mukherjee ⋅ Hui Liu ⋅ Zhen Xiang
Reasoning as State Transition: A Representational Analysis of Reasoning Evolution in Large Language Models
Siyuan Zhang ⋅ Jialian Li ⋅ Yichi Zhang ⋅ Xiao Yang ⋅ Yinpeng Dong ⋅ Hang Su
LLMs can annotate attribution graphs
Ameen Patel ⋅ Max Zhang ⋅ Nathan Z Hu
How Do Language Models Compose Functions?
Apoorv Khandelwal ⋅ Ellie Pavlick
Making LLMs Say What They Think: Measuring and Improving CoT-Interpretability Alignment
Yihuai Hong ⋅ Shauli Ravfogel ⋅ Chen Zhao ⋅ Eunsol Choi
Validating Causal Abstraction Metrics on Simulated Complex Systems
Maxime Méloux ⋅ Tiago Pimentel ⋅ François Portet ⋅ Maxime Peyrard
Rational Sparse Autoencoder
Naiyu Yin ⋅ Yue Yu
Local Causal Attribution of Chain-of-Thought Reasoning
Dennis Wei ⋅ Yannis Belkhiter ⋅ Erik Miehling ⋅ Radu Marinescu
Logit Distance Bounds Representational Similarity
Beatrix M. G. Nielsen ⋅ Emanuele Marconato ⋅ Luigi Gresele ⋅ Andrea Dittadi ⋅ Simon Buchholz
Where Do Reasoning Models Refuse?
Kureha Yamaguchi ⋅ Benjamin Etheridge ⋅ Andy Arditi
Latent Undertow: How Ordinary Typos Break Probes
Elad David ⋅ Max Fomin ⋅ Amit LeVi
Mitigating Reward Hacking via Task Representations
Lillian Sun ⋅ Joe Benton ⋅ Eric Easley
Ghost Heads Across Training: When Greedy and Distributional Patching Disagree
Unai Zulaika ⋅ Miguel Fernandez-de-Retana ⋅ Rubén Sánchez Corcuera ⋅ Aritz Bilbao Jayo ⋅ Aitor Almeida
Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs
Sinie van der Ben ⋅ Raphaël Baur ⋅ Yannick Metz ⋅ Mennatallah El-Assady
Probing Clinical Concepts in an EHR Foundation Model via Sparse Autoencoders
Shashank Yadav ⋅ David M Routman ⋅ Andrew Y. K. Foong
Lyapunov Spectral Analysis of Loop Transformer Dynamics
Fynn Kiwitt ⋅ Christopher Irwin ⋅ Riccardo Maria Ali ⋅ Pietro Lió
A Mechanistic Study of Transformers Training Dynamics
Ambroise Odonnat ⋅ Wassim Bouaziz ⋅ Vivien Cabannnes
Emergent Symbolic Structure in Health Foundation Models: Extraction, Alignment, and Cross-Modal Transfer
Gajendra Katuwal ⋅ Advait Koparkar ⋅ Salar Abbaspourazad ⋅ Anshuman Mishra ⋅ sarvesh kirthivasan
Quantifying Subliminal Behavioral Transfer Ratios in Language Model Distillation
Uwe König ⋅ Hamza Kazmi ⋅ Ruizhe Li ⋅ Maheep Chaudhary
Reading Between the Dots: Decoding Hidden Computation across Filler Tokens
Kaley Brauer ⋅ Claudio Mayrink Verdun ⋅ Samuel Marks
Decoded but Unused: Instruction Tuning Routes Moral Framing into the Judgment Readout
Phongsakon Mark Konrad ⋅ Toygar Tanyel ⋅ Serkan Ayvaz
Position: Causality is Key for Interpretability Claims to Generalise
Shruti Joshi ⋅ Aaron Mueller ⋅ David Klindt ⋅ Wieland Brendel ⋅ Patrik Reizinger ⋅ Dhanya Sridhar
Pretraining Data Statistics Shape the Phases of Learning Entity Comparison in Language Models
Yik Siu Chan ⋅ Jing Huang ⋅ Yanai Elazar ⋅ Atticus Geiger
Detecting Whether an LLM has been Backdoored
Anthony Hughes ⋅ Nicole Xing ⋅ Andy Kim ⋅ Collin Francel ⋅ Andrew Draganov
Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?
Anton Korznikov ⋅ Andrey V Galichin ⋅ Alexey Dontsov ⋅ Oleg Rogov ⋅ Ivan Oseledets ⋅ Elena Tutubalina
Bigger Is Not Better: Inverse Scaling and Arbitration Failure in Counterfactual Visual Grounding
Fabian Grob ⋅ Sanghwan Kim ⋅ Cordelia Schmid ⋅ Zeynep Akata
Subliminal Learning is Non-Semantic Distillation
Ethan Hadley ⋅ Eren Gultepe
Probing the Misaligned Thinking Process of Language Models
Kaiwen Zhou ⋅ Constantin Venhoff ⋅ Jonathan Michala ⋅ Xin (Eric) Wang ⋅ William Saunders
A Low-Rank Subspace Analysis of LLM Interventions
Angira Sharma ⋅ Christian Schroeder de Witt ⋅ Phil Torr ⋅ Anisoara Calinescu ⋅ Jialin Yu
Preference Instability in Reward Models: Detection and Mitigation via Sparse Autoencoders
Shunchang Liu ⋅ Xin Chen ⋅ Belen Martin Urcelay ⋅ Francesco Croce
From Tokens to Policy: Minimal and Sufficient Heterogeneous Treatment Effects Identification
Riccardo Cadei ⋅ Frank Otchere ⋅ Nyasha Tirivayi ⋅ Gustavo A Tagliaferro ⋅ Falco J. Bargagli-Stoffi ⋅ Francesco Locatello
How feature learning breaks the curse of dimensionality in isotropic data:\\ A mean field approach
Niclas Göring ⋅ Yoonsoo Nam ⋅ Chris Mingard ⋅ Jake Reid ⋅ Ard Louis
Mechanistic Evidence for Preserved-but-Misaligned Representations in Non-IID FedAvg
Muhammad Haseeb ⋅ Salaar Masood ⋅ Muhammad Abdullah Sohail ⋅ Mohammad F Shoaib ⋅ Muhammad Tahir
A loss curvature account of fine-tuning fragility
Ivaylo Dimitrov ⋅ Leo Karoubi ⋅ Sunny Howard ⋅ Dmitrii Krasheninnikov
ExPLAIND: Unifying Model, Data, and Training Attribution to Study Model Behavior
Florian Eichin ⋅ Yupei Du ⋅ Philipp Mondorf ⋅ Maria Matveev ⋅ Barbara Plank ⋅ Michael Hedderich
SHIFT: Steering Hidden Intermediates in Flow Transformers
Nina Konovalova ⋅ Ibragim Idrisov ⋅ Aibek Alanov
Histoscope: Expert-Grounded Inspection of Sparse Autoencoder Features in Histopathology Foundation Models
Mirza Nasir Hossain ⋅ Sarah L Bell ⋅ Gareth Bryson ⋅ David Harris-Birtill
Look Before You Steer: Geometry Predicts SAE Feature Steerability
Muhammad Khan ⋅ Shlok Channawar ⋅ Akshaj Gurugubelli ⋅ Girish Gupta ⋅ Aditya Shah
Compiling to transformers
Joey Velez-Ginorio
Interpretability Transfer from Language to Vision via Sparse Autoencoders
Alexey Kravets ⋅ Da Li ⋅ Chuan Li ⋅ Da Chen ⋅ Vinay Namboodiri
The Progress Helix Tracks Reasoning Depth in Language Models
Andrew Rufail ⋅ Andy Huang ⋅ Zarreen Reza ⋅ Henry Lue ⋅ Laurentia Liennart
The data manifold under the microscope
Marios Koulakis ⋅ Constantin Seibold
Scaling Laws for SAE Training Data
Nikita Koriagin ⋅ Nikita Balagansky ⋅ Daniil Gavrilov
Sparse Autoencoders are Capable LLM Jailbreak Mitigators
Yannick Assogba ⋅ Jacopo cortellazzi ⋅ Javier Abad ⋅ Pau Rodriguez ⋅ Xavi Suau ⋅ Arno Blaas
Mechanistic Interpretability of Adversarial Suffixes Reveals Non-Robust Shortcuts of Safety Monitors
David S Cheung ⋅ Jason Vega ⋅ Hunar Pasricha ⋅ Brian Kim ⋅ Milind K Lalwani ⋅ Enyi Jiang ⋅ Gagandeep Singh
Fixed Universal Transformer
Jingwen Liu ⋅ Alexandr Andoni ⋅ Daniel Hsu
The parameters in weight-sparse transformers are interpretable
Arnau Marin-Llobet ⋅ Stefan Heimersheim
Reasoning aligns language models to human inference
Gonçalo Guiomar ⋅ Elia Torre ⋅ Pehuen Moure ⋅ Victoria Shavina ⋅ Mario Giulianelli ⋅ Shih-Chii Liu ⋅ Valerio Mante
LatentCompass: T2I Diffusion Steering via Orthogonal Attribute Spaces for Debiasing, Concept Erasure, and Red Teaming
Sepehr Dehdashtian ⋅ Mashrur M. Morshed ⋅ Milad Afshari ⋅ Jacob Seidman ⋅ Vishnu Boddeti
Cross-Model Circuit Discovery
Harrish Thasarathan ⋅ Matthew Kowal ⋅ Thomas Fel ⋅ Konstantinos Derpanis
Position: Interpretability Can Be Actionable
Hadas Orgad ⋅ Fazl Barez ⋅ Tal Haklay ⋅ Isabelle Lee ⋅ Marius Mosbach ⋅ Anja Reusch ⋅ Naomi Saphra ⋅ Byron Wallace ⋅ Sarah Wiegreffe ⋅ Eric Wong ⋅ Ian Tenney ⋅ Mor Geva
Residue-Level Attributions in Protein Allergenicity Models Do Not Recover Allergen Epitopes
Jianzhou Yao ⋅ Anxiong Song ⋅ Katja Baerenfaller ⋅ Damir Zhakparov
Moir: Let the Model Direct Its Own Story for Robust Cross-Domain Knowledge Editing
Jea Kwon ⋅ Jiwon Kim ⋅ Dong-Kyum Kim ⋅ MEEYOUNG CHA
How Large Reasoning Models Solve Problems: A Mechanistic Study on Tower of Hanoi
Yukang Yang ⋅ Mengdi Wang ⋅ Jonathan Cohen ⋅ Taylor Webb
When Graph Tokens Sink: A Mechanistic Analysis of Graph Language Models
Ding Zhang ⋅ Runtao Zhou ⋅ Wenqing Zheng ⋅ Rizal Fathony ⋅ C. Bayan Bruss ⋅ Chirag Agarwal
The Platonic Universe: Do Foundation Models See the Same Sky?
Trinidad Borrell ⋅ Steven Dillmann ⋅ Kshitij Duraphe ⋅ Furkan Eris ⋅ Ashod Khederlarian ⋅ Aman Kumar ⋅ Giovanni F Marraffini ⋅ Michael J Smith ⋅ Shashwat Sourav ⋅ Rocco Di Tella ⋅ John F. Wu
Censored LLMs as a Natural Testbed for Secret Knowledge Elicitation
Helena Casademunt ⋅ Bartosz Cywiński ⋅ Arya Jakkli ⋅ Samuel Marks ⋅ Neel Nanda
Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior
Daniel Wurgaft ⋅ Can Rager ⋅ Matthew Kowal ⋅ Vasudev Shyam ⋅ Sheridan Feucht ⋅ Usha Bhalla ⋅ Tal Haklay ⋅ Eric Bigelow ⋅ Raphaël Sarfati ⋅ Thomas McGrath ⋅ Owen Lewis ⋅ Jack Merullo ⋅ Noah Goodman ⋅ Thomas Fel ⋅ Atticus Geiger ⋅ Ekdeep Singh Lubana
Steering at the Source: Style Modulation Heads for Robust Persona Control
Yoshihiro Izawa ⋅ Gouki Minegishi ⋅ Koshi Eguchi ⋅ Sosuke Hosokawa ⋅ Kenjiro Taura
When Is an Attention Head a Computational Unit? Shared Circuits Despite Orthogonal Representations
Casper Christensen ⋅ Loren Amdahl-Culleton ⋅ Kyle Ray ⋅ Javan Tahir ⋅ Paul Riechers ⋅ Adam Shai
Finding Interpretable Prompt-Specific Circuits in Language Models
Gabriel Franco ⋅ Lucas M Tassis ⋅ Azalea Rohr ⋅ Mark Crovella
Out of Context Obfuscation: What Facts Matter for Probe Evasion?
Fadi Benzaima ⋅ Faraz Ahmed ⋅ William Soylemez ⋅ Adrians Skapars ⋅ Rohan Gupta
NeuroFaith: Evaluating Mechanistic Faithfulness of LLM Free Text Self-Explanation at the Concept Level
Milan Bhan ⋅ Jean-Noël Vittaut ⋅ Nicolas CHESNEAU ⋅ Sarath Chandar ⋅ Marie-Jeanne Lesot
VASAE: Naming SAE Dictionary Directions with Vocabulary-Aligned Anchoring
Kairui Zhang ⋅ Ziwen Yu ⋅ Zahraa S Abdallah ⋅ Martha Lewis
Language Triggers Hijack Language Circuits: A Mechanistic Analysis of Backdoor Behaviors in Large Language Models
Théo Lasnier ⋅ Wissam Antoun ⋅ Francis Kulumba ⋅ Benoît Sagot ⋅ Djamé Seddah
Different Heads, Same Fragile Tasks: A Cross-model Retrieval Head Correspondence
Ananya Anand ⋅ Harry Ilanyan ⋅ Ira Pathak ⋅ Derrick Sual ⋅ Eric Xia
How Language Models Choose Sides: Internal Representations of Instruction Hierarchy
Enrique Balp-Straffon ⋅ Chih-Hao Hsu ⋅ Rushiraj Gadhvi ⋅ Sunishchal Dev ⋅ Callum McDougall ⋅ Anusha Mujumdar
C-Δθ: Circuit-Restricted Weight Arithmetic for Selective Refusal
Aditya Kasliwal ⋅ Pratinav Seth ⋅ Vinay k Sankarapu
Iterative Distillation for Feature-Consistent Sparse Autoencoders
Cristina P Martin-Linares ⋅ Jonathan Ling
Why do Irrelevant Instructions Inhibit Refusal?
Sedona N Ewbank ⋅ Jakob Hansen
From Actions to Understanding: Conformal Interpretability of Temporal Concepts in LLM Agents
Trilok Padhi ⋅ Ramneet Kaur ⋅ Krishiv Agarwal ⋅ Adam Cobb ⋅ Daniel Elenius ⋅ Manoj Acharya ⋅ Colin Samplawski ⋅ Alexander M Berenbeim ⋅ Nathaniel Bastian ⋅ Susmit Jha ⋅ Anirban Roy
Disentangling meaning from language in LLM-based machine translation
Théo Lasnier ⋅ Armel Randy Zebaze ⋅ Djamé Seddah ⋅ Rachel Bawden ⋅ Benoît Sagot
Sink vs. diagonal patterns as mechanisms for attention switch and oversmoothing prevention
Peter Súkeník ⋅ Cristina Lopez Amado ⋅ Christoph Lampert ⋅ Marco Mondelli
Dual Path Attribution: Efficient Attribution for SwiGLU-Transformers through Layer-Wise Target Propagation
Lasse Marten Jantsch ⋅ Dong-Jae Koh ⋅ Seonghyeon Lee ⋅ Young-Kyoon Suh
FTC: Fourier Transcoder for Monosemantic Sparse Feature Decomposition in Vision Language Model
Kyeongjun Cho ⋅ Juheon Kang ⋅ Hyunseo Shin ⋅ Hongseok Oh ⋅ Wonseok Hwang
Sparse Fourier Regularization for Modular Arithmetic Models
Amogh Dalal ⋅ Adityasinh Rathod ⋅ Akshay Rangamani
A Geometric Account of Activation Steering
Georgii Aparin ⋅ Tatiana Gaintseva
Fine-tuning with Harmfulness Probes Leads to Natural Refusals
Lena Libon ⋅ Alexander Panfilov ⋅ Ben Rank ⋅ Jonas Geiping ⋅ Maksym Andriushchenko
Where Reliability Lives in Vision-Language Models: A Mechanistic Study of Attention, Hidden States, and Causal Circuits
Logan Mann ⋅ Ajit Saravanan ⋅ Ishan Dave ⋅ Shikhar Shiromani ⋅ Saadullah ismail ⋅ Yi Xia ⋅ Emily Huang
Is One Layer Enough? Understanding Inference Dynamics in Tabular Foundation Models
Amir Rezaei Balef ⋅ Mykhailo Koshil ⋅ Katharina Eggensperger
$\mathcal{D}^2$-Monitor: $\mathcal{D}$ynamic Safety Monitoring for $\mathcal{D}$iffusion LLMs via Hesitation-Aware Routing
Aoxi Liu ⋅ Yupeng Chen ⋅ James Oldfield ⋅ Guan Zhe Hong ⋅ Junchi Yu ⋅ Baoyuan Wu ⋅ Phil Torr ⋅ Adel Bibi
Decoding Alignment without Encoding Alignment: A critique of similarity analysis in neuroscience
Johannes Bertram ⋅ Luciano Dyballa ⋅ T. Anderson Keller ⋅ Savik Kinger ⋅ Steven W Zucker
Mirage Probes: Selective Construction of False Images in Vision Language Model Inference
Daniel Ben-Levi ⋅ Judah A Goldfeder ⋅ Weiliang Zhao ⋅ Raz Lapid ⋅ Amit LeVi ⋅ Allen Roush ⋅ Ravid Shwartz-Ziv ⋅ Hod Lipson
Probing Hybrid Language Models for In-Context Recall
Pol Puigdemont ⋅ Volkan Cevher
How Do LLMs Distinguish Normative Ethical Frameworks Internally?
Weilun Xu ⋅ Alexander M Rusnak ⋅ Frederic Kaplan
From Directions to Regions: Decomposing Activations in Language Models via Local Geometry
Or Shafran ⋅ Shaked Ronen ⋅ Omri Fahn ⋅ Shauli Ravfogel ⋅ Atticus Geiger ⋅ Mor Geva
Monitoring the Internal Monologue: Probe Trajectories Reveal Reasoning Dynamics
Maciej Chrabaszcz ⋅ Aleksander Szymczyk ⋅ Marcin Sendera ⋅ Tomasz Trzcinski ⋅ Sebastian Cygert
Crosscoding Through Time: Sparse Feature Discovery Across Sequence Positions
Dmitry Manning-Coe ⋅ Han Xuanyuan ⋅ Aniket Deshpande ⋅ Andrii Shportko ⋅ William Fei
Beyond a Single Direction: Chain-of-Thought Disrupts Simple Steering of Refusal
Kia-Jüng Yang ⋅ Dominik Meier ⋅ Jiachen Zhao ⋅ Terry Ruas ⋅ Bela Gipp
Language Models Learn Universal Representations of Numbers and Why You Should Care
Michal Štefánik ⋅ Timothee Mickus ⋅ Marek Kadlčík ⋅ Bertram Højer ⋅ Michal Spiegel ⋅ Raúl Vázquez ⋅ Aman Sinha ⋅ Josef Kuchař ⋅ Philipp Mondorf ⋅ Pontus Stenetorp
Interpreting Genomic Language Models using Sparse Autoencoders
Akira Nair ⋅ Jaehyun Joo ⋅ Jonghyun Lee ⋅ Lina Takemaru ⋅ Yidi Huang ⋅ Manu Shivakumar ⋅ Matthew Lee ⋅ Jaesik Kim ⋅ Sokratis Apostolidis ⋅ Dokyoon Kim
Backtracking is Decorative: A Mechanistic Account
Unai Zulaika ⋅ Miguel Fernandez-de-Retana ⋅ Rubén Sánchez Corcuera ⋅ Aritz Bilbao Jayo ⋅ Aitor Almeida
Contrastive Conceptor Activation Steering (COAST): Unlocking Vision-Language-Action Models through Hidden States
Miranda Miao ⋅ Subin Kim ⋅ Brandon Y Yang ⋅ Dinesh Jayaraman ⋅ Lyle Ungar
Valence–Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Control
Lihao Sun ⋅ Lewen Yan ⋅ Xiaoya Lu ⋅ Andrew Lee ⋅ Jie Zhang ⋅ Jing Shao
Analysis-by-Proxy: Localization Signals in VLMs Operating as Condition Encoders
Yoav Baron ⋅ Sara Dorfman ⋅ Roni Paiss ⋅ Daniel Cohen-Or ⋅ Or Patashnik
Building Fast, Evaluating Slow: Pipeline Choices Dominate Autointerpretability Score Variance
Sinie van der Ben ⋅ Neele Roch ⋅ Anna Hedström ⋅ Mennatallah El-Assady
Decompose Sparsely Where You Should, Absorb Densely Where You Should Not
Ruixuan Deng ⋅ Zehao Jin ⋅ Zekun Wang ⋅ Zihan Dong
A Critical Analysis of Color Neurons in Vision Models
Fernando Javier Aguilar Canto ⋅ Hiram Calvo ⋅ Ricardo Menchaca-Mendez
Emergent Analogical Reasoning in Transformers
Gouki Minegishi ⋅ Jingyuan Feng ⋅ Hiroki Furuta ⋅ Takeshi Kojima ⋅ Yusuke Iwasawa ⋅ Yutaka Matsuo
User Persona Subspaces Modulate Refusal Behavior in Language Models
Yan Zhou ⋅ Shichang Zhang ⋅ Zidi Xiong ⋅ Himabindu Lakkaraju
When Reasoning Traces Become Performative: Step-Level Evidence that Chain-of-Thought Is an Imperfect Oversight Channel
Wenkai Li ⋅ Fan Yang ⋅ Ananya Hazarika ⋅ Shaunak A Mehta ⋅ Koichi Onoue
Decomposing Query-Key Feature Interactions Using Contrastive Covariances
Andrew Lee ⋅ Yonatan Belinkov ⋅ Fernanda Viégas ⋅ Martin Wattenberg
Transformers learn factored representations
Adam Shai ⋅ Loren Amdahl-Culleton ⋅ Casper Christensen ⋅ Henry R Bigelow ⋅ Fernando Rosas ⋅ Alexander Boyd ⋅ Eric Alt ⋅ Kyle Ray ⋅ Paul Riechers
Sparse Autoencoders for LoRA-Adapted CLIP Under Domain Shift
Sunayana Samavedam ⋅ Deepika Vemuri ⋅ Venkat A Amula ⋅ P J Narayanan
When Does Interleaving Prevent Emergent Misalignment?
Samy Mammeri ⋅ Rose N Kwon ⋅ Chen Sun ⋅ Sabyasachi Sahoo ⋅ Christian Gagne ⋅ Thomas Jiralerspong
Sample-Level White-Box Detection of Alignment Faking
Lakshya Chaudhry ⋅ Tianqin Meng ⋅ Anthony Nguyen ⋅ Yashraj Panwar ⋅ Yuqi Sun ⋅ Zhuofan Ying
Reading Calibrated Uncertainty from Language Model Trajectories
Aliai Eusebi ⋅ Alexander Herzog ⋅ Xiaoyu Liang ⋅ Marie Vasek ⋅ Enrico Mariconti ⋅ Lorenzo Cavallaro
Reasoning Models Know What’s Important, and Encode It in Their Activations
Yaniv Nikankin ⋅ Martin Tutek ⋅ Tomer Ashuach ⋅ Jonathan S Rosenfeld ⋅ Yonatan Belinkov
Scalable Circuit Learning for Interpreting Large Language Models
Naiyu Yin ⋅ Dennis Wei ⋅ Tian Gao ⋅ Amit Dhurandhar ⋅ Karthikeyan Natesan Ramamurthy ⋅ Yue Yu
Latent Introspection: Models Can Detect Prior Concept Injections
Theia Pearson-Vogel ⋅ Martin Vanek ⋅ Raymond Douglas ⋅ Jan Kulveit
ReasonOps: Operator Segmentation for LLM Reasoning Traces
Daniel Lee ⋅ Owen Queen ⋅ James Zou
Pretraining Numerical Frequency and Number-Line in Language Models
Mohammed Ibrahim Awad ⋅ Ahmed Elshehaby ⋅ Velibor Bojkovic ⋅ Hilal AlQuabeh
Interpreting Physics in Video World Models
Sonia Joseph ⋅ Quentin Garrido ⋅ Randall Balestriero ⋅ Matthew Kowal ⋅ Thomas Fel ⋅ Shahab Bakhtiari ⋅ Blake Richards ⋅ Michael Rabbat
Don't Lose Focus: Activation Steering via Key-Orthogonal Projections
Haoyan Luo ⋅ Mateo Espinosa Zarlenga ⋅ Mateja Jamnik
Latent Mechanisms of Code-Switching in Large Language Models
Ryo Mitsuhashi ⋅ Sabri Boughorbel ⋅ Majd Hawasly
Interaction-Aware Influence Functions for Group Attribution
Jaeseung Heo ⋅ Kyeongheung Yun ⋅ Youngbin Choi ⋅ SEHYUN HWANG ⋅ Jungseul Ok ⋅ Dongwoo Kim
Circuit Oracle: Automating Attribution Graph Analysis via Natural-Language Queries
Hong Kiat Tan ⋅ Shariar Kabir ⋅ Swastik Agrawal ⋅ Sai V Reddy Chereddy ⋅ Sriram Balasubramanian
OPIUM: Mitigating Steering Externalities and Over-Refusal via Dual Objective Latent Optimization
Kavin Aravindan ⋅ Arihant Rastogi ⋅ Aadi Prasad ⋅ Krishak Aneja ⋅ Saiyam Jain ⋅ Vaishnavi Shivkumar ⋅ Ponnurangam Kumaraguru
Circuit Tracing in Autoregressive Protein Language Models
Darin Tsui ⋅ William Deinzer ⋅ Daniel Saeedi ⋅ Amirali Aghazadeh
Demystifying Classifier-Free Guidance for Auto-Regressive Image Generation
Zhiling Zhou ⋅ Jiachun Pan ⋅ Fengzhuo Zhang ⋅ Dirk Bergemann ⋅ Zhuoran Yang
MIDSTEER: Optimal Affine Framework for Steering Generative Models
Tatiana Gaintseva ⋅ Andrew Stepanov ⋅ Ziquan Liu ⋅ Martin Benning ⋅ Gregory Slabaugh ⋅ Jiankang Deng ⋅ Ismail Elezi
Predictable Steering: Leveraging Geometric Proxies for Layer Selection and Per-Instance Success Prediction
Nishkal Hundia ⋅ Swastik Agrawal ⋅ Navita Goyal ⋅ Sarah Wiegreffe
Long Range Dependency Understanding in State Space Models
Srividya Ravikumar ⋅ Shweta Verma ⋅ Abhinav Anand ⋅ Mira Mezini
Vision-Language Binding in In-Context Image Generation
Chris Ge ⋅ Rohit Gandikota ⋅ Antonio Torralba ⋅ Tamar Rott Shaham
Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts
Sheridan Feucht ⋅ Tal Haklay ⋅ Usha Bhalla ⋅ Daniel Wurgaft ⋅ Can Rager ⋅ Jack Merullo ⋅ Thomas McGrath ⋅ Raphaël Sarfati ⋅ Owen Lewis ⋅ Ekdeep Singh Lubana ⋅ Thomas Fel ⋅ Atticus Geiger
Round-Trip Latent Geometry in Diffusion VAEs Enables Covert Channels
Catherine Ge-Wang ⋅ Tushar Nagar ⋅ Joy Z. Yang
When Reading the Chain of Thought Falls Short: A Testbed for Reasoning Trace Analysis
Daria Ivanova ⋅ Riya Tyagi ⋅ Josh Engels ⋅ Neel Nanda
What, Where, and How: Probing Spatiotemporal Representations in Video Foundation Models
Sharon Musa ⋅ Fereshteh Forghani ⋅ Harrish Thasarathan ⋅ Sonia Joseph ⋅ Matthew Kowal ⋅ Konstantinos Derpanis
Current Activation Oracles Are Hard to Use on Safety-Relevant Tasks
Arya Jakkli ⋅ Senthooran Rajamanoharan ⋅ Neel Nanda
Mechanistic Evidence for Spectral Structures in Prior-Data Fitted Networks
Kaustubh Sharma ⋅ Srijan Tiwari ⋅ Ojasva Nema ⋅ Parikshit Pareek
Induction Meets Biology: Mechanisms of Repeat Detection in Protein Language Models
Gal Pomerants ⋅ Yaniv Nikankin ⋅ Anja Reusch ⋅ Tomer Tsaban ⋅ Ora Schueler-Furman ⋅ Yonatan Belinkov
PolySAE: Modeling Feature Interactions in Sparse Autoencoders via Polynomial Decoding
Panagiotis Koromilas ⋅ Andreas Demou ⋅ James Oldfield ⋅ Yannis Panagakis ⋅ Mihalis Nicolaou
Building Better Activation Oracles
Jan Bauer ⋅ Celeste De Schamphelaere ⋅ Adam Karvonen ⋅ Niclas Luick ⋅ Neel Nanda
The Dead Salmons of AI Interpretability
Maxime Méloux ⋅ Giada Dirupo ⋅ François Portet ⋅ Maxime Peyrard
Singular Vectors of Attention Heads Align with Features
Gabriel Franco ⋅ Carson Loughridge ⋅ Mark Crovella
The Dark Subspace of Fine-Tuning Memorisation
Niloy Purkait ⋅ Gonzalo Nápoles ⋅ Emmanuel Keuleers ⋅ Henry Brighton
What Do Mixture-of-Depth Routers Learn? Routing Patterns in Gemma 2
Stein Pleiter ⋅ Ege Erdogan ⋅ Ana Lucic
Additive Relational Bindings in Transformers: What Sparse Autoencoders Miss
Sebastian Hönig ⋅ Kushal Jain ⋅ Su Ji Park ⋅ Bart Bussmann ⋅ Patrick Leask
The ACUTE Protocol: Operationalizing Language Model Activations for Better Calibration, Utility, and Trust
Nishant Subramani ⋅ Palash Goyal ⋅ Yiwen Song ⋅ Mani Malek ⋅ Yuan Xue ⋅ Tomas Pfister ⋅ Hamid Palangi
Do Clinical VLMs Need Dense Visual Tokens? Probing Spatial Grounding in Radiology Report Generation
L. Julián Lechuga López ⋅ Tim G. J. Rudner ⋅ Farah E. Shamout
LAWFUL: Law-Aligned Witness for Faithful Use of Latents
Kevin Chen ⋅ Kenneth Parker ⋅ Anish Arora
Predicting Future Behaviors in Reasoning Models Enables Better Steering
Evgenii Kortukov ⋅ Piotr Komorowski ⋅ Florian Klein ⋅ Paula Engl ⋅ Gabriele Sarti ⋅ Seong Joon Oh ⋅ Sebastian Lapuschkin ⋅ Wojciech Samek
Lossy Superposition: Predicting Compositional Errors Without Seeing Inputs
Jennifer M Lu ⋅ Ruochen Zhang ⋅ Isabelle Lee ⋅ David Alvarez-Melis ⋅ Ellie Pavlick ⋅ Naomi Saphra
Stress Tests REVEAL Fragile Temporal and Visual Grounding in Video-Language Models
Sethuraman T V ⋅ Savya Khosla ⋅ Aditi Tiwari ⋅ Vidya Ganesh ⋅ Rakshana Jayaprakash ⋅ Aditya Jain ⋅ Vignesh Srinivasakumar ⋅ Onkar Susladkar ⋅ Srinidhi Sunkara ⋅ Aditya Shanmugham ⋅ Abbaas Alif Mohamed Nishar ⋅ Rakesh Vaideeswaran Mahesh ⋅ Simon Jenni ⋅ Derek Hoiem
Position: Use Sparse Autoencoders to Discover Unknowns
Kenny Peng ⋅ Rajiv Movva ⋅ Jon Kleinberg ⋅ Emma Pierson ⋅ Nikhil Garg
Position: Don't Just "Fix it in Post'': A Science of AI Must Study Learning Dynamics
Stella Biderman ⋅ Mohammad Aflah Khan ⋅ Niloofar Mireshghallah ⋅ Catherine Arnett ⋅ Fazl Barez ⋅ Naomi Saphra
The Geometric Wall: Manifold Structure Predicts Layerwise Sparse Autoencoder Scaling Laws
Eslam Zaher ⋅ Maciej Trzaskowski ⋅ Quan Nguyen ⋅ Fred Roosta
MetaOthello: A Controlled Study of Multiple World Models in Transformers
Aviral Chawla ⋅ Galen Hall ⋅ Juniper Lovato
Retrieval Heads Meet Vision: Uncovering How VLMs Locate and Extract Visual Information
Chanho Park ⋅ Daehyeon Choi ⋅ Jihyun Lee ⋅ Minhyuk Sung
Spectral Stratification of Semantic Abstraction in Vision-Language Models
Jeonghwan Cheon ⋅ Marin Vogelsang ⋅ Lukas Vogelsang ⋅ Pawan Sinha
Model Incrimination: Investigating Whether Concerning Behavior Reflects Misalignment
Gerson Kroiz ⋅ Aditya Singh ⋅ Senthooran Rajamanoharan ⋅ Neel Nanda
Localizing and Evaluating Multinomial Concepts with Affine Subspaces
Divya Appapogu ⋅ Freya Behrens ⋅ Yonatan Belinkov ⋅ Aaron Mueller
From Noise to Diversity: Random Embedding Injection in LLM Reasoning
Heejun Kim ⋅ Seungpil Lee ⋅ Je Won Yeom ⋅ Jaewon Sok ⋅ Park Seonghyeon ⋅ Jeongjae Park ⋅ Taesup Kim ⋅ Sundong Kim
Multi-Granular Node Pruning for Causal Circuit Discovery
Muhammad Umair Haider ⋅ Hammad Rizwan ⋅ Hassan Sajjad ⋅ A.B. Siddique
Multiplication Beyond Groups: Stratified Fourier Mechanisms in Transformer Circuits
Zitong Chen ⋅ Junaid Hasan ⋅ Akhil Srinivasan ⋅ Hemkesh Bandi ⋅ Jarod Alper
Mechanistically Interpreting Compression in Vision-Language Models
Veeraraju Elluru ⋅ Arth Ashish Singh ⋅ Roberto Aguero ⋅ Debojyoti Das ⋅ Hreetam Paul ⋅ Ajay Agarwal
Mechanistic Analysis and Inference-Time Control of Modality Conflict in VLMs
Jiahang He ⋅ Dia'a AL-Dweikat ⋅ Artur Shyutts ⋅ Abdullah AlDahoul ⋅ Oyedeji Sonuga ⋅ Kevin Zhu ⋅ Ruizhe Li ⋅ Anusha Mujumdar
Do Thinking Tokens Help with Safety?
Narutatsu Ri ⋅ Abhishek Panigrahi ⋅ Sanjeev Arora
ProtoMech: Protein Circuit Tracing via Cross-layer Transcoders
Darin Tsui ⋅ Kunal Talreja ⋅ Daniel Saeedi ⋅ Amirali Aghazadeh
Discovering Mechanisms in Tokenized Graph Transformers
Yong-Min Shin ⋅ Dae-Woong Jeong ⋅ Sehui Han
LLM Jailbreaks Exploit Attention Sinks
Jonah Mackey ⋅ Stephen Zhang ⋅ Vardan Papyan
Crosscoders Identify Shared or Specific Features between the Human Brain and Language Models
Koshiro Aoki ⋅ Itsuki Hamada ⋅ Naho Orita ⋅ Daisuke Kawahara ⋅ Hiromu Sakai