Influential Papers

Advocates for understanding LLM behavior as indicative of nuances in human social behavior.

Nice overview of the AI for computational social sciences landscape.

Interesting analysis that Google, Tiktok, and ChatGPT are all ranking information, but the former two rank more bundled information than the latter. Bundled information has clearer economic, social, and institutional properties, such as notions of originality, labor, etc. Many 'fixes' to AI involve bundling information. Information-bundling and -splitting is therefore relevant for thinking about the economics of AI.

Super interesting and illuminating perspective explaining why supposedly deep-learning-unique phenomena like deep double descent, overparametrization, etc. can be explained using soft inductive biases and existing generalization frameworks. The references are a treasure trove!

Cool classic HCI-style exploration into a new interaction for code editing using visual sketching, interpreted by VLMs.

A method to probe structure in language models without any notion of ground truth, relying instead on the consistency property of tru statements.

Really nice technical details and knowledge on training and understanding large sparse autoencoders,

This paper uses sparse autoencoder features to identify possible hypotheses to explain relationships between text and a dependent variable.

Interesting way to automatically label datasets using mutual predictability and logical consistency.

Asks an interesting question and gives some helpful directions to start thinking about what progress means in philosophy and how 'premise deniability' might be a relevant factor.

Challenges the assumption that implicit measures are superior to self-reports in psychological research. Argues that self-reports demonstrate greater reliability, stronger predictive validity for both deliberate and spontaneous behaviors, and unmatched flexibility in exploring complex psychological constructs.

Identifies four cognitive behaviors (verification, backtracking, subgoal setting, backward chaining) that predict whether a model can self-improve via RL. The key finding is striking: it's the presence of reasoning behaviors, not answer correctness, that matters. Models exposed to training data with proper reasoning patterns -- even incorrect answers -- matched the improvement of models that had these behaviors naturally. A useful framing for thinking about what 'reasoning' actually is in these systems.

By creating an LM architecture in which input tokens have a direct log-linear effect on the output, we can intervene precisely on the model output.

Advocates for rhetoric over composition as the methodology for writing pedagogy in the AI era.

Uses an extremely entertaining and well-focused example of understanding the 'V - E + F = 2' Euler characterization of polyhedra to illustrate how mathemematics develops -- dialectically between criticism (proof-analysis, refutation, counterexamples) and development (proof, lemma-incorporation, etc.). Entertainingly written as a Socratic conversation among a classroom of incredibly bright students.

Language modeling operating in embedding rather than token space.

Really incredible work on discovering and visualizing feature decompositions of neuron layers with sparse autoencoders. Gorgeous visualizations and interfaces, and thoughtful reflections on interpretability methodology. I am a big fan of this publication style.

A recognition that not only must AI 'align to' human (values, behavior, knowledge, etc.) (-- whatever this means), but we also need to think about how humans might 'align' to AI by working with AI-structured systems. This paper recognizes social 'looping effects' brought about by AI and its behavior.