R.R.1. Theoretical Reading List A

Author

Fujimiya Amane

Published

April 7, 2026

Modified

June 8, 2026

This reading list is a themed reading list, focus on the topic of theoretical learning theory, with the main bulk as double descent in this case. It contains documents and interesting reading (finished of not) that are relevant per context of the laboratory works. The first few would rather be about double descent, because that is what the lab was initially tested of, so much of it would be the foundational groundwork reading that has been here since perhaps the beginning.

Certain section of the theoretical learning theory, statistical learning theory and computational learning theories, are in discussion of the preceding reading lists. Particularly, the more accessible literature would be almost trivially around double descent, the problem of grokking, catastrophic forgetting, indetermination, and so on.

Learning theory

Some of those literatures are principle materials and papers. For catastrophic forgetting, because it has become a wild horse of artificial intelligence studies

Double descent

Fairly a lot of things are there to read in double descent section. For the canonical introduction and first appearance of the problem itself, see:

Most of them are done for now, with foundational papers being Belkin’s paper and the Deep Double Descent one, though that one is more nuanced on their take. The updated list contents are rather in line with the documented analysis,

  • Reconciling modern machine-learning practice (Belkin et al.) – link here
  • Deep Double Descent (Nakkiran et al.) – link here
  • Surprises in High-Dimensional Ridgeless Least Squares Interpolation – link here
  • Changing the Kernel During Training Leads to Double Descent in Kernel Regression – link here
  • Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle – link here
  • Multi-scale Feature Learning Dynamics: Insights for Double Descent – link here
  • More Data Can Hurt for Linear Regression: Sample-wise Double Descent – link here
  • On Double Descent in Reinforcement Learning with LSTD and Random Features – link here
  • Homophily modulates double descent generalization in graph convolution networks – link here
  • An Overview of Double Descent and Overparameterization – link here
  • Unified View of Grokking, Double Descent and Emergent Abilities: A Perspective from Circuits Competition – link here
  • Kernel regression in high dimensions: Refined analysis beyond double descent – link here
  • Double Descent: Understanding Linear Model Estimation of Nonidentifiable Parameters and a Model for Overfitting – link here
  • Two models of double descent for weak features – link here
  • Double Descent of Discrepancy: A Task-, Data-, and Model-Agnostic Phenomenon – link here
  • Manipulating Sparse Double Descent – link here
  • A U-turn on Double Descent: Rethinking Parameter Counting in Statistical Learning – link here
  • Double Descent Meets Out-of-Distribution Detection: Theoretical Insights and Empirical Analysis on the role of model complexity – link here
  • Dropout Drops Double Descent – link here
  • The Double Descent Behavior in Two Layer Neural Network for Binary Classification – link here
  • Asymptotic Risk of Overparameterized Likelihood Models: Double Descent Theory for Deep Neural Networks – link here
  • Double-Descent Curves in Neural Networks: A New Perspective Using Gaussian Processes – link here
  • To understand double descent, we need to understand VC theory – link here

Some of which are already analyzed, including the reinforcement learning paper on double descent identificaton and potential explanatory isometry to the wider general theory. Most of them would be in the manuscript already, and we defer to there for such analysis.