Want to understand feature learning and the final network weights?

Part 5 of A Quickstart Guide to Learning Mechanics (prev | next)
[long list of authors]

[TO BE WRITTEN]

this one’s hard. it’s the subject of a lot of speculation, and a lot of people want to know. it’s been the subject of a lot of mechinterp, but it’s proven quite hard to get an analytically.

A Quickstart Guide to Learning Mechanics

  1. Introduction: what do you want to understand?
  2. ...the average size of hidden representations?
  3. ...hyperparameter selection (and why should theorists care)?
  4. 🚧 ...the convergence and stability of optimization?
  5. 🚧 ...feature learning and the final network weights?
  6. 🚧 ...generalization?
  7. 🚧 ...neuron-level sparsity?
  8. 🚧 ...the structure in the data?
  9. 🚧 Places to make a difference

Comments