Textbooks on Various Subjects

Well-written textbooks (or even theses) are the fastest way to learn technical topics that have achieved critical mass. Inspired by a similarly titled post, I have my own evolving list.

Numerical Methods

Cannot learn enough of numerical methods.

• Numerical Linear Algebra by Lloyd N. Trefethen and David Bau, III (1997)
• Numerical Optimization by Jorge Nocedal, Stephen J. Wright (2006)
• Numerical Algorithms by Justin Solomon (2015)
• Mathematical Methods for Physics and Engineering by K. F. Riley, M. P. Hobson, S. J. Bence (2006)

General Mathematics

• All the Math You Missed (But Need to Know for Graduate School) by Thomas A. Garrity (2021)

Optimization

• Convex Optimization by Stephen Boyd and Lieven Vandenberghe (2004)

Differential Equations

• Applied Stochastic Differential Equations by Simo Särkkä and Arno Solin (2019)

Linear Algebra

• Fundamentals of Matrix Computations by David S. Watkins (2010)
• Matrix Computations by G.H. Golub and C.F. Van Loan (2013)
• Linear Algebra Done Right by Sheldon Axler (1995; 2015)

Miscellaneous

So You Want to Study Mathematics… has a starter list for some core topics in mathematics. 30 Best Math Books to Learn Advanced Mathematics for Self-Learners for some classic introductory books. Mathematics books that are perfect as introductions to a particular field/topic just for their writing. Math books that made you significantly better at math from HN.

Machine Learning

• Pattern Recognition and Machine Learning by Christopher Bishop (2006)
• Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville (2016)
• Patterns, Predictions, and Actions: A Story about Machine Learning by Moritz Hardt and Benjamin Recht (2021)
• Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto (1998; 2018)
• Deep Learning Foundations and Concepts by Christopher Bishop and Hugh Bishop (2024)

Bayesian Inference

• Information Theory, Inference and Learning Algorithms by David J. C. MacKay (2003)
• Probabilistic Machine Learning by Kevin Murphy (book series 2012, 2021, 2022)
• Gaussian Processes for Machine Learning by Carl Edward Rasmussen and Christopher K. I. Williams (2006)

Learning Theory

• Foundations of Machine Learning by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar (2012; 2018)
• Understanding Machine Learning: From Theory to Algorithms by Shai Shalev-Shwartz and Shai Ben-David (2014)

Statistics

• Design and Analysis of Experiments by Douglas C. Montgomery (2019)

Visualization

• Scientific Visualization: Python + Matplotlib by Nicolas P. Rougier, Bordeaux (2021)