Textbooks on Various Subjects

Well-written textbooks (or even theses) are the fastest way to learn technical topics that have achieved critical mass. Inspired by a similarly titled post, I have my own evolving list.

While I have not read most books cover to cover, I have consistently referred back to more than a few chapters of relevance. I am convinced that the rest of the book is worth reading. Often, multiple books cater to overlapping topics, and provide complementary strengths to aid understanding. When multiple books are specified within each (sub-)section, it is safe to assume that as a "soft" recommendation order.

General Mathematics

• Mathematical Methods for Physics and Engineering by K. F. Riley, M. P. Hobson, S. J. Bence (2006)

Optimization

• Numerical Optimization by Jorge Nocedal, Stephen J. Wright (2006)

• Convex Optimization by Stephen Boyd and Lieven Vandenberghe (2004)

Differential Equations

• Applied Stochastic Differential Equations by Simo Särkkä and Arno Solin (2019)

Miscellaneous

So You Want to Study Mathematics… has a starter list for some core topics in mathematics.

Mathematics books that are perfect as introductions to a particular field/topic just for their writing. from r/math.

Ask HN: Math books that made you significantly better at math? from HN.

Linear Algebra

• Linear Algebra Done Right by Sheldon Axler (1995; 2015)

• Introduction to Linear Algebra by Gilbert Strang (1993; 2016)

Numerical Methods

• Numerical Linear Algebra by Lloyd N. Trefethen and David Bau, III (1997)

• Numerical Algorithms by Justin Solomon (2015)

• Fundamentals of Matrix Computations by David S. Watkins (2010)

• Matrix Computations by G.H. Golub and C.F. Van Loan (2013)

Machine Learning

• Pattern Recognition and Machine Learning by Christopher Bishop (2006)

• Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville (2016)

• Patterns, Predictions, and Actions: A Story about Machine Learning by Moritz Hardt and Benjamin Recht (2021)

• Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto (1998; 2018)

Bayesian Inference

• Information Theory, Inference and Learning Algorithms by David J. C. MacKay (2003)

• Probabilistic Machine Learning by Kevin Murphy (book series 2012, 2021, 2022)

• Gaussian Processes for Machine Learning by Carl Edward Rasmussen and Christopher K. I. Williams (2006)

• Bayesian Reasoning and Machine Learning by David Barber (2012)

Learning Theory

• Foundations of Machine Learning by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar (2012; 2018)

• Understanding Machine Learning: From Theory to Algorithms by Shai Shalev-Shwartz and Shai Ben-David (2014)

Visualization

• Scientific Visualization: Python + Matplotlib by Nicolas P. Rougier, Bordeaux (2021)