ML Fragments

Raw unstructured thoughts and ideas.
Jul 5, 2020Last updated: Aug 23, 2020

These are just raw keywords which may eventually evolve into their own pages if I dive deep enough. For now they are just disconnected "fragments", interesting directions that I may want to pursue. These are intentionally abstract. Please don't hesitate to reach out if you'd like to discuss more!

There is non-trivial chance that prior work has already posed questions similar but then I haven't spent enough time studying these in detail.

Three-Way Markets

Economy (and "micro-"economies if you will) seem to be running on three-way markets. i) The stock market ii) Gig economy - the likes of Uber, AirBnB. Each transaction can most likely be modeled as consisting of three components - a buyer, a seller and a mediator where each component could be an individual or an institution.

Much like the reward hypothesis in RL, there appears to be a similar hypothesis in stock markets - stock price contains all the information one needs (I'm still trying to understand the nuance involved in this hypothesis). We certainly would want to model the micro and macro dynamics. What tools does machine learning provide?

Reinforcement Learning

Model-Based

  • Fixing objective mismatch in MBRL using Expectation Maximization.
  • Connections to classic control theory

Bayesian Inference

D(p(x)p(y∣x)∣∣p(y)p(x∣y))\mathcal{D}\left( p(x)p(y|x) \Big|\Big| p(y)p(x|y) \right)

Learned invariances

  • It's probably become more important now than ever to have priors in Neural Networks that satisfy invariances we care about instead of just using N(0,I)\mathcal{N}(\mathbf{0}, \mathbf{I}). how do we do this? e.g. Learning Invariances using the Marginal Likelihood

Model Misspecification

Uncertainty Calibration

Uncertainty Estimation

Gaussian Processes

Implicit Distributions

Linear Algebra

  • Circulant (in general Toeplitz) matrices allow much faster matrix-vector multiplications. For non-Toeplitz ones, we have a notion of "asymptotically Toeplitz" under the weak matrix norm (Frobenius). What problems families afford such a structure? If they do, can we leverage non-asymptotic guarantees?

© 2023 Sanyam Kapoor