Me and My Research

Hard things are hard because there is no clear path forward. If there is, almost anyone who has the motivation and requisite ability can pursue it. Eventually, the hard thing often doesn’t remain so hard after all.

We, as scientists, aim to forge this path through the process of research. In this note, I document some meta-principles which I believe that if compounded over time will make us better.

This living document has a self-improvement aesthetic. Its success, therefore, relies on being more precise than abstract. I therefore keep behavioral suggestions than merely feel-good platitudes. All principles are categorized by their intrinsic nature - Frog Principles address the get-your-hands-dirty bits, and Bird Princples address the big picture bits.

👻 Also see Memes for my life for some related fun.

Frog Principles

Frogs are always in the weeds, at the bottom of the forest floor. This is where progress happens.

Note Keeping

I have shied away from note making for the better part of my education - up until my Masters degree. This goes to show the kind of structure there is education programs, that one can get away with the bare minimum.

In the knowledge creation business, note-making is a superpower. The brain can only handle first-order connections from reading, and that too only a few. Having a solid note-taking regimen for everything read allows the brain to avoid recency bias and forgetting ideas. Over time, these notes become the raw materials for developing higher-order connections - precisely what allows scientists to create new knowledge.

I prescribe a three-stage procedure to note-making:

• Stage I: Find an easy way to dump raw notes. This should have minimal friction to get the ball rolling. I use Obsidian.
• 2Stage II: Reconcile/organize all notes into a personal notebook. Keep this2 private. I use extended Markdown and store all notes via Git.
• Stage III: Distill knowledge by making connections.

Stages I and II is where we will spend most of our time. Stage III will and should happen only intermittently.

Read Deep

Erdös was one of the most prolific mathematicians, most famous for his around-the-world collaborations. Paul Halmos, another great mathematician and mathematical expositor, recounts his genius in his automathography. Erdös was extremely good at certain kind of combinatoric-geometric-arithmetic problems.

It would happen that someone would ask him a question in a field that he knew nothing about; he then demanded that the basic words be defined for him, and if it turned out that his set-theoretic “counting” techniques were at all pertinent, he proceeded to find the answer.

It is increasingly clear to me that faster progress is made by diving deeper. The reasons are simple - aiming to cover breadth as an early-career researcher makes for shallow learning. Success in research, like any other profession, relies on accummulating social capital. This is only built by first demonstrating expertise. No one calls a generalist an “expert”.

Breadth subsequently becomes easier to tackle because it is easier to contribute when riding the confidence wave of a successful track record.

As an actionable takeaway, during the working week, strictly defer reading everything not directly connected to immediate research. This will allow building an emotional connect to the problem, a prerequisite for mastery. Reading deep also opens up avenues for differentiation when complemented with the next principle.

Read Wide

Jeff Bezos, in his final 2020 Letter to Shareholder as the CEO of Amazon, emphasizes that typicality is easy to achieve and the universe wants it. By only reading deep, one opens up to the risk of becoming a low entropy individual. To be atypical, one needs to supplement depth with breadth. Creativity demands breadth, or else be stuck with the Two cultures problem.

The good news is that there is more than enough to discover and learn. To this end, during the weekend, strictly defer reading anything related to immediate research project(s). Open up the mind by reading those deferred research articles, making just enough notes to catch up in the near future.

For a more elaborate thesis on the importance of breadth, see this excellent book Range: Why Generalists Triumph in a Specialized World by David Epstein. We need more Rogers.

Bird Principles

Unlike frogs, the birds have the metaphorical 30000 feet view. This is where progress is consolidated.

Introspection

Paul Horgan, a Pulitzer Prize-winner author and artist, emphasizes¹

Illusion is first of all needed to find the powers of which self is capable.

Introspection is necessary to identify capabilities. This cannot happen during the hustle to get the next experiment working, or churn the next paper out. It can only happen with a quiet, undistracted mind. Identifying one’s superpowers is crucial; even an illusion of superpower can be impactful. Conducting research from a position of strength is beneficial for both the outcome and happiness.

Therefore, strive to spend atleast an hour every week to introspect about the desired aspirations and outcomes.

Everything is Sales

In his essay Harder Than It Looks, Not As Fun as It Seems, Morgan Housel really drives home the point - everything is sales. This is true for business, and just about everything else. This is why academic Twitter has now become an exercise in supplementing every new paper with a tweet thread. All the posturing aside, I don’t think this is a bad thing. If you do not upsell your research, who else will?

I believe many ideas win the sales lottery ². Good sales increases the chance of compouding network effects. I am certain many career researchers would be inspired by Michael Bronstein’s excellent keynote at ICLR 2021 to either pivot their research towards geometric deep learning or at least borrow ideas from this field.

Most research talks are terrible, aimed at sounding smart, than appealing to the audience and making them feel smarter. In such a world, solid sales becomes a great differentiator. Narratives are a powerful differentiating force. Remember,

facts don’t sell, stories about facts do; ideas don’t sell, narratives about ideas do.

Read the Think Different cult manifesto for some inspiration. Alas, this is an area that I really need to work hard at.

Minimum Viable Research

This is a twist on the classic product development strategy called minimum viable product (MVP). Colin Bryar and Bill Carr, in their new book Working Backwards: Insights, Stories, and Secrets from Inside Amazon, emphasize that we (including me) have focused too much on the minimum of MVP. The crucial aspect instead has always been viable.

Minimum viable research (MVR) is then the kind of research that has elements of viability, and a low friction entry point for others to build upon. It is almost easy to think how one’s research could be viable, but exceptionally hard to actually execute one.

While not always feasible, one way to judge when your research will be MVR is simply to ask the classic question - would you use it yourself?

The Overton Window of Research Ideas

The decade starting in 2010 marked the arrival of deep learning with a bang. Many computer vision purists were left behind, along with computational linguists who did not embrace the representation learning afforded by large-scale neural networks. While, only time will tell which school of thought stands the test of time, the medium-term survival of the inflexible researchers was certainly put at great risk. For now, deep learning has convincingly won, at least a significant mindshare of AI research community.

Overton Window, originally used in the context of policy-making, refers to the range of policies acceptable to the mainstream population at any given time. Despite all the popular science depiction of scientists working on crazy ideas and pushing boundaries, research is mostly incremental. The perceived impact, however, crucially relies on what ideas are considered acceptable by the research community. For the computer vision purists and computational linguists of the past, publishing old methods was no more fashionable.

One must always grapple with maintaining relevance and vitality ³. This is only possible when we have keen sense of the spirit of the times. Identifying the Overton window of research ideas is therefore imperative in research. Thomas Kuhn would probably call displacements in the Overton window of research ideas a paradigm shift. One must know where the wind blows.

The search for a finite basis of research principles lives on…