Useful Blogs (written by other people)

This is a carefully curated (and still incomplete) collection of blogposts which in my opinion are worth a read. I’m positively surprised by the sheer amount of influential (and not so influential) people that spend a lot of their free time writing down their thoughts and learnings. Enjoy!

---

AI-related (Technical)

Classics

• Richard Sutton’s The Bitter Lesson. My biggest takeaway from it is that

  “building how we think we think does not work in the long run”

Non-Classics (Yet)?

• Evan Miller´s Attention is Off By One

• Aidan McLaughlin’s The Problem with (LLM) Reasoners

• Yi Tay’s What happened to BERT & T5? On Transformer Encoders, PrefixLM and Denoising Objectives

• Kevin Liu’s Large Language Models can Simulate Everything

---

AI-related (Non-Technical)

• Tim Rocktäschel’s Advice for Short-term Machine Learning Research Projects

• Mor Harchol-Balter’s Applying to Ph.D. Programs in CS

• Andrej Karpathy’s A Survival Guide to a PhD, which in my opinion is completely generalisable to doing research in general, not only during a PhD. In fact, I think some of his thoughts (such as developing taste) are applicable to working in almost any job in general.

• Jason Eisner’s How to Read a Paper

• Jason Eisner’s Write the Paper First

  There are 2 reasons a paper will be cited. 1) If you have a great implementation that people can just use as a black box. 2) Otherwrise, your paper is only useful for the ideas that it provides.

• Sebastian Ruder’s 10 Tips for Research and a PhD

• Richard Hamming’s You and Your Research

• Jakob Foerster’s How to ML Paper

• Tim Dettmers’s Machine Learning PhD Applications - Everything You Need to Know

Personal Life

• Spruce Campbell’s Don´t Ask Successful People for Advice, instead ask for criticism.

  (If you ask for criticism, ) what they say to you is probably going to be much more actionable, direct and sensible than if you asked for generic advice.

• Spruce Campbell’s Talking to Geniuses

• Paul Graham’s The Right Kind of Stubborn

  The persistent (good kind of stubborn) are attached to the goal. The obstinate (bad kind of stubborn) are attached to their ideas about how to reach it.

• Sam Altman’s How to Be Successful

  It is much more important to work on the right thing than it is to work many hours.

• Paul Graham’s Taste for Makers

  When you’re forced to be simple, you’re forced to face the real problem. When you can’t deliver ornament, you have to deliver substance.

• ‘The Secretary Problem’

• Robert Wiblin’s ‘The Secretary Problem’ is too bad a match for real life to usefully inform our decisions

• Kevin Liu’s How do you find your people, when searching is itself an antipattern?

• Sam Altman’s Advice for ambitious 19 year olds