1. Overture

For the first fifteen years of my reading life, whenever someone asked me who my favorite novelist was, I would tell them Tolstoy, because I’d read Anna Karenina. Now I am inclined to say it is Proust. I finished In Search of Lost Time a few weeks ago, and can’t stop thinking about it.

This is surprising. I read Book 1, Swann’s Way, a few years ago and although I liked it very much, I didn’t see why it was all the way up to the pinnacle of literature. I’m not the only one: after universal rejection, Proust had to pay publishers to publish his first couple of volumes; one of the publishers who rejected it said, famously: “I can’t imagine why anybody would read 50 pages about somebody falling asleep”.

Undertaking Proust was an act of faith. Reading even 10 pages of Proust tires you out as much as reading 100 pages of an ordinary writer. And unfortunately, to see why it’s so great, you have to finish all 7 books. I say unfortunately because the novel is 3,000 pages long and at times tough to read.

Yet not a word is wasted. It sounds paradoxical, but Proust is economical with his prose. He is simply trying to describe things that are extremely fine-grained and high-dimensional, and that takes many words. He is trying to pin down things that have never been pinned down before. And it turns out you can, indeed, write 100 pages about the experience of falling asleep, and find all kinds of richness in that experience.

I am not sure how to convey the delight of the prose. Try:

…the memories which two people preserve of each other, even in love, are not the same. I had seen Albertine reproduce with perfect accuracy some remark which I had made to her at one of our first meetings and which I had entirely forgotten. Of some other incident, lodged for ever in my head like a pebble flung with force, she had no recollection.

Or:

I replied in a melancholy tone: ‘No, I’m not going to the theater just now, I’ve lost a friend to whom I was greatly attached.’ I almost had tears in my eyes as I said this, and yet for the first time it gave me a sort of pleasure to speak about it. It was from that moment that I began to write to everyone saying I had just experienced a great sorrow, and to cease to feel it.

2. The writing

I want to point out a few features of Proust’s writing.

First, constant use of analogies and metaphors. Everything is compared to physical, concrete objects, and sometimes he will use two or three different metaphors or analogies in succession to try and get at the same thing. Recall the “lodged for ever in my head like a pebble flung with force” from earlier. This is a form of extreme precision – he is always trying to get at the essence of the thing from multiple angles. (It is a curious fact about language that whoever seeks extreme precision is forced to use metaphor.)

As an aside, I tend to think that all great writers have this in common, and the frequency of it is especially striking when reading Homer, Shakespeare, and Dante.

The strongest oaths are straw / To th’ fire i’ th’ blood. (Shakespeare)

Anger, which, far sweeter than trickling drops of honey, rises in the bosom of a man like smoke. (Homer)

There are many famous examples I could give here, but here is a funny one. Marcel finally meets an aristocratic object of his desire, Oriane de Guermantes:

She showered me with the light of her azure gaze, hesitated for a moment, unfolded and stretched towards me the stem of her arm, and leaned forward her body which sprang rapidly backwards like a bush that has been pulled down to the ground and, on being released, returns to its natural position.

Like all of Proust’s descriptions, this is instantly vivid and precise.

Second, a clear-sightedness on human vanity and a total willingness to embarrass himself. There are passages in the Albertine sections which are shocking – such as the extended stretch, around 50 pages long, in which he describes watching her sleep – and, reading them, you start to understand that this was written by a dying man who did not care about anything apart from telling the whole truth in as merciless way as possible.

Third, hypotaxis in sentences. The opposite of hypotaxis is parataxis, which you often find in Hemingway, as in: “The rain stopped and the crowd went away and the square was empty.” Each item here is side by side, simple, clean. The Bible often uses such types of sentences: (“And God said, Let there be light: and there was light. And God saw the light, that it was good: and God divided the light from the darkness.”).

Hypotaxis, by contrast, describes sentences with many subordinate clauses, like nesting dolls. It is the domain of authors such as Henry James, or many of the great eighteenth century prose stylists such as De Quincey or Sir Thomas Browne. This type of sentence requires the utmost attention to parse, because clauses are often left hanging and modifying each other, and it is these cascading waterfalls of subordinate clauses that can make Proust’s long sentences hard to read.

This, for example, is the sentence that opens Book 2, which you will likely have to reread at least twice to understand:

My mother, when it was a question of our having M. de Norpois to dinner for the first time, having expressed her regret that Professor Cottard was away from home, and that she herself had quite ceased to see anything of Swann, since either of these might have helped to entertain the old Ambassador, my father replied that so eminent a guest, so distinguished a man of science as Cottard could never be out of place at a dinner-table, but that Swann, with his ostentation, his habit of crying aloud from the housetops the name of everyone that he knew, however slightly, was an impossible vulgarian whom the Marquis de Norpois would be sure to dismiss as—to use his own epithet—a ‘pestilent’ fellow.

The whole experience of reading this requires your utmost attention the way that rock climbing does. But the benefit is that you inhabit a consciousness more sensitive and greater than your own; you gain the experience of thinking thoughts you were not capable of forming; and this eventually becomes delightful, especially since you get used to it as you go further on in the book.

3. The inner ring

This is a novel that describes its own creation. The plot of the novel, in very banal terms, is how Marcel Proust realizes his vocation in life, which is to be a writer. He realizes this after a series of false starts: first, he becomes a social climber, who thinks the goal of life is to be witty and artistic and in the best salons with the noblest people. Eventually he realizes this pursuit is hollow and never-ending, and once you’re in the “inner ring” it turns out that everyone is vain and boring. Then he falls grievously in love, and he thinks the goal of life is to be in love. But he’s possessive and constantly unhappy as a result, and eventually he grows disillusioned with that, too. Finally, he goes to a party, realizes he’s gotten old, and everyone around him has gotten old too, and he receives a series of shocks, or epiphanies, that force him to realize what he really needs to do is write his novel and put everything into it. So then he does that, which is the book you’re reading.

Told this way, the through-line of the book is the question of what you should want out of life, and one way of looking at the book is it shows how these desires are often constructed by people around you. This is an intensely Girardian theme, and Girard was heavily influenced by Proust’s novel, and may well have derived his core ideas from it. Desire in this novel is mimetic, and driven by scarcity, in a way that is constant and undeniable.

In the domain of romance, Proust’s narrator, and characters like Swann, only love people to the degree that they are (a) unavailable or (b) desired by others; as soon as they gain full possession of their lover, they get bored, and the love affairs in this book are just this cycle repeating again and again. There is no healthy love, just possessiveness and triangular desire. Most memorable is Swann’s obsession with the courtesan Odette, in book 1, a chapter often published as a small novel in its own right (“Swann in Love”) and an excellent, and approachable, gateway drug for those who want to understand what the fuss is about.

On other Girardian themes, there are also examinations of scapegoating: the Dreyfus affair, and people’s attitude to Jewishness/anti-semitism, are a constant theme in the book. (Proust was Jewish on his mother’s side.) Similarly, snobbery: in society, the best salons are the most exclusive, and these gradations matter more to the narrator the more trivial they are in reality. (See the narcissism of small differences.)

It is only through sickness and the shock of seeing everyone aging and dying around him, after the war, that Proust’s narrator is finally freed of all these desires and receives, as though through grace, a transmission of his true purpose in life. The novel can thus be thought of as a long conversion narrative in which art is the religion.

4. Memory

Part of the shock the narrator receives at the climax is a realization of the nature of memory. The narrator realizes that there is a type of involuntary memory, typically released by a sensory experience of some kind, which gives you a true recollection, a kind of full-body memory; this full-body memory is truer, more artistically valuable, more complex, than our usual memory which gets activated when you try and remember things. For Proust, ‘normal’ memory falsifies and flattens, and is too influenced by consensus reality: it is only through this richer, involuntary, sensory memory, often accessed through smell or taste, and usually unanticipated – it is only by pulling on that thread, that you get to the real, soft essence of yourself, and the artistic material that is in you. All true art comes from that murky essence. But because that murk is only unlocked through involuntary memory, you could in theory live your entire life not remembering it:

But when from a long-distant past nothing subsists, after the people are dead, after the things are broken and scattered, still, alone, more fragile, but with more vitality, more unsubstantial, more persistent, more faithful, the smell and taste of things remain poised a long time, like souls, ready to remind us, waiting and hoping for their moment, amid the ruins of all the rest; and bear unfaltering, in the tiny and almost impalpable drop of their essence, the vast structure of recollection.

This is why Proust becomes totally nocturnal in real life, because it is only at night that he can extract this raw material reliably. And this is what is famously dramatized by the bite of madeleine dipped into tea, which results in an entire chapter of his childhood coming back to him in a shudder:

And just as the Japanese amuse themselves by filling a porcelain bowl with water and steeping in it little crumbs of paper which until then are without character or form, but, the moment they become wet, stretch themselves and bend and take on colour and distinctive shape, become flowers or houses or people, permanent and recognisable, so in that moment all the flowers in our garden and in M. Swann’s park, and the water-lilies on the Vivonne and the good folk of the village and their little dwellings and the parish church and the whole of Combray and of its surroundings, taking their proper shapes and growing solid, sprang into being, town and gardens alike, from my cup of tea.

5. Tactical advice

A brief aside on tactics. This book is long, and I had to make it through – while running a company, no less – so here is how I did it.

First, translation: I picked Moncrieff and Kilmartin, revised by Enright, in the Random House three-volume set. There are the Penguin editions, but I did not like the idea of a different translator doing each volume, and I found Lydia Davis, who translates Book 1, unmusical and clumsy. The new Oxford World’s Classics translations look promising, but also suffer from the multiple translators problem. (That said, the Charlotte Mandell translation of Book 2 is amazing.)

Second, how to read it. The key is consistency. I made sure to always read 10 pages a day, which was usually a mix of (a) read a bit before going to bed (b) download a copy of the book on my phone and read a few pages on the way to work every day. The phone thing sounds weird — who reads Proust on an iPhone?— but this helped me stay consistent by widening the range of situations in which I could do the key thing, which was get a few pages in. I got the idea from this great Atlantic article, “Reading Proust on my Cellphone” (link). (I didn’t use audiobook, because I struggle to retain anything I hear on audio, but anything that helps a reader make forward progress is good.)

On weekends, I’d do longer stretches and knock out 100 pages a day. In this way the whole series took me about 6 months to read. I mostly did not read other books the whole time, which was the most painful part: but I figured this discipline would be necessary, otherwise I’d just get distracted.

There are some tedious stretches: in book 1, the Combray section goes on too long; there’s an interminable salon scene in book 3; and I found some of the endless misery and claustrophobia of book 5 and 6 hard to tolerate. But I promise you Book 7 (Time Found Again) is worth it, a pinnacle of literature, and moreover it justifies the entire series of books. Without book 7, the whole thing is just good; but book 7 is the payoff, book 7 helps you understand why he is doing all of it. I can’t really say more without spoiling it too much, but book 7 is where all the treasure is hidden. You must make it to book 7.

6. Modernity

Most discourse on the novel focuses on interiority, but Proust is as great a novelist about the exterior world, and the book is a fascinating record of the death of aristocratic manners, the rise of technology, the disruption of the war, Dreyfus, Jewishness, homosexuality, society and social habits, and more. This is, in part, a novel about the death of the old world, and the rise of the new.

In book 3, the narrator tries a telephone for the first time, and, after expressing awe at the miracle, immediately gets used to it:

The telephone was not yet at that date as commonly in use as it is to-day. And yet habit requires so short a time to divest of their mystery the sacred forces with which we are in contact, that, not having had my call at once, the only thought in my mind was that it was very slow, and badly managed, and I almost decided to lodge a complaint.

In book 4, he sees an aeroplane for the first time:

Suddenly my horse reared; he had heard a strange sound; it was all I could do to hold him... then I raised my tear-filled eyes to the point from which the sound seemed to come and saw, not two hundred feet above my head, against the sun, between two great wings of flashing metal which were bearing him aloft, a creature whose indistinct face appeared to me to resemble that of a man. I was as deeply moved as an ancient Greek on seeing for the first time a demi-god... I wept... at the thought that what I was going to see for the first time was an aeroplane.

And, in book 7, you get these lovely descriptions of the narrator walking through Paris during World War I:

The night was as beautiful as in 1914 when Paris was equally menaced. The moonbeams seemed like soft, continuous magnesium-light offering for the last time nocturnal visions of beautiful sites such as the Place Vendôme and the Place de la Concorde, to which my fear of shells which might destroy them lent a contrasting richness of as yet untouched beauty as though they were offering up their defenceless architecture to the coming blows.

7. Proust’s philosophy

There is a philosophy in this novel that’s implicit but never fully outlined. It is something like: the inner life is all that matters. What’s stored in you, your deepest impressions, are what counts; these are when you live most vividly, and it is this ‘feeling aliveness’ that art is really supposed to channel and convey.

Note that often you don’t know what counts until later: e.g. the madeleine/tea moment, when he experienced it the first time, was not obviously important. It was only the later moment that ‘activated’ it as an important memory.

There is a passage I find memorable around the middle of the book, in which Marcel reads a (made-up) excerpt from the Goncourt brothers’ journal about a dinner party at the Verdurin’s salon, which he himself has been to many times. He becomes depressed after reading this, because the Goncourts, after attending this dinner party once, are able to produce many glorious pages stuffed with specific detail and precise, funny observations, and Marcel has been going to the exact same salon for months and is not able to say much about it at all beyond the fact that he finds it tedious. In this way Marcel learns that his own life, as banal as he feels it, can in fact be raw material for art if he learns to see, and examine, his own memory more deeply.

Proust can sometimes seem oddly Buddhist, e.g. at one point he refers to “the incurable imperfection in the very essence of the present moment”. But Proust’s philosophy is anti-Buddhist. For Buddhism, the (external) present is all that counts: for Proust, it is only the (internal) past. All value accretes in the memory and imagination, and ultimately all of life is to serve that, with artistic creation being the highest point of life, the transmutation of memory and imagination into an externalized object, the work of art. The present moment, paradoxically, doesn’t matter much, until later when you recollect it.

Even people are just material for art. This is the part of Proust’s philosophy that can seem anti-human: all of the people in his life are like puppets for him (and he says this explicitly in Book 7). This is clearest with his lover, Albertine: much of book 5 is him trying to control her and make her match the original, strong impression he had of her when he first saw her at the beach, and instead having to deal with the real, living Albertine-in-flux that’s right in front of him, stubborn and uncontrollable, with ‘fugitive’ and contradictory desires. And Marcel prefers her asleep, since then his imagination can take over entirely, and the present, real person in front of him does not interfere with the one he wants to see.

In a similar vein, Proust is against friendship to a shocking degree, describing it as “an abdication of duty”. In various parts of the novel, he describes friendship as a waste of time for the artist; only solitude spent in recollection and artistic creation counts. Solitude allows you go deeper into yourself, where you might draw out your own treasures. “The artist who gives up an hour of work for an hour of conversation with a friend knows that he is sacrificing a reality [i.e. art] for something that does not exist.“

I don’t know if I agree with all of this – and I certainly don’t live my life this way – but I do understand it. And like any great work of art, Proust constantly works against his own stated purposes; and by putting it down in such exhaustive, beautiful detail, the book celebrates life. The weight of the specifics in this novel, the sheer richness of it, and the force of the epiphany in the final book marked my soul deeply.

Later in the book, one of the writers the narrator models himself against, Bergotte, is looking at this Vermeer painting, View of Delft, and he becomes obsessed with how, even in this large landscape, Vermeer paints the little patch of yellow wall towards the right of the painting so perfectly.

At last he came to the Vermeer which he remembered as more striking, more different from anything else he knew, but in which, thanks to the critic’s article, he noticed for the first time some small figures in blue, that the sand was pink, and, finally, the precious substance of the tiny patch of yellow wall. His dizziness increased; he fixed his gaze, like a child upon a yellow butterfly that it wants to catch, on the precious little patch of wall.

‘That’s how I ought to have written,’ he said. ‘My last books are too dry, I ought to have gone over them with a few layers of colour, made my language precious in itself, like this little patch of yellow wall.’

Meanwhile he was not unconscious of the gravity of his condition. In a celestial pair of scales there appeared to him, weighing down one of the pans, his own life, while the other contained the little patch of wall so beautifully painted in yellow. He felt that he had rashly sacrificed the former for the latter. ... He repeated to himself: ‘Little patch of yellow wall, with a sloping roof, little patch of yellow wall.’

Who knows if the religion of art was really enough for Proust; was he happy and fulfilled, after all? Better than any author before him, Proust exposes the hollowness of society, the inner ring, all driven by mimetic desire; and in love, too, he realizes he is chasing phantoms that will never satisfy him. It is not an exaggeration to describe the ending of the book as a sort of conversion, and Proust even slips in John 12:24:

And often I asked myself not only whether there was still time but whether I was in a condition to accomplish my work. Illness which had rendered me a service by making me die to the world (for if the grain does not die when it is sown, it remains barren but if it dies it will bear much fruit), was now perhaps going to save me from idleness…

But he does not go all the way. Ultimately, Proust locates his salvation in art and in his own memories, for him the most sacred thing of all; whereas Girard, like Dostoyevsky’s heroes, found his own salvation in Christ.

Photograph of Proust on his deathbed, by Man Ray.

My thanks to Tyler Cowen, Henry Oliver, and Jessie Li for reading a draft of this piece.

A cursed fact of the world is that the most important life lessons you learn are the hardest to communicate to others. They always sound like clichés. In any case, these are a few things I’ve learned from experience and that I try and keep in mind.

1. Think about what makes you ‘imbalanced’ as a personality, & do things where this gives you an edge.

2. Once you are ok with people telling you ‘no’, you can ask for whatever you want. (Make reality say no to you.)

3. Fun is underrated. The best and most creative work comes from a root of joy and excitement. You can feel this in your body.

4. Environment matters a lot; move to where you flourish maximally. Put yourself in environments where you have to perform to your utmost; if you can get by being average, you probably will. (Greek saying: “A captain only shows during a storm.”)

5. Later, you’ll be nostalgic for right now.

6. Do things fast. Things don’t actually take much time (as measured by a stopwatch); resistance/procrastination does. “Slow is fake”. If no urgency exists, impose some.

7. Moving fast forces you to strip things down to the bare bones.

8. Wealth can be created, there is not a fixed amount of it in the world. Somebody doing well doesn’t always come at someone else’s expense.

9. “The world is a museum of passion projects.”

10. Doing as much as you can every day is a form of life extension.

11. Always be high integrity, even when it costs you. The shortcuts aren’t worth it.

12. Figure out what your primary focus is and make progress on that every day, first thing in the morning, no exceptions. Days with 0 output are the killers. (Tyler Cowen)

13. Pay attention to your production/consumption balance. If you’re only consuming and not producing, fix that.

14. You don’t do anyone any favors by lurking, put yourself out there!

15. If you don’t “get” a classic book or movie, 90% of the time it’s your fault. (It might just not be the right time for you to appreciate that thing.)

16. If you find yourself dreading Mondays, quit.

17. Lean into the good kind of fear.

18. Pick some kind of fitness/athletic activity to get addicted to, and get addicted to it for its own sake. (For me, this is running. Zone 2 cardio is underrated.)

19. Learn how to meditate, even if you don’t end up doing it regularly. The techniques are useful. (99% of books/resources on this are quite bad - I’d recommend looking at Rob Burbea’s talks and jhana practice as a way in.)

20. No matter how bad things seem, everything passes.

21. You are probably too risk-averse. Write out the worst things that can happen, realize they’re not that bad, then take the leap.

22. Do a review of your year, every year, write it out, figure out what was good and what was bad, use this to make your goals for the next year.

23. Doing things is energizing, wasting time is depressing. You don’t need that much ‘rest’.

24. Being able to travel is one of the key ways the modern world is better than the old world. Learn to travel well.

25. Form opinions on things and then find the strongest critique of those opinions. Repeat.

26. If you really can’t disprove something, it has a chance of being right. (Fallibilism.)

27. Memorize a few old poems, or texts that mean a lot to you.

28. At some point in your life, work on a startup, or at least a thing driven by a small group. Small group energy is amazing.

29. Be careful about rationalizing something that does not feel right. (Utilitarians, this means you!)

30. Know your ‘triggers’ / what makes you the worst version of yourself.

31. Figure out what creates enduring value. (A non exclusive list: great cultural artifacts such as books; great companies/institutions).

32. Don’t let anyone make you feel small.

33. Working with people you really respect, and are secretly worried are much better than you and will figure out how dumb you are, is the best.

34. “Aim for Chartres” (Christopher Alexander) — when doing something, aim to be the best there ever was at it. This compensates for your natural bias, which is to do something mediocre. You have to really aim to be as good as the greats.

35. Send more cold emails. People respond! Assume everyone’s your friend.

36. Have a lot of crazy experiences in your 20s.

37. There are some people who, after you talk to them, you feel more energized and you want to conquer the world or climb a mountain or something. They’re rare but they exist. Go find them and make friends with them.

38. Move to where the action is. Agglomeration effects are powerful.

39. Status is fake and transient. Just focus on substance and doing valuable work. Talk about it in public. Beware the inner ring fallacy.

40. Ask dumb questions. The people who matter won’t judge you for it, and you’ll learn things as a result. (See: How To Understand Things)

41. Don’t over-index on trends. Just figure out your first-principles view of what’s actually important for the world, and go from there.

42. There is some wisdom in “fake it till you make it”.

43. Don’t “slow down” as you get older, speed up. Lean into changes, be curious about new things. Most people seem to go the other way.

44. Be specific.

45. Understand power laws. Outlier math rules all.

46. Stop asking for approval and permission from others. School and work trains people to have this mindset. Instead, figure out what you want to do, and plant the “this is happening” flag. People will come along for the ride.

47. Werner Herzog to Errol Morris: “when it comes to filmmaking, money isn’t important, the intensity of your wishes and faith alone are the deciding factors.”

48. Don’t network, make friends. Writing online is great for bringing interesting people your way. Having a wide network of friends really makes a difference to the opportunities you get and how easy it is to launch your projects.

49. Figure out what gives you new ideas, and make sure you incorporate that into your routine. For me this is talking to people, tweeting, writing in my notebook, long conversations with friends (especially late night or while walking). For other people this is showering, baths, long walks, runs, etc. Make sure you “harvest” these ideas too, i.e. write them down somewhere so they don’t get lost.

50. When writing, separate the “creator” and the “editor”. The “creator” just writes, and doesn’t worry about quality; the goal is words on the page. Later, you can be the “editor” and shape it into something good.

51. Be honest about whether something is learning or entertainment. Real learning is extremely hard and effortful. (Podcasts, Atlantic articles, pop science books, anything that’s a bit too digestible is more “entertainment” than real learning).

52. Any given “bet” you take is likely to fail. Success is making lots of “bets” and trying as hard as possible at each of them. P(success) is higher the more bets you take & the better your execution per bet. (This is also why fast cycle time is so important). Ravin first articulated this to me.

53. Think in writing. Write Google Docs, scrawl in notebooks. This extends working memory arbitrarily and allows your thoughts to compound on each other. (”The difference between a Turing machine and a finite state machine is the tape.”)

54. Once in awhile, put away all concepts you read about online and reason “up” from the base of your experience and what you’ve seen and done. (e.g. What were some of the best decisions you made? The worst? Why? Can you apply those lessons now? etc.)

55. 80% utilitarian, 15% deontologist, 5% virtue ethics.

56. Most intelligent people want status, prestige, wealth and the like -- even the ones who claim not to. “Genius” is a distinct category to intelligence and geniuses are motivated by entirely different things and are socially strange (and often selfish) people. Ayn Rand is a bad writer but is one of the very few people who grokked this and made it explicit.

57. Scrolling and reading too much drowns out your inner voice.

58. Be very suspicious of a priori arguments on empirical matters (e.g. “AI will certainly kill everyone”), they’re usually wrong no matter how convincing. Reality is endlessly surprising. (Cf. Knightian uncertainty.)

59. There’s a lot of alpha in being willing to do “menial” work (take notes, send out agendas, order pizza, manually inspect raw data, whatever). Beware over-delegation and being too far from the details.

60. One key to effective negotiation is to have multiple options and be OK with it not working out. Even if you really need the thing, it’s never your only choice: reframe and create alternative options until you have multiple outcomes you’re ok with. (See BATNA.)

61. Luck isn’t a constant, it increases with surface area: be in the right places, have lots of conversations, put yourself out there, ask for what you want and be optimistic and positive. See #52.

62. The most valuable feedback usually hurts a lot.

63. If you want to think originally and differently, seek uncorrelated inputs. Read minor works, older things, obscure journals.

For long-time employees and alumni of the company, this feels deeply weird. During the 2016-2020 era especially, telling people you worked at Palantir was unpopular. The company was seen as spy tech, NSA surveillance, or worse. There were regular protests outside the office. Even among people who didn’t have a problem with it morally, the company was dismissed as a consulting company masquerading as software, or, at best, a sophisticated form of talent arbitrage.

I left last year, but never wrote publicly about what I learned there. There’s also just a lot about the company people don’t understand. So this is my effort to explain some of that, as someone who worked there for eight years.

(Note: I’m writing this in my personal capacity, and don’t have a formal relationship with the company anymore. I’m long $PLTR.)

1. Why I joined

I joined in summer 2015, initially in the newly-opened London office, before moving to Silicon Valley, and finally DC – as a forward deployed engineer. For context, the company was around 1500 people at the time; it had offices in Palo Alto (HQ), NYC, London, and a few other places. (It’s now 4000 or so people, and headquartered in Denver.)

Why did I join?

First, I wanted to work in ‘difficult’ industries on real, meaningful problems. My area of interest – for personal reasons - was healthcare and bio, which the company had a nascent presence in. The company was talking about working in industries like healthcare, aerospace, manufacturing, cybersecurity, and other industries that I felt were very important but that most people were not, at the time, working on. At the time the hot things were social networks (Facebook, LinkedIn, Quora, etc.) and other miscellaneous consumer apps (Dropbox, Uber, Airbnb) but very few companies were tackling what felt like the real, thorny parts of the economy. If you wanted to work on these ‘harder’ areas of the economy but also wanted a Silicon Valley work culture, Palantir was basically your only option for awhile.

My goal was to start a company, but I wanted (1) to go deep in one of these industries for a while first and learn real things about it; (2) to work for a US company and get a green card that way. Palantir offered both. That made it an easy choice.

Second, talent density. I talked to some of the early people who started the healthcare vertical (Nick Perry, Lekan Wang, and Andrew Girvin) and was extremely impressed. I then interviewed with a bunch of the early business operations and strategy folks and came away even more impressed. These were seriously intense, competitive people who wanted to win, true believers; weird, fascinating people who read philosophy in their spare time, went on weird diets, and did 100-mile bike rides for fun. This, it turned out, was an inheritance from the Paypal mafia. Yishan Wong, who was early at Paypal, wrote about the importance of intensity:

"In general, as I begin to survey more startups, I find that the talent level at PayPal is not uncommon for a Silicon Valley startup, but the differentiating factor may have been the level of intensity from the top: both Peter Thiel and Max Levchin were extremely intense people - hyper-competitive, hard-working, and unwilling to accept defeat. I think this sort of leadership is what pushes the "standard" talented team to be able to do great things and, subsequently, contributes to producing a wellspring of later achievements."

Palantir was an unusually intense and weird place. I remember my first time I talked to Stephen Cohen he had the A/C in his office set at 60, several weird-looking devices for minimizing CO2 content in the room, and had a giant pile of ice in a cup. Throughout the conversation, he kept chewing pieces of ice. (Apparently there are cognitive benefits to this.)

I also interviewed with the CEO, Alex Karp and talked to other members of the leadership team. I probably don’t need to convince you that Karp is weird - just watch an interview with him. I can’t say what Karp and I talked about, but he gives a good flavor for his style in a 2012 interview:

I like to meet candidates with no data about them: no résumé, no preliminary discussions or job description, just the candidate and me in a room. I ask a fairly random question, one that is orthogonal to anything they would be doing at Palantir. I then watch how they disaggregate the question, if they appreciate how many different ways there are to see the same thing. I like to keep interviews short, about 10 minutes. Otherwise, people move into their learned responses and you don’t get a sense of who they really are.

My interviews were often not about work or software at all – one of my interviews we just spent an hour talking about Wittgenstein. Note that both Peter Thiel and Alex Karp were philosophy grads. Thiel’s lecture notes had come out not long before (https://blakemasters.tumblr.com/peter-thiels-cs183-startup) and they discussed Shakespeare, Tolstoy, Girard (then unknown, now a cliché) and more.

The combo of intellectual grandiosity and intense competitiveness was a perfect fit for me. It’s still hard to find today, in fact - many people have copied the ‘hardcore’ working culture and the ‘this is the Marines’ vibe, but few have the intellectual atmosphere, the sense of being involved in a rich set of ideas. This is hard to LARP - your founders and early employees have to be genuinely interesting intellectual thinkers. The main companies that come to mind which have nailed this combination today are OpenAI and Anthropic. It’s no surprise they’re talent magnets.1

2. Forward deployed

When I joined, Palantir was divided up into two types of engineers:

1. Engineers who work with customers, sometimes known as FDEs, forward deployed engineers.

2. Engineers who work on the core product team (product development - PD), and rarely go visit customers.

FDEs were typically expected to ‘go onsite’ to the customer’s offices and work from there 3-4 days per week, which meant a ton of travel. This is, and was, highly unusual for a Silicon Valley company.

There’s a lot to unpack about this model, but the key idea is that you gain intricate knowledge of business processes in difficult industries (manufacturing, healthcare, intel, aerospace, etc.) and then use that knowledge to design software that actually solves the problem. The PD engineers then ‘productize’ what the FDEs build, and – more generally – build software that provides leverage for the FDEs to do their work better and faster.2

This is how much of the Foundry product took initial shape: FDEs went to customer sites, had to do a bunch of cruft work manually, and PD engineers built tools that automated the cruft work. Need to bring in data from SAP or AWS? Here’s Magritte (a data ingestion tool). Need to visualize data? Here’s Contour (a point and click visualization tool). Need to spin up a quick web app? Here’s Workshop (a Retool-like UI for making webapps). Eventually, you had a damn good set of tools clustered around the loose theme of ‘integrate data and make it useful somehow’.

At the time, it was seen as a radical step to give customers access to these tools — they weren’t in a state for that — but now this drives 50%+ of the company’s revenue, and it’s called Foundry. Viewed this way, Palantir pulled off a rare services company → product company pivot: in 2016, descriptions of it as a Silicon Valley services company were not totally off the mark, but in 2024 they are deeply off the mark, because the company successfully built an enterprise data platform using the lessons from those early years, and it shows in the gross margins - 80% gross margins in 2023. These are software margins. Compare to Accenture: 32%.

Tyler Cowen has a wonderful saying, ‘context is that which is scarce’, and you could say it’s the foundational insight of this model. Going onsite to your customers – the startup guru Steve Blank calls this “getting out of the building” – means you capture the tacit knowledge of how they work, not just the flattened ‘list of requirements’ model that enterprise software typically relies on. The company believed this to a hilarious degree: it was routine to get a call from someone and have to book a first-thing-next-morning flight to somewhere extremely random; “get on a plane first, ask questions later” was the cultural bias. This resulted in out of control travel spend for a long time — many of us ended up getting United 1K or similar — but it also meant an intense decade-long learning cycle which eventually paid off.

My first real customer engagement was with Airbus, the airplane manufacturer based in France, and I moved out to Toulouse for a year and worked in the factory alongside the manufacturing people four days a week to help build the version of our software there.

My first month in Toulouse, I couldn’t fly out of the city because the air traffic controllers were on strike every weekend. Welcome to France. (I jest - France is great. Also, Airbus planes are magnificent. It’s a truly engineering-centric company. The CEO is always a trained aeronautical engineer, not some MBA. Unlike… anyway.)

The CEO told us his biggest problem was scaling up A350 manufacturing. So we ended up building software to directly tackle that problem. I sometimes describe it as “Asana, but for building planes”. You took disparate sources of data — work orders, missing parts, quality issues (“non-conformities”) — and put them in a nice interface, with the ability to check off work and see what other teams are doing, where the parts are, what the schedule is, and so on. Allow them the ability to search (including fuzzy/semantic search) previous quality issues and see how they were addressed. These are all sort of basic software things, but you’ve seen how crappy enterprise software can be - just deploying these ‘best practice’ UIs to the real world is insanely powerful. This ended up helping to drive the A350 manufacturing surge and successfully 4x’ing the pace of manufacturing while keeping Airbus’s high standards of quality.

This made the software hard to describe concisely - it wasn’t just a database or a spreadsheet, it was an end-to-end solution to that specific problem, and to hell with generalizability. Your job was to solve the problem, and not worry about overfitting; PD’s job was to take whatever you’d built and generalize it, with the goal of selling it elsewhere.

The A350 final assembly line, in Toulouse. I hung out here most days. It was awe-inspiring. Screenshot from here.

FDEs tend to write code that gets the job done fast, which usually means – politely – technical debt and hacky workarounds. PD engineers write software that scales cleanly, works for multiple use cases, and doesn’t break. One of the key ‘secrets’ of the company is that generating deep, sustaining enterprise value requires both. BD engineers tend to have high pain tolerance, the social and political skills needed to embed yourself deep in a foreign company and gain customer trust, and high velocity – you need to build something that delivers a kernel of value fast so that customers realize you’re the real deal. It helped that customers had hilariously low expectations of most software contractors, who were typically implementors of SAP or other software like that, and worked on years-long ‘waterfall’ style timescales. So when a ragtag team of 20-something kids showed up to the customer site and built real software that people could use within a week or two, people noticed.

This two-pronged model made for a powerful engine. Customer teams were often small (4-5 people) and operated fast and autonomously; there were many of them, all learning fast, and the core product team’s job was to take those learnings and build the main platform.

When we were allowed to work within an organization, this tended to work very well. The obstacles were mostly political. Every time you see the government give another $110 million contract to Deloitte for building a website that doesn’t work, or a healthcare.gov style debacle, or SFUSD spending $40 million to implement a payroll system that - again - doesn’t work, you are seeing politics beat substance. See SpaceX vs. NASA as another example.

The world needs more companies like SpaceX, and Palantir, that differentiate on execution - achieving the outcome - not on playing political games or building narrow point solutions that don’t hit the goal.

3. Secrets

Another key thing FDEs did was data integration, a term that puts most people to sleep. This was (and still is) the core of what the company does, and its importance was underrated by most observers for years. In fact, it’s only now with the advent of AI that people are starting to realize the importance of having clean, curated, easy-to-access data for the enterprise. (See: the ‘it’ in AI models is the dataset).

In simple terms, ‘data integration’ means (a) gaining access to enterprise data, which usually means negotiating with ‘data owners’ in an organization (b) cleaning it and sometimes transforming it so that it’s usable (c) putting it somewhere everyone can access it. Much of the base, foundational software in Palantir’s main software platform (Foundry) is just tooling to make this task easier and faster.

Why is data integration so hard? The data is often in different formats that aren’t easily analyzed by computers – PDFs, notebooks, Excel files (my god, so many Excel files) and so on. But often what really gets in the way is organizational politics: a team, or group, controls a key data source, the reason for their existence is that they are the gatekeepers to that data source, and they typically justify their existence in a corporation by being the gatekeepers of that data source (and, often, providing analyses of that data).3 This politics can be a formidable obstacle to overcome, and in some cases led to hilarious outcomes – you’d have a company buying an 8-12 week pilot, and we’d spend all 8-12 weeks just getting data access, and the final week scrambling to have something to demo.

The other ‘secret’ Palantir figured out early is that data access tussles were partly about genuine data security concerns, and could be alleviated through building security controls into the data integration layer of the platform - at all levels. This meant role-based access controls, row-level policies, security markings, audit trails, and a ton of other data security features that other companies are still catching up to. Because of these features, implementing Palantir often made companies’ data more secure, not less.4

4. Notes on culture

The overall ‘vibe’ of the company was more of a messianic cult than a normal software company. But importantly, it seemed that criticism was highly tolerated and welcomed – one person showed me an email chain where an entry-level software engineer was having an open, contentious argument with a Director of the company with the entire company (around a thousand people) cc’d. As a rationalist-brained philosophy graduate, this particular point was deeply important to me – I wasn’t interested in joining an uncritical cult. But a cult of skeptical people who cared deeply and wanted to argue about where the world was going and how software fit into it – existentially – that was interesting to me.5

I’m not sure if they still do this, but at the time when you joined they sent you a copy of Impro, The Looming Tower (9/11 book), Interviewing Users, and Getting Things Done. I also got an early PDF version of what became Ray Dalio’s Principles. This set the tone. The Looming Tower was obvious enough – the company was founded partly as a response to 9/11 and what Peter felt were the inevitable violations of civil liberties that would follow, and the context was valuable. But why Impro?

Being a successful FDE required an unusual sensitivity to social context – what you really had to do was partner with your corporate (or government) counterparts at the highest level and gain their trust, which often required playing political games. Impro is popular with nerds partly because it breaks down social behavior mechanistically. The vocabulary of the company was saturated with Impro-isms – ‘casting’ is an example. Johnstone discusses how the same actor can play ‘high status’ or ‘low status’ just by changing parts of their physical behavior – for example, keeping your head still while talking is high status, whereas moving your head side to side a lot is low status. Standing tall with your hands showing is high status, slouching with your hands in your pocket is low status. And so on. If you didn’t know all this, you were unlikely to succeed in a customer environment. Which meant you were unlikely to integrate customer data or get people to use your software. Which meant failure.

This is one reason why former FDEs tend to be great founders. (There are usually more ex-Palantir founders than there are ex-Googlers in each YC batch, despite there being ~50x more Google employees.) Good founders have an instinct for reading rooms, group dynamics, and power. This isn’t usually talked about, but it’s critical: founding a successful company is about taking part in negotiation after negotiation after negotiation, and winning (on net). Hiring, sales, fundraising are all negotiations at their core. It’s hard to be great at negotiating without having these instincts for human behavior. This is something Palantir teaches FDEs, and is hard to learn at other Valley companies.

Another is that FDEs have to be good at understanding things. Your effectiveness directly correlates to how quickly you can learn to speak the customer’s language and really drill down into how their business works. If you’re working with hospitals, you quickly learn to talk about capacity management and patient throughput vs. just saying “help you improve your healthcare”. Same with drug discovery, health insurance, informatics, cancer immunotherapy, and so on; all have specialized vocabularies, and the people who do well tend to be great at learning them fast.

One of my favorite insights from Tyler Cowen’s book ‘Talent’ is that the most talented people tend to develop their own vocabularies and memes, and these serve as entry points to a whole intellectual world constructed by that person. Tyler himself is of course a great example of this. Any MR reader can name 10+ Tylerisms instantly - ‘model this’, ‘context is that which is scarce’, ‘solve for the equilibrium’, ‘the great stagnation’ are all examples. You can find others who are great at this. Thiel is one. Elon is another (“multiplanetary species”, “preserving the light of consciousness”, etc. are all memes). Trump, Yudkowsky, gwern, SSC, Paul Graham, all of them regularly coin memes. It turns out that this is a good proxy for impact.

This insight goes for companies, too, and Palantir had its own, vast set of terms, some of which are obscure enough that “what does Palantir actually do?” became a meme online. ‘Ontology’ is an old one, but then there is ‘impl’, ‘artist’s colony’, ‘compounding’, ‘the 36 chambers’, ‘dots’, ‘metabolizing pain’, ‘gamma radiation’, and so on. The point isn’t to explain all of these terms, each of which compresses a whole set of rich insights; it’s that when you’re looking for companies to join, you could do worse than look for a rich internal language or vocabulary that helps you think about things in a more interesting way.

When Palantir’s name comes up, most people think of Peter Thiel. But many of these terms came from early employees, especially Shyam Sankar, who’s now the President of the company. Still, Peter is deeply influential in the company culture, even though he wasn’t operationally involved with the company at all during the time I was there. This document, written by Joe Lonsdale, was previously an internal document but made public at some point and gives a flavor for the type of cultural principles.

One of the things that (I think) came from Peter was the idea of not giving people titles. When I was there, everyone had the “forward deployed engineer” title, more or less, and apart from that there were five or six Directors and the CEO. Occasionally someone would make up a different title (one guy I know called himself “Head of Special Situations”, which I thought was hilarious) but these never really caught on. It’s straightforward to trace this back to Peter’s Girardian beliefs: if you create titles, people start coveting them, and this ends up creating competitive politics inside the company that undermines internal unity. Better to just give everyone the same title and make them go focus on the goal instead.

There are plenty of good critiques of the ‘flat hierarchy’ stance -- The Tyranny of Structurelessness is a great one – and it largely seems to have fallen out of fashion in modern startups, where you quickly get CEO, COO, VPs, Founding Engineers, and so on. But my experience is that it worked well at Palantir. Some people were more influential than others, but the influence was usually based on some impressive accomplishment, and most importantly nobody could tell anyone else what to do. So it didn’t matter if somebody was influential or thought your idea was dumb, you could ignore them and go build something if you thought it was the right thing to do. On top of that, the culture valorized such people: stories were told of some engineer ignoring a Director and building something that ended up being a critical piece of infrastructure, and this was held up as an example to imitate.

The cost of this was that the company often felt like there was no clear strategy or direction, more like a Petri dish of smart people building little fiefdoms and going off in random directions. But it was incredibly generative. It’s underrated just how many novel UI concepts and ideas came out of that company. Only some of these now have non-Palantir equivalents, e.g. Hex, Retool, Airflow all have some components that were first developed at Palantir. The company’s doing the same for AI now – the tooling for deploying LLMs at large enterprises is powerful.

The ‘no titles’ thing also meant that people came in and out of fashion very quickly, inside the company. Because everyone had the same title, you had to gauge influence through other means, and those were things like “who seems really tight with this Director right now” or “who is leading this product initiative which seems important”, not “this person is the VP of so and so”. The result was a sort of hero-shithead rollercoaster at scale – somebody would be very influential for awhile, then mysteriously disappear and not be working on anything visible for months, and you wouldn’t ever be totally sure what happened.

5. Bat-signals

Another thing I can trace back to Peter is the idea of talent bat-signals. Having started my own company now (in stealth for the moment), I appreciate this a lot more: recruiting good people is hard, and you need a differentiated source of talent. If you’re just competing against Facebook/Google for the same set of Stanford CS grads every year, you’re going to lose. That means you need a set of talent that is (a) interested in joining you in particular, over other companies (b) a way of reaching them at scale. Palantir had several differentiated sources of recruiting alpha.

First, there were all the people who were pro defense/intelligence work back when that wasn’t fashionable, which selected for, e.g., smart engineers from the Midwest or red states more than usual, and also plenty of smart ex-army, ex-CIA/NSA types who wanted to serve the USA but also saw the appeal in working for a Silicon Valley company. My first day at the company, I was at my team’s internal onboarding with another guy, who looked a bit older than me. I asked him what he’d done before Palantir. With a deadpan expression, he looked me in the eye and said “I worked at the agency for 15 years”. I was then introduced to my first lead, who was a former SWAT cop in Ohio (!) and an Army vet.

There were lots of these people, many extremely talented, and they mostly weren’t joining Google. Palantir was the only real ‘beacon’ for these types, and the company was loud about supporting the military, being patriotic, and so on, when that was deeply unfashionable. That set up a highly effective, unique bat-signal. (Now there’s Anduril, and a plethora of defence and manufacturing startups).6

Second, you had to be weird to want to join the company, at least after the initial hype wave died down (and especially during the Trump years, when the company was a pariah). Partly this was the aggressive ‘mission focus’ type branding back when this was uncommon, but also the company was loud about the fact that people worked long hours, were paid lower than market, and had to travel a lot. Meanwhile, we were being kicked out of Silicon Valley job fairs for working with the government. All of this selected for a certain type of person: somebody who can think for themselves, and doesn’t over-index on a bad news story.

6. Morality

The morality question is a fascinating one. The company is unabashedly pro-West, a stance I mostly agree with – a world more CCP-aligned or Russia-aligned seems like a bad one to me, and that’s the choice that’s on the table.7 It’s easy to critique free countries when you live in one, harder when you’ve experienced the alternative (as I have - I spent a few childhood years in a repressive country). So I had no problem with the company helping the military, even when I disagreed with some of the things the military was doing.

But doesn’t the military sometimes do bad things? Of course - I was opposed to the Iraq war. This gets to the crux of the matter: working at the company was neither 100% morally good — because sometimes we’d be helping agencies that had goals I’d disagree with — nor 100% bad: the government does a lot of good things, and helping them do it more efficiently by providing software that doesn’t suck is a noble thing. One way of clarifying the morality question is to break down the company’s work into three buckets – these categories aren’t perfect, but bear with me:

1. Morally neutral. Normal corporate work, e.g. FedEx, CVS, finance companies, tech companies, and so on. Some people might have a problem with it, but on the whole people feel fine about these things.

2. Unambiguously good. For example, anti-pandemic response with the CDC; anti-child pornography work with NCMEC; and so on. Most people would agree these are good things to work on.

3. Grey areas. By this I mean ‘involve morally thorny, difficult decisions’: examples include health insurance, immigration enforcement, oil companies, the military, spy agencies, police/crime, and so on.

Every engineer faces a choice: you can work on things like Google search or the Facebook news feed, all of which seem like marginally good things and basically fall into category 1. You can also go work on category 2 things like GiveDirectly or OpenPhilanthropy or whatever.

The critical case against Palantir seemed to be something like “you shouldn’t work on category 3 things, because sometimes this involves making morally bad decisions”. An example was immigration enforcement during 2016-2020, aspects of which many people were uncomfortable with.

But it seems to me that ignoring category 3 entirely, and just disengaging with it, is also an abdication of responsibility. Institutions in category 3 need to exist. The USA is defended by people with guns. The police have to enforce the law, and - in my experience - even people who are morally uncomfortable with some aspects of policing are quick to call the police if their own home has been robbed. Oil companies have to provide energy. Health insurers have to make difficult decisions all the time. Yes, there are unsavory aspects to all of these things. But do we just disengage from all of these institutions entirely, and let them sort themselves out?

I don’t believe there is a clear answer to whether you should work with category 3 customers; it’s a case by case thing. Palantir’s answer to this is something like “we will work with most category 3 organizations, unless they’re clearly bad, and we’ll trust the democratic process to get them trending in a good direction over time”. Thus:

• On the ICE question, they disengaged from ERO (Enforcement and Removal Operations) during the Trump era, while continuing to work with HSI (Homeland Security Investigations).

• They did work with most other category 3 organizations, on the argument that they’re mostly doing good in the world, even though it’s easy to point to bad things they did as well.

  • I can’t speak to specific details here, but Palantir software is partly responsible for stopping multiple terror attacks. I believe this fact alone vindicates this stance.

This is an uncomfortable stance for many, precisely because you’re not guaranteed to be doing 100% good at all times. You’re at the mercy of history, in some ways, and you’re betting that (a) more good is being done than bad (b) being in the room is better than not. This was good enough for me. Others preferred to go elsewhere.

The danger of this stance, of course, is that it becomes a fully general argument for doing whatever the power structure wants. You are just amplifying existing processes. This is where the ‘case by case’ comes in: there’s no general answer, you have to be specific. For my own part, I spent most of my time there working on healthcare and bio stuff, and I feel good about my contributions. I’m betting the people who stopped the terror attacks feel good about theirs, too. Or the people who distributed medicines during the pandemic.

Even though the tide has shifted and working on these ‘thorny’ areas is now trendy, these remain relevant questions for technologists. AI is a good example – many people are uncomfortable with some of the consequences of deploying AI. Maybe AI gets used for hacking; maybe deepfakes make the world worse in all these ways; maybe it causes job losses. But there are also major benefits to AI (Dario Amodei articulates some of these well in a recent essay).

As with Palantir, working on AI probably isn’t 100% morally good, nor is it 100% evil. Not engaging with it – or calling for a pause/stop, which is a fantasy – is unlikely to be the best stance. Even if you don’t work at OpenAI or Anthropic, if you’re someone who could plausibly work in AI-related issues, you probably want to do so in some way. There are easy cases: build evals, work on alignment, work on societal resilience. But my claim here is that the grey area is worth engaging in too: work on government AI policy. Deploy AI into areas like healthcare. Sure, it’ll be difficult. Plunge in.8

When I think about the most influential people in AI today, they are almost all people in the room - whether at an AI lab, in government, or at an influential think tank. I’d rather be one of those than one of the pontificators. Sure, it’ll involve difficult decisions. But it’s better to be in the room when things happen, even if you later have to leave and sound the alarm.

7. What next?

Do I remain bullish on the company? Yes.

The big productivity gains of this AI cycle are going to come when AI starts providing leverage to the large companies and businesses of this era - in industries like manufacturing, defense, logistics, healthcare and more. Palantir has spent a decade working with these companies. AI agents will eventually drive many core business workflows, and these agents will rely on read/write access to critical business data. Spending a decade integrating enterprise data is the critical foundation for deploying AI to the enterprise. The opportunity is massive.

As for me, I’m carrying out my long-awaited master plan and starting a company next. Yes, there will be a government component to it. The team is great, and yes we’re hiring. We even talk about Wittgenstein sometimes.

Thanks to Rohit Krishnan, Tyler Cowen, Samir Unni, Sebastian Caliri, Mark Bissell, and Vipul Shekhawat for their feedback on this post.

2

Ted Mabrey’s essay on the FDE model here is good:

Ted’s Substack

Sorry, that isn't an FDE

One of Palantir’s secrets in the Zero to One sense of the word has been the FDE. We were criticized for a very long time for this approach to building and delivering software. This criticism was annoying in the moment but in a full accounting quite valuable for Palantir. The criticism created a chilling effect and resulting conformity in how software…

Read more

2 years ago · 40 likes · 3 comments · Ted Mabrey

3

Sarah Constantin – also an ex-Palantirian - goes into greater detail on this point in her great essay:

Rough Diamonds

The Great Data Integration Schlep

This is a little rant I like to give, because it’s something I learned on the job that I’ve never seen written up explicitly…

Read more

2 years ago · 130 likes · 24 comments · Sarah Constantin

I did not know what a "jhana" was before Twitter. I made various COVID-related decisions in January 2020 because of threads on Twitter. I met 100+ people through Twitter, many of whom are now good friends. Random famous people have invited me to dark smoky rooms full of interesting people and parties I would never have gone to otherwise -- because I tweeted something. It’s a serendipity machine.

But most importantly, it's just plain fun. If you’re curious about practically any topic, you can follow 10-20 experts in it, and immediately get a felt sense of some of the ‘live issues’ on the edge of that field. That’s amazing!

Onboarding to Twitter is challenging. It's not a rewarding experience for most people; your posts barely get any engagement at all until you have a few hundred followers at least, and the process to get there is punishing. It takes a lot of time to curate a good Twitter feed; the median content on there is trash.

But a well-curated Twitter feed is worth a lot of IQ points -- so I think it's worth doing. 

Although you can use Twitter in read-only mode and get value out of it, if you're a curious person who wants to make friends and meet people in your niche, writing tweets is also a good practice. I think of each tweet as an option with uncapped upside and little downside; each tweet is thus a ‘free option’, in finance-speak. So I think you should tweet more! (More on this below).

Here are some rules I try and follow. Maybe they’ll be helpful to you:

On reading tweets

Follow people who post insightful or interesting or amusing things. You can seed this list by just going through the list of a few 'index' accounts who have well-curated Twitter feeds and following the people they follow to start with (I suggest browsing Tyler Cowen or Paul Graham, but it depends on your interests), and then pruning from there + hitting 'follow' on new accounts if you like the look of their tweets. Don't follow accounts just because they're official or prominent on some way; often the best accounts have ~5,000 followers or even less. I think the sweet spot is following around 500-1,000 people; too few and I find you don’t get enough interesting stuff, too many and you start missing things and drowning in noise.

For some accounts, it helps to disable their retweets, especially they retweet a lot of stuff that’s irrelevant to you but otherwise have good insights.

Ignore and aggressively mute any content designed to make you angry or that does not make you feel good. Especially don't get sucked into culture wars. The algorithm can be your friend of you train it, but it's not your friend by default. On the ‘For You’ feed, you can tell the algorithm to downrank tweets you don’t like. This works ok. Some people prefer using lists or just using the ‘following’ tab; whatever works for you.

Replying on Twitter is an art, and most people are bad at it. Practice 'good reply game'. You can think of this as applying a 'yes and' mentality to the whole thing: contribute novel and interesting observations. Be cautious about getting into arguments, it's not a great medium for them; you can push back on people who you've built some trust with, but otherwise I find disagreements tend to go poorly.

Twitter search is incredibly under-used. For whatever you're currently interested in, try using the Twitter search bar and scrolling; often you'll find something delightful related to it. It’s higher-variance than Google, but that ways in both directions.

Some specific tips for search:

• Use filters in your search, e.g. adding "min_faves:10" means only tweets with >10 likes will show up, which filters out dross. If you want to find a user’s most popular tweets, do “from:username min_faves:250”. 

• If you find a particular user whose thoughts you love, try “from:username [keyword]” for various values of ‘keyword’ and see what comes up; often it leads you to delightful places.

• You can combine these two. For example, here are Karpathy’s tweets on LLMs, with a popularity filter included. 

• Adding "filter:follows" is insanely powerful once you've built up a good following feed. For example, say I’m organizing an unconference and I want to get tips for it; I search “filter:follows unconference” and magic happens. (This works best when you follow the right people, obviously.)

• Search through various Twitter niches -- film Twitter, Shakespeare Twitter, Rust Twitter, econ Twitter -- and enjoy the rich content. 

• For specific things, it can beat Google if you’re clever about it; for example, if you're trying to cook a great steak, you can often get better advice in Twitter threads than on recipe websites. 

On writing tweets

It can be scary to tweet, and you may feel pressure to play a certain character or tweet for engagement. I personally got over this by thinking of tweets as a type of public timestamped journal of notes to myself; an index of what I was thinking about at the time. Over time, this builds up and can be a powerful creative tool. You’re best off just letting go of all self-consciousness -- which happens by doing it a lot -- and then treating it as a creative exercise for yourself.

I do think most people who have read this far should tweet more, as long as they can stop themselves from getting too distracted by the whole thing.

Tweet for the kinds of followers you want. Over time you build up a sense of what kinds of tweets 'do well' and what doesn't. Be suspicious of this! It's fine to tweet for engagement occasionally, but I suggest taking the 'journal' idea seriously and tweeting for the kinds of followers you want. In practice, you'll find that viral tweets don't get you high-quality followers anyway. The real “game” is meeting the kind of people you’ll get along with and making friends with them; tweet the kinds of thoughts that attract those people, which are usually not the type of tweets that go viral.

If you're the type of person who wants to write more, having a practice of tweeting regularly can be a great way of generating essay ideas. Tweeting spawns a mental process of the type "generate things to tweet" -- it serves as a wick for ideas -- and sometimes these ideas don't fit cleanly into Twitter or are interesting enough that you want to pursue them in longer form.

You will sometimes be surprised by what tweets get popular. These tweets are often the seed of great essay ideas, too.

You can think of tweeting as summoning up a virtual conference on a topic, at will. I owe this idea to Michael Nielsen’s essay on creative contexts. It's an art, but you can ask questions on Twitter and get a staggering variety of interesting answers to questions if you do it right.

Two recent examples of mine: (a) why don't education startups work out? (b) why don't more people use spaced repetition? I don't know how I would have gotten so many interesting answers to these questions without Twitter. 

Early on, replying > tweeting, if you want to gain followers and make friends. A lot of low-follower accounts try tweeting and get discouraged when nobody engages. The correct way to gain more followers is to tweet a lot of insightful stuff, yes, but you should also be replying to high-follower accounts and making friends via good reply game for a long time; you will get way more impressions this way than just via tweeting. As a bonus, if you're cool, you will make friends and get embedded in a niche.

Twitter can be a big time-sink, so figure out ways to limit your time on it to the right kind of engagement. This will vary for everyone -- e.g. some people don’t use the mobile app. 

If you want to build a professional following, you can go a long way by building cool stuff and tweeting about it. Post a demo video or a Github link, talk about the process of building it, etc. There are tons of accounts that do this super well (e.g. Sharif Shameem, Linus Lee); go copy them. It's a cheat code for the ambitious.

Do cool shit first, then tweet about it as ‘exhaust’; not the other way round. Some people become "Twitter personalities" and I think this is a trap. By this I mean: they live on Twitter first and foremost, and spend most of their time tweeting, but they don't do interesting things in real life. The best ordering is the reverse: do interesting and valuable things, or learn interesting things, and then tweet about them. Think of the tweets as exhaust from the interesting things you do; don't think of tweets as the primary product.

A simple example: many of my favorite accounts read a lot of books or papers, and just tweet out interesting paragraphs or summaries or things they learned. This is a great way to use Twitter: tweet screenshots of books you’re reading.

Don't be sarcastic or mean, don't dunk on people, minimize negativity. Be careful quote-tweeting people, it's easy to be perceived as passive-aggressive or hostile. Generally just don’t take Twitter too seriously; treat it like a fun game, and take a break if you find yourself getting upset by anything on it.

Bookmark tweets that are especially wise, so you can find them again later. I love Karpathy’s best tweets, for example. Michael Nielsen on spaced repetition. Emmett Shear on burnout. There are too many to list here.

Twitter is of great (and underrated) societal importance. Patrick McKenzie articulates it well:

“I will say, I’m an enormous fan of Twitter. I think it is actually descriptively one of the most important products that exists on the internet because it has not solved, but ameliorated a lot of cross-organization coordination problems that happen in very important places. [laughs]

It is broadly under-appreciated how Twitter is the message bus, the sub rosa coordination mechanism for the United States federal government, for every counterparty that the United States federal government or any agency or individual faces, for the media, for everything. People see Twitter, and they see celebrities posting and the phenomenon that is shitposting, et cetera, et cetera. To a very real degree, it is an integrated part of the operating system that is the world...” (Link)

There is a deep point here: Twitter is one of the few places in the world you can, if you want, create common knowledge about important and neglected things. This can make it a powerful lever for action, if you use it correctly.   

What a wonderful place! A gem of the internet.

Thanks to Tyler Cowen, Rohit Krishnan, and Matt Clifford for reading a draft of this, all of whom I met or became closer to via Twitter, and all of whom have great Twitter accounts that you should follow.

1.

I mostly don’t play chess anymore — it’s too addictive, and tends to take over your brain in a way I don’t like — but one habit I’ve retained is solving puzzles. It’s a mental warm-up, a way of occupying my brain when I don’t want to mindlessly scroll.

Along the way, I came across a fantastic book called Think Like A Super-GM, by Michael Adams and Philip Hurtado. The authors take 100 or so chess puzzles, of varying difficulty, and then ask chess players of different skill levels (amateur to grandmaster) to solve them, while recording their thinking process out loud.

It turns out comparing the thought process of less skilled vs. more skilled players gives you many useful insights! (I’d love to see this concept used for books in other disciplines.)

The lesson I found the most striking is this: there’s a direct correlation between how skilled you are as a chess player, and how much time you spend falsifying your ideas. The authors find that grandmasters spend longer falsifying their idea for a move than they do coming up with the move in the first place, whereas amateur players tend to identify a solution and then play it shortly after without trying their hardest to falsify it first. (Often amateurs, find reasons for playing the move -- ‘hope chess’.)

Call this the ‘falsification ratio’: the ratio of time you spend trying to falsify your idea to the time you took coming up with it in the first place. For grandmasters, this is 4:1 — they’ll spend 1 minute finding the right move, and another 4 minutes trying to falsify it, whereas for amateurs this is something like 0.5:1 — 1 minute finding the move, 30 seconds making a cursory effort to falsify it.

This is a really interesting finding!

Let me explain with an example. (Obviously, you’ll need to understand the rules of chess, and chess notation, to follow all this; skip to section 2 if not.) Here is White to move:

Try and solve it yourself first. (Here’s a Lichess board link. Note: the puzzle is rated advanced, and a 2000 ELO person got it wrong in the book).

Here is the difference between an amateur and a good player: 

The amateur: “Huh, f7 seems weak, what if I attack it again? Bd5 does that. Ok, so Bd5 and next move I play Qf7+. That’s got to be winning”. At this point the amateur will conclude they’ve solved the puzzle. (Note that they didn’t consider the opponent’s move at all.)

The good player: “Ok, I notice f7’s weak. Let me analyse Bd5. What’s the strongest reply — can I prove this wrong somehow? What would I play if I were Black? Oh wow, there’s Bxh4! Now if gxh4 then Black has a perpetual check (repeat Qg4 - Qd1 forever), and it’s a draw.”

Note that the good player hits on a potential right answer but immediately dives into trying their hardest to prove it wrong, as though they were the opponent, and they find that Bd5 leads to a draw (not a win). This leads them to the correct answer, which in this case is Qd7 -- this covers the g4 square, so that now if black plays Bxh4 white can safely recapture the bishop without worrying about perpetual check.

This skill of finding the best move for your opponent turns out to be very hard in practice. [1]

2.

It’s hard in real life, too: vanishingly few people are meta-rational enough to try really hard to falsify their own ideas. Your brain really wants to find reasons to support what you believe. It’s an unfamiliar and strange mental motion to invert that, and find the strongest arguments against your own beliefs.  

But this isn’t a catch-all trick. There are important disanalogies to this in life, too. Many scenarios in life have the following properties:

(1) You have to “try it and see”; there aren’t definitive, computable answers
(2) Your degree of belief partially determines how successful you are

For example, should you launch that risky project - say, a startup?

If you spend 1 hour coming up with the idea and 4 hours coming up with reasons not to pursue it… chances are you’ll probably find some compelling-seeming reason why you shouldn’t bother.

The most effective people seem to be good at accepting the dissonance and going forward with things anyway (after thinking rigorously about them!), knowing that you can always find reasons why a thing won’t work. I think startups are an example of #2: your success is partially determined by how much conviction you have. 

The “you have to try it and see” property is also counter-intuitive; it implies that your initial idea often doesn’t matter except as a starting point. This concept is often taken to mean “ideas don’t matter”, which is not what I’m saying here. It’s more like: the idea determines what maze you’re exploring, which is important — if you start out trying to find a cure for diabetes you’re unlikely to end up working on a payroll SaaS company, for example — but it’s going to change as you go along and learn more about reality. 

This is another area where I find that the personality of founders diverges from some smart engineers or scientists I’ve known: the scientist/engineers are good at analyzing the idea and finding reasons it won’t work (witness the negative Hacker News comments in virtually every successful startup launch, e.g. Dropbox), because that’s how they’re trained to think. This skeptical mindset is adaptive in the engineering and science context — you have to find all the weak points in a system or theory, or else you’re not doing your job right.

It’s not as simple as “founders are optimists, scientists are skeptics”, though. I once asked a few founder friends what they’d have told themselves about startups, and the most common one was that you know when something isn’t working way before you’re actually able to admit it to yourself, and it’s best to save yourself time by admitting that it’s not working sooner, cutting your losses, and iterating quicker. You want the cycle time to be as fast as possible: the more cycles, the higher your chances of success overall. [2]

3.

Conviction that there is an answer. There’s a cruelty to problem-solving: if you are solving exercises that are collected in a book, it’s implicit that each of those problems has a solution. Puzzle books are a forgiving problem-solving environment in that way. It means that you can bang your head against the problem for hours, because you know that there is a findable answer, and that you will find it if you simply apply enough determination and search hard enough. Unlike real life, where you have to try it (and sometimes waste a lot of time).

This lesson hit home when I was trying the following problem from Larson’s book, “Problem Solving Through Problems” which is a collection of math problems (mostly Putnam or Olympiad):

I beat my head against this one for ages, and couldn’t find any traction. It was very annoying. (You should try and write out the sequence yourself, and see what approaches you’d take to prove that “6” occurs an infinite number of times here. I found it quite slippery. Noting that I’m not a mathematician at all, so I’m probably missing some obvious-to-mathematician techniques here.)

Probably if it wasn’t in a puzzle book I’d have given up; but I knew there must be some trick. Larson groups his problems under various headings, and this one, early in the book, is under “check concrete cases”.

So I wrote a short program that would print out the first 300 items in this sequence, and stared at it:

I noticed there was a pattern – a bunch of 8s, then a bunch of 6/4s, then a bunch of 2/4s, and then a bunch of 8s again. I started playing around with the 8s, noting that 8*8 (=64) contains a 6. If you could show this keeps recurring, you’d have proven that there are an infinite number of 6.

Two 8s peters out (8, 8 -> 6, 4 -> 2, 4 -> 8).

But three 8s:

8, 8, 8 -> 6, 4, 6, 4, 6, 4 -> 2, 4, 2, 4, 2, 4 -> 8, 8, 8...

This is a cycle! It recurs infinitely! And it contains a 6! QED. But I’m not sure I’d have found this without the conviction that there was a possible solution (otherwise, why would Larson have included it in the book?). 

Real life is not this clean; frequently you dive into an idea maze and the answer turns out to be “it’s too hard for these structural reasons, and unless you fix these structural reasons there isn’t a sufficiently scalable thing here” or something like that — this seems to be the fate of a lot of startups that tackle sectors like education or health, for example. This can be painful and suck up years of people’s lives. 

This is why I think of founders and scientists as courageous in precisely the same way: they are driven by a faith that there’s a findable answer there, and they’re willing to dash themselves against the rocks and risk failure to get to that answer. And in many cases it turns out that there isn’t one, at least not by the paths of the maze that they’ve been exploring.

4.

I think of this when reading Newton’s notebooks. They’re not structured, they’re more him writing down questions about literally everything, and sketching out little proofs / thoughts.

They’re breathtaking – the sheer diversity of questions he asks, and the audacious scope of his conjectures. And what’s striking is he wonders – profusely – about very small physical things that a child might wonder about (I modernized the spelling):

• “Why is breath or sweat seen in winter more than in summer”

• “Why water is clearer than Vapors”

• “Why flints do break upon a soft thing sooner than a hard one”

• “Why does water freeze first & most next the Air”

-- but in between these very physical questions are questions of much grander scope, like:

• Whether the soul is immortal

• What the first matter would have consisted of

• “Whether Moses his saying in Gen the 1st that the evening & the morning were the first day etc. do prove that God created time.”

It feels weird to read these intertwined, since they seem like very different kinds of question to us. But this is only because we know that the physical questions are answerable in terms of science (because we have pretty good answers to them now -- often thanks to Newton!), and the second type of question isn’t, at least as far as we know.

But it’s possible that from Newton’s perspective, both of these kind of questions were equally mysterious, and for all he knew he could find answers to both kinds by trying really hard! [3] Before Newton, maybe people were more confident about the theological questions – (“yes, obviously the soul is immortal”) – than about the physical kind – (“well, I have no idea what makes the planets move!”). Newton made the mysterious banal. 

5.

Another thing I’m struck by, in puzzles, is the role of really wanting to know the answer. You might say, facetiously, that a scientist is someone who really wants to know the answer, to a degree that is pathological (and maladaptive) to a normal person. (Elsewhere I referred to this trait as the ‘will to think’, after Shockley).

Complacency afflicts us all, to some extent. Solving chess puzzles makes this blindingly clear, because how good you are at them is inversely related to how complacent you are. Our brains want to stop thinking as soon as possible, probably because thinking hard takes up a lot of energy and if you didn’t curb that impulse… well, you’d get a very impractical scientist or a mathematician, who spent all their time lost in thought. For most of us, you have to actively fight complacency, which comes in the form of “yeah this answer seems good enough”. 

There’s a striking passage in Jonathan Rowson’s book “Chess for Zebras” where he’s tutoring a student on a puzzle. The student gets the ideas in the puzzle, but doesn’t quite make the leap to the answer. The tutor gets frustrated, because he knows the student can get to the answer (paraphrased): 

Tutor: No. You were sort of right about shortcomings. You see this position is about problem-solving, as most chess is. You have to be really persistent if you want to get somewhere...

[Student says something about needing to improve on board vision]

Tutor: Well, to be honest, I think you have that. You can see that White has more space, the better bishop, etc. And you can see that you need to get the king in ... thus far you are up there with the best. You also notice that Bxg6 is an issue.

Student: But it’s how to do it!

Me: Well that’s what we are on about here. What stops you, I think, is a combination of not really believing you’ll get it and not really caring. Is that too harsh – or is it somewhere close to the truth?

Complacency hits especially hard in king and pawn endgames, when king moves that seem “good enough” can in fact be losing or drawing, and you really have to calculate each and every move very exactly to figure out a win. You have to be unreasonably thorough and check every single move, to a degree that seems quite pedantic. [4]

I notice this in people who are good at science, too: they are really, really thorough and check every edge case.

This tendency can, again, be maladaptive in some context. I’ve noticed that sometimes good scientists have trouble when cast into more CEO-like / founder-like roles, including things like:

• Going down ‘rabbit holes’, i.e. getting super interested in one particular aspect of the business to the exclusion of all else

• Difficulty in making themselves interested in “less interesting” problems (e.g. sales, operations), resulting in e.g. mismanagement of finances because they weren’t willing to get into the weeds of the ‘boring’ area

• Wanting more information before making a decision than it’s possible to really get and moving too slowly as a result

I think this is a consequence of what I’ll call “scientist-brain” vs. “founder-brain”. (Yes, I’m generalizing, many exceptions exist and it’s not this clean in real life.)

Scientist-brain is unreasonably thorough and wants to check every single case; is very curious about things and willing to follow that curiosity, but doesn’t have a natural check on this process; and prefers working on things that are interesting to them, finding it extremely painful if this is not the case. This is great when you are doing science, because you need the ‘driver’ to be curiosity, and small details matter.  

But it’s often bad in CEO or founder type roles, where you often need to move fast, decide based on imperfect information, maintain context on 10+ workstreams including sometimes intellectually boring ones, and above all not go down rabbit holes unless they’re super critical to the business. The tragedy of CEO roles is that you’re often not spending much time on the most interesting things at all.

(This is also why CEOs/founders want you to be terse in communication and give them the bottom line up front. CEOs are holding so much in their heads already that unnecessary details have a high marginal cost.)

Again, these are very, very broad generalizations! And there are some very successful scientist-founders. But I think these are two very different archetypes, and paired together they often make a stellar team.

6.

Chess players usually end up converging on a consistent thought process for every move. [5] Thought process is useful, and it’s often helpful to have a laundry list of these techniques to apply to the problem at hand. But it’s not as if giving you the thought process is magically going to lead you to the right answer.

One constraint you run up against hard when doing chess puzzles is working memory. The more context you have to hold in your head, the harder it is to make more calculations.

Once, frustrated by my accuracy rate, I decided to allow myself to use a scratch Google Docs page to record all my calculations, as an exercise. My accuracy shot up; I rarely got anything wrong. Being able to write things down freed up ‘mental scratchpad’ space, which made my calculations more accurate overall, and made it easier to check I hadn’t missed anything. [6]

I’m pretty sure people who are great at chess have better working memory than people who aren’t. And it seems like working memory is innate and fixed, though I’d welcome any counter-evidence. [7] Indeed, Adams and Hurtado find that grandmasters have better intuition and identify the right answer very quickly; they calculate far quicker and more accurately than amateurs; in general, it seems that visualizing and thinking far ahead is much easier for them than the amateurs, the result of thousands of hours of practice and probably innate talent. I think this is clearly true of math too: more gifted people traverse the maze faster, and seem to have a preternatural instinct for what’s going to turn out to be a blind alley versus what’s not.

I’ve always been struck by Jonathan Rowson’s account of playing Grischuk, a super-grandmaster, and analyzing the game together with him afterwards:

During the post-mortem I was deeply impressed by the fluency and depth of his analysis, and the sense of balance in his assessment of positions. To be honest, it left me feeling a little deflated. It was abundantly clear to me that he was a different class of player. I could perhaps compete with him in a single game, but not in the long haul. There aren’t many players I feel that way about, but Morozevich made the same impression, and to a lesser extent, Kasimdzhanov, Aronian and Sutovsky. There are some players who just seem to function a few orders of magnitude higher – as if they are qualitatively stronger, speaking a different language. (Jonathan Rowson, Chess for Zebras)

Some people are just different. [8] Still: lean into what you’re good at, practice, and you can always get better!

7. 

Anyway, there are other lessons that I didn’t go into here, because I don’t want to spoil the entire book(s). Needless to say, they contain many excellent hacks for problem-solving. So if all of this piqued your interest, I’d highly recommend:

Think Like A Super-GM, Adams and Hurtado (Amazon)
Problem Solving Through Problems, Larson (Amazon)
Think Like A Grandmaster, Kotov (Amazon)
How To Solve It, Polya (Wikipedia)
Chess for Zebras, Jonathan Rowson (Amazon)

Thanks to Tyler Cowen for feedback on a draft. 

Footnotes

[1] In online chess games, a nice hack to make this easier for yourself is to flip the board to the opponent’s perspective when you’re trying to decide if a move works or not. Say you play the move - what would you play if you were the opponent now?

[2] This is one thing that makes founding hard: what drives you to search is blind faith that a solution is possible, but you also have to know when to call off the search, and it’s hard to calibrate when to do this because the feedback signal is sparse. Maybe you gave up two weeks too early, and if you’d simply persisted you’d have struck gold. Didn’t the Airbnb guys persist for a whole year with no traction or signals of success before taking off? Wouldn’t they have been rational to give up after the fifth time it didn’t work? Well, it would have been, but they didn’t.

[3] I originally read this point in a PG essay, “The Risk of Discovery”.

[4] Two examples of complacency, one from chess and one from math, which I omitted from the main text because they’re wonkish. (Lichess link):

You notice a few things immediately – the open b-file with the white rook pointing at the king, the white knight constraining the king’s escape on d7 – and you start looking for mating moves. The obvious move that comes to mind is Qb4, lining up more pressure along the b-file and threatening Nxa6 followed by Qb7, which “looks winning” -- note the bishop pointing at c7 too.

In an online blitz game, with no time to think, I’d play Qb4 100 times over without thinking twice. It’s the sort of move chess players would call “natural”.

But this is complacency – note that:

(1) I haven’t examined other moves, including more forcing ones
(2) I don’t actually have a concrete winning line. I haven’t bothered to think hard enough to get to the level of working out my opponent’s replies. I’ve hand-waved “this looks winning”.

It turns out Qb4 is a fine move for White, but Black can defend. The best move for White is actually Rxb7!, and after Bxb7 Qxa7 Black can only stop checkmate on b7 by giving up their queen.

Another example of complanency from the math context. Here’s another Larson problem:

Since you want to maximize the product, you can guess intuitively that smaller numbers (2s and 3s) are going to get you to a larger final product, since that gives you more multiplications. A 4 is just 2*2 (and 2+2) so the solution won’t involve any 4s, and anything larger than 4 is going to be too big.

It’s easy – but wrong – to guess from this that the maximizing answer is probably either 3^333 + 1, or 2^500. You can then go down a rabbit hole to figure out which one is bigger, or use a calculator and find that it’s 3^333+1. Complacency would stop here and conclude that that’s probably the answer.

But it’s not! It’s helpful to work out some smaller examples first, such as 10:

• 5+4+1 = 10, and 5*4*1 = 20

• 4+3+2+1 = 10, and the product is 24

• 3+3+2+2 = 10, and the product is 36

Clearly, the right answer never contains a 1. So revisiting the 3^333 + 1 answer, the ‘next’ solution would be 3^332 + 2^2, which is indeed the correct answer.

[5] When doing a puzzle, you’re typically told to examine checks, captures and threats first -- in that order.

The Russian chess player Alexander Kotov wrote a famous book, Think Like a Grandmaster, which he outlines a more general thought process for every move and introduces the concept of the candidate move -- essentially, a plausible move in the position. His point is that you should list out all candidate moves first, before going into analysis of each one. Amateurs tend to find an interesting-looking move, and then dive in and analyze it, but it’s best to list all the plausible moves first so that you have an organized list of all the possibilities.

Larson has a similar set of techniques for math problems too. Unlike chess, it’s hard to have a consistent thought process for all math problems because they’re so different, so these lists tend to look more like a bunch of useful hacks: search for a pattern, draw a figure, modify the problem, choose good notation, consider extreme cases, and so on.

[6] Hence the GTD recommendation to write everything down: “your mind is for having ideas, not holding them”.

[7] Gwern’s review of dual n-back is interesting here, but as far as I can tell the conclusion is that it’s unclear whether this has any real benefit to working memory. 

[8] When you watch interviews with grandmasters about Magnus Carlsen, they usually identify a few traits of his that make him exceptional.

One of these is his insane chess memory (see this YouTube video for a vivid demonstration of this). Clearly, it’s the result of many hours and days of hard work. But there’s also some innate thing going on here.

Another is his chess intuition, which is chess player speak for “a disturbing knack for getting the right answer without any apparent effort, at sub-second speed”. There’s a great video showing this at work here -- most players end up memorizing these endgames, but Magnus doesn’t, he works them out on the spot. At one point, he makes a very weird-looking move, the interviewer asks him why he made it, and Magnus says “That wins because, well honestly I don't know.” Similarly, at around 15:00, the interviewer asks if Magnus found the solution using the ‘key squares’ concept -- which most GMs would study explicitly -- and Magnus says in effect: no, it’s obvious to him on sight.

Another is that he’s insanely competitive, wants to win, and doesn’t get tired or let up even in extremely boring positions.

I’m sure he cultivates all of these things. But I’m also pretty sure they’re innate.

Apparently yes. Later, the Yudkowsky “we’re all doomed” TIME magazine piece came out, and talk of AI regulation was everywhere, and the doomers were dominating the AI discourse.

Merits of the arguments aside, they were winning for two main reasons: first, everyone has been primed by years of Hollywood pessimism to think of killer robots and AI going rogue anyway, making AI doom a compelling viral meme. And second, they were appealing to governments to do something, which is music to the ears of many.

I’m glad to see more debate on these arguments, but it’s mostly on Twitter – e.g. here is Adam D’Angelo arguing that teaching AIs to respect human life will not be challenging. And I’ve never found the arguments for certain doom persuasive. So I wanted to write, publicly, about some ways in which AI might go well. When Angela at WIRED magazine asked if I wanted to write for them, this is what I pitched her. We ended up with my recent essay – The Case for Moral AI. 

Writing this piece was challenging. I think it went through at least 10 rewrites. The piece had to work for laypeople; but there’s a lot of context you have to convey to a layperson on this topics. But I also wanted to avoid oversimplification or sensationalism. This balance turned out to be hard.

I decided the Waluigi Effect was an interesting hook into the broader issues – the “you can’t create an angel without also creating a demon, because a demon is just an angel with a ‘not’ before every operation” problem. This phenomenon had only been discussed on Twitter and LessWrong, and I thought a mainstream explanation of it would be fun.

I’d also noticed some funny links between it and psychoanalytic theory (Freud’s theory of taboo relies on a similar mechanism – the brain’s subconscious censor needs to know about what its censoring, which implies many horrors lurking in our subconscious…). So that connection was the beginning of it.

Still, I wasn’t able to go into the weeds on the various technical solutions to alignment as I wanted. Constitutional AI, is a clever mechanism by which we can train AIs, using feedback from AIs, to be more helpful and honest. Paul Christiano’s research agenda includes several approaches to alignment – scalable oversight, checks and balances, and others – all mentioned in the space of a paragraph, but all plausible and interesting paths. And I expect AI simulations to be a big part of testing AI agents.

Given all the constraints, I’m pleased with how it turned out. But I confess I’m excited to go back to plain old blogging, too; writing for a broader audience has trade-offs, and it’s fun to be obscure and let your people find you. Still, you can read it here:

WIRED essay

Other miscellaneous notes:

• I’m interested in analyses of how AI will affect Kaldor’s Facts. I found this old article (2014!) from Paul Christiano which does a first-pass analysis of this, but I’m curious if economists have better ideas. The facts here affect how we should feel about many of the normative questions involved (e.g. how desirable is widespread open-source AI vs. a future where 3-5 megacorps control AI? Will we really need UBI in the far future?)

• After two months in Asia, I’m heading to London on Thursday and will be in the UK for the month of June. Let me know if you’d like to hang out – I’m considering organizing a meetup with Rohit, Matt Clifford, and some other friends.

• I’m mostly trying to avoid writing code or doing technical side projects, since I plan to start doing serious idea exploration in Q4 anyway, so these few months are a nice opportunity to catch up on reading. Main readings this month have been Naipaul’s “Enigma of Arrival”, Shakespeare’s “Troilus and Cressida”, Coetzee’s “Waiting for the Barbarians”, Thurber’s “The Years With Ross” (on the early days of the New Yorker, a nice case study in how founder effects persist decades later), and “For Blood and Money”, a fun biopharma case study on the eccentric scientologist billionaire Bob Duggan and his company, Pharmacyclics. Reviews will come in the quarterly post as usual.

• Taipei is a vastly underrated city: top-tier food, friendly warm people, and it’s on a tropical island so it’s beautiful. I recommend everyone visit. We went to Taroko Gorge, too, which was extraordinary. By contrast, Japan was choked with tourists this time around; Kyoto reminded me of Venice or Rome in how there were ten tourists for every native. (It’s still possible to have a great trip in Japan if you avoid the beaten path, and Tokyo remains one of the great world cities. But I’d urge everyone to visit Taiwan.)

Films

Paris, Texas (5) - Wim Wenders, 1984. An instant favorite. So many unforgettable shots, you could frame any random screen-grab from this movie. A nice example of how some of the best, and most loving, work about America can come from foreigners (Wenders is German; de Tocqueville is another example).

L'Argent (4.5) - Robert Bresson, 1983. Visually and aurally arresting, especially the final part in the rural house. I feel like he is not exceeding the Russian novelists in depth, but a pleasure to watch for his technique and brilliance.

The Green Ray (4.5) - Eric Rohmer, 1986. Really good. Mike Leigh-esque character study of a woman who pushes people away, a depressive introvert.

Tropical Malady (4.4) - Apichatpong Weerasethakul, 2004. Benedict Anderson makes the nice point that this movie made perfect sense to rural Thai but less so to urbanites: the second section is a very literal account of being in a jungle at night. Dreamy Apichatpong, very similar to Boonmee, love as a tiger and as a dreamlike state of abandon, his most romantic movie.

Breaking the Waves (4.4) - Lars von Trier, 1996. The first two hours I hated it; it made me tense and uncomfortable, the way von Trier films tend to do. Then the ending changed my mind. Belongs with 'Ordet', 'The Sacrifice', 'Stalker'. But maybe just short of them, too. Art reflects the maker, and there’s some shortcoming in this I can’t articulate, some perversity or sadism in the director himself.

Rashomon (4.2) - Akira Kurosawa, 1950. Respected this more than I loved it, but it's very accomplished; it messes with the viewer in all kinds of ways and is a beautiful education in the craft of film. More than a little Shakespearean, too, with the framing narrative. There’s a nice essay about this on A Sharper Focus.

Wings of Desire (4.2) - Wim Wenders, 1987. Mid-80s Berlin, the last 30 mins or so are delightful and will give you great joy in the everyday. The "Homer" parts were clunky for me, although I understood what he was trying to do. Very intellectual, Walter Benjamin among others hangs over it. Worth watching if only to see Berlin in its pre-1989 state.

Return to Seoul (4) - Davy Chou, 2023. The main actress in this one is very charismatic and conveys volumes of pain with her face. Realist human drama, reminded me of Farhadi in style. One or two false notes in the writing (why does she become an arms dealer?). Excellent, likely this year’s best?

Gates of Heaven (4) - Errol Morris, 1978. Little gem documentary about a pet cemetery. Between this and Paris, Texas, you could do worse in movies about America.

Pickpocket (4) - Robert Bresson, 1959. Bresson's technique is impeccable. His movies contain a surprising amount of concrete, specific detail, in this case about the process of picking pockets. His later movies become more explicitly religious, but this one is French existentialism and nihilism without even a hint of religion, and therefore arid. (What does this add that you can’t find in “Crime and Punishment”?) I preferred A Man Escaped.

Claire's Knee (3.8) - Eric Rohmer, 1970. Light, fun human drama; the directness of youth vs. the “meta”, layered way we experience everything in age. Set in Annecy. French and flighty, I prefer Bergman's heaviness. Rather creepy, too.

Tár (3.8) - Todd Field, 2022. A pretty good movie about a conductor's gradual descent into disgrace. What a joy to watch an actual high-brow movie in an American cinema and have the audience laughing and gasping. I enjoyed the first half more; the second descended into classic psychological melodrama.

Beau Travail (3.5) - Claire Denis, 1999. I didn't enjoy this at the time, felt it unnecessarily artificial in how it was filmed and a little choreographed, but images from it have stuck with me, as has its overall rhythm and dreamlike vibe.

The Banshees of Inisherin (3.5) - Martin McDonagh, 2022. Blackly comic little tale. Opposes the christlike simplicity of the idiot with the Nietzschean dark/despair of art; the world forces us all into 'eye for an eye'; veiled commentary on Ireland itself. Good acting all round, both leads and also the sister. Maybe too simple a movie to be really great. Still - movies are not dead!

Lost in Translation (3) - Sofia Coppola, 2003. Iconic but also aged poorly. Should be seen at age 18. I found the characters annoying and self-absorbed this time round.

Turn Every Page: The Adventures of Robert Caro and Robert Gottlieb (2.5) - Lizzie Gottlieb, 2022. Documentary on Robert Caro and his relationship to his editor, Robert Gottlieb. Tight fusion of writer/editor. Nicely shows Caro's industriousness. Too long for the slim amount of material she actually had, though.

Fallen Angels (2) - Wong Kar-Wai, 1995. I’m a huge fan of the director but I hated this one, which showcased his worst tendencies. All aesthetics, no real emotion!

Books

A Mathematician's Apology (5) - GH Hardy, 1940. Celebrates ambition and the value of creating permanent, enduring, lasting work! Written in aphorisms, very engaging. I wish more books in this genre existed: an accomplished scientist reviewing his career and drawing principles from it.

The Iliad (5) - Homer, Lattimore translation. What to say about this? Maybe my favorite book? I don't know why it "hit" me so electrically this time round, but it did. I came upon the Lattimore translation by chance, but after looking at every other translation I concluded it was the best one.

Autobiography of Charles Darwin (4) - Charles Darwin, 1887. Darwin is a lucid writer. Most fascinating passages are where he analyzes himself and his strengths and weaknesses. In his early life, he goes to extreme lengths to collect and classify beetles, and isn’t really interested in anything else; this extreme interest in, and patience for, very specific facts about the world seems characteristic of the best scientists I’ve met.

Inadvertent (4) - Karl Ove Knausgaard, 2018. A nice lecture about writing and what makes him write, with some notes about his influences (Ponge, Proust, Tolstoy of War and Peace, Turgenev). Talks about the value of art and the contrast between what is infinite in us vs the materiality of our existence. Many false starts as a writer before he did 'My Struggle'.

The Greeks (4) - RDF Kitto, 1951. Accessible introduction to the Greeks, especially the chapter on Homer, which is a nice chapter-level summation of what makes him so good. Greece in 5 B.C. is one of those baffling wonders human history occasionally produces: Socrates, Plato, Sophocles, Aeschylus, Euripides, Herodotus, Hippocrates, Pericles, Thucydides, Xenophon, and more, all wandering around Athens at the same time!

Babel (3.5) - Rebecca Kuang, 2022. Fun read, best were the passages on language and translation, preachy and didactic and heavy-handed in parts, some of the characterization felt weak (Letty). Clearly a huge talent, excited to see what she does next.

An Angel Walks Through the Stage and Other Essays (3.5) - Jon Fosse, 2018. Some stimulating reflections on the most inward, intuitive parts of being a writer. I didn’t know he was so into Derrida and critical theory, I could not have told you that from reading his masterpiece “Septology”.

Economy of the Unlost (3.5) - Anne Carson, 1999. Study comparing Simonedes with Celan. Oblique and hard to summarize, but stimulating and beautiful as is all her prose.

Tomorrow, and Tomorrow, and Tomorrow (3) - Gabriella Zevin, 2022. A fun but light read, ending felt forced. I didn't resonate with her descriptions of games, it's not how I experience them. Very ‘young adult’, will appeal to people who like that kind of thing.

I’m a big fan of “advice posts” and productivity guides. A small, but meaningful, upgrade to your daily routine is worth a lot in the long run. So here’s my contribution to the genre.

1. Maximize your baseline energy levels. There is the obvious stuff: figure out a personal exercise practice and do it at least five days a week (I like running). If you don’t, you’re just leaving a bunch of power on the table.

But there are also less intuitive truths here.

Energy compounds on itself. If you start the morning by getting something done (a workout, an important task, writing) then you’re going to have a higher baseline energy day overall. It’s as though the initial thing gives you a persistent ‘boost’ throughout the day. Doing additional things becomes easier. Without this boost, there’s a good chance I get nothing important done that day.

Most people’s mental models of energy are flawed: they think there’s a ‘tank’ of energy that gets depleted as you spend it. This may be roughly true for physical energy, but mental energy is different: spending mental energy on things that you consider productive or important gives you more mental energy for other things: a positive feedback loop. On the other hand, procrastinating, spending all day scrolling Twitter, or staying in bed all day reduces the amount of energy you have to spend; this means you are less likely to get anything done.

It’s common to get trapped in this negative energy feedback loop: you don’t feel like doing something, so you check Twitter for awhile, which reduces your energy levels, which makes you feel worse, but you try and do something anyway, but you’re even less energized now, so you decide to go to bed for a bit to rest, but the rest isn’t restful… etc.

The way to get out of these energy ruts is to just do something really small (empty the dishwasher! Write one sentence!) and get that tiny reward of accomplishment. This generates a little bit more energy. Use that spark to get something slightly bigger done, and so on.

2. Do the most important thing first thing in the morning, and don’t check social media until you’ve done it. Because energy compounds, the first actions in the day matter a lot: the right actions get you into a positive spiral, the wrong actions get you into a negative spiral. The further into a negative spiral you get, the harder it is to get out. So if you start the morning by doing something you care about (e.g. writing a page of an essay), you are way more likely to have a good, productive and happy day overall, because you’ve gotten yourself into a positive spiral.

Moreover, even if you don’t subsequently do anything, you have at least done that one thing in the day; most people fail more from many days of zero output than they do from not maximizing output on any given day, so the key is to stay consistent. [1]

I’m pro social media and think it makes us smarter, but I think it’s a bad idea to check it first thing in the morning. A simple rule is just to check it after you’ve done your ‘first thing in the morning’. I don’t know why this works, but my folk-theoretic model is something like: you want to get rewards from doing a productive thing, not from doing an unproductive thing, and social media gives you rewards in a way that perniciously substitutes for the kinds of rewards you actually want. Once you have the ‘reward’, your drive to do things lessens, so by checking Twitter you kill your drive to do the more productive thing. (The analogy to sugar is apt: you want calories from the good stuff, not the sugary stuff.)

3. Tell the right stories about yourself. One of the more underrated insights you can get from reading self-help books is that the story you tell yourself about yourself matters. If you tell yourself you’re very energetic person, that feeling tired is temporary, and you keep this belief going, you actually become more energetic. If you tell yourself you don’t get jetlagged, you might get less jetlagged on average. This goes for many other traits, too -- you can psyop yourself into believing lots of things about yourself which then come true, and make you more like the person you want to be. 

(Obviously, tons of disclaimers here. But this sort of thing has worked for me, so I’m writing it down.)

4. Don’t sleep too much. Oversleeping also comes at the cost of time you’ll never get back. Moreover, there’s a good amount of evidence that sleeping more than you need is bad for you. Usually people will tell you to get more sleep, but I think this is the wrong direction for many people. The sweet spot seems to be somewhere around 7-7.30 hours, although again this is a genetic thing and can vary!

Some people claim that this is purely psychological and that you can train yourself to get by on four hours a night, but this has never worked for me. I’m making the weaker claim: there is a big difference between 7 hours and 9 hours, you can do a lot with those 2 hours, and often if you’re a 9 hour sleeper you can become a 7 hour sleeper, which will end up compounding if you use those extra 2 hours well. [2] So it’s worth adopting the resolution to sleep less than you want to.

5. Get in the habit of Fermi estimation, looking up key quantities, and using upper and lower bounds. I’ve noticed a lot of the smartest people I know do this: they don’t take any claims at face value, and check for themselves whether they’re plausible. This means, e.g. when they hear a fact, they’ll look it up to assure themselves that it’s true, because often people cite things that are false or partial.

Engineers and physicists are trained to do this, everyone else has to learn. The usual name for this is Fermi estimation: estimating the rough order of magnitude of an unknown quantity using information that you already know.

Most people don’t do this at all — they don’t “think in numbers”, and the result is that they’re easily fooled. Every now and then there’s a Twitter meme that goes like “If Elon sent money to every American instead of spending it all on the rockets, [etc.]” and this will go viral because people do not do the math. I noticed this during COVID, too, during those periods in between COVID waves: the risk of getting COVID would drop a hundredfold or more, but people would ‘under-correct’ their behavior, remaining too risk-averse during relatively safe times, and too risk-loving during times of high infection risk, simply because they didn’t get a sense of local case numbers and convert those into probabilities.

Some people do this all the time. For example, SBF models this in his Conversation with Tyler to answer the question of whether blockchain could replace the world’s infrastructure (emphasis mine):

BANKMAN-FRIED: Basically, if you look at what would it take to have a global-scale blockchain that was replacing substantial fractions of the world’s infrastructure. If that’s where you start, you can just look at, “Well, okay, what scale would that have to be at?” (...)

Answering it varies some, but you’re generally going to hear answers that sound like a million, somewhere like 100,000 and 10 million, depending on which thing we’re thinking about, and it makes sense. If you have a billion users, for instance, which is the kind of thing you might expect of a gigantically successful internet thing —

COWEN: Sure.

BANKMAN-FRIED: — then a billion users. If there’s, what, 100,000 seconds in a day, if each user does 10 clicks per day, on average, then I think that gets you a million transactions per second, ballpark. (...)

Then you say, “Well, okay, what’s that mean for a blockchain?” Well, it means that if you want a blockchain to really get to a truly global scale, you want that blockchain to be processing millions of transactions a second. (...)

If you look at most blockchains today, they cannot support millions of transactions per second. Ethereum cannot support millions of transactions per second. Today, it can support ten. Ten is the number of transactions per second it can support. Ten is off by six orders of magnitude from where it needs to get. It’s not close.

As you can see, the above exercise helps you understand, very quickly, just how much scaling is required for blockchains to handle critical infrastructure. If you ask that question and don’t do the estimation, you’re just making up fluff: “well, if we work on a scaling for a year, maybe then?” or whatever. Whereas if you realize there’s a six order of magnitude improvement required, then you can start figuring out if Ethereum’s current plans are going to get them that amount, and you’ll understand something more deeply about the world as a result. [3]

A related helpful thinking technique is putting bounds on things: “well, it’s not as high as X, and it’s not as low as Y, so given that it’s somewhere in between those numbers we can assume it’s roughly Z”. 

For example, if I ask you when strong human level AGI will come about, you might experience some mental paralysis, or just answer honestly that you don’t know. But you do have some intuitions here, which this exercise draws out: you’d be very surprised if it came about next month, and you’d probably also be surprised if humanity hadn’t invented it by 2150, say, so you can at least conclude that your model of the world predicts that it’s likely to fall somewhere in between next month and 2150. Well, that’s already a surprising conclusion! It implies that your grandkids will probably see AGI. Not so far away, is it?

6. Whenever you make a claim or an argument, try and imagine the strongest possible argument against it. In chess, a thing that distinguishes a good player from a worse one is that the good player tests their potential move against the opponent’s strongest possible replies. Weak players often play ‘hope chess’: they launch an attack, and assume their opponents will go along with their plans by playing weak responses. As a result they don’t see how their attack might fail, and they lose.

Stronger players make a mental habit of checking their opponent’s strongest responses and this allows them to pick the move that is in fact the best one.

This sounds obvious, but it’s actually very hard to do and requires practice and training: it’s as though your brain resists the mental motion of modeling your opponent as strong too, as though it ‘wants’ the idea to work and blocks you from thinking about the resources your opponent actually has.

A similar thing goes in life: people very rarely model out the very best arguments against their positions and then adjust accordingly. It’s hard to do, it’s painful, and the rewards are not obvious unless you strongly care about truth. But if you do want true beliefs, this mental habit is essential.

7. Write regularly, and learn to ‘think in writing’. This is true for literally everyone, regardless of whether you want to be a writer or not, whether you want to publish or not. Just have a Google Doc in which you add a page a day of whatever’s on your mind. This has a million benefits, but a simple one is just clearing your cache: if you don’t do this, your brain sort of gets clogged by all the things you have on your mind, whereas if you ‘empty’ your brain onto a page that creates room for new thoughts.

If you really want to be a clear thinker, you need to learn to ‘think in writing’. I like Holden Karnofsky’s guide to this.

8. Do a weekly review. Every Sunday, sit down for an hour with a text editor and review your week. What went well, what went poorly, and what you’re aiming for in the next coming week. I find this is a useful way to force myself to get out of ‘doing mode’ and into ‘reflection mode’, and often surfaces useful insights / things that I could be improving about my life. This goes into my plan for the next week, which sets off a set of slowly compounding improvements. 

9. Synthesize things as you read. Just because you’ve read something, doesn’t mean you’ve understood it; your brain has to come up with its own encoding. Whatever understanding things is, it’s related to compression. Which implies that you want to read and then restate in your own words, so that your mind is forced to compress the thing. Ideally several times, in varying ways.

Once you’ve done this, you are much more likely to retain the thing, and to actually grasp it; and if you’re struggling with this exercise, then you don’t understand the thing and should go back and look at it again. (This is also a useful bullshit filter -- try and restate someone’s claim in a different way, and see if it still holds up).

When I say ‘restate it in several different ways’, one useful way would be drawing it. Just draw a schematic representation of what you think is being said. Another would be to state it as though you’re writing an article for simple words Wikipedia.  

10. Map out problems using logic trees. This is a classic problem-solving and brainstorming technique, also known as morphological analysis. It’ll be familiar to any consultant, as it’s 80% of their secret sauce.  

Take a problem, say analyzing a business’s profits (as in consulting). Break it down into logically exhaustive possibilities, e.g. “revenue” and “costs”. Break down each branch further into its component parts, e.g. revenue becomes price * quantity. Follow this process recursively, each time breaking the tree down into components.

Now you have a full map of the possibilities and can start to answer questions like “how do we increase profits?” by listing out all available options. This often helps you spot options that other people will overlook.

You might consider this example simplistic and MBA-ish, but Ed Boyden uses this in a scientific/invention context, and demonstrates an example applied to climate/energy around minute 14 of this video.

I found this technique especially useful when tackling ambiguous problems in a startup. Questions that seem like “how do we grow faster?”, can be reduced to lower-level components that are easier to reason and brainstorm about, and because you’re making sure each ‘layer’ of the tree is mutually exhaustive, you’re not missing anything. 

Footnotes

[1] I owe this insight to Tyler Cowen, whose own advice posts on this topic are very good and influenced this one. 

[2] As Alexey Guzey has pointed out, sleep can be thought of as similar to hunger in this way: everyone accepts that you shouldn’t just eat whenever you feel like it or you’re likely to overeat, but for some reason people don’t apply the same reasoning to sleep.

[3] Tyler points out that trial lawyers also do this very well.

I.

The smartest person I’ve ever known had a habit that, as a teenager, I found striking. After he’d prove a theorem, or solve a problem, he’d go back and continue thinking about the problem and try to figure out different proofs of the same thing. Sometimes he’d spend hours on a problem he’d already solved. 

I had the opposite tendency: as soon as I’d reached the end of the proof, I’d stop since I’d “gotten the answer”.

Afterwards, he’d come out with three or four proofs of the same thing, plus some explanation of why each proof is connected somehow. In this way, he got a much deeper understanding of things than I did.

I concluded that what we call 'intelligence' is as much about virtues such as honesty, integrity, and bravery, as it is about 'raw intellect’.

Intelligent people simply aren’t willing to accept answers that they don’t understand — no matter how many other people try to convince them of it, or how many other people believe it, if they aren’t able to convince them selves of it, they won’t accept it.

Importantly, this is a ‘software’ trait & is independent of more ‘hardware’ traits such as processing speed, working memory, and other such things.

Moreover, I have noticed that these ‘hardware’ traits vary greatly in the smartest people I know -- some are remarkably quick thinkers, calculators, readers, whereas others are ‘slow’. The software traits, though, they all have in common -- and can, with effort, be learned. 

What this means is that you can internalize good intellectual habits that, in effect, “increase your intelligence”. ‘Intelligence’ is not fixed.

II.

This quality of “not stopping at an unsatisfactory answer” deserves some examination.

One component of it is energy: thinking hard takes effort, and it’s much easier to just stop at an answer that seems to make sense, than to pursue everything that you don’t quite get down an endless, and rapidly proliferating, series of rabbit holes.

It’s also so easy to think that you understand something, when you actually don’t. So even figuring out whether you understand something or not requires you to attack the thing from multiple angles and test your own understanding.

This requires a lot of intrinsic motivation, because it’s so hard; so most people simply don’t do it.  

The Nobel Prize winner William Shockley was fond of talking about “the will to think”:

Motivation is at least as important as method for the serious thinker, Shockley believed...the essential element for successful work in any field was “the will to think”. This was a phrase he learned from the nuclear physicist Enrico Fermi and never forgot. “In these four words,” Shockley wrote later, “[Fermi] distilled the essence of a very significant insight: A competent thinker will be reluctant to commit himself to the effort that tedious and precise thinking demands -- he will lack ‘the will to think’ -- unless he has the conviction that something worthwhile will be done with the results of his efforts.” The discipline of competent thinking is important throughout life... (source)

But it’s not just energy. You have to be able to motivate yourself to spend large quantities of energy on a problem, which means on some level that not understanding something — or having a bug in your thinking — bothers you a lot. You have the drive, the will to know.

Related to this is honesty, or integrity: a sort of compulsive unwillingness, or inability, to lie to yourself. Feynman said that the first rule of science is that you do not fool yourself, and you are the easiest person to fool. It is uniquely easy to lie to yourself because there is no external force keeping you honest; only you can run the constant loop of asking “do I really understand this?”.  

(This is why writing is important. It’s harder to fool yourself that you understand something when you sit down to write about it and it comes out all disjointed and confused. Writing forces clarity.)

III.

The physicist Michael Faraday believed nothing without being able to experimentally demonstrate it himself, no matter how tedious the demonstation.

Simply hearing or reading of such things was never enough for Faraday. When assessing the work of others, he always had to repeat, and perhaps extend, their experiments. It became a lifelong habit—his way of establishing ownership over an idea. Just as he did countless times later in other settings, he set out to demonstrate this new phenomenon to his own satisfaction. When he had saved enough money to buy the materials, he made a battery from seven copper halfpennies and seven discs cut from a sheet of zinc, interleaved with pieces of paper soaked in salt water. He fixed a copper wire to each end plate, dipped the other ends of the wires in a solution of Epsom salts (magnesium sulfate), and watched. (source)

Understanding something really deeply is connected to our physical intuition. A simple “words based” understanding can only go so far. Visualizing something, in three dimensions, can help you with a concrete “hook” that your brain can grasp onto and use as a model; understanding then has a physical context that it can “take place in”.

This is why Jesus speaks in parables throughout the New Testament — in ways that stick with you long after you’ve read them — rather than just stating the abstract principle. “Are not two sparrows sold for a cent? And yet not one of them will fall to the ground apart from your Father.” can stick with you forever in a way that “God watches over all living beings” will not.

Faraday, again, had this quality in spades -- the book makes clear that this is partly because he was bad at mathematics and thus understood everything through the medium of experiments, and contrasts this with the French scientists (such as Ampere) who understood everything in a highly abstract way.

But Faraday’s physical intuition led him to some of the most crucial discoveries in all of science:

Much as he admired Ampère's work, Faraday began to develop his own views on the nature of the force between a current-carrying wire and the magnetic needle it deflected. Ampère's mathematics (which he had no reason to doubt) showed that the motion of the magnetic needle was the result of repulsions and attractions between it and the wire. But, to Faraday, this seemed wrong, or, at least, the wrong way around. What happened, he felt, was that the wire induced a circular force in the space around itself, and that everything else followed from this. The next step beautifully illustrates Faraday's genius. Taking Sarah's fourteen-year-old brother George with him down to the laboratory, he stuck an iron bar magnet into hot wax in the bottom of a basin and, when the wax had hardened, filled the basin with mercury until only the top of the magnet was exposed. He dangled a short length of wire from an insulated stand so that its bottom end dipped in the mercury, and then he connected one terminal of a battery to the top end of the wire and the other to the mercury. The wire and the mercury now formed part of a circuit that would remain unbroken even if the bottom end of the wire moved. And move it did—in rapid circles around the magnet! (source)

Being able to generate these concrete examples, even when you’re not physically doing experiments, is important.

I recently saw this striking representation of the “bag of words” model in NLP. If you were reading this in the usual dry mathematical way these things are represented, and then forced yourself to come up with a visualization like this, then you’d be much further on your way to really grasping the thing.

Conversely, if you’re not coming up with visuals like this, and your understanding of the thing remains on the level of equations or abstract concepts, you probably do not understand the concept deeply and should dig further.

Another quality I have noticed in very intelligent people is being unafraid to look stupid.

Malcolm Gladwell on his father:

My father has zero intellectual insecurities... It has never crossed his mind to be concerned that the world thinks he’s an idiot. He’s not in that game. So if he doesn’t understand something, he just asks you. He doesn’t care if he sounds foolish. He will ask the most obvious question without any sort of concern about it... So he asks lots and lots of dumb, in the best sense of that word, questions. He’ll say to someone, ‘I don’t understand. Explain that to me.’ He’ll just keep asking questions until he gets it right, and I grew up listening to him do this in every conceivable setting. If my father had met Bernie Madoff, he would never have invested money with him because he would have said, ‘I don’t understand’ a hundred times. ‘I don’t understand how that works’, in this kind of dumb, slow voice. ‘I don’t understand, sir. What is going on?’

Most people are not willing to do this -- looking stupid takes courage, and sometimes it’s easier to just let things slide. It is striking how many situations I am in where I start asking basic questions, feel guilty for slowing the group down, and it turns out that nobody understood what was going on to begin with (often people message me privately saying that they’re relieved I asked), but I was the only one who actually spoke up and asked about it. 

This is a habit. It’s easy to pick up. And it makes you smarter.

IV.

I remember being taught calculus at school and getting stuck on the “dy/dx” notation (aka Leibniz notation) for calculus.

The “dy/dx” just looked like a fraction, it looked like we were doing division, but we weren’t actually doing division. “dy/dx” doesn’t mean “dy” divided by “dx”, it means “the value of an infinitesimal change in y with respect to an infinitesimal change in x”, and I didn’t see how you could break this thing apart as though it was simple division.

At one point the proof of the fundamental theorem of calculus involved multiplying out a polynomial, and along the way you could cancel out “dy*dx” because “both of these quantities are infinitesimal, so in effect this can be cancelled out”. This reasoning did not make sense.

The “proof” of the chain rule we were given looked like this. 

(Amusingly, you can even get correct results using invalid mathematics, like this. Even though this is clearly invalid, it doesn’t feel far off the “valid” proof of the chain rule I was taught.)

It turns out that my misgivings were right, and that the Leibniz notation is basically just a convenient shorthand and that you more or less can treat those things “as if” they are fractions, but the proof is super complicated etc. Moreover, the Leibniz shorthand is actually far more powerful and easier to work with than Newton’s functions-based shorthand, which is why mainland Europe got way ahead of England (which stuck with Newton’s notation) in calculus. And then all of the logical problems didn’t really get sorted out until Riemann came along 200 years later and formulated calculus in terms of limits. But all of that went over my head in high school.

At the time, I was infuriated by these inadequate proofs, but I was under time pressure to just learn the operations so that I could answer exam questions because the class needed to move onto the next thing.

And since you actually can answer the exam questions and mechanically perform calculus operations without ever deeply understanding calculus, it’s much easier to just get by and do the exam without really questioning the concepts deeply -- which is in fact what happens for most people. (See my essay on education.)

How many people actually go back and try and understand this, or other such topics, in a deeper way? Very few. Moreover, the ‘meta’ lesson is: don’t question it too deeply, you’ll fall behind. Just learn the algorithm, plug in the numbers, and pass your exams. Speed is of the essence. In this way, school kills the “will to understanding” in people.

My countervailing advice to people trying to understand something is: go slow. Read slowly, think slowly, really spend time pondering the thing. Start by thinking about the question yourself before reading a bunch of stuff about it. A week or a month of continuous pondering about a question will get you surprisingly far.

And you’ll have a semantic mental ‘framework’ in your brain on which to then hang all the great things you learn from your reading, which makes it more likely that you’ll retain that stuff as well. I read somewhere that Bill Gates structures his famous “reading weeks” around an outline of important questions he’s thought about and broken down into pieces. e.g. he’ll think about “water scarcity” and then break it down into questions like “how much water is there in the world?”, “where does existing drinking water come from?”, “how do you turn ocean water into drinking water”, etc., and only then will he pick reading to address those questions.

This method is far more effective than just reading random things and letting them pass through you.

V.

The best thing I have read on really understanding things is the Sequences, especially the section on Noticing Confusion.

There are some mantra-like questions it can be helpful to ask as you’re thinking through things. Some examples:

• But what exactly is X? What is it? (h/t Laura Deming’s post)

• Why must X be true? Why does this have to be the case? What is the single, fundamental reason?

• Do I really believe that this is true, deep down? Would I bet a large amount of money on it with a friend?

VI.

Two parables:

First, Ezra Pound’s parable of Agassiz, from his “ABC of Reading” (incidentally one of the most underrated books about literature). I’ve preserved his quirky formatting:

No man is equipped for modern thinking until he has understood the anecdote of Agassiz and the fish:

A post-graduate student equipped with honours and diplomas went to Agassiz to receive the final and finishing touches.

The great man offered him a small fish and told him to describe it.

Post-Graduate Student: “That’s only a sun-fish”

Agassiz: “I know that. Write a description of it.”

After a few minutes the student returned with the description of the Ichthus Heliodiplodokus, or whatever term is used to conceal the common sunfish from vulgar knowledge, family of Heliichterinkus, etc., as found in textbooks of the subject.

Agassiz again told the student to describe the fish.The student produced a four-page essay.

Agassiz then told him to look at the fish. At the end of the three weeks the fish was in an advanced state of decomposition, but the student knew something about it.

The second, one of my favorite passages from “Zen and the Art of Motorcycle Maintenance”:

He'd been having trouble with students who had nothing to say. At first he thought it was laziness but later it became apparent that it wasn't. They just couldn't think of anything to say.

One of them, a girl with strong-lensed glasses, wanted to write a five-hundredword essay about the United States. He was used to the sinking feeling that comes from statements like this, and suggested without disparagement that she narrow it down to just Bozeman.

When the paper came due she didn't have it and was quite upset. She had tried and tried but she just couldn't think of anything to say.

He had already discussed her with her previous instructors and they'd confirmed his impressions of her. She was very serious, disciplined and hardworking, but extremely dull. Not a spark of creativity in her anywhere. Her eyes, behind the thick-lensed glasses, were the eyes of a drudge. She wasn't bluffing him, she really couldn't think of anything to say, and was upset by her inability to do as she was told.

It just stumped him. Now he couldn't think of anything to say. A silence occurred, and then a peculiar answer: "Narrow it down to the main street of Bozeman." It was a stroke of insight.

She nodded dutifully and went out. But just before her next class she came back in real distress, tears this time, distress that had obviously been there for a long time. She still couldn't think of anything to say, and couldn't understand why, if she couldn't think of anything about all of Bozeman, she should be able to think of something about just one street.

He was furious. "You're not looking!" he said. A memory came back of his own dismissal from the University for having too much to say. For every fact there is an infinity of hypotheses. The more you look the more you see. She really wasn't looking and yet somehow didn't understand this.

He told her angrily, "Narrow it down to the front of one building on the main street of Bozeman. The Opera House. Start with the upper left-hand brick."

Her eyes, behind the thick-lensed glasses, opened wide. She came in the next class with a puzzled look and handed him a five- thousand-word essay on the front of the Opera House on the main street of Bozeman, Montana. "I sat in the hamburger stand across the street," she said, "and started writing about the first brick, and the second brick, and then by the third brick it all started to come and I couldn't stop. They thought I was crazy, and they kept kidding me, but here it all is. I don't understand it."

Neither did he, but on long walks through the streets of town he thought about it and concluded she was evidently stopped with the same kind of blockage that had paralyzed him on his first day of teaching. She was blocked because she was trying to repeat, in her writing, things she had already heard, just as on the first day he had tried to repeat things he had already decided to say. She couldn't think of anything to write about Bozeman because she couldn't recall anything she had heard worth repeating. She was strangely unaware that she could look and see freshly for herself, as she wrote, without primary regard for what had been said before. The narrowing down to one brick destroyed the blockage because it was so obvious she had to do some original and direct seeing.

The point of both of these parables: nothing beats direct experience. Get the data yourself. This is why I wanted to analyze the coronavirus genome directly, for example. You develop some basis in reality by getting some first-hand data, and reasoning up from there, versus starting with somebody else’s lossy compression of a messy, evolving phenomenon and then wondering why events keep surprising you.

People who have not experienced the thing are unlikely to be generating truth. More likely, they’re resurfacing cached thoughts and narratives. Reading popular science books or news articles is not a substitute for understanding, and may make you stupider, by filling your mind with narratives and stories that don’t represent your own synthesis.

Even if you can’t experience the thing directly, try going for information-dense sources with high amounts of detail and facts, and then reason up from those facts. On foreign policy, read books published by university presses -- not The Atlantic or The Economist or whatever. You can read those after you’ve developed a model of the thing yourself, against which you can judge the popular narratives.

Another thing the parable about the bricks tells us: understanding is not a binary “yes/no”. It has layers of depth. My friend understood Pythagoras’s theorem far more deeply than I did; he could prove it six different ways and had simply thought about it for longer.

The simplest things can reward close study. Michael Nielsen has a nice example of this -- the equals sign:

I first really appreciated this after reading an essay by the mathematician Andrey Kolmogorov. You might suppose a great mathematician such as Kolmogorov would be writing about some very complicated piece of mathematics, but his subject was the humble equals sign: what made it a good piece of notation, and what its deficiencies were. Kolmogorov discussed this in loving detail, and made many beautiful points along the way, e.g., that the invention of the equals sign helped make possible notions such as equations (and algebraic manipulations of equations).

Prior to reading the essay I thought I understood the equals sign. Indeed, I would have been offended by the suggestion that I did not. But the essay showed convincingly that I could understand the equals sign much more deeply. (link)

The photographer Robert Capa advised beginning photographers: “If your pictures aren't good enough, you're not close enough”. (This is good fiction writing advice, by the way.)

It is also good advice for understanding things. When in doubt, go closer.

Thanks to Jose-Luis Ricon for reading a draft of this essay.

My real education as a teenager was:

1. Books I chose to read myself

2. Learning to program computers (taught myself)

3. Video games (found them myself)

4. Maths (school)

#1-3 happened despite formal schooling, not because of it, something Paul Graham says here:

This suggests the following conclusion:

1. The things you learn by yourself stick; the things that are “taught” to you do not stick.

The fundamental principle of education is to give students an environment, and tools, where they can make discoveries themselves. This requires space, and time, and autonomy.

Students also need to be able to choose what they learn and how they learn it, something that modern rigid curriculums and prison-like school environments do not permit. 

However, I would go farther than this and say:

2. Video games provide a much deeper understanding of most subjects than classical education does.

I play a fair amount of chess (not well) and one thing you develop after playing a lot of chess is that you start to see “lines of force” on the board, e.g. the force a bishop exerts on an enemy pawn; and start to sense “weak points” in the opponent’s structure in a very physical way, in the way that you can sense the shakiest part of a Jenga tower in the physical world. 

This is the classical “scholar’s mate” position. As soon as you move your bishop to c4, you just “feel” that the black f-pawn is under pressure. The force exerted by the white bishop and Queen on the f-pawn can be felt as a weakness in the body of a good chess player.

Developing this touch-based, proprioceptive “sense” of a thing is often key to a really deep understanding of it. When you understand something very well it’s almost as though you can play around with it using all your senses — touch, feeling, space.

Feynman:

“If you can’t see it, it’s hard to explain why—but if you try to hold something up with a ladder, say, and you get the ladder directly under the thing, it’s easy to keep the ladder from sliding out. But if the ladder is leaning way waaaaay out, so that the far end of the ladder is only a very tiny distance from the ground, you’ll find a nearly infinite horizontal force is required to hold the thing up at a very slight angle. Now, all these things you can feel. You don’t have to feel them; you can work them out by making diagrams and calculations, but as problems get more and more difficult, and as you try to understand nature in more and more complicated situations, the more you can guess at, feel, and understand without actually calculating, the much better off you are!”

There are stories of Feynman rolling around on the floor, eyes closed in concentration, simulating physical processes with his body. His biographer, James Gleick, writes:

Intuition was not just visual but also auditory and kinesthetic. Those who watched Feynman in moments of intense concentration came away with a strong, even disturbing sense of the physicality of the process, as though his brain did not stop with the gray matter but extended through every muscle in his body. A Cornell dormitory neighbor opened Feynman’s door to find him rolling about on the floor beside his bed as he worked on a problem....In part the process of scientific visualization is a process of putting onself in nature: in an imagined beam of light, in a relativistic electron.

There is also the famous story of Einstein at 16 imagining chasing after a beam of light, the seed of special relativity. 

When you really understand something, it is almost always accompanied by this deep tacit “high-dimensional” grasp of the thing. But:

3. Schooling mostly fails at giving you this deep understanding.

Most adults have the incredibly painful experience of realizing that after 15 years of formal schooling we still can’t answer very basic questions about everyday scientific questions (e.g. why the sky is actually blue) and have to go look them up.

Most of the things I was “taught” at school simply didn’t stick. We were just told that an atom consisted of protons/neutrons/electrons, but we had no idea how anybody had figured this out (or, for that matter, why this was important). I have painful memories of studying the “plum pudding model” (aka Rutherford model) of atoms at school. 

It took humans thousands of years to really figure out that reality was made of atoms, and understand why this was true. Reality has a surprising amount of detail. The deeper you get into these questions, the more you realize that nothing is simple. But school didn’t leave us room to dive deep into these questions; we had to pass exams, which meant dutifully learning to calculate molecular weights, etc, without actually understanding a thing.

The net result is: no understanding, years of waste.

But this is entirely preventable. Consider if Pokemon had been “taught” in the usual way:

Playing with a simulation (i.e. the Pokemon games) gets you this deep, “bone-level” understanding very quickly. 

Thus, I believe that:

4. Video games will become a core component of education.

This sounds absurd, but consider that simulations are already used widely for learning:

• Kerbal Space Program is notorious for turning its players into experts on orbital mechanics

• Flight simulators and combat simulators are used to train pilots and army personnel

• The game “Factorio” teaches you about capitalism, production, and the economy

• Programming environments are simulations of the “universe” of that programming language, with fast feedback loops (write code -> debug)

A video game is just:

(a) a simulation of reality

(b) with fast feedback loops.

Learning is just the act of engaging with an external thing and performing many conjecture/criticism loops, forming conclusions, and building on them to form a body of knowledge.

So it makes sense that video games would be the primary educational environment of the future: they are the best way we have of (a) creating simulations of reality (b) with fast feedback loops (c) accessible at low cost.

(By the way, this is true not just for abstract concepts such as orbital mechanics, but also for ethical traits. e.g. my understanding of heroism is to this day shaped by games like Final Fantasy and Metal Gear Solid. This also helps explain why reading fiction is valuable: you can gain “bodily knowledge” about despair by reading Dostoyevsky, about jealousy by reading Swann’s Way, etc.)

5. Where games mostly fall short is that they’re not that transferable to the real world. The skills you learn are highly specific to that game. This will change.

Most chess knowledge is specific to chess; what you learn from getting good at chess is not really transferable to real life, it instead consists of highly technical understandings of various chess positions. This is because chess is not an accurate model of reality. The lessons you learn from chess are generalizable only at a high level (e.g. a bad plan is better than no plan).

But if you have games that are (a) fun and (b) accurate for some aspects of reality, such as KSP or Factorio, you do get learning that is real and transferable. The challenge is in making games that satisfy both constraints.

There are not that many examples of this just yet, but I believe there will be more in the future. So why hasn’t this happened already? I think the answer is:

6. It is currently too hard to make video games. Making it easier to create video games will massively increase the supply of good video games and cause a gradual revolution in education. 

One thesis of the internet that always stuck with me comes from Evan Williams, who founded Twitter/Blogger/Medium. He said that the best way to create a giant internet company is to take something people want to do and make it 10x easier.

This was the core insight behind all of his companies. People already want to be creative, they want to make music, they want to make video games, they want to write (clearly!), they want to publish things online, but the barriers to entry for most of these things is currently too high. 

"We often think of the internet enables you to do new things," Williams said. "But people just want to do the same things they’ve always done."

(...) "Here’s the formula if you want to build a billion-dollar internet company," he said. "Take a human desire, preferably one that has been around for a really long time...Identify that desire and use modern technology to take out steps." (link)

“Making things easier to create” is a mega-trend of the internet era:

• Blogger/Medium/Wordpress make it easy for anyone to write on the internet;

• Substack makes it easy for people to set up newsletters;

• YouTube and TikTok make it easy to make videos;

• Ableton makes it easier to create music; etc.

Another insight is that making things easier has nonlinear effects. Making something 10x easier can cause 1000x more of that thing to happen. Hence the explosion of online creativity you see on YouTube, with chess, Minecraft, math videos, Khan Academy, Twitchstreams, Soundcloud, etc; you remove a small bit of friction and get a large result.

This trend has a long way to go, though. Making video games is colossally hard and expensive; you  need to be a crazy genius indie game developer with ultra-high risk tolerance, or else a mega-corporation like Unity or Steam or Valve, to do it. Video games haven’t yet had their “creator revolution”.

This will not always be the case. Large companies will be built that make it easier for anyone to make videos, or video games, or podcasts, and so on, unleashing a ton more latent creativity. This will make the idea of video games for education seem less and less insane over time, as better games get built to help students understand important things like Newtonian mechanics, maths, chemistry, architecture etc. on an intuitive level that books and teachers rarely manage to reach.

Some caveats:

First, I suspect that optimizing directly for ‘being educational’ is a bad idea for video game creators. Most video games I’ve played that tried that were boring. It seems better to optimize for some combination of fun and complexity. 

Second, this post is not a call to “gamify” education or to slap gaming dynamics on top of education as an after-thought. That won’t work.

One last thought: this is also why AI is important for human productivity. Most of the discourse is about how AI will “replace” humans. I prefer the Licklider school of thought: human-computer symbiosis. AI will make humans vastly more effective by automating tedious tasks. For example, humans can use text AI such as GPT-3 to generate ideas/boilerplate writing to get around the terror of the blank page, and then simply pick the best ones and refine/iterate on those. (AI Dril, which was based on GPT-2, was an early example of this). As AI gets better, “assistive creativity” will become a bigger thing, enabling humans to create sophisticated artifacts (including video games!) easier and better than ever.

The potential energy of human creativity is vast. We just need to give people tools, and creation will follow. Onwards!

Thanks for Saku P for reading a draft of this.

Update: Some excellent discussion in the HN thread.

(NB. I don’t endorse many of these views, but I found it interesting to write about them.)

The gods did not reveal, from the beginning,

All things to us, but in the course of time

Through seeking we may learn and know things better.

But as for certain truth, no man has known it,

Nor shall he know it, neither of the gods

Nor yet of all the things of which I speak.

For even if by chance he were to utter

The final truth, he would himself not know it:

For all is but a woven web of guesses.

— Xenophanes

This quotation from Xenophanes, who lived around 500 BC, contains the essence of Karl Popper’s philosophy: fallibilism; gradual improvement via a process of conjecture and criticism; scientific knowledge as an externalized, objective entity (a "woven web of guesses"). 

Popper is one of the most underrated philosophers, because he was right about several important things. These include democracy, the paradox of tolerance, scientific method, fallibilism, and open society/anti-Marxism [1]. In addition, Popper managed to solve several important philosophical problems (e.g. induction) in convincing ways, a feat which has eluded most philosophers. So I think most people should know more about his ideas.

Let’s go into them a bit more.

Fallibilism

The essence of fallibilism is: you can never know whether you have reached certain truth. Every assertion is provisional, and may be false.

The best theories we have are those which have so far withstood a barrage of criticism in the form of arguments, experimental tests, and so on, and have so far not been found to be false. (Not: “found to be true”.)

In some cases, we even know that our best theories are false -- for example, we know that at least one of general relativity or quantum theory is false in current form, since they contradict each other.

Why might fallibilism be true? There are two main arguments [2]:

First, the asymmetry between verification and falsification: if your theory predicts X, then observing not-X can, under certain conditions, “falsify” your theory [3]; whereas any number of observations of “X” never conclusively proves your theory correct. (The cliche example is the theory “all swans are white”: you cannot prove it for sure! A black swan may always be lurking...) [4]

Since we cannot rule out observations that haven’t been made yet, theories will always remain provisional and may contain hidden errors.

Second, the usual alternative to fallibilism is to start with some axioms that one knows to be true; or to designate a particular source of beliefs (such as the senses) to be foundational, such that any beliefs that come from that source may be considered true. But the idea of a "foundational" belief is problematic because of the infinite regress problem: what does that foundational belief rest on? Either it rests on something else, in which case it isn’t foundational, or it is “self-justifying”. And the idea of a particular type of belief being “self-justifying” is arbitrary & question-begging.  

Popper’s solution to this problem is to dismiss the notion of “justification” altogether: you can judge which of two theories is better using criticism, without having to appeal of whether belief in either theory is justified. Any idea, from any source, may or may not be valid, but judging ideas from the source is not valid. 

(People sometimes dismiss emotions and prefer reason as a reliable source. According to Popperian epistemology, this is a fallacy: sometimes your emotions are right and your reason is wrong, and vice versa. Ideas must be judged on their merits, not on where they came from! So listen to your emotions.)

So if fallibilism is true, then certain knowledge is impossible, which implies that science works by attempt to disprove theories (error-correction), as hard as possible.

Spending your time looking for “confirmatory evidence” is a waste of time. What you should do, and what good scientists do, is sketch out what your theory forbids, and then try your hardest to find evidence of that; try to disconfirm your theory. [5][6]

This doesn't mean all knowledge is hopeless, but it does mean that we can never get absolute certainty about anything, because a criticism may always come along and destroy it. And if we ever did reach the truth, we would never be able to tell, because there isn't any known way of conclusively proving that our current state of knowledge really is the final state.

More interestingly, it implies that science is an infinite process; we will never know whether have reached the end, and may never. In practice, the more knowledge we accumulate, the more ignorance we recognize around us; solving a problem with a new theory raises a host of new, more interesting problems.

(A contrasting view would be: there are a finite number of important questions, with a finite set of knowable answers; ignorance decreases in constant proportion to our increase in knowledge; and there will eventually be an end to knowledge discovery. Popper opposes this.)

As Einstein wrote:

[in the struggle for new solutions] new and deeper problems have been created. Our knowledge is now wider and more profound than that of the physicist of the nineteenth century, but so our doubts and difficulties.

Around the same time as Einstein was inventing special and general relativity, Lord Kelvin was saying that physics had no further surprises to yield; Einstein was proven right.

The history of science seems to bear this out: Newton's theories were thought to be true until Einstein's theories came along and explained the facts in a new and better way. Similarly, at some point Einstein's theories will be augmented and superseded by even better theories. The history of science is one of a long train of false theories that were replaced by better ones. Constant improvement is possible.

One may object to this by reaching for the most obvious form of certain knowledge we have, i.e. mathematical proof. Surely we know that 2+2 = 4, for example?

The counter-argument here is to ask: why that example; it’s not because it comes with a shiny “truth” tag on it. It’s because you can’t see any way that it could be false. In short, you’ve tried your hardest to criticize it, and you can’t disprove it.

We may be wrong at any moment! Even mathematical proofs are fallible; we could all believe that a certain proof was watertight, & then discover a critical mistake in it at some later point. 

Conjecture and criticism

For Popper, the growth of knowledge is very much like biological evolution.

In evolution, we are presented with an environment. Various organisms attempt to fill the niche, and random mutations happen. Some of those mutations and organisms fit the environment; some don’t, and die off. Over time, the process converges towards optimal organism/environment fit. Until the environment changes, and so on...

In knowledge (or science), we are presented with a problem. Various theories (and variants of these theories) attempt to solve the problem. Some of the theories are criticized and fail, so they die off; the good theories remain. Over time, the process converges towards truth, and the problem is gradually solved. Until the new theories give rise to new problems, and so on... 

Since no theory is final, each theory presents us with a set of further problems. The best theories are very fertile: theye give rise to a host of new, interesting, and deeper problems, e.g. Newton’s account of gravity then gives rise to the problem of how gravity can act at a distance; Maxwell’s electromagnetic theory gives rise to the problem of whether Galilean relativity applies to the speed of light, which in turn gives rise to special relativity, and so on... [7]

A nice implication of this “evolutionary” view is that problem selection and problem generation are at least as important as problem solving in the activity of science; organisms in barren environments will take much longer to flourish compared to fertile environments. It’s important for scientists to pick the right problems and then try and solve those.

Einstein again:

[solving problems] may be merely a matter of mathematical or experimental skill. To raise new questions, new possibilities, to regard old problems from a new angle, requires creative imagination and works real advance in science

Szent-Gyorgi:

A scientific researcher has to be attracted to these (blank) spots on the map of human knowledge, and if need be, be willing to give up his life for filling them in.

Magee:

Popper considered it a waste of time for a thinker to address himself merely to a topic...there is often a feeling of so-what-ness hanging in the air, since no particular problem has been solved, or question answered. The whole procedure is arbitrary. So Popper suggests as a general principle that a thinker should address himself not to a topic but to a problem, which he chooses for its practical importance or its intrinsic interest, and which he tries to formulate as clearly and as consequentially as he can. His task is then manifest, namely to solve this problem…

Objective knowledge

Economists sometimes distinguish between human capital (stored in neuronal connections) and knowledge (stored in books, explicit, codified). Writing then becomes the act of converting human capital to knowledge, which in turn creates more human capital. (This is the starting point for Paul Romer’s Nobel-winning work on endogenous growth theory).

Just as the evolutionary phylogenetic tree is an objective fact, so too is scientific knowledge: all the books, theories and papers that constitute scientific knowledge exist independently of any human's mental state, even though they are originally created by humans.

Popper anticipated this insight in his “three worlds” argument. The three worlds are:

• World 1: Physical objects

• World 2: Mental states

• World 3: Science, art, philosophy, objective knowledge... things that we created, but which exist independently of humans

Consider two possible scenarios: (1) all technology, civilizational artifacts, and human memories of these things are destroyed, but our books, libraries, scientific papers, and engineering manuals remain; (2) same as #1, but our books, libraries, science papers and so on are all destroyed too.

In case #1, we would be able to recover civilization painfully but within some reasonable timeframe, whereas in scenario #2 we would have to rediscover everything from the beginning, which would take us millenia. This shows that the world of scientific knowledge exists in those books/libraries/papers, as an entity that exists independently of human mental states — even though they are all the creations of human creativity.

Given the existence of world 3, Popper suggests that the best way to make a contribution to scientific knowledge is to understand the current set of theories and problems that constitute the current state of scientific knowledge, and then aim to extend it by considering a particular problem or theory and aiming to solve that problem. In doing so, one may create new theories and subject them to critique. One may put forward a scientific theory without actually believing the theory; in Popper's parlance, we can "let the theories die in our stead". (Contrasted to evolution, where organisms literally die if they are maladapted to their environments.)

Implications for other areas

Popper’s philosophy has a large number of implications for other areas of life. (The physicist David Deutsch has done a lot of work drawing this out & the below is indebted to reading and discussions with him.)

(1) Politics

In political philosophy, Popper is famous for (1) his theory of democracy (2) the paradox of tolerance. Both more or less follow from the above epistemology.

The theory of democracy goes like this: typically, political philosophers ask the question "who should rule?" But mistakes are inevitable, and nobody actually knows the best way to govern. The most important criterion for a society overall is that it makes progress and creates knowledge at the fastest rate possible on how to make its citizens affluent. In order to do this, citizens must be able to make mistakes and correct them. Thus, the essence of democracy isn't "who should rule", but *the ability to remove a bad ruler*, which is the same as being able to correct a mistake.

(For those of you who follow UK Politics, this appears to be Dominic Cummings’s worldview as set out in his blog. Current political systems aren’t error-correcting fast enough to adapt to a rapidly changing world, which means they have to be reshaped to be able to do so. Technology helps get a better view of what’s actually going on — such as the effects of various policies — which in turns makes correcting errors easier; hence his emphasis on tech.)

The process of governance should follow the same process as that of all knowledge creation: a government commits to trying a set of policies to solve certain problems. After a sane period of time, say 5 years, they are assessed on whether or not they solved those problems, whether citizens believe they will continue to solve problems well, and so on. If citizens think they made a mistake electing that government, they should be able to kick them out (without violence) and enact a new government. The most important thing in knowledge creation, including political knowledge, is error-correction. This is Popper's criterion.

One interesting implication of it is that First Past the Post electoral systems are superior to Proportional Representation systems. In brief, this is because PR typically leads to coalition governments where it is not clear who is to blame for a given policy, and as a result the country as a whole doesn't learn much and doesn't make progress. FPTP leads to a clearly accountable government with the ability to enact its policies and an easy means of removing them if they fail. (David Deutsch explains this more in this YouTube video.)

The "paradox of tolerance": because of fallibilism, nobody actually knows anything, so everybody must be free to make guesses, criticize others' guesses, and engage in reasoning and free speech, so that we can improve our theories over time. However, anything that impedes this process should not be tolerated, because impeding this process means impeding the entire process of learning itself, which means that we will never make progress. Thus, we should tolerate people and ways of life but we cannot tolerate those that are intolerant, i.e. that seek to shut off criticism, free speech, and the means of making progress. Liberal democracies thus need to guard things like freedom of speech and critical thought jealously, otherwise they may end up failing. In short, running a tolerant polity requires a certain degree of intolerance towards anything that would threaten the polity's existence.

(2) Parenting & Education

Although Popper didn't explore this himself, I think his philosophy entails taking movements such as Taking Children Seriously, Unschooling, and the homeschooling movement more seriously, at least at the margin.

The premise of these movements is that most parenting/education philosophies take it for granted that you have to coerce children into doing various things for their own good. But fallibilism implies that nobody knows for sure what good is, and good criticism must be taken seriously as an objective idea, regardless of the source. So in the case where the parent and child disagree, they must work it out using reason, which means not forcing children to do a certain thing. Coercion cuts off the reasoning/learning process by saying "person X is right because they are person X", which violates the rule that the source of an idea doesn’t matter, only the idea itself does.

Popperian epistemology also implies an account of education as being about each person creating knowledge for themselves, rather than “receiving” knowledge passively (which he called ‘the bucket theory of the mind’, i.e. the idea that you can just pour ideas into the mind the way you pour water into a bucket). Reading instruction, or hearing them, is merely the beginning of the process; you then have to guess what the meaning of what you’re reading/hearing is, and synthesize the essential “thing” behind the words, which is a highly active process.

For example, suppose we’re both sitting in a lecture by Popper, and I ask you to imitate Popper’s way of speaking; as any AI researcher knows, this instruction does not speak for itself. Should I copy Popper’s Austrian accent? Should I be standing? Should I face the back of the room, as Popper is doing? Should I be copying Popper’s formal diction, or just the content of his thoughts? Etc. As this set of questions makes obvious, understanding pretty much anything involves an active process of interpreting the meaning behind such statements, and this understanding may be revised as further thinking occurs. This is what Popper means when he says:

In fact, I contend that there is no such thing as instruction from without the structure, or the passive reception of a flow of information that impresses itself on our sense organs.

(3) What to work on

Nobody can predict where important knowledge is going to come from -- we can guess, but there will always be an element of surprise. This is notoriously true in scientific and technology fields, which are full of entertaining examples of random nobodies/underdogs coming out of nowhere and reinventing fields based on seemingly ridiculous ideas. (See this excellent Quora answer, on how startup ideas often seem dumb, for an entertaining example.)

It is often said that people should work on the most impactful thing. But by the above, one doesn't know a priori what the most impactful thing is, because one doesn't know the state of future knowledge. Lots of impactful innovations come from working on areas that seem remarkably unpromising but are interesting/fun to the researcher.

For example, most of the 20th revolution in biology was driven by a bunch of nerds playing around with fruit flies, and the entire field of genetics comes out of an obscure Swiss monk called Gregor Mendel who was cross-breeding various strains of peas. Szent-Gyorgyi won the Nobel Prize for discovering Vitamin C (ascorbic acid) by investigating why bananas turn brown but some other fruits don’t. Newton discovered much of contemporary mechanics, optics, and cosmology, but he also spent time on alchemy and Biblical numerology. As Paul Graham has pointed out, he couldn’t see in advance which of these would bear fruit; things that are “huge if true” are worth working on, but one cannot guarantee that they will lead to important knowledge!

What you are interested in — what is “fun” — is an important signal of which part of the knowledge tree you should be trying to grow. As Popper says, seek fun problems and fall in love with them.

(4) Creativity

Another implication: there is no known way of reliably generating ideas that are only good; “idea generation” software has an error rate that is irreducible. (If there were such a thing as always generating only correct ideas, then one would be infallible, which we have shown is not possible). Nobody knows how ideas are generated; but if you want to generate good ones, you are best served by generating a lot of them (create mutations) and then filtering hard (selection) and pursuing the good/promising ones.

This has practical implications. From Arthur Jensen’s paper on creativity:

I once asked another Nobel Prize winner, William Shockley, whose cre­ativity resulted in about a hundred patented inventions in electronics, what he considered the main factors involved in his success. He said there were two: (1) he had an ability to generate, with respect to any given problem , a good many hypotheses, with little initial constraint by previous knowledge as to their plausibility or feasibility; and (2) he worked much harder than most people would at trying to figure out how a zany idea might be shaped into something technically feasible. Some of the ideas that eventually proved most fruitful, he said, were even a physical impossibility in their initial conception. For that very reason, most knowledgeable people would have dismissed such unrealistic ideas immediately, before searching their imaginations for transformations that might make them feasible.

Jensen again:

The individuals in whom this mental manipulation process turns out to be truly creative most often are those who are relatively rich in each of three sources of variance in creativity: (1) ideational fluency, or the capacity to tap a flow of relevant ideas, themes, or images, and to play with them, also known as “brainstorming”; (2) what Eysenck (1995) has termed the individuals’ relevance horizon; that is, the range or variety of ele­ments, ideas, and associations that seem relevant to the problem (creativity involves a wide relevance horizon); and (3) suspension of critical judgment. Creative persons are intellectually high risk takers. They are not afraid of zany ideas and can hold the inhibitions of self-criticism temporarily in abey­ance. Both Darwin and Freud mentioned their gullibility and receptiveness to highly speculative ideas and believed that these traits were probably charac­teristic of creative thinkers in general. Darwin occasionally performed what he called “fool’s experiments,” trying out improbable ideas that most people would have instantly dismissed as foolish. Francis Crick once told me that Linus Pauling’s scientific ideas turned out to be wrong about 80 percent of the time, but the other 20 percent finally proved to be so important that it would be a mistake to ignore any of his hunches.

Further Reading

Bryan Magee’s “Popper” is a nice short overview of Popper’s work.

“Popper Selections” is a great primary source: excerpts from various of Karl Popper’s books, most of which are collections of essays.

The best Popper books to start with are “Conjectures and Refutations” and “Objective Knowledge”.

Many more implications of Popper's philosophy are sketched out in David Deutsch's “Beginning of Infinity”.

Thanks to David Deutsch and Karl Wilzen for commenting on drafts of this.

----

[1] Any thinker who was strongly against Marxism, for reasons which remain valid today, at a time when Marxism was in intellectual fashion and indeed seemed like a good bet, is impressive in my view.

[2] I don’t attempt to prove this thesis rigorously; this essay is just a sketch of Popper’s viewpoints.

[3] “Falsify” is a slight simplification. It makes the theory worse; theories can’t be decisively falsified, but there is such a thing as a “really good” theory and a “really bad” theory, and rationality means we pick the best ones.

[4] There is an elegant analogue of this asymmetry in Buddhism. Buddhism observes that there is an asymmetry between suffering and pleasure; suffering is more bad than pleasure is good. A small amount of suffering can ruin any amount of pleasure, but pleasure does not fix suffering. Moreover, suffering and death are inevitable (for now), and no amount of pleasure changes this fact. One can cope with an infinity of pleasures and never have perfect satiation -- one can never reach the perfect upper bound -- but there is a lower bound we eventually reach, i.e. death. Human life can be thought of as a downward-biased random walk in which suffering and bad feeling are inevitable; the only rational response is to stop playing the game and reach perfect equanimity, etc.

[5] Chapter 22 of “Harry Potter and the Methods of Rationality” by Eliezer Yudkowsky is a memorable dramatization of this way of doing science.

[6] This, in short, is why astrology isn’t a science: it seeks confirmatory evidence, but never sets itself up to fail. It explains everything, and therefore nothing! Good theories take epistemological risks.

[7] This account contrasts radically with the induction-based conception of science, which says that one starts with data and then finds patterns in that data. The problem with this account is that science doesn’t have to start with data. Moreover, all 'observing' involves a human making decisions -- implicitly or explicitly -- about what features of the observed sense data are relevant, and interesting, and require explanation. In that sense, "all observation is theory-laden". Popper used to demonstrate this in lectures by asking his students to just “observe!”, prompting the question “what should we observe”?

This post contains everything I wish I’d known, before I started, about doing sales at a startup. It’s really hard to find good information about sales without a lot of fluff, so I set out to write down as much as I could about it.

The biggest mental shift I had to make to sell successfully was to realize that:

Your job is not to “sell”, your job is *qualification*. 

The actual “selling” process goes extremely smoothly if you’ve qualified the person correctly. Most pain is caused by people not doing this qualification process correctly, overselling or underselling their product, and then being mystified by the inevitable “no” / being ghosted by their customer.

Once you’ve qualified somebody as a good fit for you, actually selling to them is remarkably easy and seems to “just happen” effortlessly. This is one of the most surprising things about sales.

Qualification means asking the correct set of questions to determine if the prospect is a good fit for your product or not. Once your startup has product-market fit, you will have a good sense of what these “qualification questions” are. In our case at GoCardless [1] (where I learned how to do this), these were questions such as: “how many payments do you take every month?”, “how do your customers pay you at the moment”, “where are your customers physically based”; these quickly enabled us to figure out how big the customer/prospect was & whether they were the right fit for the specific payments solution we had.

The #1 rule of doing a sales call is: be quiet.

Most people talk way too much in sales calls. You should aim to be talking for about 20% of the time, with your customer talking about 80%. If you didn’t get this ratio, it wasn’t a good sales call. Sales is fundamentally about listening, not about talking.

Why is this true? Sales is a process of matching: somebody with a certain pain point or need finds a product that meets this need for the right price, and goes away happy. To figure out if somebody has this pain point and needs your product, you need them to tell you about their business and how they currently solve this problem.

Since people do not typically have a well-formed list of things they want out of a product in their head — often they’re just curious / have a vague sense that something is wrong and have just heard of your product from press or whatever — you need to elicit information from them so that you can help them figure out if your product is the right fit. Hence: they need to do most of the talking.

Every sales call needs a structure.

Not only should you have a structure laid out in front of you on a notepad/paper, but you need to lay this structure out verbally from the outset.

The structure we used was:

1. Introductions

2. Current process

3. Pain points & Ideal solution / requirements

4. Your solution

5. Next steps & timeline

We made sales reps print out worksheets with these sections on them along with key elicitation questions / soundbites, and they had to physically fill out these worksheets for their first hundred or so sales calls. If you miss out any of these sections, it’s typically a bad sales call.

As an aside, the most helpful book I read on sales was “SPIN Selling”. I didn’t use the specific framework in the book, but it has a lot of valuable stuff around the mental shift you need to make to do sales correctly. 

0) Pre-call research

10-15 minutes of prep makes a world of difference. LinkedIn is still the main method here: search the person, find their title, Google them, Google their boss, try and construct a clear picture of the org chart. For example, if their title is “Head of Payments” figure out if there is a “VP” level in the org (typically more senior) — you could try searching {company name} + “payments” as an example and get everyone with the word “payments” in the title. 

You want to have a clear picture of how senior the person is i.e. whether or not they have buying power and who they would need to convince to get the deal done; more on this later. 

The smaller the business, the more important it is to be talking to the Founder/CEO; the larger the business, the more you want to be talking to whoever specializes / is in charge of the function you are targeting. 

Now, onto the call itself:

1) Introductions

Very important and often not done correctly.

First, always ask if it’s still a good time to chat — “we have 30 minutes scheduled to chat, is now still a good time to chat?”. Yes, the meeting was scheduled so it probably is, but there is a nice element of asking permission here and getting someone’s consent here and getting them settled into the call. It also shows respect for their time.

Second, once they’ve said yes, you want to:

• Briefly introduce yourself and your title/role at the company

• Set out exactly the things you want to talk about. Something like: “So I was hoping to run through: your current process for XYZ, any pain points or issues you have with it, your ideal solution or requirements for a new thing, and then run through next steps if that makes sense. Is there anything else that’d be useful for you to run through?”

• Ask if they have anything else they want to add.

This, again, has the benefit of consent: nothing is going to be a surprise; you have freedom now to ask exactly the questions that you want to ask under those categories, and if they want the call to go a different way, they can just say so.

More importantly, laying out the call like this signals that you’re a competent and respectable counterparty who knows what they’re doing. This “signalling effect” is super important: orgs are much more likely to buy from you if they actually think the salesperson was smart and competent, so signalling that you are increases your likelihood of success.

2) Current process

This is by far the most important part of the call. If you do this part correctly, then you know exactly what they do at the moment, whether they are a good fit, and whether they’re likely to convert. In sales this part is most of “qualification”: understanding if somebody is a good fit for your product or not. (If you don’t have this figured this out yet, then you are probably not ready to scale up a sales process).

The key rule for this process is to ask questions that aren’t “yes/no”, i.e. get people to describe what they do right now, expansively.

So if you’re working for Zoom, you want to ask something like “how do you currently do videoconferencing?” Or “how would somebody at your company currently take a video call?” You would then get them to expand on their answer using prompts like “can you tell me a bit more about that?”

Get them in the mindset of telling you the story. Once they finish one part of the answer, you can cue them by saying “OK, so sounds they do X first, what happens then?”

The success criterion for this phase is: by the end of it, you should be able to “play the movie” in your head of exactly how the relevant process you’re targeting looks.

If you’re selling Zoom, you know exactly how they take remote calls. If you’re selling Box, you know exactly how they share files with each other. If you’re a payments provider, you know exactly what their payment flow looks like and you can draw it.

At this point, if somebody clearly isn’t a good fit for your product, tell them so, and refer them to the best companies (sometimes your competitors!) that would be a good fit!

This counter-intuitive point is one of the best things I learned about doing sales. It has a host of benefits: honesty; people are often pleasantly surprised; it builds so much good will that you often get referrals out of it; and it saves everybody time. A lot of sales heartache comes from ignoring this rule and trying to “convert the sale” anyway when it’s clearly a bad idea for everybody involved.

For example, at GoCardless there was a time when we didn’t support customers who used paper instructions, so if a business wanted to take payments from customers where >10% of payments by volume were paper-based, they weren’t a good fit for us. In those cases I’d simply cut the call short and say something like: “A couple things to note: it sounds like you need paper payments support, and we actually don’t support that, so we’re probably not the best fit for you here. If you want recommendations for who else to talk to about that, we recommend chatting to {competitors} as they’re very good.”

One last thing: if you’ve done this part right, you should have some hypotheses for what they’re looking for / what’s bothering them about their current solution. That will come in handy for the next part...

3) Pain points & ideal solution/requirements

Once you’ve figured out their current process, you need to understand what’s broken about it. Note that these can be two discrete sections, but I found they typically blended into the same thing. The key questions for this section are:

• What’s the biggest thing you want to fix about your [videoconferencing, payments] at the moment? (-> Why that thing? etc.)

• So obviously you’re on the phone with me, and we’re talking about [payments, videoconferencing] How come you chose to take this call today?

• What are you looking for? If you could wave a magic wand and have the perfect solution here, what are the top 3 things that come to mind?

• What’s important to you here? (If they’re blanking, you can feel free to suggest, but *only if they’re really blanking*. You can say something like “E.g. speed? Support for European countries? Mobile/iPad support? API integration?”)

(Note that these are in my voice; you should adapt them to your voice rather than say them verbatim.)

If none of these are working, you can try and get more specific by referencing their current process — “so you mentioned that people typically do this to start with. Is that a pain point for you?” I usually don’t find this works so well — you’re fishing at this point — but it is worth doing just in case it elicits additional information.

This phase has been successful when you have clear conviction in the 1-3 things they care about most. This has the additional benefit of better qualification: if those 1-3 things are the things you are best known for, great! If not, they may not be a great fit. For example, customers who really cared about a clean/easy API integration were a perfect fit for GoCardless since that was our main advantage early on — a nice REST API. Conversely, if what they really care about is “high touch support” or something like that, then they might not be right for you. Etc.

4) Your solution

You’ll note that at no point thus far have you done a “sales pitch”. Well, if you are ever going to do one then now is the perfect time to do it. That said, “sales pitch” is the wrong way to think of it.

At this point in a call, the person either clearly IS or clearly ISN’T the right fit for you. Moreover, they’ve given you a bunch of information if you’ve done their job right, so typically they’ll be expecting more information from you by this point, so you have “permission” to talk (in any conversation, people get uncomfortable if the ratio is too out of whack, i.e if either person is talking too much).

I hate aggressive pitches so at this point I’ll usually lead into it with something soft like “Great! That gives me everything I need, and [sounding pleasantly surprised here] I actually think we’re a great fit for you based on what you’ve said. Do you have any questions about us / have you had a chance to read up, or do you want a quick 1 minute intro and go from there?”

The “quick 1 minute intro” avoids the dread from their end of some salesperson droning on forever about their thing. This also helps because some people have in fact done research and just want to know a few specific things, whereas others are just curious and need more education.

If they want my blurb at this point, I just tell them what the company does, and then the “pitch” part is easy: confirm the 1-3 things they care about back to them, and then explain how your thing meets those needs, with credible proof points.

• “So if I heard correctly, it sounds like you care about a quick/easy technical integration, payments resolving within a day, and France support — is that right?” → give them a chance to confirm

• Then go through each trait…

  • “You mentioned that you care about XYZ. That’s actually a big strength of ours, [clear proof point].” For example — “you mentioned that you care about easy technical integration because you don’t have a ton of developers. Since we’re a tech company, we actually have a really easy REST API that usually takes people less than 2 hours to integrate. I’ll send you the link to the docs after this call so you can take a look.”

5) Next steps and timeline

Don’t forget this part.

The main thing here is you want to leave the call with definite next steps to the extent that you can. E.g. if they need to talk to some other people (typically true) then you want to know exactly who those people are and roughly when they’ll talk to them. You then want to get verbal consent for a follow up call in whatever timeframe makes sense — something like “would a follow-up chat next Monday work for you?”

This is all very contextual at this point and depends on how they’re responding, some people are more non-committal at this stage and then you don’t want to be too pushy about scheduling a call but you do want to be pushy about what information they need to make a decision and how exactly they’re going to proceed here. If they’re not telling you those things, they’re extremely unlikely to end up buying and mostly just want to politely save face / soft decline. (Note that this lesson extends well beyond just sales).

Conversely, if you’ve done your job right and they’re a good prospect then they’ll typically like and trust you by this point and want to get the deal done, so they become your ally and give you the information on what they’re doing next, how long things take, etc. Best case they really give you the inside info on who exactly needs convincing, how to convince that person, how to get on their calendar, etc and will make intros / set up follow up meetings for you. The latter piece obviously becomes more important in larger enterprise sales (a separate topic).

6) Follow-up

Always follow up same-day. No excuses. Do not leave your follow-ups till tomorrow. Just get it done then and there, it shows you’re on top of things and the signalling value is super important, plus lags are the death of sales so you always want to take care of your end of things as quickly as is humanly possible.

Your follow-up should be short, gracious, and include a clear call to action for the agreed next step. For example, if you decided a follow-up call made sense, you want something along the lines of “You mentioned a call next Monday afternoon would be good; I’ll send an invite for 3.30 and just let me know if you’d prefer a different time.”

Finally — follow up relentlessly. People drop off the radar even when the sales call went incredibly well, and it’s usually just because they’re busy, not because they hate you. I can’t tell you how many big deals I closed where I had to follow up >15 times to get the deal done. Be shameless here — you want to get a clear “no” or a clear “yes” or a clear “I’m doing XYZ and it will take me 5 days, follow up next week”. I usually followed up every 3 days or so, but people differ and the specifics matter here.

Thanks to Sarah Constantin, Alexey Guzey, and Nikhil Krishnan for reading drafts of this.

Footnotes

[1] About me: I was first BD/salesperson at GoCardless, a YC 2011 company that’s now a large payments company backed by GV, Accel and so on. I set up and ran the sales, BD and marketing teams and closed deals with many companies you’ve heard of (Box, the UK government, The Guardian, TripAdvisor, SurveyMonkey, etc.) as well as thousands of small/mid-size businesses. I’ve taken many hundreds of sales calls & trained sales reps.

However, I am more of a generalist and do not consider myself a sales “expert”; treat this as “the core basics to get you from technical/non-technical founder to decently good at sales”.

These short lectures on Proust were given by Jozef Czapski, a Polish artist and soldier, who was captured by the Soviets during WWII. The lectures were given to his fellow officers inside the prison. He describes them hunched, squatting on the floor after a long day of picking potatoes in freezing temperatures, listening to these by candlelight under large portraits of Marx & Lenin.

Czapski didn't have reference to the text, nor any notes, references, etc. This turned out to be a really nice constraint; it forces him to reconstruct the book from memory as best he can and focus on what’s interesting about it to him (since that is what is available to his memory), so the result is not dry.

I was wondering whether I would be able to give a series of lectures on anything that would hold up the way these have, in the absence of references. It turns out that you can recall more than you can think. Apparently, Czapski "found that a prisoner's constant state of vigilance was surprisingly conducive to the reclamation of memories":

"After a certain length of time, facts and details emerge on the surface of our consciousness which we had not the slightest idea were filed away somewhere in our brains...These memories rising from the subsconsious are fuller, more intimately, more personally tied to one another... Far away from anything that could recall Proust's world, my memories of him, at the beginning so tenuous, started growing stronger and then suddenly with even more power and clarity, completely independent of my will."

There are some moving parts about how, trapped in these inhumane circumstances, the thing that made them feel the most human was precisely these lectures on subjects that were "useless" to their survival (Proust, mountain-climbing, etc.) but which were the most precious things for these captive soldiers. 

Czapski has a good knowledge of Proust's life, and he argues that one of Proust's superpowers is having an insane attention to, and memory for, detail. He recounts a story where Proust goes to the opera, sits in corner, facing away from stage & talks the entire time, but can recount pretty much everything that happened. (pp24)

"...in the final years of his life, the Duchess de Clermont-Tonnerre secured a box at the Opera for a large charity event with the idea of allowing Proust to observe once again...Proust arrived late, seated himself in a corner of the box, turned his back to the stage, and never stopped talking. The next day the duchess remarked to him that it had hardly been worth the bother of taking a box to help him venture out if he had had no intention of paying attention to what was going on. With a sly smile, Proust proceeded to recount, with meticulous precision, everything that had occurred in the theater and on the stage, piling up a wealth of details that no one had noticed, and then added: 'Don't worry, when it comes to my work, I'm as busy as a bee.'"

This reminds me of those YouTube clips of LeBron or other basketball players who can recall the exact sequences of moves from games from years ago. Same with Magnus Carlsen on chess.

By this measure, Czapski really lived and breathed Proust. He read him during a bout of typhoid fever, for weeks he had nothing to do but read Proust, & it seems like the only way anybody reads Proust is during a period of extended recovery from illness. (Which is the only way Proust was able to write his book, by the way, more on that later...)

Just as Czapski can remember large swathes of Proust in prison, isolated from the rest of the world, Proust used isolation and solitude -- caused a delibitating illness -- to build this huge castle of detail in his memory/imagination. 

It is not actually known what illness Proust had. There's a moving story of how once he missed seeing apple blossoms so had someone drive him and had to look at them through the windows (later in life, he couldn't take smell -- someone would come in to visit him wearing a scented handkerchief & he'd order them to take it off / leave.)

"One spring day, the author of memorable passages on apple trees in bloom wanted to see an orchard once more. He decided to make a trip outside Paris in a closed car, and only by keeping the windows shut was he able to admire his beloved trees in bloom."

(You take in more than you know, and you need isolation and a steady, persistent nurturing of your imagination to bring it out.)

Like everyone else, Czapski mentions the cork-lined room:

"Buried in his work, he found the slightest noise untenable. Proust spent his final years of labor in a cork-lined room, stretched out on a bed beside a piano, the piano piled high with a mountain of books. Medicine bottles littered his night table along with sheets of paper covered in his nervous handwriting.”

Proust went completely against the fashion of his day with his long sentences, obsession with the aristocracy, etc; the fashion of the time was "telegraph prose", i.e. terse and to the point, + naturalism (Zola et al). The antithesis of this was the impressionists. Proust synthesizes these two, with his combo of interiority and precision of detail. As usual, it’s notable how many of the most popular writers of the day aren’t read so much anymore. 

"[For his contemporaries] Proust was off-putting, strange, and absolutely unacceptable".

For Proust, a single thing could bloom into a giant web of associations that could go on for pages and pages. Interconnections. Thus Proust's style is not "microscopic naturalism", it's more like what are his thoughts upon running into a fact, what comes to his mind, and tracing out that web.

Before A La Recherche, Proust was, basically, a failure; his life looks like a whole lot of nothing -- and then, suddenly, this masterpiece.

In fact, Proust's epiphany on becoming a writer comes right after he has given up completely. He decides to devote himself to socializing and enjoying time with friends, since he lacks talent. He goes out to a social function, trips on two uneven cobblestones, is immediately transported back several years to a time when he did exactly this in Venice, St Mark's Square... the whole thing is worth quoting in full:

This is all recounted in the final volume of A La Recherche, which Czapski says was written before the others.

Meta note on notetaking

A meta note, inspired both by Proust and by this book about Proust: after reading a book, when you're making notes, don't refer to the book; just write down the most interesting things that come to mind. This is a better way of digging out what actually struck you about the book; as soon as you have the book to reference, you will start looking up the bits you "should" write about, and end up aiming at comprehensiveness rather than interestingness. Your actual criterion should be whatever interested you. Later, you can fill in quotations & references.

Another thing about note-taking that the Zettelkasten guy says is that you have to write your notes in prose, as though you are writing an essay, and spend the extra effort to cite things correctly. This ensures that your notes become a growing organism and don't decay over time. (As a bonus, this makes writing easy: simply compile your notes, edit them lightly, and publish.)