Hi all,
I’ve been reflecting on Twyman’s Law this week:
“The more unusual or interesting the data, the more likely they are to have been the result of an error of one kind or another'.”
Calibrated skepticism—a BS-detecting system—can make all the difference.
Tyler Shultz, the Theranos whistleblower, was one of the few employees who was skeptical of the misleading experimental data from Theranos’s faulty products. Both Holmes and Balwani are going to prison as a result of his (and the other whisteblowers’) calibrated skepticism.
With calibrated skepticism, we can heed Feynman’s warning:
“The first principle is that you must not fool yourself, and you are the easiest person to fool.”
Have a good weekend,
—Brendan
“I waited so long
Listening for
Something to workI'm making bad decisions
Really, really bad decisions
I'm making bad decisions”
“Bad Decisions”, The Strokes
This is my second article on my series on decision making. I know, a sErIeS. If you’d like some context on the rather cheeky name, check out Would you like chips or cookies? *Speech over*
As someone who agonizes over decisions, I’ve found the reversible vs. irreversible decision framework helpful for discerning whether I should just pull the trigger or pause to further think things over.
The idea is that for a reversible decision, you should feel more comfortable proceeding with imperfect information, knowing that you can always reverse course if needed. But if the decision is irreversible, or what Jeff Bezos calls a “one-way door”, you should be as methodical and thoughtful as possible. Marriage is the stereotypical irreversible decision. It can’t be undone without much pain or cost, so it (usually) requires many years of measured consideration.
Here's a helpful heuristic from Shane Parrish summarizing the reversible vs. irreversible framework:
“When decisions are reversible, make them fast. Your biggest risk is dragging your feet and not making a decision. The cost to acquire additional information isn’t worth the effort.
When decisions are irreversible, slow them down. The biggest risk is making the wrong decision. The cost to get the information we need to reduce uncertainty is worth the time and effort.”
Shane goes on to explain how he applied this framework in his own life:
“I used this heuristic to quit my job and start my first company. While some people saw this as a huge risk, I didn’t. I realized that if the company failed, I would have learned a lot and could pretty easily return to my old job. It was a risk, but it wasn’t as big as some thought.”
This framework can also be helpful for addressing the opposite problem: over-analyzing a simple, reversible decision. As a data scientist, I sometimes run into this problem when deciding on an approach to solve a difficult problem. I’m tempted to treat the decision as irreversible and spend several hours thinking through all of the possible considerations. In truth, I don’t have that kind of time to waste. It’s better to just build a quick, imperfect POC (proof of concept) and adjust (or possibly reverse) from there. For reversible decisions, over-experimentation (POC’s, A/B tests, iteration, etc.) trumps over-thinking.
"The real measure of success is the number of experiments that can be crowded into 24 hours.”
—Thomas Edison
There is one critical challenge with the reversible vs. irreversible framework, though, and one I haven’t seen further examined. Sometimes a decision’s reversibility can change over time. This can be pernicious when a decision was made under the assumption that it would always be reversible… until it’s uncovered that it has slowly congealed, over time, into panic-inducing irreversibility.
Apologies for the biology side-bar, but I immediately think of this video of an amoeba engulfing two paramecia:
I think it’s common for many people to take a high-paying job they don’t really like as a temporary financial boost. After they habituate to a higher standard of living and develop attachments, that temporary plan can harden into a nearly irreversible lifestyle. Reversible —> Irreversible.
Reversible to irreversible is also currently playing out in China. Xi Jinping’s Zero-Covid policy was originally intended to be reversible, a harsh temporary solution to curb Covid’s spread until his re-election was in the bag. Post-election, he found Zero-Covid congealed into near irreversibility. As Joseph Sternberg recently put it, he now can only unwind Zero-Covid by, “letting the virus rip through a population with limited immunity and developed-country expectations of China’s developing-country healthcare system.”
A reversible decision might not be reversible forever. It might be a two-way door today that slowly closes over time. When using the reversible vs. irreversible framework, it’s important to re-evaluate and re-think as those decisions may move categories over time.
Thanks for reading! I love when these thoughts lead to conversations with readers. Did you find anything interesting or surprising? Reply to me and let me know.