​THE IMPORTANCE OF BAYESIANISM​

As discussed elsewhere, in many important contexts, we need to form accurate judgments about the world: this is true of medical diagnosis and treatment, of law proceedings, of policy analysis and indeed of a myriad other domains. And as discussed elsewhere, more accurate judgments often means better decisions, including in contexts where they can be a matter of life and death—such as medicine and law.

In analytic epistemology and philosophy of science, “Bayesianism” is the dominant theory of how we should form rational judgments of probability. Additionally, as I discuss elsewhere, Bayesian thinking can help us recognize strong evidence and find the truth in cases where others cannot. 

But there’s ample evidence that humans are not Bayesians, and there’s ample arguments that Bayesians can still end up with inaccurate judgments if they start from the wrong place (i.e. the wrong “priors”).

So, given the importance of accurate judgments and given Bayesianism’s potential to facilitate such accuracy, how can one be an accurate Bayesian?

Here, I argue that there are seven requirements of highly accurate Bayesians (somewhat carrying on the Steven Covey-styled characterization of rationality which I outlined here). Some requirements will be well-known to relevant experts (such as requirements 1 to 3) while others might be less so (such as requirements 4 to 7). In any case, this post is written for both the expert and novice, hoping to say something unfamiliar to both—while the familiar remainder can be easily skipped.

With that caveat, let us consider the first requirement.

• We all make judgments about the world and then use these to make important decisions
• But if someone made perfectly accurate judgments and sound decisions, would we recognize it?
• The science and philosophy of judgment and decision-making suggests that the answer is often “no”
• This post identifies 7 habits of highly rational people—habits which others often label as “irrational”:

               1. Highly rational people are confident in things despite “no good evidence” for them
               2. They are confident in things which are outright false
               3. They countenance the “impossible” and are “paranoid”
               4. They avoid risks that don’t happen
               5. They pursue opportunities that fail
               6. They are often irrational 
               7. They do things that are often “crazy” or “unconventional” 

• I lastly conclude with some evidence-based suggestions about how we can distinguish the genuinely rational from the irrational:

               1. Measure calibration
               2. Learn norms of reasoning
               3. Think in terms of expected utility theory

• This might help us to both recognize and make trustworthy judgments and decisions in our lives

​
If someone was as rational as could be—with many accurate and trustworthy judgments about the world, and with sound decisions—would we recognize it? There are reasons to think the answer is “No”. In this piece, I aim to challenge prevailing intuitions about rationality: I will argue that the philosophy and science of judgment and decision-making reveal a number of ways in which what appears to be rational diverges from what actually is rational. 

This piece takes its title from Steven Covey’s well-known book “The Seven Habits of Highly Effective People”. I will argue that, similarly, there are seven habits of highly rational people—but these habits can appear so counter-intuitive that others label them as “irrational”. Of course, the rationality of these habits might be obvious to specialists in judgment and decision-making, but I find they are often not so obvious to others of the sort for whom this piece is written.

In any case, not only are these habits potentially interesting in their own right, but recognizing them may also help to open our minds, to help us better understand the nature of rationality and to better identify the judgments and decisions we should trust—or not trust—in our own lives.

Without further ado, then, I present…

 
​          1. Highly rational people are confident in things despite “no good evidence” for them

The first habit of highly rational people is that they are sometimes confident in things when others think there is “no good evidence” for them.

​We all need to form accurate judgments about the world in many diverse and important contexts. What is the correct diagnosis for someone’s medical condition? Does someone have a crush on you? Did the defendant kill the victim? Here, I will discuss how the evidence can reveal the truth about these questions--and potentially others which you might care about--but only if we think in the right ways.

It’s well-documented that various biases can hinder us in our quest for truth. In a recently published paper in Judgment and Decision Making (freely available here), I introduce a new cognitive bias: likelihood neglect bias. Understanding this bias, and how to overcome it, can help us recognize good evidence and find the truth in numerous cases where others might not.

To show this, though, I’ll use a well-known brain-teaser which reveals this bias—the Monty Hall problem—and then I’ll apply the emerging ideas to show how we can find the truth in other realistic cases—including medicine, law and more mundane topics. You might then want to apply these ideas to other cases which you might care about.

​
​As I mentioned elsewhere, I recently published a paper arguing for calibrationism, the idea that judgments of probability are trustworthy only if there’s evidence they are produced in ways that are calibrated—that is, only if there is evidence that the things one assigns probabilities of, say, 90% to happen approximately 90% of the time.

Below is an example of such evidence; it is a graph which depicts the calibration of a forecaster from the Good Judgment project—user 3559:

​The graph shows how often the things they assign probabilities to turn out to be true. For example, the top right dot represents all the unique events which they assigned a probability of around 97.5% to before they did or didn’t occur: that Mozambique would experience an onset of insurgency between October 2013 and March 2014, that France would deliver a Mistral-class ship to a particular country before January 1st, 2015 and so on for 17 other events. Now, out of all of these 19 events which they assigned a probability of about 97%, it turns out that about 95% of those events occurred. Likewise, if you look at all the events this person assigned a probability of approximately 0%, it turns out that about 0% of those events occurred. 

However, not all people are like this, below is a particular individual, user 4566, who assigned probabilities of around 97% to things which were true merely 21% of time, such as Chad experiencing insurgency by March 2014 and so on.

​THE IMPORTANCE OF TRUSTWORTHY JUDGMENTS

We all make judgments of probability and depend on them for our decision-making.

However, it is not always obvious which judgments to trust, especially since a range of studies suggest these judgments can sometimes be more inaccurate than we might hope or expect. For example, scholars have argued at least 4% of death sentence convictions in the US are false convictions, that tens or even hundreds of thousands of Americans die of misdiagnoses each year and that sometimes experts can be 100% sure of predictions which turn out to be false 19% of the time. So we want trustworthy judgments, or else bad outcomes can occur.

How do, then, can we determine which judgments to trust—either from ourselves or others? In a paper recently published here and freely available here, I argue for an answer called “inclusive calibrationism”—or just “calibrationism” for short. Calibrationism says trustworthiness requires two ingredients—calibration and inclusivity. 
​