​  US focus on welfare and happiness

Central to the US constitution are concerns for welfare and happiness. This can be seen in several places, as italicized below:

• The Constitution states that “We the People of the United States… do ordain and establish this Constitution for the United States of America” in order to “promote the general Welfare” (among other ends such as “Tranquility”, “Justice” and “common defense”).
• Again, the Constitution states that Congress has the power to collect taxes in order to “to pay the Debts and provide for the common Defence and general Welfare of the United States”.
• Likewise, the Declaration of Independence states that “Governments are instituted among Men” to secure “certain unalienable Rights” including “Life, Liberty and the pursuit of Happiness”.
• It further declares that “whenever any Form of Government becomes destructive of these ends”, then the people have the right to “alter or abolish” that government in favor of a new government predicated on the principles and powers that “to them shall seem most likely to effect their Safety and Happiness”.

Interestingly, then, references to welfare and happiness are central to the founding principles of the United States--but not so much New Zealand.


        ​  Common focus on human rights

Nevertheless, both countries have an emphasis on rights. For example, the Bill of Rights Act 1990 enshrines numerous rights, such as freedom of speech (Section 14) and thought and freedom from torture (Section 9) or unjust deprivation of life (Section 8). Other components of New Zealand’s constitution—such as the Constitution Act 1986—focus less on the ethical dimension of government and instead specify the structure of its administrative machinery.


        ​  Common focus on public service

Perhaps needless to say, both constitutions explicitly state that the purposes of government are not self-serving. Instead, the government exists to serve the people, such as by securing human rights or happiness. The government does not exist to serve the interests of the governors at the expense of the governed, so to speak.


        ​  NZ focus on indigenous partnership

One way in which New Zealand differs, however, is that the Treaty of Waitangi—or its principles—is often regarded as a foundational component of New Zealand’s government and constitution. At least in principle, the Treaty reflects an explicit partnership between, and regard for, the indigenous people of New Zealand—the Maori people. Interestingly, according to the Treaty of Waitangi Act 1975, “Maori” is statutorily defined as “a person of the Maori race of New Zealand”, including “any descendant of such a person”. The implication is that if one has any Maori blood—no matter how little—they are legally defined as Maori under that act. 


        ​  US focus on grievances

Unlike New Zealand, however, the constitution of the United States has an explicit emphasis on grievances—27 in fact—which, in the eyes of the Founding Fathers, justified the creation of the United States. Understandably, these grievances involved violations of the very purpose government that is endorsed by the US constitution, such as refusal to assent to laws which the colonies deemed necessary “for the public good”, or the unpunished murder of two Maryland residents by British marines. That said, a lack of representation in the UK’s parliament is only one of these grievances; for that reason, it is not entirely accurate to say American independence was motivated solely by the issue of “taxation without representation”—although that issue was undoubtedly an important one.


        ​  US focus on God

Lastly, unlike New Zealand, the US constitution has an explicit emphasis on God. As one example, the Declaration concludes with a statement of reliance of God, especially since signing it was considered an act of treason that was punishable by death:
​

​My life is great in a lot of ways: I’m lucky to have a wonderful family and many projects and activities which I enjoy, for example. Thankfully,  then, I'm very happy with my life overall. 

But despite that, one thing about my life can at times be very difficult and frustrating: being a judgment and decision making scholar. 

A judgment and decision making scholar—or a “JDM scholar” for short—is someone who professionally studies judgment and decision making: that is, someone who studies how we do, or how we should, make judgments and decisions.  I’m a JDM scholar because I want to improve judgments and decisions, both the ones from myself and those from others in their domains.

And there are many domains where judgment and decision making could be improved: a JDM scholar could reduce false death sentence convictions in law, fatal misdiagnoses in medicine or disastrous policies in politics, to take a few of countless examples. 

But despite both the possibility and promise of improving judgment and decision making in these domains, there are many reasons why this can be difficult or impossible for JDM scholars.

I will explore some of them here, as well as why I think they sometimes stem from assumptions that are specious—that is, superficially plausible but actually wrong.

(A caveat, though: while this blogpost talks of "people", it is not written with any specific "people" in mind--unless otherwise stated.)
​

​JDM scholars are often generalists: they study how to improve judgment and decision making in ways that are "domain general"—that is, applicable across many domains rather than just one.

However, judgment and decision making, as it happens in practice, always happens within a specific domain, whether it is medicine, law, politics or something else.  There is simply no such thing as “applied judgment and decision making” without it “applying” to "some" specific domain. 

But because of this, any time a JDM scholar aims to improve JDM in a given domain, those within the domain could criticize the scholar as lacking any specialization in that domain.  They might say, “She’s not a doctor”, or “a lawyer”, or “a political analyst”, for example. 

Concomitant with this is often the claim that the JDM scholar lacks “experience” in the relevant domain, where experience is typically operationalized in terms of the number of years working in the given domain. 

This, then, is one reason why being a JDM scholar is difficult. Sometimes this reason is right; it's known meteorologists are often accurate about their domain, and it's not obvious to me that JDM scholars generally have much to contribute there.

But sometimes this reason is specious. 

First, the available evidence suggests years of experience is a weak predictor of judgment accuracy in some domains; for example, Philip Tetlock found that years of experience in geopolitical forecasting didn’t predict accuracy at all, while another study found it had only a mild influence. The problem with some (but not all) domains is that—as I argue elsewhere—they do not rigorously track judgment or decision quality, nor what correlates with it. It is analogous to how pre-modern physicians practiced bloodletting for millennia without rigorously studying whether it worked. Only with the development of modern medical science did we realize what truly worked and what didn’t.

The same point holds with judgment and decision making, and that brings me to a second point: the same literature shows that other things aside from domain experience or specialization matter more. For example, the literature suggests far stronger predictors of accuracy in domains like geopolitics are past records of accuracy, use of reference classes and other domain general features that are studied by JDM scholars. 

So although the JDM scholar often lacks the specialization or personal experience for a specific domain, what they do have is insight into the decades of rigorous scientific research about how to best make judgments and decisions, and sometimes in the specific domains too.

Consequently, while it is sometimes correct to claim that a JDM scholar is useless simply because they are unspecialized or inexperienced, as with meteorology, there are other cases where this claim is like a medieval bloodletting doctor claiming that a modern medical student is useless because the doctor has years of experience and specialization in practices whose efficacy has not been rigorously studied.

Unfortunately, however, it gets worse than that for another reason…
​

​
REASON #2: PEOPLE MAY THINK YOU ARE WRONG

​Not only will people think you are useless, but people will think you are wrong. 

Of course, sometimes the reason for this is that JDM scholars are wrong, and as JDM scholars, we have to be genuinely open to letting others correct our views through open-minded debate. However, a lot of the time people think you are wrong for another reason: namely, that humans are programmed to think in ways which are intuitively compelling but demonstrably unreliable. 

This comes up in countless ways—in availability biases, in representativeness biases and in many others.  In my own work, it comes up especially strongly for the so-called Monty Hall problem—a brain-teaser where my experiments found that every untrained participant got the wrong answer. And when I taught logic at Stanford, I clearly remember one of my own students insisting—so strongly—on why I was wrong about the correct answer to the very problem which I specialized in!

She, however, was one of my better critics: although she thought I was wrong, she was at least willing to engage and argue with me about why that is so.

Much of the time, people simply assume you’re wrong from a distance, foreclose any possibility of questioning or challenging that assumption, and then they move on.

This is problematic, since sometimes seeing the correct answer or reasoning in a given context requires the time, patience and open-mindedness to debate or undertake some training—and even then it might not work. For example, only a third of the trained participants in my experiments got the right answers to the Monty Hall problem (and some of them probably cheated even then!).

Note that this differs from other specializations. For cryptologists or engineers, for instance, it’s obvious to the non-specialist that they do not know what the right answer is; the average person would have no idea how to decode a given string of symbols or how to engineer a rocket ship. It’s different in JDM research because the non-specialist is psychologically programmed to think in ways where they believe they already know what the right judgment or decision is; as David Mandel puts it, they already have an intuitive theory of how to make judgments and decisions.

This is not to say those theories are never right; sometimes they are and can contribute beneficial insights, but there are also many ways in which they can potentially be wrong as well. Unfortunately, though, all the ways in which these theories could be wrong are not obvious: if they were, then JDM scholars wouldn’t need to study them for years or decades at a time. 

But unfortunately, it gets even worse than that for another reason…
​

​
REASON #3: PEOPLE HATE MAY HATE BECAUSE YOU THINK THEY'RE WRONG

​Not only will some people think you are useless and wrong, but they will sometimes hate you because you think they are wrong.  For example, if you are a good JDM scholar, then you have to think uninitiated people are wrong when they first encounter the Monty Hall problem, no matter how confident, competent or qualified they themselves think they are. And of course, many of them would dislike you for thinking they are wrong—both because it insults them and because you appear arrogant. They might also be convinced your qualifications contribute only arrogance to your outlook instead of merit to your views.

And unfortunately, this same phenomenon can come up in many other domains, because the patterns of thinking that lead to incorrect or correct reasoning in the Monty Hall and other problems are also potentially present in countless other domains, including medicine, law and many more (as I discuss here).

Yet not only would they hate you for thinking they are wrong, but they might feel that if you are right about their wrongness, then their job security, self-perceived performance or other things might be at risk—in which case they could hate you even more. What’s more, we could see how the hate would be only worse if you think there may be horrible consequences because of their wrongness—like how innocent people die because of false diagnoses or false death sentence convictions.

SUMMARY AND SOLUTIONS? 

To summarize, then, being a JDM scholar is hard for various reasons: people may think you’re useless because you’re “unspecialized” or “inexperienced” in their domain, because they are psychologically programmed to think you are wrong and because you think they are wrong in ways that can make them hate you.
 
Consequently, being a JDM scholar might entail that if you try to help others by explaining how their judgments and decisions could be improved in their domain, then they might just think you’re a "useless, inexperienced, incorrect and arrogant [insert your preferred personal swearword here]"—far from an outcome that benefits anyone.
 
That, then, brings me to the final part of this blog: what are the solutions?

So far, my answer is mainly, “Yeah, I don’t know”.

That said, I surmise two things may help when trying to improve judgment and decision making in a given domain.
 
The first is finding people in that domain with three things: 1) the time, 2) the patience and 3) the open-minded humility to explore how JDM science can benefit them. We might call such people the “champions” of those domains, like how Brian Nosek uses the term to refer to the advocates of “open science” in specific disciplines. (In fact, that more I think about it, the more I realize all my JDM complaints might apply to metascience scholars too.) Unfortunately, however, many people lack at least one of the qualities to be champions of JDM science in their domain, and sometimes through no fault of their own—often we are too busy to take on extra work, for example. Whether there are JDM champions in a given domain is not something JDM scholars can easily control, however.
 
The second thing that JDM scholars can do--and something that they can control--is how to make their communications as tactful and accessible as possible.
 
Typically such tact may often require passing two tests. The first is the "no criticism" test: for example, no one should read your work and feel criticized by it. Sometimes such criticism is appropriate—like in a typical philosophy department—but sometimes it only counter-productively alienates people. The second test is the "positivity" test: people should read your work and feel positive afterwards.

And additionally, JDM work should often pass an "accessibility" test: a non-specialist should be able to read your work and understand both what you are saying and why it might add value to their domain.
 
However, these things are not easy, and I myself am guilty of violating both of these requirements at times.
 
But as JDM scholars, perhaps aspiring to these standards might make life easier for us, as well as improving our likelihood of benefitting those we aspire to help.

​THE IMPORTANCE OF BAYESIANISM​

As discussed elsewhere, in many important contexts, we need to form accurate judgments about the world: this is true of medical diagnosis and treatment, of law proceedings, of policy analysis and indeed of a myriad other domains. And as discussed elsewhere, more accurate judgments often means better decisions, including in contexts where they can be a matter of life and death—such as medicine and law.

In analytic epistemology and philosophy of science, “Bayesianism” is the dominant theory of how we should form rational judgments of probability. Additionally, as I discuss elsewhere, Bayesian thinking can help us recognize strong evidence and find the truth in cases where others cannot. 

But there’s ample evidence that humans are not Bayesians, and there’s ample arguments that Bayesians can still end up with inaccurate judgments if they start from the wrong place (i.e. the wrong “priors”).

So, given the importance of accurate judgments and given Bayesianism’s potential to facilitate such accuracy, how can one be an accurate Bayesian?

Here, I argue that there are seven requirements of highly accurate Bayesians (somewhat carrying on the Steven Covey-styled characterization of rationality which I outlined here). Some requirements will be well-known to relevant experts (such as requirements 1 to 3) while others might be less so (such as requirements 4 to 7). In any case, this post is written for both the expert and novice, hoping to say something unfamiliar to both—while the familiar remainder can be easily skipped.

With that caveat, let us consider the first requirement.

        ​  Requirement #1: Assigning Likelihoods 

Bayesianism requires us to consider the likelihood of the evidence we have given various hypotheses. Here, a “likelihood” is a technical term that refers to the probability of the evidence assuming the truth of some hypothesis that we are uncertain about. Examples include the likelihood of an improvement in your symptoms given the uncertain hypothesis that your medication is effective, the likelihood someone would smile at you given the uncertain hypothesis that they have a crush on you, or the likelihood someone’s fingerprints would be at the scene of the crime given the uncertain hypothesis that they are guilty.

In another blogpost, I discuss evidence humans sometimes don’t do this: they “neglect” the likelihoods in ways which make them oblivious of even strong evidence when they see it. For example, in one experiment, participants are aware that the evidence is 10 times more likely if one hypothesis is true compared to another, but when they receive that evidence, they assign the hypotheses equal probabilities instead assigning the correct probability of 91% to one of them. There, I also discuss how this evidence about likelihoods can arise in more realistic and natural settings too. 


          Requirement #2: Assigning Priors 

The second requirement is to assign prior probabilities to hypotheses, especially since sometimes multiple implausible hypotheses can make the evidence likely. For example, suppose you are at home, seemingly alone, and you hear the door slam. This could be quite likely if there is an intruder in your house, in which case it should somewhat raise your probability that there is such an intruder in the house. 

However, if there is a sufficiently low prior probability of an intruder, and if other explanations have a higher prior probability—like the wind slamming the door shut—then the evidence won’t necessarily make the intruder hypotheses probable all things considered, even though it somewhat raises its probability.
​
This illustrates that when we consider the likelihood of the evidence, we also need to consider the prior probability of the relevant explanations in order to know just how confident we can be in those explanations—as I do in the three "realistic" examples here.


          Requirement #3: Updating Using Bayes’ Theorem

But Bayesianism tells us not just to consider likelihoods and to assign priors, but also to update our probabilities in specific ways once we receive our evidence. This often requires us to use Bayes’ theorem, since if the priors and likelihoods are accurate, then the only accurate posterior probability is the one prescribed by Bayes' theorem. Updating with Bayes' theorem can be complicated, but I give some examples of how this can be done here, as well as a calculator here to easily facilitate these calculations.


          Requirement #4: Using Calibrated Probabilities

So Bayesianism requires us to assign initial probabilities—including prior probabilities for hypotheses and likelihoods for evidence—and to then update our probabilities when we receive that evidence.

However, we also need to assign initial probabilities that are in some sense “correct”—an issue relating to the notorious “problem of the priors”. After all, if we assign likelihoods and prior probabilities, and these turn out to be inaccurate, then the outputs of the Bayes’ theorem will also be inaccurate. 

Elsewhere, I argue for a particular solution to this problem called calibrationism. Calibrationism basically says we should trust probabilities—including the input probabilities in Bayesian calculations—only if we have evidence that the ways we have assigned them are "well calibrated".  

Probabilities are well calibrated when they correspond to the objective frequency with which things are true; for example, here I discuss examples of forecasters who assign 97% probabilities to events which happen about 95% of the time—even events that appear “unique” and that might have looked like we couldn’t assign numerically precise probabilities to. It is in principle easy to get this evidence about how calibrated we are; I explain how to do so in this other post here.

The implication is that if we have evidence that we assign calibrated probabilities, and we input these probabilities into Bayesian calculations, then we can likewise trust the outputs of these calculations, even in cases where the calculations instruct us to be highly confident or certain in sometimes counter-intuitive ways (like with the new Monty Hall problem here).

We can also verify the trustworthiness of outputs from Bayesian calculations in the same calibrationist way too: that is, by seeing whether the resulting probabilities similarly correspond to the frequency with which things are true. (And we can mathematically prove that they will be under certain assumptions.)

There are various well-supported ways to get more accurate and calibrated probabilities, as I discuss here. One is to use statistics or frequencies where relevant: for example, in assigning prior probabilities about whether the wind or an intruder caused the door to slam, we could consider statistics about how often you or others have experienced intruders, how often the wind slams doors shut in your house and so on. (Of course, we need not meticulously record these statistics, but even our experience can roughly tell us something about what the statistics would be, such as wind slamming the door every few days when the windows are open.)


          Requirement #5: Recognizing Auxiliary Hypotheses and Successful Accommodation

But being an accurate Bayesian also requires us to understand so-called “auxiliary hypotheses” and the role they play in trying to accommodate evidence. Auxiliary hypotheses are hypotheses that are distinct from the central ones we care about but which nevertheless may have implications for how we interpret the evidence.

For example, here I use an example where there is a low likelihood of a person sitting at the table in front of you if they did not have a crush on you.

To make this likelihood appear higher, however, you might appeal to the auxiliary hypothesis that, say, that particular table just happens to be their favorite table. Now, we could suppose that this likelihood is higher: if the person does not have a crush on you and that table is their favorite table, then there is a 100% likelihood they would sit at that table. In this case, one might argue the evidence isn’t so strong when they can appeal to an auxiliary hypothesis like this.

In another article, I discuss in detail how to make sense of attempts to accommodate evidence with auxiliary hypotheses like this. However, the main message is that an attempt like this would fail if the initial probability of the auxiliary hypothesis is sufficiently low and it will succeed if the initial probability of the auxiliary hypothesis is sufficiently high.

For example, if we have no independent evidence or good reason to think that that particular table is their favorite table out of the 64 other tables, then we can assign it a 1/65 initial probability. And if that is the case, then this accommodation attempt fails because it attempts to raise the likelihood of the evidence only by appealing to an improbable auxiliary hypothesis.

We can prove this using a theorem from that paper called the theorem of successful accommodation. The theorem basically says that an auxiliary hypothesis (symbolized with a) and central hypothesis (symbolized with h1) accommodate some evidence (symbolized with e) as successfully as—or more successfully than—some alternative hypothesis h2 just in case:

​In words, this says that the accommodation attempt is successful just in case the likelihood of the evidence given h2 is less than or equal to the result of multiplying the initial probability of the auxiliary hypothesis (supposing h1 is true) and the likelihood of the evidence given h1 and the auxiliary a. 

In this case, we could suppose the evidence e is that the person sits at the table in front of you, P(e|h2) is the likelihood they would sit at that table (denoted with e) if they have a crush on you (denoted with h2), P(e|h1&a) is the likelihood they would sit at that table if they did not have a crush on you (denoted with h1) but that table was still their favorite table (denoted with a) and P(a|h1) is the probability that that table is their favorite table (assuming they do not have a crush on you). 

And if we suppose that we have no reason to think that table is their favorite table, then P(a|h1) is low, we could plug in some probabilities as below and the evidence strongly favors the hypothesis that they have a crush on you:

In this way, we could use these theorems and principles to determine when appealing to auxiliary hypotheses successfully accommodates the evidence in this and countless other cases.

​
          Requirement #6: Recognizing Consilience

Auxiliary hypotheses play an especially important role when a central hypothesis is potentially supported by multiple kinds of evidence—in which case we say the central hypothesis “consiliates” the evidence. I also discuss consilience in detail that article. 

There, a main take away is that sometimes the evidence will strongly support a hypothesis which explains multiple kinds of evidence if the probability of the alternative hypotheses are sufficiently low—even if each alternative hypothesis seems potentially plausible when considered in isolation.

I use one example to illustrate this: Darwin’s arguments in favor of evolution over special creationism (the idea that God created each species in a separate creative act).

Darwin thought evolution was probable because it could explain diverse evidence:

• on his theory, some so-called “non-aquatic” birds had webbed feet even though they didn’t go in or near water because they evolved from other birds with webbed feet that were in the water;
  
• many animals (such as frogs and lizards) share similar bone structures because they evolved from common ancestors;
  
• blind insects inside European caves are more similar to those outside of their caves than to insects in similar caves elsewhere in the world, and this is because they evolved from the insects immediately outside their caves;

...and so on and so forth.

However, Darwin noted that the special creationist could in principle offer another explanation for each piece of evidence: that God simply desired it to be that way—that God simply desired the similarities in webbed feet among specific birds, the similarities in bone structures among some animals, and the similarities among some insects—even though he created them all separately.

As I discuss in the article, the problem with the creationist explanation is that, for each piece of evidence, it appeals to an auxiliary assumption about God’s desires that—while potentially plausible in isolation—is still less than certain. And combining uncertainty with uncertainty results in even more uncertainty. 

We can use an analogy to illustrate the point: if each auxiliary hypothesis has the probability of a coin flip landing heads—50%—then the probability they are all true is much lower, even though they each might seem plausible or possible in isolation. For example, the probability of three coin-flips landing heads is 50%×50%×50%=12.5%, and one can be 87.5% confident that not all coin flips landed heads. 

Appealing to independent multiple auxiliary hypotheses to accommodate evidence likewise faces the same problem. If those auxiliary hypotheses are not sufficiently supported, then the appeal will fail, regardless of whether it’s auxiliaries about whether creationism is true, or about whether someone does not have a crush on you, or about something else—even when those auxiliary hypotheses might seem plausible in isolation. And when this is the case, then the evidence can favor a consiliating alternative explanation—like evolution—that explains multiple kinds of evidence.


          Requirement #7: Caution about Pervasive but Fallible Heuristics

Above, I’ve discussed many ideas about how to assign probabilities, but there’s ample evidence that we often assign probabilities using so-called heuristics—that is, efficient but often sub-optimal thinking processes. And while sometimes useful, they can cause us to assign inaccurate probabilities.

Below, then, are a set of heuristics to be cautious of:

• We all make judgments about the world and then use these to make important decisions
• But if someone made perfectly accurate judgments and sound decisions, would we recognize it?
• The science and philosophy of judgment and decision-making suggests that the answer is often “no”
• This post identifies 7 habits of highly rational people—habits which others often label as “irrational”:

               1. Highly rational people are confident in things despite “no good evidence” for them
               2. They are confident in things which are outright false
               3. They countenance the “impossible” and are “paranoid”
               4. They avoid risks that don’t happen
               5. They pursue opportunities that fail
               6. They are often irrational 
               7. They do things that are often “crazy” or “unconventional” 

• I lastly conclude with some evidence-based suggestions about how we can distinguish the genuinely rational from the irrational:

               1. Measure calibration
               2. Learn norms of reasoning
               3. Think in terms of expected utility theory

• This might help us to both recognize and make trustworthy judgments and decisions in our lives

​
If someone was as rational as could be—with many accurate and trustworthy judgments about the world, and with sound decisions—would we recognize it? There are reasons to think the answer is “No”. In this piece, I aim to challenge prevailing intuitions about rationality: I will argue that the philosophy and science of judgment and decision-making reveal a number of ways in which what appears to be rational diverges from what actually is rational. 

This piece takes its title from Steven Covey’s well-known book “The Seven Habits of Highly Effective People”. I will argue that, similarly, there are seven habits of highly rational people—but these habits can appear so counter-intuitive that others label them as “irrational”. Of course, the rationality of these habits might be obvious to specialists in judgment and decision-making, but I find they are often not so obvious to others of the sort for whom this piece is written.

In any case, not only are these habits potentially interesting in their own right, but recognizing them may also help to open our minds, to help us better understand the nature of rationality and to better identify the judgments and decisions we should trust—or not trust—in our own lives.

Without further ado, then, I present…

 
​          1. Highly rational people are confident in things despite “no good evidence” for them

The first habit of highly rational people is that they are sometimes confident in things when others think there is “no good evidence” for them.

One case where this shows up extremely clearly is the Monty Hall problem, as I discuss in detail in a blogpost here.

In the problem, a prize is randomly placed behind one of three doors, you select a door and then the gameshow host—named Monty Hall—will open one of the other doors that does not conceal the prize. If you select a door and that door conceals the prize, then Monty Hall will open either of the other two doors with an equal likelihood. But if the door you select does not conceal the prize, then Monty Hall must open the only other door that does not conceal the prize and which you did not select.

In these circumstances, as I explain in the blogpost, if you select door A and Monty Hall opens door C, then there’s a 2/3 probability that door B conceals the prize. In this case, then, door C being opened constitutes “evidence” that door B conceals the prize. Furthermore, let us consider an adaptation called the “new Monty Hall problem”: in this case, door C would be opened with a 10% likelihood if door A conceals the prize, in which case there’s provably a 91% probability that door B conceals the prize after door C is opened. In this version, the truly rational response is to be very confident that door B conceals the prize.

But despite this, in my experiments, everyone without training who encountered these problems got the wrong answer, and the vast majority thought door B had only a 50% probability of concealing the prize in both versions of the problem. This effectively means they thought there was no good evidence for door B concealing the prize when there in fact was!

What’s more, the studies found that not only did participants not recognize this good evidence, but they were also more confident in their incorrect answers. Compared to participants who were trained and more likely to get the correct answers to these problems, the other participants were on average both more confident in the correctness of their (actually incorrect) answers and they thought they had a better understanding of why their (actually incorrect) answers were correct.

What this shows is that truly rational people may recognize objectively good evidence for hypotheses where others think there is none—leading them to be confident in things in ways which others think are irrational. In the blogpost, I also discuss some more realistic scenarios where this in principle could occur—including some from medicine, law and daily life.


          2. They are confident in things that are outright false

But even if someone is rationally confident in something, that thing might be false a particular proportion of the time.

In fact, according to one norm of trustworthy judgments, a perfectly accurate person would often be 90% confident in things that are false approximately 10% of the time. In other words, a perfectly accurate person would be “well calibrated” in the rough sense that, in normal circumstances, anything they assign 90% probabilities to will be true approximately 90% of the time, anything they assign 80% probabilities to will be true approximately 80% of the time and so on.

We can see this when we look at well calibrated forecasters who might assign high probabilities to a bunch of unique events, and while most of those will happen, some of them will not—as I discuss in detail here. Yet if we focus on a small sample of cases, they might look less rational than they are since they will be confident in things that are outright false.


          3. They countenance the “impossible” and are “paranoid”

However, studies suggest many people—including experts with PhDs about their domain, doctors, jurors and the general public—are not so well calibrated. One example of this is miscalibrated certainty—that is, when people are certain (or virtually certain) of things which turn out to be false.

For instance, Philip Tetlock tracked the accuracy of a group of political experts’ long-term predictions, and he found that out of all the things they were 100% certain would not occur, those things actually did occur 19% of the time. Other studies likewise suggest people can be highly confident in things which are false a significant portion of the time.

But a perfectly rational person wouldn’t be so miscalibrated about these things which others are certain about, and so they would assign higher probabilities to things which others would think are “impossible”. For example, a perfectly calibrated person would perhaps assign 19% probabilities to the events which Tetlock’s experts were inaccurately certain would not happen—or they might even assign some of them much higher probabilities, like 99%, if they had sufficiently good evidence for them. In such a case, the perfectly rational person would look quite “irrational” from the perspective of Tetlock’s experts.

But insofar as miscalibrated certainty is widespread among experts or the general public, so too would be the perception that truly rational people are “irrational” in virtue of them countenancing what others irrationally consider to be “improbable” at best or “impossible” at worst.

Furthermore, when one has miscalibrated certainty about outcomes that are “bad”, not only will a rational person look like they believe in the possibility of “impossible” outcomes, but the rational person will look irrationally “paranoid” in doing so since the supposedly “impossible” outcomes are bad.


          4. They avoid risks that don’t happen

But not only will a rational person look “irrational” or “paranoid” in virtue of thinking the “impossible” is possible or even (probably) true, but they will also act to reduce risks which never actually happen.

This is because our leading theory of rational decision-making claims that we should make decisions not just based on how probable or improbable outcomes are, but rather based on the so-called expected utility of those outcomes, where the expected utility of an outcome is the product of both its probability and how good or bad it is.

This sometimes entails that we should avoid decisions if there is an improbable chance of something really bad happening. For example, it can be rational to avoid playing Russian roulette even if the gun is unlikely to fire a bullet, simply because the off-chance of a bullet firing is so bad that this outcome has high expected negative utility. 

Likewise, for many other decisions in life, it may be rational to avoid decisions if they have improbable outcomes that are sufficiently bad. This consequently means a rational person could often act to avoid many risks that never actually happen.

But as is well-known, people often evaluate the goodness of a decision based on its outcome, and if the bad thing does not happen, the average person might evaluate that decision as “irrational”. 

This kind of thing could happen quite often too: for example, if for every highly negative outcome with a probability of 10% and which a rational decision-maker would avoid, then they will be avoiding negative outcomes which simply don’t happen 90% of the time, potentially making them look quite irrational. 

The situation is even worse if the evaluator has the "miscalibrated certainty" we considered earlier, and the outcome not only does not happen, but rather it looks like it was always “impossible” from the perspective of the evaluator.


          5. They pursue opportunities that fail

But the same moral holds not only for decisions that avoid risk, but also for decisions that pursue reward. For example, a rational decision-maker might accept an amazing job offer which has merely a 10% chance of continued employment if the possibility of continued employment is sufficiently good. But of course, that decision to accept that job has a 90% chance of resulting in unemployment, potentially making the decision again seem like a “failure” if the probable happens.

More generally, a rational decision-maker would pursue risky options with 90% chances of failure if the options are sufficiently good all things considered: it is like buying a lottery ticket with a mere 10% chance of winning but with a sufficiently high reward.

But again, the rational decision-maker could look highly “irrational” in the 90% of cases where those decisions lead to the less-than-ideal outcomes. 

In any case, what both this habit and the preceding one have in common is that rational decision-making requires making decisions that lead to the best outcomes over many decisions in the long-run, but humans often evaluate decision-making strategies based on mere one-off cases.


          6. They are often irrational

Despite that, though, arguably any realistic person who is as rational as could be would still be genuinely irrational to some degree. This is because our dominant theory of judgment and decision-making—dual process theory—entails that while we often make reflective judgments and decisions, there are countless situations where we do not and simply cannot. 

Instead, the literature commonly affirms that everyone employs a set of so-called heuristics for judgment and decision-making which—while often adequate—also often lead to sub-optimal outcomes. Consequently, even if someone was as rational as could be, they would still make irrational judgments and decisions in countless other contexts where they cannot be expected to rely on their more reflective faculties.

If we then focus solely on these unreflective contexts, we would get an inaccurate impression of how rational they are overall.


          7. They do things that are often “crazy” or “unconventional”

All of the preceding thoughts then entail that rational people may do things that seem “crazy” or “unconventional” by common standards: they might believe in seemingly impossible things, or act to reduce risks that never happen, or pursue opportunities that never materialize, and so on. This might express itself in weird habits, beliefs, or in many other ways.

But this shouldn’t be too surprising. After all, the history of humanity is a history of common practices which later generations appraise as unjustified or irrational. Large portions of humanity once believed that the earth was flat, that the earth was at the center of the universe, that women were supposedly incapable or unsuitable for voting, and so on and so forth.

Have we then finally reached the apex of understanding in humanity’s evolution, a point where everything we now do and say will appear perfectly rational by future standards? If history is anything to go by, then surely the answer is “No”. And if that is the case, then perhaps the truly "rational" will be ahead of the rest—believing or doing things which seem crazy or irrational by our currently common standards.
​

What I hope to have conveyed, then, is just how frequently our untrained intuitions about what is rational may diverge from what is truly rational: what’s rational might appear “irrational”, and vice versa. In a world where these intuitions might lead us astray, then, how can we really tell rational from irrational, accurate from inaccurate or wisdom from unwisdom?

Some common rules of thumb might not work too well. For example, sometimes the evidence fails to find that years of experience, age or educational degrees improve accuracy, at least in domains like geopolitical forecasting. 

What follows, then, are some suggestions which I think are supported by the evidence:


          Suggestion #1: Measure calibration

First, track the calibration of the judgments you care about—whether they are yours or others. I provide some tools and ideas for how to do this here. This can help us to put things in perspective, to avoid focusing on single-cases and to detect pervasive miscalibration which can afflict our decision-making. And as other studies suggest, past accuracy is the greatest predictor of future accuracy.


          Suggestion #2: Learn norms of reasoning

Additionally, I would suggest learning and practicing various norms of reasoning. These include the evidence-based suggestions for forming more accurate judgments in my book Human Judgment, such as practicing active open-minded thinking and thinking in terms of statistics. It also includes other norms, such as so-called “Bayesian reasoning” which can produce more accurate judgments in the Monty Hall problem and potentially other contexts, as I discuss here and here.


          Suggestion #3: Think in terms of expected utility

Finally, when evaluating the rationality of someone’s decisions, think in terms of expected utility theory. Expected utility theory is complicated, but here is a potentially helpful introduction to it (and from my former PhD advisor—a really awesome person!). In short, though, expected utility theory requires us to ask what probabilities people attach to outcomes, how much they value those outcomes and—on my preferred version of it—whether their probabilities are calibrated and their values are in some sense objectively “correct”. Then, we can ask whether they are making decisions which lead to the best possible outcomes in the long-run.

In these ways, I think we can better tell what’s rational from what’s not in a world where our intuitions can otherwise lead us astray.

​We all need to form accurate judgments about the world in many diverse and important contexts. What is the correct diagnosis for someone’s medical condition? Does someone have a crush on you? Did the defendant kill the victim? Here, I will discuss how the evidence can reveal the truth about these questions—and potentially others which you might care about—but only if we think in the right ways.

It’s well-documented that various biases can hinder us in our quest for truth. In a recently published paper in Judgment and Decision Making (freely available here), I introduce a new cognitive bias: likelihood neglect bias. Understanding this bias, and how to overcome it, can help us recognize good evidence and find the truth in numerous cases where others might not.

To show this, though, I’ll use a well-known brain-teaser which reveals this bias—the Monty Hall problem—and then I’ll apply the emerging ideas to show how we can find the truth in other realistic cases—including medicine, law and more mundane topics. You might then want to apply these ideas to other cases which you might care about.

​In the Monty Hall problem, a prize is randomly placed behind one of three doors. You select a door, which remains closed for the time being. If the prize is behind the door which you selected, then the gameshow host, Monty Hall, will randomly open one of the other two doors with an equal probability. But if the prize is behind one of the doors which you did not initially select, then he will open the other door which does not conceal the prize and which you did not select.

In my experiments, participants read a description of the problem and then are asked to imagine that they select door A and that Monty Hall opens door C, like in the very high-quality picture below:

​The question, then, is this: what is the probability that the prize is behind door B after Monty Hall opens door C?

Most people think the answer is 1/2 or 50%. But actually, the correct answer is that it has a 2/3 or 66% probability of concealing the prize.

However, virtually no one without training gets it right when they first encounter the problem, including my once younger self and all the 50 participants in one group of my experiments, for instance. 

In fact, it is because the right answer is so counter-intuitive that one cognitive scientist has called the Monty Hall problem “the most expressive example of cognitive illusions or mental tunnels in which even the finest and best-trained minds get trapped” (Piattelli-Palmarini, 1994a, p. 161).

To see how the correct solution is indeed correct, though, let us consider the probabilistic explanation before applying the same ideas to some more important real-life problems.
​
​

​From a probabilistic standpoint, the sole reason door B more probably conceals the prize than door A is because of the likelihood that Monty Hall would open door C if door B conceals the prize. Here, a “likelihood” is a technical term used in probability theory: it refers to the probability of the evidence that we are certain about if we assume the truth of the hypothesis that we are not certain about. In this case, we are certain about door C being opened, but not about which door conceals the prize, so the likelihood refers to door C being opened given that a particular door conceals the prize.

The likelihood of the evidence differs for whether door A or door B conceals the prize. If door A conceals the prize, then there’s a 50% likelihood that Monty Hall would open door C, since he could have opened either door B or C. But if door B conceals the prize, then there’s a 100% likelihood that Monty Hall would open door C, since he cannot open the door that you selected (door A) nor the door that conceals the prize (door B), so he must open the only remaining door (door C).

From a probabilistic perspective, the fact that door C being opened is twice as likely if door B conceals the prize (and the fact the prize was randomly placed behind one of the doors) makes it the case that door B is twice as probable to conceal the prize than door A. Mathematicians calculate this using Bayes’ theorem as below (although you don’t need to know Bayes’ theorem or anything aside from the importance of likelihoods here):

Again, the formula isn’t too important, but what is important is that door B more probably conceals the prize because of the likelihoods—because P(c|A)=50% and P(c|B)=100%.

What my experiments showed, however, is that most participants were aware of what the likelihoods were, but they didn’t realize that these likelihoods favored one hypothesis over another (and for other explanations of this and the Monty Hall problem, see the Appendix below). This, then, is likelihood neglect bias: by definition, it is being aware that the evidence is more likely given one hypothesis compared to another, all while failing to realize that the evidence therefore raises the probability of the former hypothesis relative to the latter.

Technically, likelihood neglect bias is a violation of what is known as the law of likelihood—a law of probability which states (roughly speaking) that if the evidence is more likely given one hypothesis compared to another, then the evidence necessarily raises the probability of the former hypothesis relative to the latter. The strength which the evidence provides for the hypothesis is measured by the so-called likelihood ratio—that is, how much more likely the evidence is assuming one hypothesis compared to the other. In the original Monty Hall problem, the likelihood ratio is 100%/50%=2, meaning that door C is twice as likely to be opened if door B conceals the prize than if door A conceals the prize. 

Overcoming likelihood neglect bias requires us to apply the law of likelihood in our assessment of the evidence, thereby raising our probabilities for hypotheses and getting closer to the truth in cases where others do not.
​

​In a sense, then, overcoming likelihood neglect bias can potentially help us recognize strong evidence and find the (probable) truth where others can not.

To see how this is, though, we could consider another innovation of the experiments: they showed that not only do people get incorrect probabilities for the original Monty Hall problem, but they also get the incorrect probabilities for an adaptation of the problem where there’s objectively strong evidence to favor door B over door A.

More specifically, in another experiment, participants were presented with what I call the “new Monty Hall problem”, a problem where the likelihood of door C being opened given that door A conceals the prize is merely 10%—not 50% like in the original Monty Hall problem. In this case, the likelihood ratio changes to 100%/10%=10, meaning door C is 10 times more likely to be opened if door B conceals the prize than if door A conceals the prize.

Under these conditions, one can prove with mathematics and simulations that the probability that door B conceals the prize after door C is opened if 91%—as I show in the paper and its supplementary materials.

But despite this, 84% of participants—or 21 of 25 of them—thought the evidence was effectively irrelevant and that door B had merely a 50% probability of concealing the prize. (And the four other participants gave flawed answers for different reasons too.)

Importantly, though, most of the participants were aware of the likelihoods—they knew that door C being opened was much more likely given that door B conceals the prize than that door A conceals the prize—but remarkably, this did not affect their probabilities for the hypotheses at all.

This, again, is likelihood neglect bias: it is realizing the evidence is more likely given one hypothesis than another, all while not realizing that this evidence favors that hypothesis over the other.

What this shows, then, is that sometimes the world can present us with strong evidence to favor some hypotheses over others, yet we might be oblivious of how this is so if we neglect the likelihood of the evidence.

The evidence could potentially favor hypotheses to the point of virtual certainty too, especially if, say, there are multiple pieces of evidence where we neglect the likelihoods. For instance, if there are just two pieces of evidence as strong as door C being opened in the new Monty Hall problem, then one can be 99% certain of the relevant hypothesis—and this is in a case where we know humans fail to think the evidence is relevant at all!

The evidence could in principle be so strong as to virtually reveal the truth, but we might fail to realize this if we neglect the likelihoods and do not think about the evidence in the right ways.
​

To see how this could be the case in more important and realistic scenarios, let us consider a few examples, each of which are adapted from real situations where subsequent evidence supported the favored hypothesis (albeit with the details changed to preserve anonymity).


          Example #1: Medical diagnosis and treatment

Suppose Tabatha has a debilitating medical condition but does not know for sure what is causing it. Suppose she furthermore has some hunches and tries a particular therapy with some suggestive but severely limited studies supporting its efficacy. She then assigns the therapy an initial 10% probability of working, but she tries it for a few months, and she starts feeling much better.

Now, if the therapy was effective, we can suppose there would be a 100% likelihood she would have a remarkable improvement (and I will discuss how we can assign these values later, although the exact values of the likelihoods do not matter so much as the general ideas). 

On the other hand, though, if the therapy was ineffective, how do we determine the likelihood she would have recovered by chance? One demonstrably accurate method of assigning probabilities is by looking at the frequency with which she has had similar improvements in her condition in the past, since study after study after study have suggested the past can help us assign probabilities. 

To do this, then, suppose Tabatha has had the disease for 3 years and has never seen a comparable improvement in her condition before. Then, using this historical information, she could accurately assign a likelihood of her remarkably improving given chance that is 1/156 or approximately 0.6%, since she had not improved remarkably for any of the past 156 months. In this case, we then have a likelihood ratio of 100%/0.6%=166, meaning the recovery is about 166 more times likely if the therapy is effective than if it is not.

If this is the case, then using Bayes’ theorem (like one can in this post here), the probability that the therapy is effective given her remarkable recovery is approximately 95%, and she can be confident that the therapy works. 

However, if she commits likelihood neglect and instead attributes her improvement to mere “chance”, then she could fail to realize the implications of the evidence which strongly favors one hypothesis over another. 

Of course, one alternative explanation is that she is improving merely because of a placebo effect: perhaps her trying the treatment gives her the hope or belief that she will improve, and this itself causes the improvement. Placebo effects are real, and that is why there are widespread experimental protocols to rule them out.

But in this case, we could suppose she has evidence to discredit this explanation. Perhaps, for example, she has had years of trying alternative therapies, some of which she was even more confident in because they were recommended by experts. But if those therapies did not improve her condition despite their potential for a placebo effect, then the probability of the placebo effect would be undermined by this evidence of historical frequencies.

Likewise, when getting into details, there may also be evidence to undermine other alternatives, in which case Tabatha can be virtually certain the therapy is effective if she avoids likelihood neglect.


          Example #2: Someone having a crush on you

Let us use another example which has—for educational purposes—turned out to engage my students’ interest in the past: whether someone has a crush on you.

Suppose you and another person notice each other a lot in a dining hall at your college; you both make eye contact a lot and you assign a conservative 30% initial probability to them having a crush on you, since you have historically noticed that at least 30% of people who make eye contact with you like that have turned out to like you.

Now suppose that one lunch time, the dining hall is virtually empty, there are 65 free tables around, but when this person comes in to get lunch, they choose to sit at the table directly in front of you. 

You are wondering, then, if them sitting in front of you means they like you, and again, the likelihoods could provide an answer.  In particular, suppose that if they do not like you, then they could have sat at any of the 65 tables, and so the likelihood of them sitting at the table in front of you is 1/65 or 1.5%—very unlikely.

However, if they like you, then we could suppose the likelihood of them sitting at the table in front of you is about 70%—and much more likely—because they might want to sit there to get your attention, to encourage you to approach them and start a conversation or whatever. 

If that is the case, then the likelihood ratio would be 70%/(1/65)=45.5 and would again mean that the evidence strongly favors the hypothesis that they like you—in this case with a probability of 95%. But again, if we merely attribute the occurrence to “chance” while neglecting the likelihoods, we might similarly not realize how strong the evidence is.

​
          Example #3: Law and the attribution of a crime

Let us use another example that is based on an actual scenario (albeit with some details modified), this time concerning law. Jemaine is an investigative journalist based in central Africa, and he publishes a news piece that is critical of a local gang. Six months later, his house is burned down. However, it is known that some in the gang did not like the critical news piece, and there have been rumors they have retaliated against others before. What, then, is the probability that Jemaine’s house was sabotaged?

Again, if we rely on well-documented heuristics and assign the “accident” hypothesis a high probability merely because it seems plausible or consistent with the evidence, then we might commit likelihood neglect and fail to recognize evidence for the truth. To think more carefully about the case, we could assign initial probabilities and then consider likelihoods.

There are multiple rumors that the gang has retaliated against critics before, and so suppose we assign an initial 10% probability to the hypothesis that Jemaine would experience retaliation before updating on the evidence about the house burning down. (That said, if the gang is sufficiently powerful and motivated, it could be very difficult to determine the true frequency with which they successfully execute covert arson.) 

Then, we can think about the likelihood of Jemaine’s house burning down given that he was or wasn’t retaliated against. Suppose that if he was retaliated against, there’s a 20% likelihood he would have his house burned down, since there are also other ways in which he could have been retaliated against—such as a plane crash, staged suicide and so on. On the other hand, suppose that if he was not retaliated against, then there is a 1/10,000 likelihood his house would burn down, since although house burnings occasionally happen, they are still exceedingly improbable relative to the vastly larger number of cases where house burnings never occur. In this case, the likelihood ratio would be 20%/(1/10,000)=2,000, meaning the house burning down is 2,000 times more likely if he was retaliated against.

Again, then, using Bayes’ theorem, we can be 99.5% confident Jemaine’s house was burned down by the gang in central Africa. And in the real-life case which this example was adapted from, an informant close to the “gang hierarchy” did indeed confirm Jemaine was retaliated against despite the gang’s attempts to cover it up. But again, if we neglect likelihoods and rely merely on what seems “plausible” or “consistent” with the evidence, we could fail to recognize strong evidence when it can reveal the truth to us.

​Of course, in the examples above, some people might naturally think the evidence is relevant and not neglect the likelihoods, and it’s an open-question as to exactly when people neglect likelihoods. However, from experience, I know that some people can neglect likelihoods in cases like these, even if others do not.

In any case, the point is that the experimental evidence suggests we can sometimes neglect likelihoods, potentially in more important contexts too.

That said, an important warning: even if we do not neglect likelihoods, there are many ways we might try to reason about them unsuccessfully. For example, we might assign incorrect likelihoods in the first place, or we might not properly take into account so-called “auxiliary hypotheses”, or we might not use valid tools like Bayes’ theorem to correctly revise our views given information about likelihoods. I discuss how to avoid these and other errors in another post here. 

For now, though, here’s an appendix of other explanations for the Monty Hall problem.
​

​Above, I gave an explanation of the Monty Hall problem in terms of probability theory, but there’s other ways to explain the correct answer to the problem too. 


          Explanation #2: The Many Doors Adaptation

One style of explanation comes from the genius Marilyn vos Savant.

Suppose we adapt the problem: there are 1,000 doors instead of 3, Monty Hall randomly places a prize behind one of the doors, you select one, and then he opens every other door except one. Suppose you select, say, door 358, and then Monty Hall opens every other door except door 721. 

Intuitively, it’s much, much more probable that door 721 conceals the prize than that the door you happened to select conceals the prize. 

In fact, the probability is provably 99.9%, and the reason is again because of the likelihoods: if door 721 conceals the prize, there’s a 100% likelihood Monty Hall would open every other unselected door except door 721, so P(721 is not opened|door 721 conceals the prize)=100%. But if the door you selected conceals the prize, then there’s a 1/999 likelihood that Monty Hall would open every door except 721 —simply because he had 999 other doors he could not have opened, and so P(721 is not opened|door 358 conceals the prize)=1/999. 

The point is that this adaptation and others like it show that as we change the number of doors—and consequently the relevant likelihoods—we can see how this affects the probability of the relevant hypotheses. And if we decrease the number of doors enough, we will eventually get to just three doors and the 2/3 probability that door B conceals the prize. 


          Explanation #3: The Mental Simulations Approach

A third way to explain the problem is with what I call the mental simulations approach in the recent article. This approach involves running a number of mental simulations of the Monty Hall problem and then proportioning the mental simulations by the relevant probabilities. 

So suppose we first run, say, 30 mental simulations—that is, we imagine that the Monty Hall problem happens 30 times. And since there is a 1/3 probability that a given door conceals the prize at the beginning, we make it so 1/3 or 10/30 simulations are those where a given door conceals the prize. Then, since the likelihoods mean door C is opened 50% of the time when door A conceals the prize and 100% of the time when door B conceals the prize, we make it so Monty Hall opens door C in 50% and 100% of those simulations respectively. We could depict this as follows:

What these mental simulations show is that 10 out of 15 times, or 2 out of 3 times, when door C is opened, it is because door B conceals the prize, so door B conceals the prize with a probability of 2/3.

The experiments further showed that training in the mental simulations approach caused 32% or 16 of 50 participants in one group of the experiment to get the right probabilities for the Monty Hall problem. Of course, this isn’t perfect, but it’s at least better than the other group where none of the participants got the correct probabilities.
​

          Explanation #4: Computer Simulations

Another explanation of the problem is actually to just simulate the problem many times using a computer. I provide some code and instructions for how you can do this yourself in Appendix C of the paper here. If you run the simulations, you can see that door B will conceal the prize 2 out of 3 times in the original Monty Hall problem and 10 out of 11 times in the new Monty Hall problem. Apparently, it was only after seeing computer simulations like these that the famous mathematician Paul Erdős finally accepted the 2/3 solution to the problem and accepted that his initial 1/2 intuition was incorrect. 

​
​As I mentioned elsewhere, I recently published a paper arguing for calibrationism, the idea that judgments of probability are trustworthy only if there’s evidence they are produced in ways that are calibrated—that is, only if there is evidence that the things one assigns probabilities of, say, 90% to happen approximately 90% of the time.

Below is an example of such evidence; it is a graph which depicts the calibration of a forecaster from the Good Judgment project—user 3559:

​The graph shows how often the things they assign probabilities to turn out to be true. For example, the top right dot represents all the unique events which they assigned a probability of around 97.5% to before they did or didn’t occur: that Mozambique would experience an onset of insurgency between October 2013 and March 2014, that France would deliver a Mistral-class ship to a particular country before January 1st, 2015 and so on for 17 other events. Now, out of all of these 19 events which they assigned a probability of about 97%, it turns out that about 95% of those events occurred. Likewise, if you look at all the events this person assigned a probability of approximately 0%, it turns out that about 0% of those events occurred. 

However, not all people are like this, below is a particular individual, user 4566, who assigned probabilities of around 97% to things which were true merely 21% of time, such as Chad experiencing insurgency by March 2014 and so on.

This graph shows that this person is poorly calibrated. 

Studies have shown people can be more or less calibrated for a range of domains, including medical diagnoses and prognoses, general knowledge about past or current affairs, geopolitical topic—virtually anything.

Calibrationism then says that judgments of probability are trustworthy only if we have evidence that they are produced by ways of thinking that are well calibrated—that is, by ways of thinking that look more like the first forecaster’s calibration graph than like the second forecaster’s calibration graph. 

A number of studies suggest humans can be miscalibrated to some extent. For example, Philip Tetlock found that some particular experts assigned 0% probabilities to events that actually did happen 19% of the time. Another study found that some students assigned probabilities of about 95% to things that we true only 73% to 87% of the time (depending on which university these particular students were from). A variety of other studies provide evidence of miscalibration in other contexts, as discussed in my book Human Judgment.
​

​​So, the evidence suggests not only that miscalibration is fairly common; what’s more, various studies also suggest that we can be unaware of just how miscalibrated or inaccurate we are.

Because of this, it is worth gathering evidence about how calibrated the judgments are that we rely on, and calibrationism says this is necessary before we can fully trust them.

Gathering this is often entirely possible, though, and the purpose of this post is to share some thoughts about how this is so.

For a start, we can plug some values into a spreadsheet. Below is an adaptation from a spreadsheet which I’ve used to track my calibration for almost a year now and which can be downloaded here:

​It might seem odd to do this, but I don’t think it’s irrational. I find myself often making probability judgments about the world, and I found it both fun and insightful to take a moment to make this tool to systematically track the accuracy of the judgments which I make regardless.  (Plus, as a cognitive scientist who specializes in judgment accuracy, it makes sense to do something like this since studying this kind of thing is literally my job!)

In the spreadsheet above, I have anonymized the content so it does not concern other people, but the topics include a broad variety of things: examples include past or current medical diagnoses and prognoses, interpersonal relationships with friends or others, career developments, events in international affairs and many other things. In that sense, it’s a truly “domain general” set of judgments, to use a term from my book on Human Judgment. Some aspects of the spreadsheet are self-explanatory (e.g. the “Topic”, “Description” and “Rationale” columns) while others are explained elsewhere (e.g. “Brier” and “Resolution” scores, both of which are discussed in the second chapter of my book here).

What matters, though, is that the spreadsheet automatically produces a table and graph to depict the calibration of a set of judgments as follows (note errors like “#DIV/0” show up in the categories where I haven’t made any judgments*):

Like the other calibration graphs, the x-axis shows the probabilities I assign to things and the y-axis shows the frequency with which those things turn out to be true. So far, the judgments are pretty well calibrated for some categories: if you take the things I have been around 97.5% confident in, for example, about 92% of those things are true (although I will say more on this shortly), while the things I have assigned 70% probabilities to are true about 71% of the time.  Other categories may appear less well calibrated: for example, out of the things I assign 80% to, all of those things are true, thus possibly indicating a degree of underconfidence. 

However, the statistics for some of the categories may be misleading, and so there are a few things to pay attention to when measuring calibration—there are several desiderata or ideal features, we might say.

          Desideratum #1: Multiplicity

One of which is what I call multiplicity—which concerns the number of judgments we have. When measuring calibration, we often need multiple judgments that can be assessed for accuracy. 

For instance, even a person who is perfectly calibrated will assign probabilities of 90% to things which are false 10% of the time, and if we only have one of their 90%-judgments for an outcome which happens to be false, then they might look terribly miscalibrated when they are actually not. 

For example, in the spreadsheet above, I assigned a 95% probability to moving into a particular apartment ahead of time; then the day to move in came, I was on my way to get the keys, and then the apartment-owner said an issue came up and I couldn’t move in. That was the one thing I assigned a high probability to in that category which turned out false, and if we focus on just that instance which (I would think) is objectively improbable, I might look much more miscalibrated than I actually am.

More generally, the so-called “law of large numbers” implies that if, say, the things we are 90% confident in would be true 90% of the time, then our actual judgments will likely converge to this proportion—but only with a sufficiently large number of judgments. It’s like how the probability of a coin landing heads is 50%, but the actual numbers reflect this only with multiple coin flips: three coin flips might land heads 100% of the time, or 66% of the time, but many more flips will eventually reveal the true probability of 50%. 

Consequently, the number of judgments in a given category is depicted in the calibration graph. There are 13 judgments in the 97.5% category and only 2 in the 60% category, meaning that the observed calibration is much more likely to reflect the true calibration for the former category than for the latter.

          Desideratum #2: Independence

However, observed calibration is more likely to reflect one’s real underlying accuracy when another desideratum is met: namely, that the things one makes judgments about are probabilistically independent of each other. Suppose, for example, that the probability the Republican candidate will win the next US presidential election is 70%. Then, if they have a dataset containing only 100 of the same judgments that are each 70% confident in his success, then either all of the predictions are true or none of them are, since they all concern the exact same thing.

More generally, then, a set of judgments could look unduly inaccurate merely because the judgments are dependent on each other, either because they are about the same topic (e.g. repeating the same forecast in a forecasting tournament), or because they are about closely related topics which affect each other (e.g. the number of COVID deaths and the number of COVID infections in a country).

One way to guarantee the right kind of independence, then, is to make sure that the judgments each concern an independent topic: you might have one judgment about the US presidential election, another about the culture of Mexico, another about a question of geography or whatever else one might care about.

          Desideratum #3: Resolvability

But beyond that, however, we would ideally assess accuracy using questions that are "resolvable", meaning we can know with virtual certainty what the correct answer is. Sometimes we can do this, like when predicting a future outcome which will eventually be knowable or when using things like DNA evidence to figure out what happened in the past.

But sometimes the questions we are concerned with are not always resolvable. For example, one study assessed the accuracy of judgments from experts vs. non-experts by comparing the two groups’ judgments to estimates from other experts.  But if the “other experts” have inaccurate judgments, as experts sometimes do, then this would give a skewed and possibly biased picture of judgment accuracy.

In any case, the upshot is that it’s very feasible to measure the calibration of multiple, independent and resolvable judgments using the free spreadsheet tool above. This could potentially help us determine both the trustworthiness of those judgments and—by extrapolation--those in other contexts where the true outcomes might not be knowable.



*Footnote: In my calibration graph above, I've "flipped" probabilities like 10% in false propositions to become 90% in true propositions so I have a larger sample of judgments for the categories above 50%.

​THE IMPORTANCE OF TRUSTWORTHY JUDGMENTS

We all make judgments of probability and depend on them for our decision-making.

However, it is not always obvious which judgments to trust, especially since a range of studies suggest these judgments can sometimes be more inaccurate than we might hope or expect. For example, scholars have argued at least 4% of death sentence convictions in the US are false convictions, that tens or even hundreds of thousands of Americans die of misdiagnoses each year and that sometimes experts can be 100% sure of predictions which turn out to be false 19% of the time. So we want trustworthy judgments, or else bad outcomes can occur.

How do, then, can we determine which judgments to trust—either from ourselves or others? In a paper recently published here and freely available here, I argue for an answer called “inclusive calibrationism”—or just “calibrationism” for short. Calibrationism says trustworthiness requires two ingredients—calibration and inclusivity. 
​

​​​The calibrationist part of “inclusive calibrationism” says that judgments of probability are trustworthy only if they are produced by ways of thinking that have evidence of their calibration. Here, “calibration” is a technical term that refers to whether the probabilities we assign to things correspond to the frequency with which those things are true. Let us consider this with an example. 

Below is a graph which depicts the calibration of a forecaster from the Good Judgment project—user 3559:

​The graph shows how often the things they assign probabilities to turn out to be true. For example, the top right dot represents all the unique events which they assigned a probability of around 97.5% to before they did or didn’t occur: that Mozambique would experience an onset of insurgency between October 2013 and March 2014, that France would deliver a Mistral-class ship to a particular country before January 1st, 2015 and so on for 17 other events. Now, out of all of these 19 events which they assigned a probability of about 97%, it turns out that about 95% of those events occurred. Likewise, if you look at all the events this person assigned a probability of approximately 0%, it turns out that about 0% of those events occurred. 

In this case, this person has a relatively good track record of calibration because they think in ways which mean that the probabilities that they assign to things correspond (roughly) to the frequency with which those things are true. 

And this is the case even when those things concern “unique” events which we might have thought we couldn’t assign probabilities too. After all, a particular insurgency either would or would not occur, it’s a unique event, and so it’s not obvious we can assign it a numerically precise number, like 67%, that—as it turns out—reflects the objective odds of its occurrence. But it turns out that we can—at least if we think in the right ways.

Yet not all of us are so well calibrated. Below is a particular individual, user 4566, who assigned probabilities of around 97% to things which were true merely 21% of time, such as Chad experiencing insurgency by March 2014 and so on.

​Studies have shown people can be more or less calibrated for a range of domains, including medical diagnoses and prognoses, general knowledge about past or current affairs, geopolitical topics—virtually anything.

Calibrationism then says that judgments of probability are trustworthy only if we have evidence that they are produced by ways of thinking that are well calibrated—that is, by ways of thinking that look more like the first forecaster’s calibration graph than like the second forecaster’s calibration graph. 

Of course, for many kinds of judgments, we lack strong evidence of calibration, and we then have little grounds for unquestioning trust in those judgements of probability. This is true in some parts of medicine or law, for instance, where we have some evidence of inaccurate judgments which can lead to misdiagnoses or false convictions at least some of the time, even if we have perfectly good judgments the rest of the time. 

But in other contexts, we simply lack evidence either way: our judgments could be as calibrated as the first forecaster or as miscalibrated as the second, and we have no good reasons to tell firmly either way. 

The calibrationist implication is that in some domains, then, we need to measure and possibly improve our calibration before we can fully trust the judgments of probability which we have in that domain. 

The good news, however, is that this is possible, as I discuss in my book elsewhere. For example, some evidence (e.g. here and here) suggests we can improve our calibration by drawing on information about statistics or the frequency with which things have happened in the past. For instance, we can better predict a recession in the future if we look at the proportion of the time that recessions have occurred in similar situations in the past. We can similarly use statistics like these to determine the probabilities of election outcomes, wars, a patient having a disease and even more mundane outcomes like whether someone has a crush on you.

The other good news is that some people are well calibrated, meaning that they can provide us with trustworthy judgments about the world. For example, take user 5265 from the Good Judgment’s forecasting tournament. In the first year of the tournament, their judgments were well calibrated, as the below graph depicts:

​If we were in the second year of the Good Judgment tournament, and we were about to ask this person another series of questions, we could then have similarly inferred the calibration and trustworthiness of their judgments about the future. And in fact, that is exactly what we see when we look at their track record of calibration for the second year of the tournament below:

​More generally, the evidence demonstrates track records of accuracy are the biggest indicator of someone’s accuracy in other contexts—and better indicators than education level, age, experience or anything else that has been scientifically tested. 

So track records of calibration are one important ingredient of trustworthiness, but it’s not the only one. 
​

​Another important ingredient is inclusivity: that is, the extent to which we consider all the evidence which is relevant.  After all, calibration isn’t everything we care about, since someone  could be perfectly well calibrated merely by assigning 50% probabilities to a series of “yes/no” questions. Additionally, some evidence suggests people form more accurate judgments when they include more evidence than others—and it’s obvious how this can improve accuracy when, for example, including DNA evidence which vindicates otherwise convicted defendants in law.

What we also care about, then, is whether judgments of probability are informative in the sense that they tell us whether something is true or not in a particular case. This in turn is largely a matter of including evidence. For example, one could be well calibrated by assigning 50% probabilities for the “yes/no” questions, but they would likely be omitting relevant evidence and not saying anything particularly informative.

Calibrationism then says that in order for judgments to be trustworthy, they must also include all the evidence which we regard as relevant.
​

​So that’s calibrationism in a rough nutshell: our judgments are trustworthy to the extent we have evidence that A) they are produced in ways that are well calibrated and B) they are inclusive of all the relevant evidence. What, then, are the implications of this? Four come to mind:
​

1. We should measure calibration to determine which judgments are trustworthy
2. We should trust individuals who have evidence of their calibration 
3. But only if they are inclusive of all the evidence
4. And where necessary, we should aim to improve our calibration

Practically, I provide more ideas about how we can do these things elsewhere. For example, one can measure calibration by plugging some judgments into a spreadsheet template, as I discuss here. Calibration can also be improved with recommendations which I discuss here. Lastly, if we want to assess the inclusivity and trustworthiness of someone’s thinking, we can list the evidence which we think is relevant, ask them questions about their reactions to each item, and if their responses seem to reflect calibrated engagement with all of the evidence, then we can trust their judgments.

Put simply, to determine which judgments to trust, we might want to see more calibration graphs and evidence checklists to determine calibration and inclusivity—at least when the stakes are high. This might help make a world with fewer fatal misdiagnoses, false criminal convictions and other expressions of inaccuracy that compromise the functioning and well-being of our societies.

We all make countless judgments, and our important life decisions depend on them.  

My new book, “Human Judgment”, investigates these judgments, and it is now available to purchase online here.

The book concerns two topics to do with human judgment, as implied by the subtitle: How accurate is it, and how can it get better?

It has two somewhat newsworthy items, one bad and the other good.

The bad news is that the science suggests that human judgment is often much more inaccurate than we might hope or expect. For example, some researchers estimated as many as 40,000 to 80,000 US citizens will die because of preventable misdiagnoses—and that’s each year. If they are right, that’s a yearly death toll at least 13 times higher than the September 11th terrorist attacks. Unfortunately, medicine is not unique too: judgmental inaccuracy can afflict a number of other areas in society as well. As another example, some researchers estimate at least 4.1% of death sentence convictions in the US are actually false convictions; this implies that some people are trialed, convicted and executed for horrific crimes that they never actually committed. So that is a few of numerous studies painting a less than ideal picture of human judgment: we make inaccurate judgments about medical diagnoses, about criminal convictions and about a number of other areas.