Overconfidence Unmasked: An Entomologist Explores Human Self-Deception

Illusions Fostered by Primates

Vivek Nityananda is an entomologist by profession. He studied katydids and their secretive communication methods, the wing rustling among leaves that eluded human ears. Somewhere along the way, his academic curiosity strayed from the natural world into a distinctly human phenomenon: overconfidence.

It was an unlikely leap for someone steeped in the natural sciences. More so, as he puts it, for one who had never penned a book earlier. The seeds of his fascination with overconfidence were planted by a scientist friend who examined cooperation among wasps. From the collaborative chatter of insects, the conversation shifted to the competitive, often self-delusional tendencies of humans. Overlaying this curiosity was the influence of evolutionary biologist Robert Trivers, who posited that self-deception—including overconfidence—was an evolutionary trick. Humans, Trivers argued, deceive themselves in order to effectively deceive others.

Overconfidence, Nityananda came to realize, was a peculiar and paradoxical form of self-deception. It was a trait he had seen up close and personal. After a high-stakes interview for a research fellowship, he left the room brimming with self-assurance, only to face the sting of rejection. In hindsight, his preparation had been inadequate. It was a hurtful reminder that confidence, no matter how fervent, could not substitute for competence.

Yet, he observes that overconfidence can be wildly effective. Politicians wield it like a sword, corporate leaders use it to rally investors, and, perhaps most famously, Donald Trump rode it to the White House. For Democrats, Vivek notes, a different kind of overconfidence—the assumption of inevitable victory—might have sealed their fate.

Nityananda probes the psychological underpinnings. The Dunning Kruger effect, he posits, reveals much. Those least competent in a domain often underestimate its complexity and overestimate their abilities. Ironically, ignorance fuels the boldest confidence. In contrast, true experts, burdened by their nuanced understanding, often appear hesitant. The result? The ignorant sometimes triumph, projecting certainty that others mistake for mastery.

Overconfidence, he suggests, is also a magnet for risk-taking. The overly confident are gamblers at heart, willing to bet big on shaky ground. It’s a dangerous game, especially in politics, where leaders often distort reality to bolster their self-image.

The Lure of Confidence

If overconfidence is the root, both distrust and a paradoxical trust are its fruits. And nowhere is this more apparent than in the peculiar world of game shows.

Sushil Kumar, a young man from the modest town of Motihari, Bihar, embodied this dynamic. At sixteen, he watched “Kaun Banega Crorepati” for the first time, dreaming of landing in the hot seat. Eleven years later, that dream, uncannily enough, became reality. Faced with a question that stumped him, Kumar turned to the audience poll lifeline. The crowd, exuding collective confidence, voted overwhelmingly for one answer. They were right.

But was their wisdom guaranteed? Vivek’s research shows otherwise. The ‘wisdom of crowds,’ he finds, can work for easy questions. But can falter on complex ones. Studies show that on such questions, the opinion of a single expert often trumps the aggregate guesses of the masses. Trusting the crowd, then, is a calculated risk—one that Kumar had been fortunate to win.

More broadly, humans are drawn to confidence like moths to light. This cognitive shortcut, the ‘confidence heuristic,’ predisposes us to trust individuals who project certainty. Psychologists Paul Price and Eric Stone tested this phenomenon with two stock trading analysts. Both analysts were equally accurate in their predictions, but one was more certain in presenting his views. Investors consistently favored the more confident advisor, even when his results were no better than his humbler counterpart’s.

This heuristic, Nityananda observes, has far-reaching effects. Confidence, regardless of its foundation, can sway elections, guide investments, and even determine life-or-death decisions in courtrooms and operating theatres. It’s a potent and yet perilous currency.

Historic Costs of Hubris

Despite or even because of its appeal, overconfidence exacts a heavy toll. It can blind individuals to their shortcomings and lead them into catastrophic failures. History, Vivek notes, is littered with examples.

The 2008 financial crisis was a case in point. Bankers and traders, intoxicated by their own success, believed they had tamed the chaotic beast of financial markets. Their confidence, bolstered by complex mathematical models, led them to take increasingly reckless risks. When the bubble burst, it wasn’t just Wall Street that paid the price; millions of ordinary people saw their savings wiped out.

In the realm of warfare, overconfidence has been equally disastrous. The hubris of military leaders has led to countless miscalculations, from Napoleon’s ill-fated invasion of Russia to the American quagmire in Vietnam. In each case, leaders underestimated their opponents and overestimated their own capabilities, with tragic consequences.

In everyday life, overconfidence can have dire consequences. Drivers who overestimate their skills are more likely to speed and ignore warning signs. Entrepreneurs who overestimate their business acumen are more likely to bankrupt their companies. The list goes on.

Categorizing Overconfidence

Naturally, there are gender differences. In her iconic essay, Men Explain Things to Me, Rebecca Solnit shares a jaw-dropping moment at a party. A man enthusiastically began explaining a book to her—her own book! Even when she politely mentioned that she was the author, he brushed it off and carried on as if she hadn’t said a word.

Jason Adam Katzenstein’s 2018 New Yorker cartoon nails this dynamic perfectly. A man and a woman are casually sipping wine. The man confidently declares, “Let me interrupt your expertise with my confidence.”

Overconfidence can be categorized further into three types:

1.     Overplacement: The “I’m better than them” syndrome. Think you can out-debate that news anchor or run the country better than its leader? You’re not alone—94% of professors once declared they were better teachers than their peers. We can ponder about who’s grading those responses.

2.     Overestimation: When you believe you can do everything. Like promising to meet that impossible deadline or signing up for a marathon after one jog around the block. (Spoiler alert: it’s harder than it looks!)

3.     Overprecision: This is the poker face of doom. You’re so sure of your winning hand, you hang on to those cards way too long—only to lose spectacularly.

For all its pitfalls, overconfidence is not without its virtues. It can inspire individuals to take bold actions and achieve the seemingly impossible. The Wright brothers, for instance, were dismissed as dreamers when they set out to build a flying machine. Their confidence, however, propelled them to success.

Nityananda is attuned to this duality. He sees confidence as a tool, one that can be wielded for good or ill. The key, he believes, is self-awareness. By recognizing their biases and limitations, individuals can harness confidence without falling prey to its excesses.

Learning from Insects

Perhaps it’s fitting that an entomologist should find parallels between the insect world and human behavior. Katydids, Nityananda’s first love, use mimicry to deceive predators, projecting a confidence that belies their vulnerability. Humans, too, are masters of camouflage, their overconfidence often a mask for insecurity.

Vivek’s journey from katydids to confidence reinforces the interconnectedness of life. His work reminds us that, while we may consider ourselves the pinnacle of evolution, we are still governed by the primal instincts that shape the natural world. Overconfidence, for all its flaws, makes us human.

Like katydids, we must learn to navigate the delicate balance between projection and reality, between confidence and competence. In that balance lies the key to understanding not just the world around us but that most complex of entities: ourselves.

References

Vivek Nityananda, Beyond Doubt: Overconfidence and What it Means for Modern Society, Context, Westland Books, 2024