Investing is full of ironies. Here's a big one: With few exceptions, everyone thinks they're a contrarian.

Which they can't be, of course. By definition, most people are the consensus they think they're outsmarting.

The core of active investing is your ability to think against the crowd, and take positions the market disagrees with. As the classic Buffett saying goes, "be greedy when others are fearful, and fearful when others are greedy."

But who are those "others"?

Not you, and not me, of course. At least that's what we tell ourselves. No one ever considers themselves part of the "others," though they must exist, somewhere. No matter our positions -- buying, selling, it doesn't matter -- we can think of a nameless group who disagrees with our views and declare ourselves contrarian.

But being a contrarian doesn't mean you found someone who disagrees with you. It's much harder and rarer than that.

If you live in New York City, you are a contrarian in that most other people don't live in New York City, and would consider doing so a miserable prospect. But living in New York isn't at all contrarian. It's quite a popular thing to do within the confines of New York City, which makes it one of the most crowded, competitive, and expensive places in the world. Lots of people disagreeing with you isn't enough to call yourself a contrarian, because even a little support can make something popular enough to squeeze out opportunity.

The same idea applies to investing.

Being bullish on stocks in 2011 seemed like a contrarian stance. So did being bullish on gold. Each group could point to investors who disagreed with their analysis as evidence of their contrarianism. But stock bulls and gold bulls were betting on opposite outcomes, and each considered themselves contrarians. In truth, both groups had lots of support within their own asset class, even if separate groups viewed the world differently.

Adam Parker, Morgan Stanley's chief equity strategist, wrote this week that many of his clients claim to be contrarians, then proceed to tell him the same story with the same views.

"Romanticizing that you are a contrarian when you are indistinguishable from consensus can't be good" he wrote.

It can't be good because a false sense of contrarianism offers assurance that you are on to something others don't see. What's particularly dangerous is when you have support from others in the contrarian community – fellow New Yorkers, to continue the analogy. It's the land of the free lunch: The power of feeling like a contrarian with the comfort and confirmation of peer support. And like most free lunches, it rarely exists.

This happened last decade during a period I'd call the "value investor bubble." Value investing became extremely popular among a group who took pride in contrarian views that went against those chasing pricey technology stocks. But value investors loved talking to each other, forming groups, and sharing ideas -- which is when true contrarianism starts to unravel. What began as a genuine attempt at independent thinking often led to the opposite, as value investors plowed into the same popular ideas and the same popular stocks. Sears Holdings was one. Bank of America was another. The same thing happens with hedge funds, where self-described contrarians pile into the same stocks.

Contrarianism is the key to huge outperformance. But we should accept that not everyone can be a contrarian, by definition. Real contrarianism is painful. It's lonely. It hurts. It's when everyone thinks you're nuts. Real contrarianism doesn't happen when you buy a stock that fell 10%. It happens when your clients threaten to sue you for negligence, your friends stop calling, and your wife wishes you'd find a new career. It's buying shares of Valeant, or Theranos, today.

The good news – the solution to the irony of contrarianism – is that deep contrarianism isn't necessary to achieve good returns. A market that generally agrees with you can still offer a nice long-term return, especially if your edge is found in being more patient than the crowd, rather than trying to outsmart it.

Too much effort is spent attempting to be contrarian for contrarian's sake, when there's plenty of room to get ahead being patient in a market where most people, most of the time, are pretty smart.

For more:

Performance vs. outcomes

Why does pessimism sound so smart?

Things I'm pretty sure about

Hard truths for investors to wrap their heads around

How the investing industry could change