Five years after the terrorist attacks of Sept. 11, 2001, we're all well versed in the statistics: nearly 3,000 people killed; $40 billion in damage; two gigantic buildings and four airliners destroyed; and another building, the Pentagon, maimed.

It's almost irresistible for politicians to resist playing the demagogue when it comes to Sept. 11. And what do you know -- most of them don't resist at all. But there's one niggling detail that politicians somehow forget: it wasn't 2,900 Americans killed on Sept. 11, it was 2,900 people. There were more Somalians killed in the terrorist attacks than there were people from North Carolina, the nation's 10th largest state by population. This matters.

Though we remember the pictures of those who took to the streets to celebrate these attacks, we must also remember that not only did Sept. 11 represent a particularly horrible introduction of a long string of global terrorist events to U.S. soil, but that it was not a singularly American event. People from 78 countries died in New York, Virginia, and Pennsylvania on that dark day.

Ask citizens of Beirut, Vienna, Rome, Belfast, Tel Aviv, and elsewhere about whether terrorism began that day, and they would say, "Of course not." Ask the people of Bali, Madrid, London, Istanbul, Karachi, Jakarta, and Mumbai if Sept. 11 represented the last big terrorist attack, and they'd say the same thing. In a sense, on that day the United States became a card-carrying member of the world's least awesome club: the location of a major international terrorist event.

Beyond the immeasurable cost in lives lost, the economic impact of Sept. 11 has spread to all corners of the globe. There are costs that are defined and enormous -- insurance companies based in Bermuda and Europe having to pay vast sums for claims; the sudden adoption of "terrorism exclusions," where before most insurers remained blissfully ignorant to the risks that they were covering on landmark buildings that could be targeted; and increased investments in security countermeasures around the world.

But what about the intangible numbers? How much has been lost around the globe as a direct or indirect result of the terrorist attacks? Prior to the attacks, it wouldn't have been out of the ordinary for me to arrive at an airport 20 minutes before a flight's departure. Immediately following Sept. 11, the game changed, and two hours became the norm. Is it possible to capture the loss of productivity among business travelers because of the additional time required to complete the curb-to-gate process of boarding an airplane? It has to be in the billions, annually. And this happens throughout the globe.

That's not the only vestige of the attacks. We've become more insular. There are people who once thought little of vacationing overseas who suddenly can no longer get on airplanes for fear. Tourism in Egypt, Israel, Bali, and Turkey has yet to fully recover to pre-2001 levels. Many guest workers simply left Saudi Arabia, Kuwait, and other Middle Eastern countries rather than risk their lives, or their families', as easy targets for terrorists.

In the name of safety, we've built our walls a little higher and retreated to the familiar a little more. We're hoping that our countries, with their porous borders and liberal immigration policies, will turn into the fortresses we dearly want them to be. And to me, as a certified internationalist, that's a little bit sad.

On the day before Sept. 11 this year, I took a flight out of Dulles Airport in Washington, DC, to San Francisco. I find myself incapable of resisting asking where people are from. As it turns out, my cab driver hailed from Pakistan.

"Ah," say I, "I've spent some time in your country. There are many beautiful places up north."

For whatever intrinsic values Pakistan has, it is nothing if not off the beaten path for most travelers. As such, my cab driver was surprised and pleased to hear me speak positively about his country.

I continued, "It's a shame that it's fairly dangerous to travel there now." Truth be told, it has always been dangerous to travel to Pakistan, but following several events where Westerners and Western institutions have been targeted, this seems truer now than ever before.

"Pah," said my driver. "Pakistan is the same as it has always been. There are crazy people everywhere, and we have some. But I ask you, were they any less crazy, or any less likely to cause problems before? No." Then he added, "It's like three years ago when everyone got afraid because of the shark attacks in the news. Were there more attacks that year, or just sensational ones? Why were we afraid then? The sharks were behaving exactly the same way as they always have."

Pakistan's economy isn't tourism-driven, so it hasn't felt the same impact of other places around the globe. But fear -- even for someone who has been there before -- still has an impact on whether I'd consider going there again. Multiply that kind of "why risk it?" decision with thousands or millions more, and there can not help but be fallout among those who do depend on foreign visitors in whole or part for their livelihoods.

The terrorist attacks were a tragedy. This is something else, and raising it to the level of tragedy is overwrought. It's a poverty, a deficiency in life that we all share. And while the "Middle East is dangerous" meme certainly didn't have its origin five years ago, it has become the default assumption for a great many more people.

For related coverage:

Bill Mann is the co-advisor to Motley Fool Hidden Gems . He welcomes your feedback.