Unemployment is by far our biggest problem. No one really cares whether GDP grows, productivity rises, or consumer confidence inches up. All that matters is if you have a job. Right now, millions don't.

Not only is it an awful problem, but many can't agree on why it's a problem. Some say taxes are too high. Some say stimulus isn't high enough. Others say regulations are scaring businesses; others that Wall Street still has Main Street by the neck.

 Above all else, we had a financial crisis. Today's unemployment is consistent with what you'd expect after a deep recession. Here are three less-obvious things holding back jobs.

1. Clingers
Think of the jobs market as a conveyor belt. People start on one end in menial jobs, and then move on to better things, making room for younger generations who jump on the belt and take up menial jobs, eventually going on to become supervisors, managers, entrepreneurs, so on and so forth. People don't stay in one place. They move down the line.

Today, that belt has slowed way down. In 2007, about 2.5 million people quit their jobs voluntarily every month. Lately, it's been 1.8 million a month. In 2009, it dipped as low as 1.6 million.

If you have a job, you're clinging to it for dear life. Part of this is out of necessity: There simply aren't many new job opportunities to pursue. But a good part of it is pure fear. Quitting a job voluntarily just feels reckless when you hear stories about people being unemployed for two years straight. There are actually more job openings today than there were in early 2004, yet voluntary quits are considerably lower. The focus isn't on getting ahead; it's on making sure you don't fall behind. Not a good recipe for growth.

2. Should-be retirees
This one's related to the first point. At the end of the jobs conveyor belt, older workers retire, making way for younger generations to fill their old positions.

Today, that's happening at a much slower pace than it did in the past. You can see this in the labor force participation rate among workers age 65 and over:

Source: Bureau of Labor Statistics.

Obviously, this trend started before the financial crisis. Still, it's powerful. Since 1995, the percentage of older workers hanging on to their jobs has jumped 50%.

There could be a few reasons for this. People are living longer, for one. Pensions are becoming scarcer, being replaced by woefully inadequate private savings accounts. The past decade has been agonizing for net worths in general, with the dot-com crash followed by the real estate crash. If you rely on investments to finance a retirement, and those investments repeatedly collapse, you need to work longer. Nearly half of working Americans don't think they'll ever be able to retire.

Regardless of the reason, more older folks staying in the workforce hurts the prospects of younger workers trying to move up. The economy needs to add about 100,000 jobs a month just to keep up with population growth. Anything less, and the unemployment rate rises. When the labor force participation rate moves higher, that figure goes up -- more new jobs are needed every month just to keep the unemployment rate flat. The overall participation rate is low these days, since unemployment is high. But it's skewed, with a very low participation rate among young workers and a high (and growing) participation rate among older workers.

The danger this poses isn't necessarily today, but in the years and decades ahead. When young workers start out their career with chronic unemployment, the odds that they'll ever fully recover drop substantially.

3. A handful of awful states
The nationwide unemployment rate is now 9%. In good economic times, the lowest that could sustainably get is about 5%, so our employment gap these days is 4%.

If you remove just three states with the worst unemployment (California, Nevada, and Michigan), the overall unemployment rate for the remaining 47 states drops by half a percentage point. So one-eighth of our employment gap is attributable to just three states.

Sure, this is basically just saying that if you ignore what's bad, things looks better. But it highlights an important point: Unemployment is far, far worse in some states than in others. The unemployment rate for the 10 worst states is over 11%. In the 10 best, it's 5.8%. North Dakota's unemployment rate is just 3.5%.

What do states with the highest unemployment have in common? They were deeply reliant on one big industry. California is home to blossoming companies like Google (Nasdaq: GOOG) and Apple (Nasdaq: AAPL), but housing made up such a large percentage of its economy during the boom years that it's now overwhelmed by rottenness. Michigan relies heavily on Chrysler, General Motors (NYSE: GM), and Ford (NYSE: F), not only directly, but through layers of tangential parts suppliers, consultants, and accountants tied to the Big Three. Nevada's only significant asset -- Americans' innumeracy -- is losing its edge.

When President Clinton took office in the early 1990s, he said, "I do not believe we can repair the basic fabric of society until people who are willing to work have work. Work organizes life. It gives structure and discipline to life." He was right, of course. And step one to fixing this mess we're in is knowing why we're in it.

Why do you think unemployment is so high? Share your thoughts below.