Radio personality Garrison Keillor often talks about a lovable locale called Lake Wobegon, "where all the women are strong, all the men are good-looking, and all the children are above average."

You may think that you and your family are above average, too -- at least financially. Permit me to offer some numbers to help you determine where you really stand in the financial pecking order, and how our nation's households are doing. The following data is from the Federal Reserve's Survey of Consumer Finances. It's released every three years. The latest survey covers 2001 to 2004.

The median net worth of the bottom 40% of families declined, while those at the top experienced gains in their net worth. Want specifics? The average family's income fell 2.3%, from $72,400 in the 1998-2001 period to $70,700 in the 2001-2004 period (adjusted for inflation). Part of this can be attributed to the stock market's slump during the period. During the boom years of 1998 to 2001, average income soared by 17.3%. Another factor is a 6.2% decline in median real wages.

Note that average numbers can be misleading, if some extreme numbers at the top or bottom skew things, so it's often good to look at median numbers, which represent the middle of the batch. In this case, median income rose a bit -- up 1.6% to $43,200.

Meanwhile, fewer families are investing in stocks, which doesn't bode well for their long-term fiscal health. The percentage of families investing in stocks fell from 51.9% to 48.6%.

Median net worth rose just 1.5%, to $93,100, between 2001 and 2004, versus a pop of 10.3% in the previous three years and a 17.4% increase in the three years before that. Meanwhile, mortgage debt increased significantly on account of the continued housing boom of recent years. (Home prices surged almost 27% during the period, and the ranks of homeowners rose to 69.1% of the population. Overall household debt jumped a whopping 33.9% to a median value of $55,300. Not surprisingly, the portion of household income devoted to paying off debts rose to 14.4% from 12.9%.

Do you get the picture? Many Americans are not doing as well as they may appear. Yes, perhaps they've bought a house, but it's come at a high cost, and overall incomes aren't rising quickly enough to make living with the debt any easier. Credit card debt is always a problem, and those not tackling it are falling deeper and deeper into a hole that's tough to dig out of. (Let us help you if you're debt-ridden.)

It's also worrisome when you think about many Americans' retirements. The median net worth stands at $93,100. Let's say that $93,100 is your personal nest egg, and all of it is in the stock market, growing at the historic average annual rate of 10%. In 25 years, you'll have $1 million! That sounds good, but in 25 years it may only have the buying power of $350,000, thanks to inflation -- hardly enough on which to base a comfy retirement. Worse still, most of us hardly have our entire net worth in stocks. Much of it is likely equity in our homes. That kind of investment tends to grow between 5% and 6%, on average, annually. (Learn more in "Prepare for a Gruesome Retirement.")

So how did you do? If you're anywhere near average, you need to make some big financial changes ASAP. Make sure you're tending to your big retirement picture. Begin planning now. We can help you reach your dreams, with our Rule Your Retirement newsletter. It's issued each month, is readable in a single sitting, and contains lots of valuable tips, as well as inspiration and motivation. (You've got little to lose and a lot to gain by trying it for free for 30 days.) (One of my favorite features in it are the frequent stories of people who retired early and well, and explain how they did it.)

And by the way, if you're not yet a homeowner but want to profit from the ongoing housing boom (which may actually come to an end one of these days), consider looking into companies that profit from it, such as lenders Wells Fargo (NYSE:WFC) and Countrywide Financial (NYSE:CFC), home builders Pulte (NYSE:PHM) and Toll Brothers (NYSE:TOL), home furnishers such as American Standard (NYSE:ASD) and American Woodmark (NASDAQ:AMWD), or home-improvement retailers such as Lowe's (NYSE:LOW).

Longtime Fool contributor Selena Maranjian owns shares of no company mentioned in this article.