Being rich means you get to worry about everything except money.

-- Johnny Cash

Most of us would love to be rich. We imagine that just about all our problems would disappear if only we were really wealthy. Having a lot of money would mean that we wouldn't have to worry about keeping a roof over our heads, sending our kids to any schools they wanted to attend, or having enough to live very comfortably in retirement. What's not to like about being rich?

A finger pointing to the word millionaire against a backdrop of city lights at night

Image source: Getty Images.

As it turns out, there are some downsides to extreme wealth. Here are 15 of them.

1. Sudden wealth catches us unprepared

If you become wealthy suddenly, such as via a lottery jackpot, you won't be ready for what it entails. News archives are full of stories about people who won lottery jackpots only to end up poor, in trouble, or even dead. I wrote about some lottery winners' sad stories a few years ago. Here's one example:

David Edwards won $27 million in a Powerball jackpot, and spent much of it on drugs, a mansion, a Learjet, and fancy new cars. With $27 million, you could live comfortably for the rest of your life without working; but Edwards spent lavishly, and ended up living in a filthy storage unit late in his life. His wife eventually left him, and he died in hospice care.

2. Many don't know how to handle windfalls

The story above is instructive. Imagine winning $27 million. Even if you lose half of it to taxes, you'd have $13.5 million. Even if you spent half of that on splurges and perhaps some gifts to loved ones, you'd have $6.75 million left. If you parked that in solid, dividend-paying stocks with an average yield of just 3%, you'd collect $202,500 annually. That's a pretty good amount of money to live on each year -- plus, the $6.75 million would still be there, invested in those stocks.

Over time, healthy and growing stocks tend to grow in value, and they typically increase their dividend payments, too, so your income and wealth would likely keep growing. If you parked, say, $3 million in an immediate fixed annuity, it would also pay you regularly -- for the rest of your life. For $3 million, at recent rates, a 50-year-old man would get around $12,500 per month, or about $150,000 per year.

3. Your wealth can overshadow your other characteristics

People can focus more on your wealth than on your other traits, and that wealth tends to become your most defining characteristic. Abigail Disney, granddaughter of Roy Disney, who co-founded the Walt Disney Company, has related her own experience as a rich person: "People do say to me, straight up, 'Oh my God, you must be really rich.' In every interaction, you don't get to make a first impression because they're already thought about what they want to think about you before you even shake their hand."

4. If you're famous, too, you'll lose a lot of privacy

This one is no surprise, but if you're famous along with being rich (and possibly famous because of your wealth), you'll lose much of your privacy. You'll no longer simply be able to go out to a restaurant or a store -- or anywhere -- without at least a chance of being recognized, disturbed, photographed, and perhaps gossiped about. Many rich people even have to deal with people going through the garbage they put out each week, with some having to find more private ways to get rid of trash.

5. You may have given up a lot to become rich

People become wealthy in different ways, such as through winning a lottery jackpot, inheriting assets, or perhaps working hard and investing. If you achieved your wealth through lots of hard work, there's a good chance that you sacrificed a lot along the way. You may have worked long hours at your job, or even worked two jobs -- so you may have missed birthday parties, chances to take vacations, and more. You may not have developed close relationships with family members and friends because of your sacrifices and long working hours. Those are all meaningful losses.

6. You may feel uneasy

If your wealth suddenly dropped into your lap, perhaps via an inheritance, you may find yourself feeling somewhat uneasy. Not unlike survivors of disasters who wonder why their lives were spared when others' were not, you may see people struggling to get by and wonder why you were spared such a fate. It may seem unfair, and that might trouble you.

7. You may have to keep a lot of secrets

Many wealthy people keep a lot more secrets than the rest of us do. You may not want your co-workers or neighbors to know that you're wealthy, for example, perhaps because of what they'll think or assume. You may not want to share your address on occasion, as it might reveal that you're very well-off. If you take a lavish vacation and enjoy lots of experiences that others can't afford, you may want to keep that to yourself in certain company.

8. You'll be criticized by people who don't even know you

Related to the problem above is that once people know you're rich, they'll often have negative thoughts or assumptions about you. After all, there are certain stereotypes about the wealthy. Some people may assume that you're greedy, shallow, or materialistic, or that you don't know what it's like to be an average person. They may assume that you became wealthy just because you're lucky -- or maybe because you cheated or took advantage of various loopholes. (Some do become wealthy due to good luck, of course; others may cheat their way to wealth, such as by evading taxes.) Some people will even see your having become wealthy as a loss for them, as if one person's good fortune is tied to the bad fortune of another.

Two transparent piggy banks, one full of pennies and the other empty

Image source: Getty Images.

9. People will treat you differently

One reason you may start being secretive when rich is because you won't want to be treated differently. When people know you're wealthy, you may become their go-to person when they need to borrow money, or they might expect you to pay every time you dine at a restaurant together. Some friends or relatives might expect costly gifts from you for birthdays or holidays -- and might be put out if you give ordinary gifts.

While many of us get relief from venting about our problems to others, you might find that people don't have much patience for your problems, assuming that since you're rich, you shouldn't have much to complain about. (Of course, often you'll receive more attention and deference because you're wealthy -- which can be unwelcome, too.) When you're wealthy, you may be an object of envy and even resentment.

10. You can grow out of touch with most of society

There are different ways to live your life once you become rich. You might not change much, remaining in a modest home and continuing to work. Or you might quit your job, buy a fancy house and car in an upscale town or neighborhood, and develop new circles of friends. That latter option might be perfectly enjoyable for you, but it can also mean that you'll lose touch with the rest of society and may have trouble relating to average people, their common experiences, and the challenges they face.

11. You can develop an identity crisis

You may not realize it, but much of our identity is tied up in what we do for work. You might be a teacher or a scientist or a nurse, and devote your time and thoughts to your field and your status within it. If you're no longer working because you're financially independent, you might find yourself experiencing an identity crisis, with your self-esteem taking a bit of a blow.

12. You can suffer a lot of disappointment

You may no longer have goals like you used to, because now you can afford just about anything you want instead of having to work for it. You'll no longer get the satisfaction of having earned something through your own efforts; because you haven't had to yearn for them or save up for them, purchases can be less satisfying. Buying something you want may not provide the delight you expected it would, and you can find yourself still unsatisfied.

Many people assume that if they had a lot of money, that they'd be happy, with all their problems solved. That's not always the case, though. If you were a generally unhappy person before becoming rich, there's a good chance you'll still be an unhappy person.

13. You'll have trouble trusting people

This is a major drawback of great wealth: You'll have trouble trusting people. With anyone you meet, whether in a business setting or social setting, you'll find yourself unsure whether they're truly interested in you and what you have to offer, or they're more interested in your money and what it might do for them. That can make dating particularly hard.

14. Your children may end up spoiled

This might not seem like a bad thing, but if you're raising your children in luxury, they'll never have to struggle to achieve things and to build a life for themselves. They may also find it hard to make friends who aren't interested in their wealth.

Warren Buffett, one of the world's richest people, has made a point of not giving his children too much of his money; he's said that he aimed to give his kids "enough money so that they would feel they could do anything, but not so much that they could do nothing." Bill and Melinda Gates have said that they're leaving their children "a minuscule portion" of their billions, explaining that "It will mean they have to find their own way."

15. Boredom

A final drawback to great wealth is this: boredom. If you don't have to work and you choose not to, you may have trouble filling your time with meaningful activity. Consider Timothy Kim, who started investing in the stock market at the age of 19; by the time he was in his early 30s, he had more than a million dollars -- and was bored: "If someone gives you $100 million and you don't have to work anymore, you're going to quickly find out that life feels a little meaningless and you have this hole."

Want to be rich, anyway?

There's a good chance that, despite the drawbacks above, you'd still love to be wealthy. That's fair. After all, being rich does have compelling upsides, such as being able to live almost anywhere you want, travel anywhere you want, and attend any shows or sporting events you want, among many other things.

Here's some good news: You may be able to get rich before you retire, or at least make yourself significantly more financially secure. Getting wealthy, or wealthier, isn't just about luck or a rich family -- you can get there by determination and effort.

Here's how much you might amass by regularly saving and investing certain sums:

Growing at 8% for...

$10,000 Invested Annually

$15,000 Invested Annually

$20,000 Invested Annually

3 years

$35,061

$52,592

$70,122

5 years

$63,359

$95,039

$126,719

10 years

$156,455

$234,682

$312,910

15 years

$293,243

$439,864

$586,486

20 years

$494,229

$741,344

$988,458

25 years

$789,544

$1.2 million

$1.6 million

Data source: Calculations by author.

So go ahead -- use the time you have before you retire to get as rich as you can.