Let's face facts: No one likes paying tax to Uncle Sam or their state. But we also know that federal, state, and local governments wouldn't be able to operate without tax revenue. This battle between honoring your civic duty and paying your fair share of taxes, while also doing everything in your power to legally avoid paying tax, is something all Americans face to some degree.

However, the importance of minimizing taxable liability is even more significant for the wealthy -- because they have a lot more to lose to taxation. Though the definition of who is and isn't "rich" is completely debatable, statistics from the Internal Revenue Service for the 2016 tax year show that nearly 25.3 million income-tax filers reported in excess of $100,000 in adjusted gross income. These taxpayers, who made up less than 17% of total filings, were responsible for paying 80.6% of all federal income taxes for 2015. Ouch! 

A hundred dollar bill, with Ben Franklin wearing Uncle Sam's America-themed hat, next to the Capitol building.

Image source: Getty Images.

These states are sticking it to the wealthy with high tax rates

And it's not just federal taxes that the rich are exposed to. All 50 states offer their own blend of state and local tax, property tax, and income tax, which can drain the pockets of wealthy individuals and couples who've not done their homework.

While debatable, the following six states (listed in no particular order) look to be the least friendly to the rich. 

A crumpled up tax form in the foreground on a table, with a man looking over his tax documents in the background.

Image source: Getty Images.

California

California might be a popular place to live for its abundant sunshine and star-studded population, but it's not a particularly friendly place for the rich to call home. The biggest drawback comes in the form of the highest peak marginal income tax rate in the country.

Once a single taxpayer crosses $268,750 in income, they've entered double-digit state income tax territory (10.3%). Single millionaires living in California will have to fork over 13.3% of their income to the state, with married couples filing jointly hitting the same tax rate on income over $1,074,996.

California also has the ninth-highest combined state and local sales-tax rate, according to the Tax Foundation. And while its average property tax rate of 0.79% is in the lower half of the 50 states, median home prices in California are well above the national average. This means the typical homeowner is paying $3,237 in annual property tax, per WalletHub. That's the 10th highest in the country, based on median home value.

An ascending stack of coins in front of a house with a hand adding another coin to the highest stack.

Image source: Getty Images.

New Jersey

Sorry, folks, but it's a Jersey thing. By "a Jersey thing," I mean high property tax rates. New Jersey's effective real estate tax rate of 2.4% is the highest in the country. When compared with median home values in the state of $316,400, it leads to an average annual property tax bill of $7,601. That's more than $2,100 higher than the next-closest state (Connecticut), and near 3 1/2 times the average property tax bill of $2,197 across America, per WalletHub.

But New Jersey isn't done shaking the change out of the wealthy's pockets. Single and married taxpayers are subject to an 8.97% state income tax rate if they earn in excess of $500,000. Thankfully, the combined state and local tax rate of 6.6% is "only" the 30th highest in the country, giving wealthy New Jersey residents perhaps the only "break" they're going to get.

A wealthy senior man in a suit scowling and showing his displeasure.

Image source: Getty Images.

New York

If you think you're going to hop across the George Washington Bridge to New York in order to escape New Jersey's high tax rates, think again. The Big Apple boasts the 10th highest effective real estate tax rate in the country, and like New Jersey and California, has an above-average median home value. Running the math, it works out to an average annual property tax bill of $4,738 -- the fourth highest in the U.S.

In terms of state income tax, individuals and married couples can earn up to $1,077,549 and $2,155,349, respectively, and still pay only a 6.85% rate based on the state's progressive income brackets. That's not too terrible, all things considered. But earn over these respective amounts, and New York will levy an 8.82% state income tax rate. Plus, its combined state and local tax rate is one spot behind California, at 8.49%, making it the 10th highest in the country.

A businessman in a tie placing crisp hundred dollar bills into two outstretched hands.

Image source: Getty Images.

Oregon

If the opportunity to purchase cannabis legally intrigues you, then Oregon is a great place for the wealthy to consider settling down. But if you're more interested in avoiding a big tax bill, Oregon may not be the place for you.

On the bright side, Oregon has no state and local sales tax, which is a big one-up on every other state mentioned here. But Oregon is notorious for siphoning money out of the rich via state income tax. Single filers earning in excess of $8,450 are subject to a 9% state income tax rate, with earned income above $125,000 jumping to a rate of 9.9%! Married couples filing jointly hit the 9.9% rate when adjusted gross income tops $250,000.

Lastly, the average Oregonian shells out $2,637 a year in property tax, based on median home values in the state. That's more than $400 above the national average.

A Social Security card wedged in between IRS tax forms, a pair of glasses, and a $20 bill.

Image source: Getty Images.

Minnesota

You'd think with Minnesota's notoriously cold winters, the state would offer tax perks to lure the wealthy to its state. But that's simply not the case.

For starters, Minnesota is one of 13 U.S. states that taxes Social Security income. And while the state has recently implemented a handful of new state tax breaks for Social Security income, these breaks phase out completely for those earning over $99,500 in taxable income, according to Kiplinger.

Aside from being taxed on Social Security income, the rich would also be subject to a whopping 9.85% peak marginal income tax rate on adjusted gross income above $156,911 (or $261,510 for married couples filing jointly). Add in the 17th highest state and local sales tax rate in the country, at 7.42%, as well as an average property tax bill that's just slightly higher than the national average, and the wealthy have a few pretty good reasons to avoid Minnesota.

A surprised senior man tightly clutching a piggy bank as outstretched hands reach for it.

Image source: Getty Images.

Vermont

Last but not least, Vermont tends to stick it to the wealthy via high tax rates. Like Minnesota, Vermont also taxes Social Security benefits. But unlike Minnesota, it allows no exemptions, with the state tax on Social Security mirroring that of the federal government. Making matters worse, retirement income, including 401(k)s, traditional IRAs, and both public and private pensions, are taxable at ordinary income tax rates.

Vermont's effective real estate tax rate of 1.78% works out to the eighth highest in the country. When compared to its median home value of $218,900, it means the average Vermonter is forking over $3,893 in property tax per year (also eighth highest).

And let's not overlook those state income taxes. Single filers making over $91,900, $191,650, and $416,700, are subject to respective income tax rates of 7.8%, 8.8%, and 8.95%.

Long story short, if you're in the top quintile of income earners in this country, it may very well pay to do your homework and avoid these six states.

The Motley Fool has a disclosure policy.