Right now, millions of Americans are crunching numbers, sorting receipts, and sharpening pencils as they prepare to file their tax returns. As you look over your tax forms and the sometimes eye-popping amounts you've handed over to the federal government, you may find yourself wondering whether you pay more in income tax than the average American.

A woman calculates her taxes.

Image source: Getty Images.

Tax payments are climbing
The economic recovery means that the IRS is collecting far more tax dollars -- and paying out far less in refunds -- than it did during the recession.

In 2013, the IRS' gross tax collections totaled $2.86 trillion. That was up from $2.52 trillion in 2012 and $2.35 trillion in 2009. The majority of that IRS tax revenue comes from individuals like you and your family, rather than businesses.

Meanwhile, the amount of money returned to taxpayers in the form of refunds has fallen steadily from $437.7 billion in 2009 to $364.3 billion in 2013.

How do you stack up?
According to the IRS, 146.24 million individual tax forms were sent in during 2013. Dividing the total amount of tax receipts collected from individual taxpayers (minus the amount of money returned to taxpayers in refunds) by the number of individual tax filers shows that the average taxpayer was on the hook for roughly $8,548.49.

However, you shouldn't take that comparison at face value. That's because averages can be misleading, given that tax brackets vary dramatically based on income. After all, many people owe no taxes at all, while the tax bill for high-income earners can be in the hundreds of thousands of dollars. That suggests that a far more useful way to compare yourself to the average taxpayer is to look at the group of taxpayers who earn rougly the same amount as you.

Fortunately, the IRS provides that data, too, but because the data is based on sampling, these estimates should be taken with a grain of salt.

While that data is intriguing, it might be even more helpful to consider what the average tax rate is for various ranges of adjusted income. That way you can simply multiply the average tax rate by an amount of income in the relevant bracket and see how you compare. For example, let's assume your reported adjusted income is $80,000. Using the following table, you'll notice that the average person earning between $73,354 and $125,195 in 2012 paid an average tax rate of 16.4%. That suggests that the average person earning $80,000 paid $13,120 in taxes that year ($80,000 x 0.164). Are you paying more? Or less?

Income Bracket Lowest Income in Bracket Average Tax Rate*
Top 0.1% $2.2 million 21.7%
Top 1% $434,682 22.8%
Top 5% $175,817 21%
Top 10% $125,195 19.2%
Top 25% $73,354 16.4%
Top 50% $36,055 14.3%
Bottom 50% <$36,055 3.3%

*Average tax rate = percentage of AGI paid in income tax. Source: Tax Foundation, IRS, and author's calculations.

Crossing your T's and dotting your I's
If the amount of income tax you're paying is much higher than the income tax being paid by similar income-earners, it may be time to start looking more closely at potential deductions.

Those deductions may include everything from mortgage interest and student loan interest to miles driven to visit doctors and charities. Also don't forget to keep detailed records of everything you donate, as charitable donations are tax-deductible; even goods donated to nonprofit thrift shops like Goodwill can be written off. Other deductions that may lower your tax bill include the home office deduction, certain taxes paid to register vehicles, and even the cost of investment newsletters.

If you take all the deductions you think you can get and still feel you're paying more in income taxes than you ought to, it may be time to visit a tax professional to make sure you're not missing out on other tax-saving opportunities.