Everyone gets a refund with their tax return, right? For some people, an annual tax refund is one of the bigger checks they'll receive all year.

But others have to pay the IRS at tax time. There tends to be a lot of confusion as to why this occurs, but it's usually for one of three reasons: Your employer withheld too little money from your paychecks, you sold investment assets for gains, or you have a side hustle or gig to supplement your income. Below, we'll look into each of these scenarios and think about the potential fix for each. 

Your employer didn't withhold enough tax

When you receive a refund come tax time, this doesn't happen out of thin air: It simply means your employer paid more money to the government on your behalf than ended up being required. A tax refund happens when you've paid more than you legally owe to the IRS; in other words, you've overpaid the government during the tax year, and your refund makes you whole come tax time. 

When your employer fails to withhold enough tax during the year, the opposite occurs. If your employer only withholds a fraction of what you owe, you'll be responsible to foot the bill come April 15 (May 17 for 2020 returns). This is really only a problem if there's a significant discrepancy in the amount your employer holds back -- if they're within 90% of the right number, no problem.  

This isn't always the fault of the employer, but it's good to double-check with your HR department to ensure your W-9 reflects the correct number of allowances for your personal situation. Sometimes, due to certain additions or deductions that can crop up when you complete your tax return, you may end up owing a small amount of money -- this isn't anyone's fault, and happens regularly without much consequence. 

Man running from papers labeled Tax

Image source: Getty Images.

You sold investment assets

Selling appreciated investment assets -- like stocks, bonds, options, and yes, cryptocurrency -- can cause short- or long-term capital gains, which won't bother you at the time you sell but will come back to bite you when you file your tax return. Inevitably, when you sell investments for a profit, the IRS is going to know about it, and they expect you to declare this income as part of your overall income. 

Let's assume your regular place of employment pays you a salary of $100,000, and they withhold exactly the right amount of tax every year. No refund, no payment due. But let's also assume you sold your heavily appreciated, long-term Bitcoin (BTC -1.32%) for a gain of $50,000 during the year. Assuming this gain is taxed at 15%, you can expect to owe $7,500 when you finally file your taxes. 

If the sale of your investment asset takes place in the beginning of the year, you may forget about it by the time you actually owe tax to the IRS -- the following April. This is why it's critical to keep track of your locked-in investment gains throughout the year and budget for the reality that you'll need to pay this tax in the future. 

You have a side hustle or freelance income

If you're a full- or part-time freelancer, you'll be intimately familiar (hopefully) with paying your own taxes. With no employer to withhold money on your behalf, you'll be responsible for a quarterly payment to the IRS in the form of "estimated tax." These are payments that you make on your own, directly to the federal and state agencies. Most tax programs will help calculate these payments for you, and if you aren't the DIY type when it comes to taxes, a good CPA will walk you through this process. 

If you fail to make any payments during the year, you'll owe the entire year's worth of tax when the filing deadline comes around. You'll not only owe the entire balance due to the federal government and to the state in which you live, you may also owe interest and penalties for not paying throughout the year. This can get very complicated and is explicitly dependent on your own situation, so make sure to conduct your own research or hire an expert if needed.

Stay on top of your finances

Very few people are perfect when it comes to paying taxes -- chances are, you'll either owe money or receive a refund when it's time to file. But you can get a very rough sense of where you stand by projecting your income for the year and thinking about what you might owe down the line. Having a plan around this will save you huge amounts of stress, so be sure to think ahead and never be afraid to ask for help if you feel you need it.