Investing in an IRA can be one of the best ways to set yourself up for a worry-free retirement, but it's important that you do it right. We have written extensively about the right kind of stocks for an IRA and strategies to maximize your contributions, but not much has been said about what you don't want to do. There are certain IRA mistakes that are very important to avoid. Here are three from our retirement experts:

Matt Frankel: One costly mistake a lot of people make in their IRA is overtrading, which not only costs a lot in the way of commissions, but defeats the purpose of what an IRA is meant for -- long-term, buy-and-hold investing.

There are two main types of overtrading to avoid. The first, and most obvious, is buying and selling stock positions frequently. Too many investors sell as soon as a stock goes up by a few hundred dollars, and then look elsewhere for another quick profit. Not only do those $9.99 commissions add up this way, but you deprive yourself of the opportunity for long-term gains in the process.

In an IRA, you should focus on stocks that you believe in for the long run. The only reason you should sell one of your position is if something changes that makes your long-term opinion of the company change. For example, if a new competitor is causing your stock's market share to erode, it might be a good reason to sell. If your stock went up and you are tempted to sell and lock in your gains, it's probably not a good reason.

Also, make sure you're not buying stock every time you deposit $100 (or another relatively small amount) into your IRA. If you buy $100 worth of stock, a $9.99 commission represents about 10% of your purchase amount. So, your shares have to appreciate by 10% just to break even. A better idea is to wait until you have a more significant amount of cash, and then invest all at once. If you deposit $100 at a time, waiting until you have $500 to invest cuts your commission from 10% of the purchase to 2%. This can have a huge effect in the long run.

Dan Caplinger: One of the least understood aspects of IRAs is what happens to the account after death. It largely depends on who you name as your beneficiary. Surviving spouses have the right to roll over inherited IRA proceeds into their own IRAs, but other family members have to follow more complex inherited IRA rules.

Specifically, you have two options when you inherit an IRA. You can take out all of the proceeds within five years of death, as long as the person you're inheriting from hadn't yet started taking required minimum distributions. Alternatively, as long as you start right away, you can stretch out IRA distributions over the course of your lifetime, using life expectancy tables to determine your own required minimum distribution each year. For instance, if the IRS life expectancy tables say that you can expect to live 50 years after you inherit the IRA, then you'll have to take 1/50th or 2% of the account the first year, then 1/49th the next year, and so on until the entire account is withdrawn.

Stretching out distributions can reduce the tax impact of IRA withdrawals, since you have to include every dollar you take out of an inherited traditional IRA as taxable income on your return. Failing to follow the rules, though, can lead to big penalties and other headaches.

Jason Hall: One of the biggest mistakes that people make in their IRAs is buying actively managed mutual funds. You may think an active fund should get better returns, but the reality is the vast majority actually perform worse than the benchmark they are measured against.

Why not just pick the best active funds? Unfortunately, the best funds don't remain the best funds. According to research by S&P Dow Jones Indices, less than 1% of large and mid-cap funds in the top quartile five years ago remained in the top quartile as of December.

So not only do these funds tend to underperform, they tend to have inconsistent results. To top it off, actively managed funds charge significantly higher fees than passive index-based funds, so you actually pay a premium for underperformance.

The fees can look small at 2% per year or less, but they can really kill your returns, especially when combined with underperformance:

Based on S&P 500 annualized total returns since 1983, and SPIVA five-year average underperformance for large cap active funds.

The data above is based on the actual underperformance -- net of fees -- for the average large-cap fund over the past five years, versus investing in a low-cost index fund like the Vanguard 500 Index Fund. As you can see, when you stretch it out over the length of a career, you're looking at a huge impact on your returns.

Don't pay a premium for underperformance. It could be your biggest IRA mistake.

Try any of our Foolish newsletter services free for 30 days. We Fools may not all hold the same opinions, but we all believe that considering a diverse range of insights makes us better investors. The Motley Fool has a disclosure policy.