A $1 million retirement portfolio might sound like a dream to many.

Just 3.2% of Americans managed to save at least $1 million in their retirement accounts by the end of last decade, according to EBRI estimates based on the latest Federal Reserve Survey of Consumer Finances. They don't know anything special the average person doesn't, and most don't have access to any special savings plan that allows them to save more in their 401(k). But there are a few simple secrets that allowed them to reach the $1 million milestone.

Here are three secrets of 401(k) millionaires.

An egg with the word 401k printed on it lying on top of $20 bills.

Image source: Getty Images.

1. They get their full company match

The biggest benefit of a 401(k) is the company match. If you aren't contributing enough to get your full matching contribution, you're practically leaving free money on the table. And with recent rule changes, it's easier than ever to qualify for a matching contribution.

The way a 401(k) match works is typically based on your salary. An employer might offer a $0.50 match for every dollar you contribute up to 6% of your salary. Or it might offer a 100% match up to 3% of your salary. Every plan is different, so be sure to read the details of your employer's 401(k) plan to know just how much you should be contributing to max out this benefit.

Importantly, the amount needed to get your full company match is usually well below the maximum allowable contribution to a 401(k). For 2024, the 401(k) deductible contribution limit is $23,000, or $30,500 for workers age 50 and older. But you don't need to contribute that much to reach $1 million for retirement.

If you save 10% of your salary between your contribution and your company match, you could end up a millionaire over the long term. With an average salary of around $60,000, you'll end up saving $6,000 per year. Do that for a full 40-year career and you're likely going to end up a 401(k) millionaire.

2. They keep their fees low

One of the biggest drags on 401(k) savings are the fees. Fees can range anywhere from almost nothing to around 2%. And if you're not paying attention, it could be costing you a lot of money.

There are three types of fees in 401(k) plans: administrative fees, investment fees, and individual service fees. There isn't much you can do about the first one, but the latter two are typically under your control, at least to some degree.

The easiest way to lower your investment fees is to review your investment options and pick a fund with one of the lowest expense ratios. The good news is the funds with the lowest expense ratios are typically the best long-term investments for a 401(k) -- broad-based index funds or exchange-traded funds (ETFs). While you might gravitate toward a target-date retirement fund or an actively managed mutual fund, the cost of those funds is often too high to justify the cost. While there are some cases where target-date funds use index funds and keep costs low, that's not always true.

Individual service fees are for accessing features of a 401(k) like a loan option or using the participant-directed option. Note that if the fund options in your 401(k) aren't very good, but it has a participant-directed option with a reasonable fee, you may be able to save more by using the self-directed account.

Fees in a 401(k) are a big reason many savers roll over their 401(k) to an IRA as soon as they can. IRAs generally don't have any fees except the expense ratios charged by your mutual funds or ETFs.

The value of keeping fees low cannot be understated. A 1% difference in fees could result in a massive difference in total savings after 40 years. Someone saving $6,000 per year, as in the example above, could miss out on hundreds of thousands of dollars over their career.

3. They don't withdraw funds unnecessarily

Famed investor Charlie Munger is often said to have remarked, "The first rule of compounding is to never interrupt it unnecessarily."

The first few years of saving may feel like pushing a boulder up a hill. A portfolio with just a few thousand dollars in it doesn't grow very fast. Even a 50% return on a $6,000 portfolio is just $3,000. You'll probably add more money of your own into your account by the end of next year.

But patiently allowing your money to compound over decades will result in a massive portfolio. That's the nature of compounding. It takes a long time for you to build enough momentum that the boulder practically pushes itself. Soon enough your portfolio will generate more in returns than you contribute.

If you want compounding to work for you, though, you can't take your money out of the market. While the stock market can fluctuate wildly, it goes up over the long run. When the market is down and you feel like taking your money and running, take a look at a chart of the S&P 500 index dating back to the 1920s. When the market is making new highs and you think it's time to get out, remember the wise words of another famous investor, Peter Lynch, "People who exit the stock market to avoid a decline are odds-on favorites to miss the next rally."

If you stay the course, you'll reap the benefits. And that's how you get to a $1 million retirement portfolio.