Gold is often positioned as an asset people should invest in to help with inflation or down markets, but the logistics behind investing in physical gold (such as transporting it) can be a turnoff for some people. Luckily, there are gold ETFs that are backed by real gold, allowing investors to get exposure to gold in their portfolios without possessing it physically.

While it may be worth considering, here are two reasons you shouldn't add gold to your portfolio.

Rows of gold bars.

Image source: Getty Images.

1. Gold ETFs face a higher maximum long-term capital gains tax rate

If you hold an investment for less than a year, any profits will be taxed at your regular income tax rate (between 10% to 37%). If you hold an investment for longer than a year and sell it for a profit, you'll have to pay capital gains. Unlike the seven tax brackets for income, capital gains primary fall into one of three rates, as seen in the chart.

Income Range Long-Term Capital Gains Tax Rate
$0 to $40,400 0%
$40,401 to $445,850 15%
$445,851 or more 20%

Data source: IRS.

If you buy 10 shares for $50 each ($500) and sell those same shares in six months for $100 each ($1,000), you'll pay your income tax rate on the $500 profit. If you make $100,000, you will pay 22%. However, if you held those same 10 shares for two years and made the same $500 profit, you'd only owe 15%.

The majority of assets fall into the 0% to 20% capital gains tax rate range, but there are a few exceptions, one of which is the rate applied to assets categorized as collectives, such as precious metals, coins, stamps, and artwork. Although gold ETFs are technically ETFs, because they're backed by gold -- labeled as precious metal -- they face a higher 28% capital gains tax.

If you earn $100,000 and invest $1,000 in a gold ETF that you later sell after longer than a year for $3,000, instead of owing $300 (your long-term rate), you'd owe $440 (the lesser of your short-term rate of 28%. Knowing the exact tax rate you'll face when selling an asset is important so you don't underestimate your bill and throw off your financial planning.

2. There are cheaper options to choose from

Like the majority of ETFs, gold ETFs come with an expense ratio. The expense ratio is charged annually as a percentage of the amount you have invested in the fund. Gold ETFs can be a bit more pricey compared to some other major ETFs.

Here are the three largest gold ETFs by assets under management that handle gold as a commodity.

Gold ETF Ticker Symbol Expense Ratio

SPDR Gold Trust

(GLD 0.68%) 0.40%

iShares Gold Trust

(IAU 0.65%)  0.25%

VanEck Vectors Junior Gold Miners ETF

(GDXJ 2.56%)

0.52%

Data source: Company websites.

At those current expense ratios, you'd pay $4, $2.50, or $5.20 per $1,000 you have invested, respectively. Although slight differences in percentages may seem minimal, they can add up to a decent amount over time. There are ETFs -- such as the iShares Core S&P 500 ETF (IVV -0.50%) and the SPDR Portfolio S&P 500 ETF (SPLG -0.51%) -- with a 0.03% expense ratio. 

If over time you managed to accumulate $100,000 into the three gold ETFs mentioned and one of the S&P 500 ETFs, here's how much you'd be charged annually.

ETF Expense Ratio Amount Charged With $100,000 Invested
GLD 0.40% $400
IAU 0.25% $250
GDXJ 0.52% $520
IVV or SPLG 0.03% $30

Data source: Author calculations.

You should not shy away from gold ETFs solely based on expense ratio, but you should always be knowledgeable about how much you're paying to hold an investment because it can eat away at your returns or add insult to injury with losses. Being knowledgeable can save you from a surprise tax bill.