We're counting down to Nov. 3. The election between Donald Trump and Joe Biden is one of the most anticipated elections in modern history. The only thing that holds more sway in the news is the ongoing COVID-19 pandemic. With a ton of votes already in early, and polls opening Tuesday morning, just what does an election do to the stock market?

What really matters

According to analysts at U.S. Bank, a division of U.S. Bancorp (USB -0.20%), equity and bond markets tend to underperform their typical 12-month trends during the year of a general election.

It's not hard to see why when you consider all the noise. Every single news source reports evolving commentary daily. Within that commentary, citizens (including investors) are flooded with a lot to think about. Will a liberal government sway the market with tighter regulations and taxes on industries? Will four more years of tough trade stances from the Trump administration weigh on international dealings?

Man Voting

Image source: Getty Images.

Following an election year, U.S. Bank noted, stock markets have a habit of closing slightly lower, while bond markets outperform. Perhaps most interesting is that the party taking office doesn't have a large impact on the trends; the bigger factor is whether there is a change in who is president. Specifically, stock market returns average around 6.5% if an incumbent party maintains hold of the presidency, or if the president is reelected. If the party changes, markets average gains of around 5%.

Jurrien Timmer, Fidelity Investments' director of global macro for their global asset allocation division, noted that the first two years of a presidential term are mostly associated with below-average returns, while the second two years tend to outperform.

But this has been a strange year, with a plethora of influences on the market and economy that has spurred a large variant from the norm. Going into election week, 2020 has basically been a wash for the the market, as COVID-19 sent the S&P 500 spiraling in the spring. The index has largely recovered, but the majority of the strength has come from tech companies, with access to the needs of remote working and social distancing. 

The speculation gets overblown 

There are so many working parts to an economy that it's hard to give too much credit to one person. In many ways, the Federal Reserve and Congress have far more influence on the economy than the President does. Congress can drive fiscal spending, and the Fed holds control over credit markets and liquidity.

Very rarely has a shift in party governance meant catastrophic results for the economy or security markets. A notable exception might be the failure of Herbert Hoover, and the government at that time, to act during the crash of 1929 and subsequent Great Depression. But the cause of that crash was not the government. The culprit was good ol' speculation run amok, with leveraged debt to buy securities that were inflated. Those types of trends are what investors should watch out for. Look at the market today. You'll find that some of the best performing stocks this year command pretty big premiums that are not in conjunction with the actual earnings of the company. The P/E ratio of Tesla (TSLA 1.85%) should influence your decisions far more than whether or not the President supports electric cars. 

If you're making long-term investment decisions based on an elected official who could be around for eight years at most, you're already thinking too small. Throughout the last 50 years, in which we have had nine Presidents, the S&P 500 has grown by over 4,000%. Again, certain industries can flourish or suffer based around certain policies, but the broader market has always muscled on.

Traders might be able to take advantage of volatility during a presidential election, but for the average investor, it doesn't pay to take part in that speculation. Maintaining a well-thought-out, diversified portfolio that isn't focused on the short term is still the intelligent way to play the game.

1 sector that could see a lot of volatility

I'll extend an olive branch to those that are heavily engaged in the impact that the election could, or could not, have on markets.

A sector that could see a wild ride is healthcare. The debate over private healthcare systems and government funded plans is an ongoing fight between Democrats and Republicans. It's no secret that the two parties don't agree on what they want when it comes to our nation's healthcare system. That ongoing debate and uncertainty could scare investors in companies like UnitedHealth Group (UNH -1.03%) or Humana (HUM 0.82%)

Joe Biden's stance on healthcare includes a call for a public option. No matter how you slice it, that creates competition for private sector insurance companies.

President Donald Trump supports replacing Obamacare with a new plan. There hasn't been a ton of information provided on what he wants exactly, but it includes the desire to lower drug costs and drive down premiums. That puts pressure on healthcare stocks as well.

As we've learned time and again, nothing is easily achieved when it comes to actually creating the legislation. Nevertheless, speculation and fear could very easily run amok within the healthcare sector. If the supreme court does take up the issue, the matter will only get more volatile.

Stick to the fundamentals 

Over the course of time, no election has resulted in permanent damage to the stock market. The S&P 500 has increased through time, regardless of the who is in the White House. There can of course be short term volatility, and certain policies can have detrimental implications for certain industries, or specific companies. That's why it will never backfire to stick to the fundamentals of diversification and long-term thinking.

Make sure you think rather than simply buying something because Joe Biden or Donald Trump won. Justify it with analysis. As pointed out, it might be best to go light on healthcare stocks. Regardless, investing in a diversified portfolio, with an emphasis on long-term performance, can withstand election volatility.