The allure of living abroad after retirement has proven too strong to resist for hundreds of thousands of Americans.

Enjoying new experiences while immersed in different cultures. Getting away from it all. Stretching your nest egg in cheaper locations. Whatever your individual reasons may be, you won't be alone in making the move.

The Social Security Administration (SSA) says that as of December 2021, it was paying benefits abroad to 695,086 people, including 443,546 retirees collecting $386 million a month. Canada led the list of the latter at 71,376 benefits-collecting retirees, followed by Japan at 53,909 and Mexico at 32,957.

A person reads an e-book while laying in a hammock.

Image source: Getty Images.

So, clearly the answer is yes, you can collect Social Security while living abroad, and for that matter, continue paying into the system if you're living abroad and still working.

Here's how the SSA puts it: "If you are a U.S. citizen, you may continue to receive payments outside the United States as long as you are eligible for payment and you are in a country where we can send payments."

The U.S. Treasury Department bans direct payments to people living in North Korea and Cuba. Several former Soviet republics complete the list, but individual exceptions can be made there.

Of course, none of this is as simple as sending in a change of address form. Here are some other considerations if moving abroad is in your retirement plans:

Double taxation

You may not be a citizen, but you still have to pay taxes. Every country has its own tax rules, and one particularly big thing to consider is how to avoid being taxed twice for the same income.

A good way to start is by focusing on the 25 countries that currently have "totalization agreements" with the United States. They're designed to benefit retirees and working folks alike by coordinating the social security and national medical insurance programs between the two countries.

The rules vary but in general, those treaties are aimed at helping to eliminate or at least reduce double taxation on your Social Security income and allow you to continue your benefits uninterrupted even after you move. If you don't live in one of those countries, not only could you pay taxes to both countries, but there could also be other restrictions on your income.

Full retirement age and "the foreign work test"

Also, the rules around retirement age and other factors that determine how much Social Security benefits you'll actually get still apply. And if you're under full retirement age and decide to work in your new country, keep in mind the "foreign work test."

That means that the SSA will withhold benefits for each month that you work more than 45 hours in jobs that aren't subject to U.S. Social Security taxes. And if you're not a U.S. citizen but have collected enough work credits to qualify for Social Security, there are rules around that, too, if you leave the U.S. permanently.

The SSA recommends that you consult with the agency or otherwise seek professional advice to help navigate the complexities and fully understand the financial implications as far as your hard-earned benefits go. For that matter, the same thing goes for legally moving abroad. Make sure you understand the residency and citizenship rules that pertain to the place you plan to retire.