Eight years ago, I bought my first condo with just under $5,000 down. We only lived in it for a few years, and when we moved, we decided not to sell and instead rent the condo out.

We made good income on it for the first few years, but you certainly couldn't call it passive income. In between renters, we would spend hours cleaning, painting, repairing, listing, showing, and screening tenants. And when we had renters, we'd spend hours and a ton of money fixing things and listening to complaints.

Since then, we've moved a few times and now have three rentals that churn out totally passive income. I talked to the property manager maybe once a quarter but get to cash checks every month.

Here are five steps to turning $10,000 (my inflation-adjusted down payment) into a passive income empire.

Two people and a dog walking past a house.

Image source: Getty Images.

1. Start small

The first step is to buy your first property. I'd recommend buying it as a residence. Because we lived in the condo for more than a year, I only had to put 5% down to purchase the condo. If you buy an investment property with the sole intent of renting it out, you'll likely have to put 20% or 25% down -- more likely 25%, if you've never owned a rental before.

With a $10,000 down payment, living in the property versus buying it as a rental is the difference between being able to buy a $200,000 property and a $40,000 property. So the advantage of living in the rental first is clear.

Living in the rental will also give you time to learn the locality, fix any easy issues, and generally get more competence for the unit. You can also use the time to save for your next property and have some cash reserves.

2. Create an automatic saving plan

Every month, one of our brokerage accounts automatically pulls cash out of the bank account. Right now, it pulls about 15% of our net salaries each month. It's been higher in the past, especially before we had to pay for day care, but the key point is that the amount is automatically pulled each month.

Once it gets into the brokerage account, I invest it in low-risk municipal bond ETFs and low-volatility dividend-paying stocks. For our "down payment account," we don't want the risk to be so low that the returns are on the level of a savings account, but we also want to be able to pounce on real estate opportunities when they come -- so any stock that can fall 30% or more in six months is out.

Over time, we build up enough cash in that account to buy another property. So far, we've moved each time and lived in the next property. Eventually, we'll stay in a residence for the long term and start using this cash to purchase rentals directly.

3. Put the properties into an LLC

Once you've accumulated a bunch of properties, it's time to get them into a limited liability corporation (LLC). This will protect you from any personal liability and help centralize the accounting and finances of your properties.

We did this by refinancing the three properties into one commercial loan and quitclaiming the deeds into the new LLC. The LLC now owns the properties and therefore has the liability. Not only does this make our risk lower, but our credit reports look a lot better, so it will be easier to buy another house in the future.

4. Use a property manager

Property managers are the key to turning your rental income into true passive income. Being a landlord is a ton of work. The property manager will find and screen new tenants, take care of little issues and hire someone to take care of big issues, keep direct property bookkeeping in line, and sometimes even help you sell the property when you're ready.

5. Build your empire

Once you have your process in place, keep the momentum going. Set aside reserves, invest some of your savings in the stock market, and then put the rest into your down payment account. The more quickly you can buy new properties, the more cash flow they'll throw off that can go into the down payment account.

Soon you'll have savings to buy a new property every few years using just the cash flow generated by your existing properties. And then, eventually, you'll have enough cash flow from your properties that you can stop buying and stop working, living just on that income.