Should You Raid Retirement to Invest in Real Estate? | White Coat Investor

[Editor’s Note: Today we’re going to take a deep dive into a topic that was briefly covered recently on the WCI Podcast. I thought this topic lent itself particularly well to the Pro/Con format so I have asked Dr. Kenji Asakura, to take on the Pro side (since he has actually taken money out of retirement accounts at least once to invest in real estate) and I will take the Con side of the argument. Dr. Asakura and his wife, Dr. Letizia Alto, blog at Semi-Retired, MD about achieving financial independence through real estate investing.]

 

Pro: Raiding Retirement Accounts Is Reasonable in Some Circumstances

By Dr. Kenji Asakura

Conventional wisdom says that all employees should maximize savings using retirement accounts. I generally agree with this, especially when an employer offers matching funds. In fact, my wife Letizia and I have been participating in our employer’s 401(k) plan for years. However, the question we are exploring today is whether or not it makes sense to tap into your retirement account and perhaps even liquidate it entirely.

For the purposes of this pro/con article, let’s focus on reasons why you might consider liquidating your retirement account. Now keep in mind, liquidating your retirement account comes at a high cost. First, there’s the 10% penalty. Then, you are taxed on the money you pull out, both at the federal and state level. So why would anyone want to liquidate their retirement account? Here are some reasons:

#1 You Are Planning to Claim Real Estate Professional Status

Let’s start here because we don’t believe anyone should consider liquidating their retirement accounts unless they have a really good plan for addressing the hit from taxes and the 10% penalty.

This is where real estate professional status comes into play. My wife and I have liquidated our 401(k)s because I have real estate professional status and, with the depreciation from recently purchased rental properties, we can shelter all of the liquidated funds and pay zero income taxes on it.

So we minimize the damage of liquidating our retirement accounts to just the 10% penalty, which we feel confident we’ll surpass in our first year with our rental properties because we invest in properties that return far greater than 10%.

Becoming a real estate professional isn’t the only way to overcome the taxes and 10% penalty. Maybe you take advantage of an opportunity zone fund or a conservation easement. Maybe you have an amazing business idea that you anticipate will set you up far better than your retirement accounts can. Whatever may be the case, be sure to have a plan for overcoming the taxes and 10% penalty before liquidating your retirement accounts.

# 2 You Want to Retire on More Than What the Calculators Tell You

If you’ve ever gone to a financial advisor, you’ve seen a retirement calculator. They take the amount you are contributing to retirement and apply an interest rate and with compounding, your nest egg grows. For most of us, this amount is relatively decent. But what if you aspire to retire on more than this amount? What if the number you see in the calculator doesn’t even come close to helping you achieve your dreams?

Let’s use a sports analogy.

Relying on your retirement account is like a team that plays defense the entire game. You’re playing a low scoring game. You’re not playing to win, you’re playing not to lose. You run the same play all game. You’re predictable. Now back to the real world. You’re a physician and you are saving for retirement. Just like I can tell you that a team playing defense won’t score a lot of points, I can tell you exactly how much you’ll have in retirement when you’re 59 ½. It’s completely predictable. And the number is “comfortable,” but in the big scheme of things, it’s not a lot.

Maybe that’s great for some but many want more than the predictability of a low scoring game. They envision more for their retirement. They want to make more so they can do more. And your retirement account is a substantial source of funds that can be used to pursue an alternative investment strategy.

It really comes down to what you want in life and realizing that you have choices. You don’t have to play defense and run the same play over and over again. You can choose to play offense and run the score up if you want. The possibilities are endless, but only if you play to win.

# 3 You Want to Enjoy Your Money Now, Rather Than Having It Tied Up in “Money Jail”

We all enjoy our freedom, so why would you willingly put your money in jail? Instead of choosing to put your money in an alternative investment vehicle, for example, investing in cash-flowing rentals, you willingly choose to put your money in a retirement account that penalizes you if you take your money out before an arbitrary 59 ½.

Some of us want more degrees of freedom than retirement accounts can offer. However, most default to retirement accounts because you don’t know that other options exist. Oh, and keep in mind, just like retirement accounts, the federal government wants you to invest in real estate. That’s why real estate has such favorable tax treatment in the form of tax deferred growth using 1031 exchanges. And, at the same time, your money isn’t in jail for what might feel like a life sentence.

# 4 You Want to Have Another Source of Income (from an Investment or Business), So You Aren’t Reliant on Your Employer

It’s hard to talk about 2020 without bringing up COVID. For many doctors, 2020 has been a time of tremendous uncertainty. Some of you got furloughed. Others had to close their offices. Many experienced a significant drop in income. Our own hospitalist team had their bonuses taken away, which represents about 20% of our income. And oh, by the way, our hospital system received over $5 billion from the federal government in COVID-related aid.

Even if you didn’t experience a loss of income, you probably know someone who did. So, for the first time, I think all of us doctors realized that our jobs and incomes aren’t safe. We can’t rely on them like we thought we could. And, for the first time, many of us started looking for alternative sources of income. But the reality is that in order to make money, it helps to have money to invest. So some, especially the students in our real estate course, started considering tapping their retirement accounts.

While we don’t encourage this in our course, we also don’t discourage it. We say it’s a personal choice and the decision should be made carefully after taking into consideration the pros/cons, which for us is one of the goals of this pro/con article. Dr. Dahle’s perspective is completely valid and his thoughts should be weighed carefully as you make this decision.

# 5 You Want to Achieve Financial Freedom More Quickly

I think one of the things that often gets lost in discussions about retirement accounts is the fact that they don’t give you any income. The whole idea behind these accounts is that any gain is reinvested and there are severe penalties for tapping the income before 59 ½.

So you save and save and create a nice nest egg, but it doesn’t pay you one cent until you’re no longer young. But what if you could invest in something that puts money in your pocket each month? Even sweeter, what if that income was tax-free? You could then choose to live off of that income while you’re still young enough to enjoy it, or you can reinvest it to achieve the same compounding growth you achieve with retirement accounts.

The nice thing about doing something different is you have a choice. What we do is a mix. We use some for ourselves so we aren’t always delaying gratification, and we reinvest some. As our cashflow grows, we don’t have to reinvest as much of it and we have more for ourselves to enjoy TODAY, not tomorrow. It gives you an opportunity to achieve financial freedom well before 59 ½.

# 6 You Believe Taxes Are “On Sale” Now

One of the touted benefits of retirement accounts is that you’ll pay less taxes on your retirement income because you’ll be making less. The problem with this argument is that it assumes tax rates will stay the same.

But what if taxes go up dramatically? Why would this happen? Well, Social Security is going bankrupt, and oh, by the way, those $1,200 COVID checks aren’t free. We’ll be paying for them at some point (or maybe our kids will). Some argue that taxes are on sale now. They say that the tax rates will never again be this low.

If that’s the case, doesn’t it make sense to pay today’s tax rates rather than a much higher rate in the future? Of course, you’d have to account for the 10% penalty but some might believe that the increase in tax rates will exceed even the penalty.

I acknowledge the fact that you can do a Roth conversion and pay today’s tax rates. But your money is still in jail. So the point is if you are planning to liquidate, better to liquidate now while taxes are on sale and you have more time to grow your alternative investments than wait until later.

# 7 You Realize That Retirement Isn’t the Goal

Finally a different twist on the pro/con argument. What if you don’t plan on retiring because you love what you do? My wife and I achieved financial freedom but keep on working on Semi-Retired MD and our real estate portfolio because we enjoy both. We’ve been there. We’ve semi-retired and traveled the world. However, what we realized early on in our “semi-retirement” was that, without continual growth and contribution, we were not fulfilled.

This means that we seek out opportunities to continually challenge ourselves to grow and we do what we do from a place of contribution. This is why we work so hard to ensure that our students are successful. So for us, retirement accounts are irrelevant. We’ll keep working, on our terms, as long as we live. So we’ve liquidated some of our retirement accounts and plan to liquidate the rest as new investment opportunities arise. It’s not for everyone, but it really depends on what you want – do you want to play defense all your life or do you want to play to win?

 

Con: Most Doctors Should Not Tap Their Retirement Accounts

By The White Coat Investor

It always makes me worry a little bit when I see a book, course, guru, insurance agent, or financial advisor suggest taking money out of a retirement account in order to do something else with it. It often means that person is trying to sell you something. A classic example is Douglas Andrew, who under the brand names of either Missed Fortune or Live Abundant has been suggesting for years that people pull money out of their retirement accounts (and home equity) in order to buy Index Universal Life Insurance, a poorly performing but high commission product similar to whole life insurance. You can see why these folks selling something would try to get you to tap those sources of money in order to buy it—that’s where a large percentage of your net worth resides!

Benefits of Retirement Accounts

The benefits of investing inside retirement accounts are numerous, particularly for a high earner such as a physician. The primary benefits are tax advantages. For a tax-deferred account like a 401(k), you get a large upfront tax deduction (usually the largest deduction for a doctor), and ongoing protection from the tax drag of having the income component of your return taxed as it grows. In addition, you can switch investments without any tax consequences. Perhaps the great benefit of a tax-deferred account for a doctor is the ability to save on taxes at a very high marginal tax rate during the peak earnings years and then use withdrawals during retirement to “fill the brackets” during retirement. Contributing at a marginal rate of 32-37% and then withdrawing at an effective rate of 10-20% is a winning formula. With a tax-free (Roth) account, a doctor does not get the up-front tax break nor (typically) any arbitrage between tax rates, but she does get tax-free withdrawals, allowing her to set her own tax break in retirement.

In addition to these tax benefits, retirement accounts facilitate estate planning with the ability to name beneficiaries and the ability for heirs to stretch those retirement account benefits out for an additional ten years after your death. There are also serious asset protection benefits for retirement accounts. In every state, 401(k)s are essentially bankruptcy-proof, and that protection extends to IRAs in many states.

Downsides to Retirement Accounts

Are there downsides to investing in retirement accounts? Sure, but they pale in comparison to the upsides in the vast majority of cases.

One downside is that if you want to use that money prior to age 59 1/2 (55 for 401(k)s once you leave the employer), you had better have a good reason because, if not, you will have to pay a 10% penalty in addition to any taxes due. Fortunately, the government has created a long list of good reasons, including early retirement, that allow you to get at that money without paying the penalty.

You also have to start taking the money out of tax-deferred accounts (and paying taxes on it) and Roth 401(k)s beginning at age 72, whether you want to spend these Required Minimum Distributions or not. But the percentage of the account that must be taken out in your 70s is small, and it isn’t like you cannot just turn around and reinvest the money in a taxable account if for some crazy reason you saved up a bunch of money for retirement but do not actually want to spend it on retirement.

A more serious issue with retirement accounts is that not every investment out there is easy to invest in (or even allowed in) a retirement account, and you generally cannot use leverage to invest in a retirement account. There are workarounds here, but they can be a bit onerous.

For example, a self-directed IRA (or even better, a self-directed individual 401(k) because it allows you to avoid Unrelated Business Income Tax on leveraged investments) can be invested in a wide range of non-traditional investments. Since money is fungible, it is also possible for the leverage to be held outside the retirement account in the form of a mortgage on your primary home or loans against other properties or a margin investing account.

The Price of Early Withdrawal

The main problem for a physician who wants to pull money from retirement accounts to invest in real estate is the heavy price required to take the money out of the retirement account. In my case, I would pay a 37% federal income tax plus a 5% state income tax plus a 10% penalty to take that money out. 52% total! So in order to get $100,000 out, I would have to pull out $208,333 and pay $108,333 in taxes just to invest $100,000 into real estate!

Paying 52% on a withdrawal that you might be able to only pay 20% on later (not to mention let it keep compounding in a tax-protected and asset-protected account for decades more) is pretty foolish. It would take massively higher returns on the new investment in order to make up for taking that tax hit.

Investing in Real Estate Without Withdrawing

In addition, it is not like you cannot invest in real estate inside retirement accounts. I mentioned above the possibility of using self-directed IRAs and individual 401(k)s to buy individual properties, but they can also be used to buy syndications and private real estate funds.

Sure, you will not get the benefits of depreciation or the ability to write off the associated property expenses, but you will still get to pay for those expenses with pre-tax dollars (in the case of a tax-deferred account) and, over decades, the retirement account associated tax protection on the income will add up to more than the benefits of the initial depreciation.

Publicly traded real estate, such as individual REITs and the mutual funds that invest in them (like the Vanguard REIT Index Fund) are also widely available. Some real estate investments that do not benefit from depreciation (such as a hard money/private loan fund) are incredibly tax-inefficient and provide much higher after-tax returns when held in a retirement account.

In addition to these methods of using retirement account money to invest in real estate, you can also take a loan out of your 401(k). The terms on these are much less onerous than they used to be. You can borrow the money for up to five years and you no longer have to pay it back within 60 days of leaving the company—you have until tax day of the following year to pay it back. Plus, for 2020, the amount you can borrow out of the 401(k) has doubled to $100K from the $50K (or 50% of the account balance, whichever is less) limit that was in place prior to the COVID associated downturn.

There is also the option that most physician real estate investors choose. They simply use their retirement accounts to hold their stock, bond, and REIT mutual funds and use their taxable money to buy properties, syndications, and equity funds. Not enough taxable money to meet your real estate goals? Try working more or spending less and you soon will likely have more than enough.

What About Super-Savers?

Drs. Alto and Asakura are real hustlers and expect to be in the highest bracket in retirement, so they do not expect to see much of a tax rate arbitrage between their tax rate at contribution and their tax rate at distribution, and, in fact, think it might be negative, especially considering their current state tax rate (0%). However, there is an alternate solution to just pulling the money out and paying the taxes due plus the 10% penalty. You can do Roth conversions on that money. You will pay the same tax bill, but you will not owe any penalty and you can now take advantage of all of the tax, estate planning, and asset protection benefits of a Roth IRA for decades. While the typical physician is not in this situation, many super-savers are.

Real Estate Professional Status

Letizia and Kenji are also careful to point out the main reason why they thought this scheme would work for them, despite fully acknowledging that it will not work well for most doctors—they qualify for Real Estate Professional Status (REPS). Under current bonus depreciation laws, one receives a ton of depreciation upfront when you buy a property. This can be used to offset real estate income, essentially allowing for tax-free income, at least until that depreciation is recaptured at the time of sale. However, that income cannot be used to offset your earned (usually highly-taxed physician) income UNLESS you qualify as a real estate professional. As you can imagine, qualifying for that status can make a huge difference on your taxes, and it certainly has for these two doctors. They used the depreciation from their investments to cover the taxes (but not the penalty) on their early retirement account withdrawals. But I think it is important to understand just how difficult it is to qualify for REPS as a practicing physician. There are basically two rules:

1. You must work in real estate more than 750 hours per year
2. You cannot work in anything else more than you work in real estate

Needless to say, most doctors are not going to qualify for this until they are basically transitioning from medicine to real estate as a career. It is slightly easier to get your non-physician spouse to qualify, but there is still the 750 hours per year rule. How many hours is 750? That’s 62.5 hours a month. Or the equivalent of eight, 8-hour days per month. It’s a half-time job. You’re not going to get that consistently with just one or two properties unless you are the world’s least efficient real estate investor. Since most people get interested in real estate (passive income!) because they want to work less rather than more, this really is not a workable solution. Go ahead. Try to talk your partner into taking on a half-time job in order to save some money on taxes. Let me know how it goes.

All in all, I am in full agreement with these two fine doctors that a circumstance where pulling money from retirement accounts in order to pursue an entrepreneurial pursuit including real estate investing is a good idea can exist. I just think it is so rare that the rule of thumb to not raid retirement accounts for anything but retirement should still be the default solution.

What do you think? Under what circumstances would you raid your retirement accounts to start a business or make an investment? Comment below!