How to Optimize Your Money in Retirement – The White Coat Investor – Investing & Personal Finance for Doctors

[Editor’s Note: Today’s guest post is from David Graham, MD, an infectious disease physician and advice-only financial planner. We have no financial relationship. In this post, David hits on a wide range of financial issues that affect physicians, particularly during the decumulation (i.e. retirement) phase.]

White Coat Investors are excellent accumulators of wealth. Once you have your accumulation plan in place, it might be time to think about some issues in physician retirement planning.

Physicians are a heterogeneous lot, this much is true. There are, however, some common issues faced in retirement by those who wear the white coat. Let’s review the epidemiology of physician retirement and then go over some common issues in retirement planning for physicians.

Epidemiology of Physician Retirement

When Do Physicians Retire?

 

Source https://www.ama-assn.org/practice-management/career-development/top-considerations-physicians-ahead-retirement

 

Surprisingly, 58% of physicians retire after the age of 65! In the US general population, women retire at age 60-62 and men 62-64. Do physicians retire later on average because of the late start we get after a decade of training?

Why Do Physicians Retire?

What does a metanalysis say as to why physicians retire? According to this study, physicians do indeed retire later than average.

Reasons to retire early include low job satisfaction, medicolegal issues, health concerns and financial troubles. Reasons given to delay retirement include career satisfaction, institutional flexibility, feeling responsible for patients, financial reasons and a lack of interests outside of medicine.

Finances show up on both sides—retire early and delay retirement. What are these financial concerns? Net worth is one of them.

Net Worth of Physicians

Let’s look at physician net worth by age.

 

Maybe you’ve seen the data in Figure 2 before. It’s from a Medscape survey of physicians and modestly depressing.

While you can easily retire with less than $5M, especially when you are traditional retirement age, look at the purple bar (net worth > $5M) increase with age. It goes from 3% up to 22%. So, about 1 in 5 physicians over the age of 70 have a net worth greater than $5M.

The percentage between $1-5M doesn’t really change much with age. Sadly, 25% of physicians older than 65 never have a nest egg larger than $1M.

Summary of Physician Retirement Epidemiology

So, on average, physicians retire later than the general population yet with more financial resources. Still, given the high incomes physicians earn, it is depressing so many of our colleagues are not better prepared for retirement. Financial concerns are cited as reasons both to retire early and to retire late.

Let’s move on now. I assume anyone reading this is doing well for themselves. What are some issues that impact physicians planning their retirement?

Selected Issues in Physician Retirement Planning

Retire To Something

Everyone needs to consider what they plan to do in retirement.

Many physicians find more than a mere calling in medicine. Letting go the title of “Doctor” is difficult. Finding purpose in retirement, however, is necessary to ensure health and sanity.

It will be interesting to see trends in physician retirement in the next decade given the industrialization of medicine and the loss of autonomy many physicians are facing. Encore careers are common for physicians. How will we find purpose?

Liquidity Events

From poor distribution options on Non-Governmental 457 plans, to liquidating investments in practices, equipment or real estate, many physicians will face liquidity events.

A liquidity event happens when you must recognize a bolus of ordinary or capital gain income all in one year.

Of course, ideally, you would be able to take a better distribution option on your 457, or spread out your payout for your share of the practice or investment over two or more tax years.

If you are forced to “suffer” a liquidity event, some planning is possible. Tax planning is important that year, perhaps to reduce your other sources of income, or find a way to take a major tax deduction in the same year.

Other possibilities include Oil and Gas, Opportunity Zone Investments, Real Estate depreciation through cost segregation and bonus depreciation, and more. Plan at least a year in advance with your CPA.

Large Pre-Tax Accounts

Many physicians have large 401k or 403b accounts and face huge future tax burdens as Required Minimum Distributions force out this deferred income. These 401k Millionaires have the opportunity to do some tax planning which may include partial Roth conversions.

After you retire and before you start social security and RMDs, you have the chance to control your income. This Tax Planning Window is the most important opportunity to control your future tax liability and should not be squandered.

Other Tax Considerations

White Coat Investor not infrequently calls retirement planning issues “first world problems.” Given his gift for tax efficiency while practicing and running a burgeoning business, I expect he will change his tune when he considers retirement and sees how much he will pay in taxes during his own retirement! [To be fair, EVERYTHING discussed on this website is a first world problem. But if I pay more in taxes in retirement than I do now, it’s going to be one heck of a retirement-ed.]

In fact, taxes can be a physician’s largest expense not only while practicing but also in retirement. It pays to be tax efficient. After all, every dollar you save is another dollar you can send or give to heirs or charity.

Aside from partial Roth conversions, there are other tax-efficient withdrawal strategies. These include:

• minimizing your taxable brokerage account (yet maintain liquidity)
• use lower tax brackets to recognize pre-tax income
• order of withdrawal from various account types
• capital gain harvesting in the 0% cap gains tax bracket
• avoiding Medicare Surcharges (IRMAA — a Cliff Tax on the Rich)
• maximizing the efficacy of charitable deductions

Charitable Planning

Since the Tax Cut and Jobs Act, most have been using the standard deduction and no longer get tax deductions for our charitable gifts.

Not that a tax deduction is the only reason to give, but if you can find a way to give tax-efficiently, you can give more!

If you have charitable intent, there are many considerations. These include:

Legacy Planning

Leaving a legacy is important to consider as many doctors will have more money than time. The loss of the Stretch IRA has engendered some new retirement planning strategies. Unfortunately, none are as good as the Stretch IRA.

Cascading beneficiaries, disclaimer planning, multi-generation spray trusts, Roth conversions, Life Insurance, and Charitable Remainder Trusts are all considerations when planning your legacy.

Long-Term Care Insurance

Since we have all seen the ravages of age, Long-Term Care Insurance is a quandary in the back of our mind. While many retired physicians can self-fund long term care needs, there is a dearth of insurance options if you want to transfer the risk of a large Long-Term Care event.

Traditional Long-Term Care Insurance suffers from issues of premium increases and future solvency.

So-called hybrid LTC/Life Insurance policies are becoming more popular but have copious downsides. One issue I cannot get the insurance industry to understand about these policies is that you are often better off NOT TO USE one once you have it. This is because medical expenses are a tax deduction above an AGI floor. You can withdraw pre-tax money tax-free if you have a large enough deductible Long-Term Care event. So, if your goal is leaving money behind for your heirs, you are better off withdrawing pre-tax money tax-free to pay for healthcare costs, and letting the tax-free death benefit of the policy go to your heirs. Or, not having a hybrid policy in the first place!

Long Term Care considerations are the most intractable problem in retirement planning today. This is because they can have truly devastating costs, but insurance is priced to cover non-devastating events. Imagine how expensive home insurance would be if we all had small or moderate fires in our house at least at some point in our lives! Insurance is best saved for risk pooling of catastrophic events, which is not how Long-Term Care insurance operates currently.

Social Security

Social Security planning should be something that all physicians are familiar with. If you have done well and have a reasonable life span, consider delaying social security to take advantage of delayed retirement credits. Social security is, perhaps, the best longevity insurance out there, with inflation adjustment and a government-backed guarantee.

Most physicians should just assume that 85% of their social security will be included with their taxable income, as the brackets for taxation of social security are absurdly low and haven’t been inflation-adjusted since Regan.

Unwinding from a Bad Financial Advice

This is, perhaps, the most interesting retirement planning conundrum. Many physicians have been suckered into bad investments or products by “financial advisors.” Unwinding these takes care, consideration, and patience.

If you have an insurance salesman masquerading as a financial advisor, you might own expensive annuities or unnecessary permanent life insurance products. Unwinding these might just involve liquidation and some “stupid tax.” There are other important considerations, though.

With annuities, if you need income in retirement, you may consider annuitizing the policy, or actually using the Income Rider that you have been paying for all these years. Or, you could 1035 into an Investment Only Variable Annuity depending on the goal of the funds. I’ve found most folks who have been sold an annuity don’t really understand “why” they own the annuity and what purpose it is supposed to serve in the future.

With Permanent Life Insurance, you may consider a 1035 into a hybrid LTC policy if you want Long Term Care Insurance. WCI writes about other options for life insurance as well. Understanding the indication for owning a Permanent Life Insurance policy is key.

If your financial advisor is a stock picker, then you might have individual stocks or crazy expensive mutual funds with a low basis. Unloading these will involve paying capital gains taxes. Many doctors will remain in the 18.8% capital gain tax bracket on future projections, so a decision must be made regarding the purpose of the money. If you plan to unwind it at some point due to the tax-inefficiencies of the funds, earlier is better as it allows you to reset your basis and chose funds more in line with your goals and chosen asset allocation.

If you have a donor-advised fund, or bunch deductions, these low basis bad investments are perfect to give away.

Should I Work a Few More Years?

Ultimately, there is no perfect answer to the question “do I have enough to retire?”

Clearly, according to figure 2, some will never have enough to retire! If you spend more than you earn, you will never get there.

Others, like Physician on Fire, prove that you can limit your spending and goose your savings rate, such that early retirement is possible after a mere decade or so of practice. This, of course, requires continued spending discipline, but I doubt there are many who question Physician on Fire’s resolve to stay disciplined.

Summary

Retirement Planning seems backward after being in the accumulation mindset.

Instead of growing your wealth, you need to consider spending some of it down!

There are additional pitfalls and traps in retirement that take some planning. Many of these issues uniquely affect physicians and other high-income professionals.

A forward-facing 20-30-year tax plan is often useful to see what your tax bracket “number” will be. For instance, if you know that RMDs will expose much of your retirement income to the 25% federal tax bracket in the future, use all your tax bracket below the 24% bracket now! A percent or two doesn’t seem like it would make all the much difference, but if you can rescue money from a pre-tax account and convert it into a Roth, all future growth is also tax-free.

The goal, of course, it not to perfectly predict how much you will be paying in taxes even five or ten years from now. Everything can and will change and many assumptions will be wrong!

The goal is to optimize the next 1-3 years so you set yourself up in the best possible position to roll with the punches as they come, all while efficiently funding retirement goals.

What are you doing to optimize your retirement planning? What mistakes have you made? Comment below!