Is It Possible You’re Saving Too Much for Retirement?

I recently sat down and had a conversation with my accountant. We talked about my family's retirement accounts and long-term investing goals. And we both came to the conclusion that my wife and I should focus on adding more to our taxable investments. That's because we already have a sizable nest egg in retirement accounts.

Although I don't plan on, really, ever formally retiring, it is possible to be lopsided and have too much in retirement accounts and not enough in taxable investments. With retirement accounts, it's possible you are saving money on taxes now only to get walloped with a bigger tax bill in the future.

I certainly believe that it's critical for your investing to be tax efficient. However, at the end of my post on that subject, I also said, “Keep in mind, like any part of investing, one should not invest solely for tax avoidance.”

Most personal finance books say you should put away the maximum possible in retirement accounts. What they don't talk about is when you have to start withdrawing money from your retirement accounts. They talk only about the accumulation phase, and all of the authors state that your expenses will be lower after you retire — another piece of Suze Orman-type financial advice that doesn't always apply to everyone.

Yes, income from a salaried position decreases, but keep in mind, for tax purposes, distributions from retirement accounts (except Roth accounts) are taxed at ordinary income rates. It's possible that your tax rate will be much higher when you retire.

Tax-deferred accounts do NOT mean tax avoidance.

I Have to Pay More in Taxes When I Retire?!

Let's use a hypothetical example. Let's say you're currently 40 years old and have $1 million saved for retirement. (That's a lot of money for that age, but it's not an impossible goal, especially if you are a married couple and you started saving for retirement right out of college.) You don't plan on retiring until at least 60 years of age, so you have 20 more years of investing with your retirement account. Assuming a 7% rate of return, and socking away an additional $25,000 annually, you'll have approximately $4.5 million by that time.

The problem with retirement accounts is twofold: You have minimum required distributions starting at age 73, and any money you do take out is taxed at ordinary income rates. Currently, long-term capital gains and dividends are taxed at a lower rate than regular income. While this may change in the future, historically, investment taxes have been at a lower rate than income taxes.

So let's assume you take out 4%, or $180,000, annually from that retirement account. At today's tax rates, that puts you in the 24% tax bracket. Obviously, this isn't accounting for inflation, so let's use the next lower tax bracket of 22%. That's still higher than if it were all investment income and/or dividends in a taxable account — currently taxed at 15%.

So while you'll save taxes when socking away this money, it's possible that you'll lose this advantage when you start taking it out. Especially if you need large sums of money in any tax calendar year. With state taxes included, it can be as high as a 50% total tax rate. Ouch!

Obviously, if you are going to invest in taxable accounts, you should use only tax-efficient investments, but they can be part of your total asset allocation. That means NOT using investments like REITs, bond funds, or dividend-paying stocks (if possible). ETFs such as Vanguard's Russell 3000 Index ETF (VTHR), which cover the entire market, are pretty tax efficient. There are also other funds that are specifically designed to minimize taxes, as well as tracking various indexes. These are perfect investments to place within taxable accounts. What you are setting up is a blend of tax-deferred and taxable accounts.

Disadvantages of Retirement Accounts

• Retirement accounts are much more restrictive. In most situations, you can only start withdrawing from them when you are 59 1/2 years old (55 in some situations). If you need the funds sooner, be prepared to pay a massive penalty.
• They assume you'll be in a lower tax bracket when you retire. While this might be true for most people, it's not the case for everyone. Retirement accounts are taxed at regular income levels. As we all know, tax rates are going up and are not expected to go down for the foreseeable future.
• Restricted investment selection. For example, I love real estate rental properties. While it's possible to invest in real estate with a self-directed IRA account, I don't consider that a viable option. Also, many retirement accounts have a poor selection of funds to choose from.
• Required distributions. With the exception of Roth accounts, you are required to take money out of your retirement account starting at 73.
• Taxed at ordinary income rates. This is perhaps the killer if you put money into only a retirement account. Depending on other sources of retirement income and the amount saved, it's possible you are in a higher tax bracket than before retirement. Also, your future taxes are an unknown.

Bottom line: It makes sense to have a mixture of tax-deferred and taxable accounts. This gives you much more flexibility when you retire. So while it makes sense to minimize your taxes while saving for your retirement, you should also be concerned after you retire. Obviously, I'm not suggesting that you stop putting money into your retirement account, especially if your company does matching.

What I am suggesting is that, once you get past some level of retirement savings, you may want to balance it out with taxable investments as well. With taxable accounts, you have not only more flexibility with investments, their intended use, but also more control over when you have to pay taxes and traditionally at a lower tax rate to boot!