When planning for retirement, it’s effectively impossible to precisely forecast the performance and timing of future investment returns, which in turn makes it challenging to accurately predict a plan’s success or failure. And while Monte Carlo simulations have made it possible for advisors to create retirement projections that seem to have a reasonable basis in math and data, there has been limited research as to whether Monte Carlo models really perform as advertised – in other words, whether the real-world results of retirees over time would have aligned with the Monte Carlo simulation’s predicted probability of success.
Given the importance of some of the recommendations that advisors may base on Monte Carlo simulations – such as when a client can retire and what kind of lifestyle they can afford to live – it seems important to pay attention to how Monte Carlo simulations perform in the real world, which can reveal ways that advisors may be able to adjust their retirement planning forecasts to optimize the recommendations they give. By conducting research assessing the performance of various Monte Carlo methodologies, Income Lab has suggested that, at a high level, Monte Carlo simulations experience significant error compared to real-world results. Additionally, certain types of Monte Carlo analyses were found to be more error-prone than others, including a Traditional Monte Carlo approach using a single set of Capital Markets Assumptions (CMAs) applied across the entire plan, and a Reduced-CMA Monte Carlo analysis, similar to the Traditional model but with CMAs reduced by 2%.
Notably, Historical and Regime-Based Monte Carlo models outperformed Traditional and Reduced-CMA models not only in general, but also throughout most of the individual time periods tested, as they had less error across many types of economic and market conditions. Furthermore, compared with the Traditional and Reduced-CMA Monte Carlo methods, the Regime-Based approach more consistently under-estimated probability of success, meaning that if a retiree did have a ‘surprise’ departure from their Monte Carlo results, it would be that they had ‘too much’ money left over at the end of their life – which most retirees would prefer over turning out to have not enough money!
Ultimately, although Historical and Regime-Based Monte Carlo models seemed to perform better than the Traditional and Reduced-CMA models, advisors are generally limited to whichever methods are used by their financial planning software (most of which currently use the Traditional model). However, as software providers update their models, it may be possible to choose alternative, less error-prone types of Monte Carlo simulations – and given the near-certainty of error with whichever model is used, it’s almost always best for advisors to revisit the results continually and make adjustments in order to take advantage of the best data available at the time!
