Single Best Answer?

A mainstay of multiple-choice test instruction is the admonition to select the “single best answer.” Now, generally there really is only one valid answer, but there are times when the questions posed are sufficiently imprecise, or the potential answers specific, that more than one response is viable. That’s where the test architect can always fall back on their notion of “best” answer– because, even if a credible argument can be made for an alternative, it’s a lot easier to grade when there’s only a single, pre-determined result.

When it comes to projecting possible outcomes in situations where there might be hundreds, perhaps thousands, or even millions of different results, it’s not uncommon to pick a single point to focus on. For example, projections in financial analysis might use the most likely rate of claim, the most likely investment return, or the most likely rate of inflation, whereas projections in engineering analysis might use both the most likely rate and the most critical rate. That choice provides a point estimate, one that ostensibly provides a best single estimate for purposes of analysis.

The downside of this approach, of course, is that it does not fully cover the fact that there is a whole range of possible outcomes, some more probable, some less. The alternative, a stochastic modeling, doesn’t just pick a single likely result, but uses random variations to look at what a broad range of conditions might be like. It does this based on a set of random outcomes, projects results, and then repeats with a new set of random variables. In fact, this process is repeated thousands of times.
When the modeling is done, you can look at a distribution of outcomes – and with that not only consider the most likely estimate, but what ranges are reasonable as well. It is, quite simply, a more complete and realistic assessment of potential outcomes, because, unlike so-called “deterministic” models that rely on picking a single point of experience, it includes a wide range of possibilities.

Deterministic models can predict outcomes under a few economic and demographic scenarios but don’t generally present a distribution of the wide range of scenarios that could arise from different combinations of economic and demographic variable values. Only stochastic models – like that embodied in the EBRI Retirement Security Projection Model® (RSPM) - can measure the "risk" of their performance measure values, because stochastic models, using Monte Carlo methods, are based on probability distributions. Said another way, stochastic modeling brings into account the volatility and variability of experience that are part of living in the real world.

Those deterministic models may offer a “single” answer, but life is rarely that simple – and projections that attempt to help us make better decisions about the future needn’t be.

- Nevin E. Adams, JD

1. The EBRI Retirement Security Projection Model®(RSPM) simulates 1,000 alternative retirement paths for each household to explicitly model investment, longevity and stochastic healthcare risks (i.e., nursing home and home healthcare costs).

2. More information on stochastic versus static/deterministic modeling can be found here.