Stressing Stress Testing

Scenario analysis, stress testing, call it what you will, it is ubiquitous in financial risk management. Risk reports are littered, both literally and figuratively, with the detailed and sometimes not so detailed results of stress tests. The question that I would like to pose is, to what extent do all those numbers and graphs aid the risk management process? Or are they in fact, just a big tick-box exercise.
First it is useful to define what I mean by stress tests. I divide the stress test world into hypothetical and historical. For clarity hypothetical refers to stated changes in some parameter or parameters that impact the valuation / liquidity / relevant metric of a portfolio. Hypothetical stress tests can be single factor – “EUR down 10% vs all other currencies” or “EUR fixed income up 200bps” or multi-factor – “EUR down 10% vs all other currencies and EUR fixed income up 200bps” or considerably more detailed that the above. I like hypothetical stress tests because the only thing that changes their output is changes in the portfolio. As always there are drawbacks, but their clarity is a valuable thing. The other side of the coin are historical stress tests. To cut to the chase, historical stress tests are just more fun. They are truly complicated with lots of changes to almost everything going on at the same time and one is fairly guaranteed to forecast large changes. As an added bonus, with the right kit, you can get to see the evolution of those changes over the period in question. Problem is setting limits is difficult and depending on the dates of the stress test, the methodology is a bit sketchy. It generally has some reliance on correlations that we know go to 1 when bad things happen, not the value you are using. Worse the realised betas that you are using to forecast behaviour bear little or no relationship to the ones you determined in less stressful times. Disconcerting, to say the least.
Getting back to the original point, as a risk manager a good way to start determining the utility of a stress test is to ask the following question: Is it possible for a fund to fail the stress test? Now let’s be honest here, no rational risk manager will design, if they thought about it in the first place, a stress test that was not going to forecast non-trivial losses for a fund. It is almost a definitional thing. Sadly, that is completely missing the point if you would like to use stress tests as a risk management tool. The purpose of a stress test is not and never has been to forecast that the fund will lose money. The purpose of a stress test is to aid the determination of whether or not the current risk profile of the fund is consistent with the risk appetite and risk capacity of the fund, both now and in the future. Which gets me back to the original point. Is it possible to fail a stress test? In the event that the relevant stress tests have been described and a risk capacity and risk appetite has been stated for each stress test, then in theory it is possible to fail a stress test. In reality however my experience is somewhat different. Stress tests are rarely specifically designed for a given sub-fund and even rarer are specific limits applied to specific stress tests. Rarer still is the risk manager with the capability to force a change in the holdings of a fund due to a “failed” stress test. So a bit of a box-tick then…
Not necessarily. Like all tools, they depend on their manufacture and the uses to which they are put. So after box-ticking (tick) and risk report padding (tick), let’s try and determine what risk management aims one is trying to achieve when applying stress tests to a portfolio. They are two-fold and they are the same as all the other risk management tools. Firstly, one is attempting to determine if the current risk profile is consistent with the stated risk profile. Secondly have there been any changes in the risk profile of the fund that might imply that the fund is in a trajectory that may lead it to have a risk profile that is inconsistent with the stated risk profile of the fund. As with all risk management tools, stress tests work best in conjunction with other tools but their specifics need to be considered in isolation.
I asked above whether or not specific stress tests were designed for a fund. This may be appropriate for a single strategy AIFM but for a multi-strategy ManCo this is almost certainly not appropriate. A broad set of stress tests uniformly applied across all funds allows the risk manager to consider the relative riskiness of all the funds on the platform. What is most important however is that for each fund, for each stress test, there is a limit on the forecast loss, below which the fund cannot go. This puts quite a lot of work on determining an appropriate limit, but this really should not be rocket science. For a hypothetical stress test, a combination of minimum asset quality and maximum leverage times size of stress generally puts the limit in the right ball park. For a historical stress test, not so obvious. My advice is to take a model portfolio and see what you get when you apply the historical stress test. Then scale according to the asset quality / leverage of the model portfolio compared to their limits.
Stress test limits are achievable. But just as important is the evolutionary data. Observing how the stress test forecasts evolve as both the portfolio and the market evolve is a key tool in understanding the evolving risk profile of the fund. Having the ability to see the normal range of outcomes of stress tests allows the risk managers to determine if the fund is moving into rather more “interesting” space. Generally, nothing to worry about. Creative managers continually adjust the risk profiles of their funds as their viewpoint and market conditions evolve. However, over time, a risk manager can observe the ranges of the stress test forecasts. Outcomes outside of a normal range may not be significant, but they are a canary in a coalmine. If accompanied by other canaries from other risk management tools, it may be time for the risk manager to worry about air-quality.