Searching for an Efficient Market Regime Filter
The probability of our long term success as traders increases when we trade with the prevailing market trend. This means when trading stocks we should be buying when the overall market is rising and / or shorting when the overall market is falling. In order to filter trading opportunities therefore, we need an efficient way of determining the current market regime. In this article we’ll focus on defining a market regime filter for use when trading large cap US stocks, so our proxy for the overall market will be the S&P500 Index.
When defining a market regime filter, we’re looking for a measurement with the following characteristics:
- Ability to discern a bull market from bear market
- An indicator than can oscillate between the two regimes
- Reactive enough to signal bear markets early and to identify bull markets quickly
- Stability, in that it doesn’t whipsaw too often
- Low number of false positives
- It should also conform to the KISS principle.
- Index close price against a simple moving average
- Dual moving averages crossing
- Moving Average Convergence Divergence (MACD) against its zero line
- Simulate a Buy in the index when the filter turns bullish
- Simulate a Sell in the index when the filter turns bearish
- Apply to the S&P500 Index ($SPX)
- Weekly timeframe
- In-sample period 2001 – 2010 inclusive
- Optimize for Compound Annual Return to Maximum Historical Drawdown ratio (CAR/MDD)
Simple Moving Average
Possibly one of the widest known methods of filtering for market regime is checking whether the close price is above or below the 200-day simple moving average. It’s reputedly even used by many professional fund managers, but how does it really stack up?
The first thing to recognise is that the period on the MA is going to be key. Here’s a 40-week MA (in green) [approx 200-day] compared to a 120-week MA (in blue). You can see the 120 period is more stable, but slow to react, whereas the 40 period is much faster to react but is more choppy and creates many more false positives.
So we’ll run an optimization on the lookback period and see what comes up best. Here’s the result of looking at all lookback periods between 25 weeks and 120 weeks, against the CAR/MDD produced.
The first thing to notice is the very low numbers and the very high numbers produced the worst results. But there is a range of decent results in the 70-95 range. Notice 40-weeks, the one supposedly most discretionary traders rely on, is one of the poorer results.
When selecting a value to use, we rarely want to select the number one best result from the optimisation, as it’s often an outlier and the result of getting lucky, so we’ll take one of the nearby results that has other similar level results around it. For the purposes of this test, I’m going to use 75-weeks and compare it to a Buy & Hold baseline.
Clearly it has done a better job than Buy & Hold (yes the index really did make a loss over the 10 year in-sample period!). In particular the filter has managed to minimise drawdown, which is always desirable, and it has done so with a low number of turnovers which is also good.
So, let’s now move on and see if the other methods are any better or worse.
Dual Simple Moving Averages
By using dual moving averages crossing, we will be eliminating some of the noisiness of using the bare close price in the above test. That you would think should lead to a more stable indication. The question of course is whether the cost of any increase in stability will be a decrease in responsiveness and more lag. We’ll run an optimisation on a range of values for both MAs and compare the results. This time because we have two parameters, the chart becomes three dimensional.
Clearly the results themselves are quite noisy, with a number of sharp peaks, indicating outlying results. So, again we’re looking for an area of stability, or so-called “high plateau” where there are a group of similarly good results nearby. The area I’ve highlighted with the white circle is about the best pick of the bunch, so we’ll select a pair of parameters from in there. I’ll select 10 and 53. Let’s see how that compares to the previous method.
As you can see it produced the same max drawdown as the simple moving average, but with less gain and more turnover, leading to a (slightly) lower return to drawdown ratio.
The MACD is well known as an oscillator, so it could be ideally suited for our purpose. It is also made up of two underlying moving averages, so it’s already smoothed. We’re not concerned with the MACD Signal Line here, we’re only interested in the MACD line being above or below zero to signal bull or bear market conditions respectively. The two inputs to the MACD are the lookback periods on the Fast and Slow EMAs that make it up, so let’s optimise for both those numbers.
This time we have a reasonably smooth result. There are still some outlying spikes, but also some stability across a range of parameter values. This is encouraging. The results are zero in the back corner, but this is because the Fast EMA period has to be less than than the Slow EMA period to register any result at all. There does appears to be a good spread of similar values in the white circle, so let’s pick a pair from there and compare that to the other methods. Let’s take 20 for the Fast EMA and 34 for the Slow EMA.
Here you can see it turnovered fewer times than the first two methods, and although it picked up the highest return, but the additional lag meant it also incurred the largest of the drawdowns, leading to a lower CAR/MDD.
Here’s a side-by-side comparison of the in-sample outcomes against $SPX for the chosen parameter pairs for each of the methods.
As you can see on the chart and in the results, they all do a reasonably similar job, so before deciding, it might also be worth taking other considerations into account, such as consistency of results for example.
Finally, we’ll take a wider look at the best candidate, ie being long when the SPX weekly close price is above its 75-week MA.
On a chart as you can see it would have kept you on the long side for most of the recent bull market, although there have been a few false positive crosses to the downside along the way.
Looking at how the performance of the index would have gone applying the SMA filter:
Clearly it wasn’t fast enough to react to the 2011 market correction, and so it’s maximum drawdown has increased in the out of sample period, but otherwise it would appear to have performed in line with expectations.
Just based on this limited test, it appears something as simple as a single simple moving average can be useful as a market filter, although as we have found, using the common 40-week lookback period has been sub-optimal historically. Longer lookback periods it would seem, appear to work better. However, the out-of-sample performance perhaps does raise some concerns about the filter’s ability to protect against a more volatile market.
There are of course many more indicators we could look at, and you should do so before deciding on which is the best filter for your own purposes. For example other types of moving averages, EMAs, WMAs etc. or other types of oscillating indicators, such as RSI or Stochastic could be considered. What about even the MACD Histogram? If you have any ideas of your own, leave it in the comments below and we could test them in a future article.
Remember of course, this is not a trading system in itself, merely a building block that could be used as part of a system.
Listen to me discussing more on the topic of Market Regime Filtering on the Better System Trader podcast.