An update on my research:
Research is becoming a sort of an obsession lately. I’ve been doing it almost every day, from after lunch to the early hours of the morning, with a few breaks in between for meals and the gym. After a morning on Atlantic City beach last weekend, the guys went to play volleyball, the girls went shopping… and I planted myself at a Starbucks to do research. It’s that intense. And exhausting.
I don’t think I’m a workaholic. It’s just that I hate the feeling of going through an entire day’s worth of research and not be able to show anything for it, which has happened several times now. Coming up with trading ideas is easy, testing a system out is easy, but trying to find a system that meets all of my criteria is hell:
1) The system needs to have Net Profit/Max Drawdown ratios of above 3 in both my in-sample and out-sample periods (out-sample periods need to be at least 2 years)
2) The system equity curve should be relatively smooth and upward sloping, and not marked by many flat/negative periods and a few big upward moves. (I know of some trading styles that adopt this sort of equity curve: losing a little for many periods and then locking in the rare, huge upward moves, but that isn’t my style.)
3) The system should have a stop in place at all times whenever I have an open position.
4) There should be at least 30 trades in the entire sample period in order for me to gauge the statistical significance of each trade’s expectancy.
5) The system should still yield a Net Profit/Max Drawdown ratio of at least 2 for both in-sample and out-sample periods even after taking away the trade with the biggest gain. This is to ensure that the good performance wasn’t due entirely to luck.
The first three criteria were relatively easy to meet. I think I have close to a hundred examples of supposedly “good” market-system pairings, but very few that meet criteria #4 and #5. In fact, I spent an entire day last week testing out criteria #5 only to realize that… I had less than 3 market-system pairings, from an initial 50-ish. Bummer.
Things have since improved since then. My ATR system seems to be the most robust compared to other systems – after a week’s worth of testing, I found that it could trade 5 markets so far: the SGD/USD, the EUR/JPY, T-Notes, the HSI, and the Nikkei 225.
Just today, from a sheer stroke of luck or brute force of will, (or maybe I’m just getting better at this), I found 2 additional markets for my Donchian Breakout system to trade. It now trades 4 markets in total (the same markets at the ATR system, with the exception of the Nikkei, which I have yet to test out).
Some things I’ve noticed/thought of:
1) I’m not sure if the amazing performance of my ATR system is due to it having a high number of parameters – there might be a danger of overfitting. However, an out-of-sample test is one of the best defenses against overfitting, and the system performs well in both in and out-of-sample test periods. Still, I’ve optimized different parameters for each market, so I must expect that the actual performance might be slightly lower than what I’ve tested.
2) It’s always good to optimize based on “normal” times (such as the run-up from year 2000 to 2007), since those are the conditions under which the market normally operates. However, it is almost critical to include fat-tailed events (such as October 2008 to present) in the out-of-sample period to see if the system can withstand such conditions. I’ve cheated a bit in this area by using Neuroshell’s papertrade function, which gives me the best performance in the out-of-sample period based on in-sample optimization. However, I guarded against this by ensuring that the in-sample period performs well too.
3) Sometimes, less rules help to improve performance. I was stuck on my Donchian system for the longest time. Today, I experimented removing the exit rule of exiting on breakouts, and simply relied on a trailing percentage stop to exit my trades, and I found that it was much easier for me to generate trades that performed well. Perhaps I should experiment removing some parameters from my ATR systems, though I don’t see how that can be done. Hmmmm.
Ok I think I’ve been working too hard. I need to take a break and watch Friends or something before I go insane.