Backtest traps – Custom Exit

In this post, I will explain why you need to be careful when you are using custom_exit function on your freqtrade strategy. Let’s start with this simple custom_exit logic

def custom_exit(...):
   if current_profit >= 0.05:
      return "roi_reached"

The logic is very simple, when the trade has at least profit of 5%, the trade exited with exit tag of “roi_reached”. In dry/live run, what gonna happen is that every throttle secs (which is 5 seconds by default, unless you set different value), the bot fetch current rate from the exchange and calculate current_profit, then supply the value to custom_exit function above.

Where the backtest trap lie is in the fact that for backtest, to make it faster and because there is no data of intra-candle movement, custom_exit function gonna be checked only on candles’ open rate, which means current_profit value also only gonna be checked once per candle. Why is this problematic? Let’s see the example of backtest logic below.

Let’s say you entered a trade of BTC/USDT coin on candle A. What happen next is the bot use candle A’s open value to calculate current_profit and supply it to custom_exit function like the one above. Since current_profit is still below 5%, the bot move to candle B and repeat the same process (use candle B’s open rate to calculate current_profit, and send the value to custom_exit function).

Let’s move forward to candle F. If we calculate the profit based of candle F’s open rate, the profit still below 5%. But then, if you look at candle F’s entire values, it’s a green candle where the close value is 10% compared to open value. Which means in case of dry/live run, the trade would exit somewhere in the middle of candle F at profit of 5%. But for backtest, since current_profit at the open of candle F is still below 5%, the bot move forward to candle G where the current_profit at the open of candle G is 15%. The value sent to custom_exit function, trigger the logic and exited the trade at 15% profit. In this case, the difference of backtest vs dry/live is 15% vs 5% profit. It’s a significant difference. So the next question is

How to avoid the traps?
There are three ways I use to avoid it, which are

1. Don’t use current_profit at all
2. Calculate the current_profit based of last candle’s close value. So for example above, instead of using candle F’s open rate, use candle E’s close rate to calculate current_profit. This will make sure backtest and dry/live run gonna be similar. The issue with this method is you won’t have intra-candle exit in dry/live run, since no matter what is the current rate of candle F (which is fetched every throttle secs), the current_profit will always stay the same (anchored to candle E’s close rate)
3. Use timeframe-detail to help minimize the effect. But since the lowest workable timeframe available now is 1 minute timeframe, that means it is still prone to such trap if you have a very long 1 minute green candle. But in general, the difference shouldn’t be as big as if you aren’t using it.

Example of how to do the second method above is

def custom_exit(self, pair: str, trade: Trade, current_time: datetime, current_rate: float, current_profit: float, **kwargs) -> Optional[Union[str, bool]]:

    dataframe, _ = self.dp.get_analyzed_dataframe(pair, self.timeframe)
    current_candle = dataframe.iloc[-1].squeeze()
    current_profit = trade.calc_profit_ratio(current_candle['close'])

The snippet above will make sure your current_profit tied to last closed candle’s close rate.