I have to admit, I don’t know why I never thought of using combining all of the available metrics into one hyperopt loss. Currently available metrics are Sharpe, Sortino, Calmar, SQN, and Expectancy (Ratio). Let me explains each metrics first. But before that, time for big important disclaimer.
All metrics are calculated based of past trades’ data. There is no guarantee that future will follow the past. The “this strategy is good based of this metric” discussions are all based of past performance and may not repeats in the future.
Sharpe Ratio
Sharpe measures how much profit you made compared to the amount of risk (ups and downs/volatility) you took to get it. Think of it as grading the “smoothness” of your strategy’s profitable ride. A high Sharpe Ratio means you got good returns for a relatively smooth ride (low volatility). A low Sharpe Ratio means the ride was very bumpy (high volatility) for the returns you got, or the returns weren’t great compared to a safe investment. “Was the profit worth the stress of the volatility?” is the question asked by Sharpe.
Use Sharpe if you prefer steady growth with (almost) uniform profit/loss with very small deviations, which means no large loss(es).
Let’s say we have these results from strategy A.
3 3 -2 3 3 -2 3The total profit is 11 USDT. Sharpe ratio is 0.6957. Now compare this with strategy B.
7 3 -2 5 2 -6 4The total profit is 13 USDT, but the sharpe ratio is 0.4507. Each people will have their own preference. Strategy A offers lower profit but more stability. Strategy B offers more profit with more volatility.
As you see in the example above, high Sharpe ratio doesn’t implies more profits, but it implies stability/eficiency. In theory, you will have less stress when the market turns opposed of what your strategy preferred because past trades suggest that the strategy can handle bad market well, such as no big loss(es). While it might not shows up on the numbers (you get smaller profit), using strategy A might be good psychologically.
Sortino Ratio
Sortino is similar to Sharpe, but Sortino only cares about the volatility/deviation of the losses. While it seems that Sharpe is better version than Sortino, but there are cases where you want to use both.
Let’s take this result of strategy A
4 4 4 4 4 4 -3 -10 -5The total profit is 6 USDT, sharpe is 0.1330, the sortino is 0.1728. Let’s see strategy B
3 5 2 6 1 7 -3 -10 -5Same total profit of 6 USDT, but lower sharpe of 0.1255. Sortino stays the same at 0.1728. If you only look at sharpe ratio, it seems like A is better than B, but that’s because the profits of A is less deviated. The losses is the same between A and B. Now let’s see strategy C
4 4 3 3 3 3 -1 -12 -1The total profit remain 6 USDT, the sharpe is 0.1380, and the sortino is 0.1655. If you based on Sharpe alone, strategy C is the best of the three. But based of Sortino, strategy C is the worst of the three because the losses of C is the most volatile. So it’s recommended to use both Sharpe and Sortino to make sure your overall trades and your losing trades aren’t highly volatile/deviated.
Calmar Ratio
It measures average annual profit compared to the worst past drawdown. It doesn’t mentioned a lot compared to Sharpe/Sortino because it only focus on a single max drawdown. It doesn’t tell anything about the other drawdowns, volatility etc.
SQN (System Quality Number)
Developed by Dr. Van Tharp, it tries to measure the overall quality of a strategy by looking at both its profitability per trade (Expectancy) and the consistency of those profits (Standard Deviation of profits/losses). Since the formula include trade number, it might favor a strategy with high number of trades with lesser profitability than a strategy with better profitability but lesser number of trades.
Expectancy and Expectancy Ratio
Expectancy is estimated amount you expect to gain or loss per trade based of previous trades. So if your expectancy is 0.34 for example, it means everytime your bot open new trade, it’s expected to gain 0.34 USDT profit.
Expectancy Ratio can be defined as the ratio of your average profit to your average loss, taking into account the winrate as well. For example if your expectancy ratio is 0.09, one losing trade will cancel out the gains from 11 winning trades.
So which metrics should be used?
Well, for now, I’m trying to use almost all of them. I omitting calmar for now. Use the loss function below for reference. It might not be suitable for your preference, so do edit it to suit yours.
from datetime import datetime
import numpy as np
from pandas import DataFrame
from freqtrade.constants import Config
from freqtrade.data.metrics import calculate_expectancy, calculate_sharpe, calculate_sortino, calculate_calmar, calculate_sqn
from freqtrade.optimize.hyperopt import IHyperOptLoss
# Set maximum values for metrics used in the calculation
max_expectancy = 4
max_profit_ratio = 10
max_avg_profit = 50
max_sharpe_ratio = 5
max_sortino_ratio = 5
# max_calmar_ratio = 5
max_sqn = 5
class JetLoss(IHyperOptLoss):
@staticmethod
def hyperopt_loss_function(
results: DataFrame,
trade_count: int,
min_date: datetime,
max_date: datetime,
config: Config,
*args,
**kwargs,
) -> float:
starting_balance = config["dry_run_wallet"]
max_profit_abs = (max_avg_profit / 100) * results["stake_amount"]
strict_profit_abs = np.minimum(max_profit_abs, results["profit_abs"])
results["profit_abs"] = strict_profit_abs
total_profit = strict_profit_abs / starting_balance
average_profit = total_profit.mean() * 100
winning_profit = results.loc[results["profit_abs"] > 0, "profit_abs"].sum()
losing_profit = results.loc[results["profit_abs"] < 0, "profit_abs"].sum()
profit_factor = winning_profit / abs(losing_profit) if losing_profit else 10
total_profit = strict_profit_abs.sum()
_, expectancy_ratio = calculate_expectancy(results)
sharpe_ratio = calculate_sharpe(results, min_date, max_date, starting_balance)
sortino_ratio = calculate_sortino(results, min_date, max_date, starting_balance)
# calmar_ratio = calculate_calmar(results, min_date, max_date, starting_balance)
sqn_ratio = calculate_sqn(results, starting_balance)
if sharpe_ratio == -100:
sharpe_ratio = max_sharpe_ratio
if sortino_ratio == -100:
sortino_ratio = max_sortino_ratio
# if calmar_ratio == -100:
# calmar_ratio = max_calmar_ratio
if sqn_ratio == -100:
sqn_ratio = max_sqn
total_trades = len(results)
backtest_days = (max_date - min_date).days or 1
average_trades_per_day = round(total_trades / backtest_days, 5)
loss_value = (
total_profit
* min(average_profit, max_avg_profit)
* min(profit_factor, max_profit_ratio)
* min(expectancy_ratio, max_expectancy)
* average_trades_per_day
* min(sharpe_ratio, max_sharpe_ratio)
* min(sortino_ratio, max_sortino_ratio)
* min(sqn_ratio, max_sqn)
)
if (total_profit < 0) and (loss_value > 0):
return loss_value
return -1 * loss_value
[…] Best metrics for hyperopt loss […]