Use locally-downloaded candles’ data on dry/live run

Credit to Dubok from discord server for this snippet. Important note, you have to use static pairlist for this method. You can’t use volume pairlist, unless you set automatic job that regularly download candles’ data for all pairs.

This is an example of how your populate_indicators should be. I am using binance futures but please modify the file path to your exchange’s folder and appropriate spot/futures pair naming

# Actual number of candle count needed for your calculation
loaded_candle_count = 2000

# adjust startup candle count. Small on dry/live
if self.config['runmode'].value in ('live', 'dry_run'):
    startup_candle_count = 20
else:
    startup_candle_count = loaded_candle_count

def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
    
    if self.config['runmode'].value in ('live', 'dry_run'):
        initial_df_length = len(dataframe)
        dataframe.drop(columns=[x for x in dataframe if x not in ["open","high","low","close","volume","date"]],inplace=True)
        pairdata_filename = f"user_data/data/binance/futures/{metadata['pair'].split('/')[0]}_USDT_USDT-{self.timeframe}-futures.feather"
        loaded_df = pd.read_feather(pairdata_filename)
        if loaded_df.date.max()<dataframe.date.min():
            raise Exception(f"LOADED DF DATA MISSMATCH. Max loaded {loaded_df.date.max()}, min current {dataframe.date.min()}")
        dataframe = pd.concat([loaded_df,dataframe]).drop_duplicates('date')[-self.loaded_candle_count:].reset_index(drop=True)
        dataframe.to_feather(pairdata_filename, compression_level=9, compression='lz4')
    dataframe['rsi'] = ta.RSI(dataframe, 14)
    if self.config['runmode'].value in ('live', 'dry_run'):
        return dataframe[-initial_df_length:]
    
    return dataframe

And in your config.json, put this inside ‘exchange’ section

"_ft_has_params":{
    "ohlcv_candle_limit":20
},

Before starting the bot, please download the required data as how you usually do for backtesting (for example freqtrade download-data --config config.json --timeframe 30m). What will happen is that you will download this data and once you run the bot, it will cut it to only last x candles (2000 candles in the example above). Change to cut-off number to suit your needs. Also while it’s possible to use similar steps with informative pairs, if you are using specific pair(s) as your informative pair(s), for example BTC, since FT gonna re-calculate informative pair(s)’ indicators for every pairs, using such steps above might be costly and unnecessary. In such case, you might want to calculate informative pair(s)’ indicators once, store it in a dict, and have a check inside informative function whether the dict already has the latest values or not.

IMPORTANT!!!!! As written in the snippet above, when you want to do backtest using that strategy, change the startup candle to 1000. Change it back to 20 when you want to run the strategy in dry/live mode. Don’t worry with such low value, because we gonna load the local data containing 2000 rows, so we don’t really need to download a lot of candles’ data. Actually you can try to set it even lower, for example 10 or even 5.

Let me know if you encounter any error, or you have better solution/improvement.