Please ask why bring my stock data into the training, the action produces nan

[longjie1101]

The resulting problems are as follows

/home/xie/Tacrypto/DCmaster/env/EnvMultipleStock_validation.py:162: RuntimeWarning: invalid value encountered in greater
buy_index = argsort_actions[::-1][:np.where(actions > 0)[0].shape[0]]
/Deep-Reinforcement-Learning-for-Automated-Stock-Trading-Ensemble-Strategy-ICAIF-2020-master/env/EnvMultipleStock_validation.py:136: RuntimeWarning: invalid value encountered in double_scalars
df_total_value['daily_return'].std()
/Deep-Reinforcement-Learning-for-Automated-Stock-Trading-Ensemble-Strategy-ICAIF-2020-master/model/models.py:128: RuntimeWarning: invalid value encountered in double_scalars
df_total_value['daily_return'].std()
A2C Sharpe Ratio: nan
======PPO Training========
Training time (PPO): 5.048629434903463 minutes
======PPO Validation from: 20201126 to 20210303
PPO Sharpe Ratio: nan
======DDPG Training========
Training time (DDPG): 0.9256621718406677 minutes
======DDPG Validation from: 20201126 to 20210303
DDPG Sharpe Ratio: 0.07492627731887684

My data did not produce missing values, only DDPG in the three algorithms can run successfully to get results, the rest of A2C and PPO and even ACER, TD3 are training a certain amount of time after the action space to produce nan, why is this?

[longjie1101]

I found that as long as I put in one or two stock data, there will be no Nan. If I put in six or more stocks, there will be Nan. I also repeatedly confirmed that there is no lack of data. Will it be a problem of training?

[longjie1101]

Now the problem seems to be much clearer. If I set A2C timesteps to 100, Nan will not appear, but set it to 1000, Nan will appear. It seems to be a convergence problem in training? Does anyone know how to solve this problem? thanks!