Hello,
I have started to use the machine learning method implemented in VASP 6.3.0 and I have a question about RMSE implemented here. I tried to find solutions on the forum, wiki, and some initial publications by Georg Kresse, but I didn't find a clear answer.
Could you please explain how the RMSE is calculated in VASP? If I understand correctly, it is a cumulative error, right? If it is, we can observe a decrease in RMSE, is this because the RMSE has been averaged over all the configurations in the datasets? If not, from both the example of melting Si provided by you and the calculation I made, I always observe a steady increase. This clearly indicates the model is performing worse and doesn't make sense. Can you please clarify this point a little bit?
Thanks a lot in advance.
Best regards,
Xiliang
MLFF RMSE question
Moderators: Global Moderator, Moderator
-
ferenc_karsai
- Global Moderator
- Posts: 460
- Joined: Mon Nov 04, 2019 12:44 pm
Re: MLFF RMSE question
The RMSE is calculated after an update of the force field over all training structures up to that step. So yes the RMSE can go up and down.
I have clarified the section for "ERR" in the monitoring part of the best practices for machine-learned force fields:
https://www.vasp.at/wiki/index.php/Best ... Monitoring
So if RMSE goes up what can cause that:
1) First check if you have a temperature ramp. If yes then increased temperature always leads to larger errors due to a broader variety of configurations.
2) If the RMSE suddenly goes up it could also mean that you encounter some kind of deformation of your cell. In that case your force field is not good enough to describe those structures. This is usually something you want to avoid if you want to train a specific phase. So in that case adjust your MD parameters. For melting for example this definitely happens. But there one would possibly use the force field with on-the-fly learning to get a melt from the crystal structure, but throw away the force field created during the melting process. Afterwards one would train on the melt and crystal structure separately.
3) The evidence approximation is unable to fit (can sometimes happen for magnetic structures). In this case some investigations is needed and some parameters for the algorithm need to be adjusted. But this is the least likely to happen since Tikhonov regularization is turned on by default.
Please check if you encounter 1) or 2).
I have clarified the section for "ERR" in the monitoring part of the best practices for machine-learned force fields:
https://www.vasp.at/wiki/index.php/Best ... Monitoring
So if RMSE goes up what can cause that:
1) First check if you have a temperature ramp. If yes then increased temperature always leads to larger errors due to a broader variety of configurations.
2) If the RMSE suddenly goes up it could also mean that you encounter some kind of deformation of your cell. In that case your force field is not good enough to describe those structures. This is usually something you want to avoid if you want to train a specific phase. So in that case adjust your MD parameters. For melting for example this definitely happens. But there one would possibly use the force field with on-the-fly learning to get a melt from the crystal structure, but throw away the force field created during the melting process. Afterwards one would train on the melt and crystal structure separately.
3) The evidence approximation is unable to fit (can sometimes happen for magnetic structures). In this case some investigations is needed and some parameters for the algorithm need to be adjusted. But this is the least likely to happen since Tikhonov regularization is turned on by default.
Please check if you encounter 1) or 2).
-
xiliang_lian
- Newbie
- Posts: 37
- Joined: Thu May 19, 2022 8:44 am
Re: MLFF RMSE question
Thank you very much. It is very clear to me.