Hi Zeyu,
Thanks for your question and for your interest in the TPP.
Yes I use MSFragger in the TPP and I did run into this exact problem last week. It required me to install java 9 jdk that I downloaded and installed from oracle. I then had to make sure that the windows PATH pointed to the correct version of java from the jdk. Let me know if you cannot find it and I would be glad to help you further.
I have found that the PeptideProphet model that seems to function best for MSFragger searches is the EXPECTSCORE based NONPARAM (semi-supervised) model, also with the option ONEFVAL to model all the charge states with a single f-value mixture model.
Hope you find this useful.
Cheers!
-David