If you search this forum using the search term "jumpy" you will find numerous threads,
recent & old, that pretty much uniformly reinforce the advice you gave yourself above,
and I quote you:
However, the video plays normally inside VLC.
Yes. VLC is the recommended player. You have given yourself the evidence that QuickTime
does not do as good a job as VLC. Draw the inevitable conclusion. Stop using QuickTime.
Stick with VLC. There are more elaborate workarounds mentioned in the threads you will
find if you search for them. It's up to you whether you will read them & follow the
advice there. The simple answer is to let go of QuickTime & hang onto VLC.