TL;DR: don't use PSI to determine whether MPS is working. Use Chrome Dev Tools or webpagetest, because (a) too much caching (b) no user-agent control.
mod_pagespeed's webp transcoding only happens when there is an Accept:image/webp request header, which Pagespeed Insights unfortunately does not send. So when mod_pagespeed sees the request from PSI it will optimize jpegs to progressive jpegs if there isn't already a fresh version in the server-side cache. So if you are interactively using Chrome, you can visit your site and see the webp version, but those can't be sent to PSI. So mod_pagespeed will deliver the unoptimized view, while it re-optimizes the images in the background. During this optimization process, mod_pagespeed sends short cache lifetimes so that proxy caches won't deliver the unoptimized view forever.
WIth PSI, the usability is confounded by a server-side results-cache in PSI, which may capture the unoptimized view from mod_pagespeed. So you have to hit PSI a few times to let its results-cache expire, while hopefully mod_pagespeed's cache doesn't.