Hello Patrick,
I'm getting a similar problem with a video. I jump to 5 different points of the video, playing only 3 seconds of each fragment.
Checking the buffered attribute I can see that exists buffer for each fragment, so I supposedly can use this buffer again.
What happens is: first fragment buffer is used, middle fragments download again (I can see on the network console), and the last one use the buffer.
Clean cache, reload. Same result.
Anybody could explain that? Why the buffer is not being used?
Thanks!