on the network. Which is good, because INP should never measure network.
However, what I do see is that network response for the images does add Long Tasks and long running rasterization work to the page, and that in turn may affect interactions. On your example page, none of your event handlers themselves are slow, it's just that if there happens to be any long running work processing these huge images while they are loading, interactions become affected by these huge workloads -- either via input delay blocked on previous frame Commit, or via presentation delay of the Interaction's animation frame, waiting for current frame Commit or off-main GPU rasterization for sync image decode.
I also noticed that even after the images are fully loaded and fully decoded once, as you switch back and forth between them each interaction may or may not continue to be slow. I think that is because your click handler is wrapped with a React.startTransition (I am not sure, I am reading minified code). State updates / effects inside startTransitions are supposed to be delayed and time sliced in idle time... but it is not actually guaranteed to wait until the next animation frame. Because your event handler + Component render() is so fast, I think your transitions sometimes apply dom updates in the same frame.
I have had that happen to me with React before. There are way to explicitly delay the transition until after next paint -- but in this case, I think you are much better served just fixing the problem: reduce the costs of these images, as you have already done.
Cheers.