Thanks! I didn't expect that selecting the images I want in the list would work, since I thought once I clicked "Link positions", the selection of photos to use or ignore for CP detection was already locked, but that's not the case.
To give more context on why both images I was looking at looked dark, below are the 3 layers for the whole panorama. I was looking at the first photos towards the left, where the difference is less noticeable than towards the sun.
Anyway, the problem is solved for me, thanks again.