This is common with multiplane panoramas. You can delete control points at the bottom and optimize (or align) again.
The objects closer to the camera are sharper, this is why the detector mainly places control points on the objects near you.
No worries, you can delete all points on trees and rocks. Even if they are not aligned, the stitcher will blend them. Nobody will count leaves on trees and cracks on the rocks :)
For you the most important is the horizon and the objects in the middle.
If you want to make your work easier, add exclude masks on the bottoms before searching for control points,
Then select "remove control points in masks", and remove the masks.
You should see if there are any important objects at the bottom - buildings, towers etc. that could be duplicated or hidden on the panorama.
For these objects, add one or two control points manually.
Take care to rotate the camera only around its axis, this is very important
for multiplane panoramas.
