Having the same issue on Jenkins 2.181, GitHub Branch Source Plugin 2.5.3, spent hours trying to understand what's wrong until found this ticket. I think this is major usability issue.