Yeah, I find setting taxon photos is very frustrating right now. Not being able to find a specific photo is the biggest annoyance.
The other issue is that there is no way to preview how a default photo will look when cropped to a square, and it seems to take a few minutes to update the picture throughout the site so you can actually check.
There are a lot of default photos I'd like to see updated but haven't because the process is really annoying.