You are not alone in realizing there are issues with gitflow and CI.
The problem with having CI pointed to development is that there is a
lot of code that does not get tested. A hotfix come from the master
branch. How do you have your build server suddenly point to these
dynamic branches? Even if you decided to merge to development branch
AFTER you merged into master; development would most likely have new
features that are not a part of the master branch yet.
We use buildbot and as it doesn't support dynamically named branches
that gitflow says to use. I think our solution will be to just use the
'release' branch for releases as well as hot fixes, but this will only
work until we are in a situation where we need to apply a hotfix while
working on a release.
I think the versioning issue you brought up is just one part of the
flaw with this flow and probably not even the worse.
I also am struggling with the issue of how to support maintenance
branches (long term supported deployments).