I have heard of a case where a former SO was awarded visitation
rights, but I forget the details.
I do not understand why you are considering this route.
Bringing up the whole issue of legal action seems way out of
line here. If you are on reasonably good terms with K's mom,
then you should be able to discuss this openly. I gather that
you have not yet. If K is bringing it up to you, then he is
probably bringing it up to her too, right? Just mention to SO
that you are going to Florida again this year and invite K to
join you. Tell her he has been asking if he can go.
Don't make this a power or control issue between you and SO.
You have good visitation as it is. It sounds to me like you are
looking for a fight. Btw, if SO does not let K go to Florida
without her, that does not make her unreasonable; he is only 4.