Well, that's a pretty good stab at some of the things that I might say.
I.e. At first glance I agree with most of what you've said I would say.
But there's more to it.
Lot's more.
And again, I hesitate to get into this for a number of reasons including
skepticism that I'll be able to explain myself adequately and/or that
I'll have the time to do so here on this ng.
But I guess I'm gonna try...
So, at it's simplest, playing something completely by ear has to involve
first being able to hear it in your mind's ear prior to executing it.
The development of this in a musician seems like it has to start with
him (or her) listening to someone else's music, assimilating that in his
ear and developing the ability to repeat those pitches in the same
sequence as the original.
I don't know how anyone can play anything at all by ear if they have not
already developed that ability to at least some extent.
Thus the importance of doing lifts when you want to become a jazz
musician, or any kind of a musician that improvises.
But this is not improvising.
The process of improvising something *of your own* involves a great many
other steps.
Some of those steps are to compose things yourself, work with them until
you can hear them, learn to play them on your instrument and then try
various ways to consciously plug them into your improvisations.
This is one of the steps in the process where good music theories might
help to guide you towards intervallic (or rhythmic) things that hang
together nicely so you can experiment with them *until you can hear them*.
But ultimately, when you're really performing and not merely practising,
you'll also have to learn how to let those ideas come out naturally,
without consciously trying to plug them into your solos (or comping).
I.e. We have to practise not practising too. lol
But there's still more involved...
What I just described is not really improvising in the truest sense.
It's just regurgitating things that you've already worked out.
In order for this process to turn into real improvising you'll need to
do a whole lot of the above so that you've got a really large palette of
musical sounds that your ears and your hands are familiar with and then
you'll need to develop the ability to seamlessly switch from one
familiar sound to another, often on the note-by-note level.
I.e. Ultimately, IMO, this becomes about hearing things on an
interval-by-interval basis, not always in whole phrases or in larger
chunks - like chord-scales or voicings or familiar chains of chords.
But there's still more...
It's one thing to play something that you hear, but what if it doesn't
fit what then rest of the band is playing?
I often use the example in my own development of trying to "play by ear"
on min7b5 chords when I made my first attempts at jazz playing while at
Berklee.
I was playing things that *I* heard, but I hadn't learned how to hear
min7b5 chords yet or how my own note choices mixed with the sound of
that chord.
So I had to spend some time studying min7b5 chords before I cold hope to
have any success with any of this.
But there's still more to it....
Mick Goodrick does this great random chord exercise with his students
where he'll play random chords, usually for a pretty long duration,
while the student tries to solo over those chords without knowing what
they are.
If the student plays a note that sounds funky on the chord-of-the-moment
he is encouraged to first resolve that note by 1/2 step before leaping
away to some new note.
FYI There's almost always a 1/2 step resolution (sometimes up, sometimes
down, sometimes either) available for any inharmonic note on any chord.
But in order for a student to be able to do this even remotely
successfully he has to already be familiar with the norms of which notes
happen to fit which chords, otherwise he may be unable to judge whether
the note he's playing "fits" or not.
E.g. To some novices a b9 on a dom7 chord sounds totally wrong.
But this exercise trains you to be in the moment more than any other
single exercise I've ever come across.
And being able to hear and spontaneously resolve these types of rubs on
the fly as you're performing is essential to being an improviser IMO.
No matter how much you've worked on and assimilated various sounds into
your own playing, you have no control over what the other guys in your
ensemble are going to play.
If the piano player plays a min7 chord with the b3 on top and you
happened to hear a maj 9th on that chord in the same octave as his b3,
there's going to be a rub and you'll probably want to be aware of that
as you're hearing it (of course you have to be listening)and be able to
resolve your note up a 1/2 step for a more successful line.
And you don't need to be aware what he's playing or what you're playing.
You just have to be listening and reacting, but reacting within a
certain framework of musical experience.
Etc., etc., etc.
But there's more....
Sometimes my fingers lead me somewhere that my ear is not really sure of.
I've been playing guitar long enough that I have a pretty good idea of
what something will sound like in that millisecond od 2 just before I
strike the note.
Sometimes this allows for good results.
But sometimes it leads me into a clam, and then Mick's random chord
exercise training kicks in and I can still save the day.
And these accidents are really the spice of jazz, as Miles himself has
been quoted as saying (in his own words of course).
So, ultimately, for me, "ear training" is really just musical experience.
The more things you've played, tried to play, listened to others play,
etc., etc., will set the basis for what it is that *you* can hear.
Back to theory again briefly...
Theory just allows you to methodically check out the way that certain
sounds hang or don't hang together.
What you do with those sounds, after you've lived with them, is your own
creative business.
But wait, there's more....
What if you can "play what you hear" with no effort all the time but
it's just not very good?
Well then you've gt some more listening, learning, thinking and
soul-searching to do.
And how do you turn off all the knee-jerk study habits you developed
when practising when you're actually performing?
Well, for me, that's only been possible when I really know the materials
of the tune I'm playing inside and out and it happens far less than I'd
like it to, especially these days because I'm not playing out nearly
enough anymore so every time I do I gig now it's like I've forgotten how
to play until maybe the 2nd set. lol
I.e. For me, I still need to tap into things that I know (or think I
know) will work until after I've had some time on an unfamiliar tune (or
even a familiar tune that I haven't played in a while).
Reacting well to my mistakes, ala Mick's exercise above, is key to this
being successful or not.
Even tunes with seemingly common chord progressions require their own
study because those tunes are not *exactly the same* as the other tunes
that I already know that use those same progressions, so certain note
choices are called for on one tune vs the other, etc., if that makes any
sense.
But wait, there's more....