Wouldn't it make sense to add a 'Reply by email' button if the person
I've been talking with went off-line? Then I'd like to either write the
reply directly in Talk's window or at least be shown the chat history
in the Gmail reply window.

PS. I'm quietly hoping for Talk to grow into an simple desktop e-mail
client for Gmail, seems like an obvious thing to do for Google, and
quite an interesting task for developers, to create a simplified e-mail
client, similar to Gmail, which itself is very good, but on desktop,
which would allow UI developers to explore further, with the help of
practically unlimited processing power, disk space, and bandwidth, and
a very consistent and powerfull windowing mechanism of a modern desktop
computer (as compared to what's available for an Javascript
application).

PS2. Or at least implement some simple local storage (via Google
Desktop), search (via Desktop), and tweak Gmail code to make it work
with a local installation of Gmail server. That would at least allow
for live search (via Desktop) and instantanious delivery and
notification of e-mails within Gmail (via Talk).

To reply to a chat by GMail, find the chat in question under 'Chats' in
your GMail folders, scroll all the way down and just start typing.

I don't know much about using GMail with local storage and search, but
I can tell you that it is possible to download your messages to a
regular desktop client using POP; I also seem to remember that Google
Desktop has an option to let you search your GMail. A locally installed
GMail server is not going to happen, though: I very much doubt that
Google is going to make it available for free, and even if they did, I
have to assume that the hardware requirements would be higher than most
home users would be comfortable with.