Python client library / simple GUI

[Shish]

I wanted a systray interface to exist, so I made one~

Currently tested against the ruby daemon, on the assumption that the client-daemon interface will be approximately the same in the C++ version (in the visible API sense -- switching from JSON to MsgPack should be trivial, as long as there's still an add_share(code, path) method, etc). I plan to switch to working with the C++ daemon as soon as it actually exists :P

https://github.com/shish/python-clearskies

https://github.com/shish/clearskies-gui

I have the infrastructure set up for building binaries, but I'm assuming they aren't wanted yet (for starters, the gui currently only covers 75% of the CLI features; the rest shouldn't be too hard, I've just not got round to them).

A couple of API questions though:

- The client communicates with the daemon via Unix socket in ~/.local/share/ -- how does this translate to windows?

- "For forwards compatibility, unrecognized keys should be ignored. Unrecognized messages should be give <sentence is cut off>"

- protocol/control.md seems very unfinished, is the protocol likely to have large changes in future? (If it's vaguely stable, I'd be happy to finish documenting what currently exists)

-- Shish

[Steven Jewel]

On 04/13/2014 07:44 AM, Shish wrote:
> I wanted a systray interface to exist, so I made one~
>
> Currently tested against the ruby daemon, on the assumption that the
> client-daemon interface will be approximately the same in the C++
> version (in the visible API sense -- switching from JSON to MsgPack
> should be trivial, as long as there's still an add_share(code, path)
> method, etc). I plan to switch to working with the C++ daemon as soon as
> it actually exists :P
>
> https://github.com/shish/python-clearskies
> https://github.com/shish/clearskies-gui

Awesome work! I will link to this from the main clearskies README so
that other people can find it.

Python seems like a good choice for the desktop GUIs.

> I have the infrastructure set up for building binaries, but I'm assuming
> they aren't wanted yet (for starters, the gui currently only covers 75%
> of the CLI features; the rest shouldn't be too hard, I've just not got
> round to them).

That's also awesome. There has been enough interest in the project that
once we start shipping binaries I think we'll see a lot more contributors.

> A couple of API questions though:
>
> - The client communicates with the daemon via Unix socket in
> ~/.local/share/ -- how does this translate to windows?

It seems like "Named Pipes" (e.g. CreateNamedPipe) work in a similar
manner. We can also have the control interface be over a local TCP
socket if there are reasons why named pipes won't work.

> - "For forwards compatibility, unrecognized keys should be ignored.
> Unrecognized messages should be give <sentence is cut off>"

Heh, so sorry about that. I type much faster than I can think. (I'll
let you decide if that's because I'm a fast typer or just a slow thinker.)

Since the protocol is request-response, unrecognized messages should be
responded to with a "not_understood" response:

{
"error": "not_understood",
"message": "Could not understand $type message"
}

I think that gives us reasonable forwards compatibility, as a newer GUI
can probe an old daemon to see what it is capable of.

> - protocol/control.md seems very unfinished, is the protocol likely to
> have large changes in future? (If it's vaguely stable, I'd be happy to
> finish documenting what currently exists)

Yes, I started documenting it and then got distracted by other things.
I figured that once we actually had a GUI that we could fill in the
blanks. If you'd like to document it, I'd be happy to merge your
changes. Also if there are parts of it you don't like, let's change
them. As the first GUI, you can decide what direction it should take.

The only big improvement that I can think of is that we should probably
have a way to push messages, so that the client doesn't have to
continually poll. I think the simplest way to do this, from an
implementation standpoint, would be to let the client open a second
connection to the server, and then request to put it in push mode. Once
it's in push mode, it just streams a news feed of all updates until the
connection is closed.

There are probably some minor changes related to the large
"protocol_cleanup" merge that I did a month or so ago. It introduced
new things that aren't in the ruby implementation, such as a method for
handling conflicts. Those new things would also need to be exposed in
the API.


One final thought: on mobile we'll be integrating the C++ core right
into the application so that it's all in a single process. Let's call
this approach "libclearskies" for clarity. It doesn't make sense to
have this be a completely new interface for controlling the clearskies
code, so I'd like to see if we can have the control protocol to work
well for both cases.

Steven

[Shish]

> - The client communicates with the daemon via Unix socket in
> ~/.local/share/ -- how does this translate to windows?

It seems like "Named Pipes" (e.g. CreateNamedPipe) work in a similar
manner.

Not used them before, but they do look suitable, will check them out when I get back to a windows box. One minor thing seems to be that they can be network-accessible; not sure if that's opt-in or opt-out, but it's something for the daemon coders to be aware of

If you'd like to document it, I'd be happy to merge your
changes.  Also if there are parts of it you don't like, let's change
them.  As the first GUI, you can decide what direction it should take.

Cool :)  TBH I'm actually quite happy with what's there and how it works -- if the ruby daemon was portable and stable I'd happily call it 1.0 as it is now; I'd rather have a minimal working product than an amazing to-do list :P

Though if ideas are being written down for the future:

- get / set arbitrary config value? Just noticing that the GUI at the moment is practically stateless; my experience building scalable websites tells me that's a good thing, though I don't know if it really applies to desktop apps? The only client-side data storage I can think of is a couple of booleans for visual preferences like start-minimised vs start-windowed, whether that's stored as ".org.shishnet.csgui.foo = bar" in the daemon's database or in an ini file of its own isn't *that* much difference. And actually, having an ini file of its own means that the gui can launch without the daemon running, which sounds sensible, so I can't think of a use for this after all :) (leaving it in here because someone else might think along similar lines and come up with a real use-case that I've missed)

- get log data? The log file's physical location just feels like a daemon implementation detail rather than a public API to me, so I don't know if a GUI should rely on it always being at that specific location (or even rely on it being a file)

 

The only big improvement that I can think of is that we should probably
have a way to push messages, so that the client doesn't have to
continually poll.

That would be nice; a second connection that just handshakes and then sends {"type": "subscribe"} (or something like that) sounds good

    -- Shish

[Cotton Seed]

Comments inline.

Shish:

>
>>
>>> - The client communicates with the daemon via Unix socket in
>>> ~/.local/share/ -- how does this translate to windows?
>>
>> It seems like "Named Pipes" (e.g. CreateNamedPipe) work in a similar
>> manner.
>>
>
> Not used them before, but they do look suitable, will check them out when I
> get back to a windows box. One minor thing seems to be that they can be
> network-accessible; not sure if that's opt-in or opt-out, but it's
> something for the daemon coders to be aware of

Just to follow up on Steven's comment, I figured the plan was to use
libuv's pipes, which are UNIX domain sockets except on Windows, where
they are named pipes:

/*
* uv_pipe_t is a subclass of uv_stream_t
*
* Representing a pipe stream or pipe server. On Windows this is a Named
* Pipe. On Unix this is a UNIX domain socket.
*/

Having a pipe abstraction in python would be great. It could be used in
my python command line client, too.

[Shish]

Library updates:

- Unit test coverage up to 100% (excluding one abstract class and a non-production fallback)

- Theoretical windows support added (I've only proven it works, not tried running it ;) )

- README updated to note that it includes a simple CLI for testing with / debugging

GUI updates:

- Remove silly 100 share limit

- Begin work on a full window power-user view (double-click the tray icon to toggle window visibility)

-- (Nothing much there yet, just the skeleton of the layout so people can see where I'm going and suggest better ideas :P)

    -- Shish

[Steven Jewel]

On 04/15/2014 08:57 AM, Shish wrote:
> Though if ideas are being written down for the future:
>
> - get / set arbitrary config value? Just noticing that the GUI at the
> moment is practically stateless; my experience building scalable
> websites tells me that's a good thing, though I don't know if it really
> applies to desktop apps? The only client-side data storage I can think
> of is a couple of booleans for visual preferences like start-minimised
> vs start-windowed, whether that's stored as ".org.shishnet.csgui.foo =
> bar" in the daemon's database or in an ini file of its own isn't *that*
> much difference. And actually, having an ini file of its own means that
> the gui can launch without the daemon running, which sounds sensible, so
> I can't think of a use for this after all :) (leaving it in here because
> someone else might think along similar lines and come up with a real
> use-case that I've missed)

There are lots of "advanced options" that we could expose to the GUI. I
think an interface similar to the about:config interface in firefox
would work best, since that means the GUI wouldn't need to do any
config-option specific programming. Some examples:

* connection timeout duration
* port numbers to listen on [1]
* whether or not to use the DHT, public tracker, etc.
* whether or not LAN connections should be rate limited
* how often a manual rescan of the filesystem should happen [2]

> - get log data? The log file's physical location just feels like a
> daemon implementation detail rather than a public API to me, so I don't
> know if a GUI should rely on it always being at that specific location
> (or even rely on it being a file)

You're right. Some daemons might just keep the latest log information
in memory and never write it to disk.

My personal preference would be that we work to eliminate the need to
expose the log in the GUI. Right now it's handy because the software is
alpha grade, but we should work to expose the information that an end
user would actually care about with status commands. For example, one
thing I'd be interested in is why I'm not connected to a peer. If in
the status details it had a list of all the peers, the last connection
time, and the most recent connection error, that'd eliminate the need to
examine the log for that particular purpose.

Steven

[1]: In theory this shouldn't matter, since between uTP and UPnP it
should work automatically, but advanced users like to these sorts of things.

[2]: My proposal for settings like these is that we make the default
value be "auto", which means that it auto-tunes based on how long the
last full file scan took.