State of the Do-nion

[Alex Launi]

Alternate title: "What About Do, the Fingerbang King".

Some of you might be wondering what the hell is up with Do. Are we dead? Are we alive? Are we in some sort of cryogenic sleep, waiting to awaken in 2012? Honestly, that's the closest, but 2012 isn't the target date- and the sleep isn't sleep. The sleep is like staying up all night hacking our fucking asses of rewriting the entirety of Do.

That's right. Rewrite. We're rewriting Do, with an entirely new architecture. 

Some of you might remember the meeting we had a while ago over Google wave. We talked about moving plugins out of process, and making Do a receiver vs. a creator itself. This work has finally begun. Chris Czsiksoy, Chris Halse Rogers and I have been talking over IRC and have been making more concrete specification as blueprints on lp. So far we have two, and a branch full of mocked up methods that need implementing.

The blueprints are https://blueprints.launchpad.net/do/+spec/service-loading and https://blueprints.launchpad.net/do/+spec/universe-management. More will be coming as needed. The next one I'm going to write up will be about Item wrapper proxies, for how sources will create items, and item sources and export them over the bus, and how Do will use them.

The code is living in lp:~alexlauni/do/dos-new-do. Until it is stable this is where development will continue. I am the gatekeeper. If you work on code, submit a merge request, I (or maybe some other member of Do core) will review, and it will be merged in. All classes must be fully documented before they are committed. Tests would be ideal, but I'm not sure about that yet. If you write tests you will most certainly be bought drinks the next time we meet.

There's a lot of work to be done, so anyone interested in helping come to #gnome-do and come be part of an awesome new future for Do. Expect more e-mails. Contact me with questions, concerns, insults, and love notes.

---
-- Alex Launi

[Nicolas Chachereau]

Alex Launi <alex....@gmail.com> wrote:
> [...]

> That's right. Rewrite. We're rewriting Do, with an entirely new
> architecture.

Cool! Do's not dead. This is great... or so I thought at first. But
now I have become a bit skeptical. I wonder: why did you make this
move? why is it important from an engineering point of view? Is this
the end of Do as we know it?

And, more importantly, what is it going to bring to the end user? As
far as I understand it, each and every plugin will have to be
rewritten... so there are big risks. Am I mistaken on this? I don't
understand the blueprints very well.

Here are a two things I'd like to see in Do. I think they have been
discussed before, even though I don't remember where and when. Will
this rewrite make it any easier to get any of that pie-in-the-sky
stuff? If not, will it bring some other goodness I'm not even thinking
of?

* A way to launch Do from Nautilus to do something on the selected
files. While I've never used Quicksilver, this seems to be one of the
feature that make it powerful. Right now, I don't use Do much to do
things with files: drilling down to the wanted file is a bit annoying,
and multiple selection really doesn't work well (see bugs #268295 and
#320365).

* An easier way to develop plugins. I guess it is not really
complicated, but for simple plugins I shouldn't have to read about the
internals of Do, about its "Universe", nor should I need to learn a
new programming language (at least for simple tasks). I am still
impressed with the way Textmate (on the mac, again I have never used
it) allows extension: write a script in your favorite little language
(bash, python, ruby, whatever), you can use some environment variables
set up by the program, tell me what your input and output look like,
and that's basically it [1]. Could this maybe inspire Do?

Right now, I can easily set up a shell script as a program for Do to
run, even with the custom icon, by writing a .desktop file. I can't
pass arguments to it, however. If we could write a configuration file
that says "present an action with this name, this description, this
icon for items of standard type text or url or file or ..., this
action runs this shell script, or runs this external application, with
those parameters, passing the item as that parameter... and doesn't
return anything/returns a filepath/a url"
A lot of plugins are just running external applications any way:
AptURL, Archive, EOGSlideShow, Locate...

It may be very easy already to do this. If so, show me. And even if it
is: could it be made easier?

There are other things I'd like to see, but mainly it's just some
plugins I wish I had come around to writing, so the above takes care
of this.

This was probably too long. My point really is: what should we expect?
This is not the best time for me to start helping on an open source
project, but if it seems worthwhile, I might try to give you guys a
hand - even though my knowledge of C# and Do's code base is limited.

Regards,
Nicolas

[1] If you've never used Textmate either, see
http://manual.macromates.com/en/, especially
http://manual.macromates.com/en/commands#commands, and the "Scope
based customization" screencast on http://macromates.com/screencasts.
(The beginning is less interesting but at about 7:18 you can see how
adding a command works).

[Alex Launi]

On Wed, Aug 25, 2010 at 5:38 PM, Nicolas Chachereau <nicolas.c...@gmail.com> wrote:

Cool! Do's not dead. This is great... or so I thought at first. But
now I have become a bit skeptical. I wonder: why did you make this
move? why is it important from an engineering point of view? Is this
the end of Do as we know it?

Not really, Do will look and behave similarly (a lot will change too though), but pretty much everything under the hood will change.

 

And, more importantly, what is it going to bring to the end user?

We want Do to become a first class part of your desktop. To implicitly integrate with your system. We want the applications themselves to have more control over what Do does. This should increase the coherence between Do and your desktop.

Another great engineering point is that it lets us also move the plugins into their own processes. Plugins will never again be able to crash Do. If a plugin crashes you won't get its items- but everything else will still work, as if nothing happened.

 

As far as I understand it, each and every plugin will have to be
rewritten... so there are big risks. Am I mistaken on this? I don't
understand the blueprints very well.

Yeah, pretty much. A lot of the code will be reusable but it will need ported. Most of the application interface logic will be the same.

 

Here are a two things I'd like to see in Do. I think they have been
discussed before, even though I don't remember where and when. Will
this rewrite make it any easier to get any of that pie-in-the-sky
stuff? If not, will it bring some other goodness I'm not even thinking
of?

* A way to launch Do from Nautilus to do something on the selected
files. While I've never used Quicksilver, this seems to be one of the
feature that make it powerful. Right now, I don't use Do much to do
things with files: drilling down to the wanted file is a bit annoying,
and

Ehh maybe. I'm not too into multiple ways of launching Do, and the selected file thing (I think) is waiting some changes in nautilus. I'd love to be able to act on selected file though. It's something I'll look back into when we're there. 

 

multiple selection really doesn't work well (see bugs #268295 and
#320365).

I'l definitely be reworking multiple selection. I'm not sure what it will look like yet but it won't be as half-assed and confusing as it is now. 

* An easier way to develop plugins. I guess it is not really
complicated, but for simple plugins I shouldn't have to read about the
internals of Do, about its "Universe",

You shouldn't really have to now. Simple plugins can be hacked up in about 20 minutes, with no real knowledge of Do's internals.

 

nor should I need to learn a
new programming language (at least for simple tasks).

This will be possible! That's part of the benefit of the d-bus based api. You'll be able to write a plugin in anything that speaks d-bus, but for some languages it will be easier than others. Once we're confident enough that the d-bus approach works well enough, and our implementation is solid, we're going to write some wrapper libraries that will handle all of the d-bus and other crap that all plugins will have to do. Essentially all you'll have to do is override or implement a couple of methods. It will be no harder than it is (or is supposed to be...) now. We're going to do a pure C# wrapper (which should mean you also can use any .NET language), and GObject wrapper, which we'll use to generate Vala, Python, and whatever other language bindings can be generated from gobject-introspection.

 

I am still
impressed with the way Textmate (on the mac, again I have never used
it) allows extension: write a script in your favorite little language
(bash, python, ruby, whatever), you can use some environment variables
set up by the program, tell me what your input and output look like,
and that's basically it [1]. Could this maybe inspire Do?

I'll look. I'm not really sure the model is the same though. It will be more like what I described above.

 

Right now, I can easily set up a shell script as a program for Do to
run, even with the custom icon, by writing a .desktop file. I can't
pass arguments to it, however. If we could write a configuration file
that says "present an action with this name, this description, this
icon for items of standard type text or url or file or ..., this
action runs this shell script, or runs this external application, with
those parameters, passing the item as that parameter... and doesn't
return anything/returns a filepath/a url"
A lot of plugins are just running external applications any way:
AptURL, Archive, EOGSlideShow, Locate...

Yeah this is a good point. I don't know yet. I'm really not into the idea of custom configuration files people need to write, but there might be a more elegant solution. The fact that it will be possible to write plugins in Python might already solve this.

 

This was probably too long. My point really is: what should we expect?
This is not the best time for me to start helping on an open source
project, but if it seems worthwhile, I might try to give you guys a
hand - even though my knowledge of C# and Do's code base is limited.

Happy to answer all of your questions, I hope I did a sufficient job. If you have any questions about blueprints, or if you can identify parts that are unclear to you I'd like to know so I can clear the up.

--
--Alex Launi

[Nicolas Chachereau]

Thank you Alex for your thoughtful answers. I have one follow-up
question. You wrote:

> We want Do to become a first class part of your desktop. To implicitly
> integrate with your system. We want the applications themselves to have more
> control over what Do does. This should increase the coherence between Do
> and your desktop.

This sounds great, but it is pretty abstract. Can you give me an
example of this tight integration? What application could benefit from
having more control over Do?

> Another great engineering point is that it lets us also move the plugins
> into their own processes. Plugins will never again be able to crash Do. If a
> plugin crashes you won't get its items- but everything else will still work,

> as if nothing happened. [...]

Sounds good, even though Do never crashes on my computer.

Thank you for taking a look at my "wishlist". Being able to write
plugins in any language is very nice! I won't comment a lot more, but
if I understand correctly, you will look into some of these issues,
which is great :)

> Happy to answer all of your questions, I hope I did a sufficient job. If you
> have any questions about blueprints, or if you can identify parts that are
> unclear to you I'd like to know so I can clear the up.

I'll look more into them, and see if I can give you a better feedback.

[Christopher James Halse Rogers]

On Thu, 2010-08-26 at 21:47 +0200, Nicolas Chachereau wrote:
> Thank you Alex for your thoughtful answers. I have one follow-up
> question. You wrote:
>
> > We want Do to become a first class part of your desktop. To implicitly
> > integrate with your system. We want the applications themselves to have more
> > control over what Do does. This should increase the coherence between Do
> > and your desktop.
>
> This sounds great, but it is pretty abstract. Can you give me an
> example of this tight integration? What application could benefit from
> having more control over Do?

Firstly, there's an overall goal here in reducing the current plugin
bottleneck. We've got *lots* of plugins, but in order for your swanky
new application to get supported in Do you additionally need to write a
Do plugin, get that merged into do-plugins, etc.

The new architecture allows the application itself to provide the
functionality a Do plugin would provide.

Since the requirements for Do are quite close to the requirements for a
GUI automation/scripting tool, I think this will create exciting
opportunities.

We're also looking at having a live-search mechanism, where rather than
the plugin/application providing all the items up front it can provide
them on-demand. Think google's search suggestions, or a full filesystem
search, or a dynamic search of Banshee's database.

[Joe Hillenbrand]

On Thu, Aug 26, 2010 at 6:00 PM, Christopher James Halse Rogers
<chalse...@gmail.com> wrote:
> On Thu, 2010-08-26 at 21:47 +0200, Nicolas Chachereau wrote:
>
> Firstly, there's an overall goal here in reducing the current plugin
> bottleneck.  We've got *lots* of plugins, but in order for your swanky
> new application to get supported in Do you additionally need to write a
> Do plugin, get that merged into do-plugins, etc.
>
> The new architecture allows the application itself to provide the
> functionality a Do plugin would provide.

This explains why the plugins I have written have not been added to
do-plugins. Good show! I'm excited to see what you come up with.

[Alex Launi]

On Fri, Aug 27, 2010 at 2:05 AM, Joe Hillenbrand <joeh...@gmail.com> wrote:

This explains why the plugins I have written have not been added to
do-plugins. Good show! I'm excited to see what you come up with.

Yeah, it might be a reason. Honestly it's just because I've been negligent. Sorry.

--
-- Alex Launi

[trusktr]

OK, awesome! Remeber to include the new Docky integration!!!

On Aug 27, 5:12 pm, Alex Launi <alex.la...@gmail.com> wrote: