Proposed Plugin API

[David Stark]

Here's a big list of all the listeners and attachment functions I've thought up that might be needed for plugin developers. Are there any others you can think of, or any clarifications that are needed? Obviously implementing all these will take some time, so I will probably start on the more basic ones, or any which you would like me to prioritise.

Listeners:

General:
- Post code load hook that executes once all code is loaded
- Pre-shutdown hook that executes before Builder is shut down

Script editing:
- Entire test script changed (new script, script switched, script discarded)
- Step added/deleted
- Step parameter changed
- Step type changed
- Parameter edit started/ended
- Type edit started/ended
- "Find a different target" started [1]
- "Find a different target" ended
- Script saved/exported to location
- Script converted to different selenium version

Suites:
- Script added/removed
- Suite discarded
- Suite saved to location

Recording:
- Recording started [1]
- User action captured [2]
- Step modified [3]
- Step recorded [4]
- Recording ended

GUI attachment API:
- Add/remove new menu item to existing menu
- Add/remove whole new top-level menu
- Add/remove step menu item
- Show dialog
- Hide dialog
- Add/remove element at top/bottom of script
- Add/remove element in step
- Add/remove element in text/locator edit
- Add/remove element in step type editor
- Add/remove element in startup view

Notes:
[1]: Can supply a visitor function that is used to attach listeners to the page.
[2]: Listener is supplied raw event and can manipulate its data or prevent the recording of the event.
[3]: Listener is supplied data structure describing the change and can manipulate its data or discard it. This data structure should probably be a general "step change" event that is also used when listening to user edits of the script.
[4]: Listener is supplied generated step and can manipulate its data or discard it.

---

- David

[Adam Christian]

This sounds pretty comprehensive, I'm curious if you have thoughts on the right way to package this stuff up, install and store it in the client?

Adam

[Zarkonnen]

Thoughts but nothing concrete yet:

AFAIK Selenium IDE uses Firefox extensions as its plugin system. So you install an extension that then modifies the IDE. We could do the same thing - in this case the extension could just register with Builder a set of JS files that it wants to have loaded. The issue with this is that it might be too tightly coupled to Firefox. But if the extension mechanism is basically "call this function with a list of paths", then that's hopefully fairly portable.

Apart from that, I really really want to store the extensions required for a given script in the script JSON. So each extension should have a unique identifier by which it is known. Probably a URL - obviously the URL of its project or download page - but *not* its AMO URL, as plugins should hopefully have per-browser implementations. So the existing JSON format would be enhanced by an extra optional top-level list of strings which are these URLs.

So when the user loads in a script they can't run due to a missing plugin, Builder can supply a download link.

The alternative is to somehow build our own plugin system and repository, which could potentially have a nicer UX, such as the ability to install plugins without having to restart the browser. But not sure that's worth it.

- David

[Adam Christian]

Having spent significant time writing plugins for IDE -- I found the firefox plugin architecture to be pretty terrible and tightly coupled to the firefox overlay internals. I feel pretty strongly that we should do this from scratch in a better way that involved no knowledge beyond JS/CSS/HTML.

I'm wondering if there is another place in the browser we can store this stuff..

Adam

[David Stark]

Ignoring questions of feasibility for now, what would seem like the nicest way of doing things would be to have a GUI within Builder that lists the extensions you have and allows you to uninstall/update them, as well as a list of known extensions you can install with a single click. (The canonical list can live as a JSON file somewhere on a server.)

Things we would need for that to become possible:
- A place to store the extensions.
- The ability to download them.
- Either the ability to unzip them after download, or keep them in a zipped state and access their innards as needed.
- A way for plugins to refer to files like images. With Se Builder, images can just be referred to using relative paths or absolute chrome paths, but how do we find images inside Se Builder extensions?
- Given that some extensions may need to modify Builder at startup/load time, restarting it may be necessary for new extensions to take effect. Can this be done cleanly without also restarting the browser?

The answers I have to these questions so far, assuming Firefox as a platform:
- We can use local storage ( https://developer.mozilla.org/en/XUL_School/Local_Storage ) to store Se Builder extensions in a directory directly inside the user's profile directory.
- Firefox does have ZIP support so we can load data from zip files. ( https://developer.mozilla.org/en/XPCOM_Interface_Reference/nsIZipReader ). There used to be code for loading and executing Javascript files from input streams, and that can be resurrected.

The big issue then is how extensions should refer to image files, etc. Older versions of Builder actually rewrote URLs found in HTML pages to point to the right location, but that seems hacky. (What about image URLs in Javascript?) We could use a combination of a special syntax for HTML files that says "replace this URL with one pointing to the right location" and an API in Builder for fetching resources from the extension - but this seems like it would add a lot of confusion for extension authors.

Anyway, it looks like it may be possible to do this. Another approach would be to find a FF extension that has its own sub-extensions in the kind of way we want, and copy whatever they're doing.

- David

[Stephen Mc Gowan]

Hey,

I meant to add to this a while back but I've been busy. 

Another nice feature for plugins would be if it was possible to distribute SEBuilder with plugins pre-packaged. So you wouldn't need to tell a user to download SEBuilder, then install plugin A,B & C....you could add just prepare an xpi with SEBuilder with plugin A, B & C bundled already.

-Steve

--
Stephen Mc Gowan.

[Adam Goucher]

That is already supported in Firefox (and is how Se-IDE is shipped).

-adam

[Adam Christian]

Greetings,

Thanks for the feedback folks. 

What I would ideally like to see is a plugin layer that is completely independent of anything Firefox. Builder needs to be ported to Chrome and so I would prefer if the plugin layer relied upon nothing beyond privileged XHR to pull down files and store them. Then on restarts of the extension they can be parsed and applied to the UI etc.

I don't like the way plugins with IDE work at all, I would like to have a HTML/CSS dialog that you pull up and allows you to navigate, search and one click install these extensions. Assuming that they will be chunks of HTML/CSS and JS I don't think it should be all that complex to do. Having written and supported to FlexPilot IDE extension, the overlays model that firefox and IDE uses is totally un-intuitive and quite nasty as they appear as separate extensions. I think we should start from scratch on this one, and I have asked our UX guy that contracts for us at Sauce to take a look at coming up with conceptual mockups for us to discuss.

David, can you do a brain dump of what would be the technology challenges of implementing something like that?

Adam

[Zarkonnen]

> David, can you do a brain dump of what would be the technology challenges of implementing something like that?
>
> Adam

I've identified the following components of the plugin system:

- Navigating and searching for extensions
- Extension format
- Extension metadata
- Downloading extensions
- Storing extensions
- Loading extensions
- Uninstalling extensions
- Referring to images / loading resources

Going through them one by one, looking at technology needed:

Navigating and searching for extensions:
This is pretty easy. We want to store the list of extensions on a server somewhere, and we want to make sure that each extension entry can point to multiple downloads for different browsers.

Extension format:
I think each extension should be a zip file with a very simple internal structure: there is a JSON header file at the top level. This contains the metadata and a list of relative paths within the zip to JS files that Builder should load in. (see https://developer.mozilla.org/en/XPCOM_Interface_Reference/nsIZipReader )

Extension metadata:
Each extension should store its name, icon, description and version compatibility information.

Downloading extensions:
As you suggested, this can be done using XHR. https://developer.mozilla.org/en/DOM/XMLHttpRequest/Sending_and_Receiving_Binary_Data describes how to cleanly read binary data. One issue is that we can't provide a progress bar for the download, but hopefully the downloads should generally be small.

Storing extensions:
In Firefox, the Correct Way of storing extra files for an extension is to put them in an appropriately-named folder directly in the profile folder. This is how major extensions like Adblock Plus and Greasemonkey do it. (see https://developer.mozilla.org/en/XUL_School/Local_Storage )

Given that we'll want to be able to refer to files inside the extensions in HTML (see "Referring to images / loading resources"), extensions will need to be unzipped into folders.

Loading extensions:
After Se Builder itself has loaded, but (probably) before the hooks trigger that cause the UI to be displayed, Builder goes through each installed extension and loads and executes the JS files referenced in the header.js file. These files can then install listeners (as discussed in the earlier email) to extend Builder. (There used to be code in Builder for reading in and executing files, and this code can be resurrected.)

One potential issue is that the code may behave differently depending on which order the extensions are loaded in. Extensions should at least be sorted by identifier to produce a consistent loading order. (Otherwise there'll be lots of hard to reproduce bugs where extensions conflict only if loaded in a specific order.)

Uninstalling extensions:
The simple way to do this is to just remove the extension from the list, and then require a restart of Builder. Will have to make sure that Builder can be restarted cleanly with no old state "hanging over" from its previous incarnation. (Firefox gets very enthusiastic about caching at times.)

Referring to images / loading resources:
The big issue is how to refer to other resources. For example, if an extension wants to add a GUI element that includes an image, what should the HREF path for it be? With a Firefox extension, this can be a chrome:// URL, and that will get resolved correctly. But I don't think it's possible to construct chrome URLs that will point to the right location inside the profile folder.

So instead, we need to use file:// URLs. How do we get these? There can be a getURLForFile function in the extension API that turns a relative file path into a valid file:// URL. This does mean that you have to go via this function each time you want to include an image, but I don't see a way around that.

The API can also have related functions for getting nsIFiles / input streams from a relative path.

Summary of main issues:
- Cleanly downloading extensions
- Unzipping extensions
- Reading extension JS via input stream and executing it
- Minimising chance of extension conflicts
- Cleanly restarting Builder when installing/uninstalling
- URLs for images inside Builder extensions/input streams for resource files

In conclusion, I don't see any huge obstacles, though getting the whole download-unzip-load-reboot cycle to work cleanly may take some time, and we'll definitely have to clearly document how you get at those file URLs.

---

- David

[Adam Christian]

Fantastic list,

This also looks like a pretty solid plan to me -- I suggest we get a proof of concept working and see what the reality is :)

Adam

[David Stark]

Shall do. I'm currently engaged in yet another round of code tidying to pave the way to having an extension API, and once that's done I'll dig into it.

- David