Getting to version 1.0

[Eric Saunders]

Greetings and salutations...

So... let me summarise the current project status and describe where I
think we go from here.

0) During the three days of the implementation session at HTN IV, we
jointly decided to implement an open source telescope embedded agent
to reduce the barrier to entry of new telescopes seeking to join an
HTN. The concept was to allow someone with only a cursory knowledge of
the HTN protocol to 'HTN-enable' their telescope by downloading our
code, following some instructions, and then figuring out how to
provide the telescope-specific things our agent needs in order to
function. An interesting by-product of such an agent would be the
creation of a number of components that could be reused in a future
user agent (next week's project) ;).

1) To this end, we designed an abstract framework to cleanly separate
out each functional component, and provide clear boundaries at both
the outward HTN face and the inward-looking telescope face.
Specifically, once we pass the RTML out to the HTN end-point
component, we're done. What happens next is up to the specific
implementation, whether written to file, sent out by SOAP over HTTP,
or piped to a socket, for example. Similarly, once we pass RTML
through the scoring, request processing and phase 0 interfaces, what
happens next is up to the telescope implementation. For example,
scoring could be as simple as "1=on, 0=off", or as complicated as
Steve Fraser's dispatch scheduler probability algorithm. Finally, the
telescope must specify whether to implement either the event listener
or the event poller mechanisms, in order to tell us when something of
note has happened, and critically, tell us when this request is over
from the telescope's point of view.

2) Having identified the bits that weren't our problem, we next
concentrated on the specific internal functionality of the embedded
agent that was required, and this was incorporated into the second
component of the project, the core. This includes the authentication
module, the parser, the delegator (determines from document type where
to send it), the document manager (to handle the document lifecycle,
and track requests in progress), and the plugin module for hooking
arbitrary external code into the outgoing data. Stub modules were
planned for the external interfaces, which would allow end to end
testing of the framework to take place using simple endpoints (e.g.
files read in and out).

3) Finally, since this would all be much more useful with examples,
tests, and actual real world use to iron out the bugs, the third
component of the project is the Liverpool/Faulkes specific
implementation, a refactoring of the existing TEA and node agent to
decouple the functionality and allow the open embedded agent to run on
the three 2 meter robotic telescopes.


So where are we at present? And in order to get to a version that we
can release to somebody, what do we need? We made tremendous progress
during the 3 days. We jointly designed and specified the agent, and
Phil generated his magic UML diagrams to document it all. Phil worked
on the interface classes, XPath library and error handling code. Mark
and Eric H. extracted the telescope-specific TEA scoring and request
implementations, and made a substantial dent in the core document
manager. Ben, Strawberry and myself extracted the TEA telemetry*
classes and folded the functionality into the event listener
component. In between enhancing their existing agent, Steve and Chris
provided the clues. ;) Bob worked on his own RTML 3 parser for
interoperability, and helped argue use cases and specify RTML. Dan
researched interfacing to our stuff using Jython, and provided the
balance to keep the design language neutral.


In terms of functionality, I don't think we're very far away from a
version 1.0 of the core framework at all. As I see it, there are three
incremental stages required to get to a useful version 1.0.

i) A working end to end infrastructure, with the simplest dummy
telescope implementation. We should be able to read an RTML document
from a file, pump it through the system, and get something sane out.
We should have a small number of system level tests that demonstrate
this is the case for each of the documents we expect, and some that we
don't expect. The authentication, parser, delegator, document manager
and plugin components should all have basic unit tests (that pass!).

ii) We should have a simple default implementation of a network
endpoint. The obvious example is the simple socket server as
implemented by Alasdair and Bob, and is described in that important
HTN transport paper Alasdair never got around to writing. ;) I think
this is important because it means anyone with a telescope can see the
point of our framework immediately - remote users can talk to them!
Also, it's much more cool sending documents remotely over a socket
than reading from a file. ;)

iii) I, Eric Saunders, should be able to download and run the dummy
telescope implementation on my local machine without the benefit of
any specialist knowledge (pretty easy to simulate, given my almost
total Java non-experience). This implies the existence of helpful
documentation. ;) When I ping my socket server with one of the
provided example RTML documents, I watch my dummy scoring/request
stubs activate, and I get RTML out the other end, the whole thing will
appear to be magic. We'll be done, and I promise to provide you all
with beer or an equivalently joyful product!


You'll note that none of these three sub-goals require any further
work on the LT/FT specific implementation. I think that porting that
successfully is much more challenging than finishing the framework. I
also think it's less of a priority, since no one else can use it.
Similarly, plugins (e.g. VOEvent, image analysis), alternate language
implementations, and an alternate end point such as one based on
Neil's web services node agent, while nice to have, are not on the
critical path. On the other hand, several of us have specific skills
related to these areas and could make a substantial contribution
working on these particular avenues.


So here's my proposal. If you agree with my analysis, start adding
tasks to the development page (under the issues tab) that get us
closer to 1.0. If you see a task that you'd like to work on, drop an
email to the group to let us know what you're pursuing, and assign
yourself to that task. Documentation and testing are tasks too! I
suspect there are a limited total number of people who can
simultaneously work on the core framework (of the order of the number
of independent tasks) before we start treading on each other's toes.
So if you feel your time is better spent working on something not on
the critical path, such as specific porting or implementation, that's
fine too - just create a task so we know what you're up to.

I know none of us have much time to devote to this, but even sporadic
progress would be excellent. I would suggest 8hrs a month (1%!) as a
reasonable minimum developer contribution, if we're serious about
seeing this project succeed.

Finally, be warned - in a week or so I will start contacting you
individually if it seems anyone needs further 'direction'.
Mwahahahaha!

Of course, this is open source. Feel free to argue, rant, point out
anything obvious I've missed, or attempt a megalomanic but bloodless
coup to replace the project leadership and install your own puppet
ruler. However, I do reserve the right to eventually tell you to shut
up. ;)

Happy hacking

Eric