Re: hooke status/roadmap
m.s.
Thanks for your excellent mail -will answer tomorrow.
On 27 Dic, 17:52, "W. Trevor King" <tvr...@gmail.com> wrote:
> Hey all,
>
> I'm doing force spectroscopy in the states,
> http://www.physics.drexel.edu/~wking/
> and I ran across your program a few weeks ago.
>
> I'm a bit confused about the status of the project.
> Points for an active project:
> * Project published March, 2009.
> * Commits by separate authors as recently as November.
> Points against an active project:
> * Mailing list full of spam & otherwise quiet
> * None of the open High priority issues have seen recent action
> http://code.google.com/p/hooke/issues/detail?id=3
> http://code.google.com/p/hooke/issues/detail?id=10
> ...
>
> I expect the quiet is because it works for you guys, and you don't
> have time to spend tweaking, but I've been looking it over and have a
> few suggestions for future work (in order of increasing difficulty):
>
> * Directory structure
>
> Dumping everything into one directory is ugly, confusing, and
> harder to package. I suggest something along the lines of
> hooke$ tree -d .
> .
> |-- conf
> |-- doc
> |-- hooke
> | |-- driver
> | |-- plugin
> | `-- test
> `-- mfp_igor_scripts
> Distutils/setuputils might have some say in this, but I don't know
> much about how they should be handling configuration files, etc.
>
> * Plugin architecture
>
> The current plugins seem too tightly tied to their command-line
> interfaces. I've been working on Bugs Everywhere
> http://bugseverywhere.org
> on the side, and I recently reworked it's the command structure to a
> more class based one inspired by Bazaar's internal structure. I can
> send details if you want, but I think defining a plugin should look
> something like
> class Autopeak (hooke.plugin.Plugin):
> """Automatically performs a number of analyses on the peaks of the given curve.
>
> >>> doctest stuff, if desired...
> """
> name = 'my-plugin'
>
> def __init__(self, *args, **kwargs):
> hooke.plugin.Plugin__init__(self, *args, **kwargs)
> self.options.extend([
> hooke.plugin.Option(name='rebase', short_name='r',
> help='Re-asks baseline interval'),
> hooke.plugin.Option(name='persistence length', short_name='p',
> help='Use a fixed persistent length for the fit in
> 'meters. If not given, the fit will be a '
> '2-variable fit. Scientific notation like '
> '0.35e-9 is fine.',
> arg=hooke.plugin.Argument(
> type='float', metavar='FLOAT', default=None,
> completion_callback=None)),
> ...
> ])
> self.arguments.extend([...])
> self.dependencies.extend(['fit', 'flatfilts'])
>
> def _run(self, **params):
> pl = params['persistence length']
> T = params['temperature']
> if T == None:
> T = self.config['temperature']
>
> slope_span=int(self.config['auto_slope_span'])
> ...
>
> def _long_help(self):
> return """
> Currently it automatically:
> - fits peaks with WLC or FJC function (depending on how the fit_function variable is set)
> - measures peak maximum forces with a baseline
> - measures slope in proximity of peak maximum
> """
> The base hooke.plugin.Plugin class defines the methods
> run(self, options={dict of options}, args=[list of args])
> params, checks options, constructs params, calls _run()
> help(self)
> returns help string with auto-generated usage and options list.
> Each user interface sets up a means for accepting options/arguments.
> For example, the command line interface would build a
> optparse.OptionParser from command.options and command.arguments and
> run
> argv = shlex.split(arg_string)
> options,args = parser.parse_args(argv)
> sys.exit(command.run(options, args))
> A GUI interface could pop a window or add a frame with widgets for
> each available option/argument.
>
> This plugin structure takes a bit of legwork to get off the ground,
> but it decouples the algorithms from the interface and makes it very
> easy to extend things later while keeping the documentation etc. up to
> date.
>
> * Driver architecture
>
> Plugins should list driver dependencies if they depend on driver
> extensions, e.g. picoforceDriver.deflection() and
> procplotsCommands.plotmanip_correct(). Hmm, what is plotmanip_correct
> for anyway? There doesn't seem to be anyway to access it...
>
> Is this the way you see Hooke going? How would you feel about me
> making these changes and emailing in a patch set? I'm tracking the
> SVN repository in Git, so I can publish my Git repo if you'd rather do
> things that way.
>
> Thanks for open-sourcing!
> Trevor
>
> --
> This email may be signed or encrypted with GPG (http://www.gnupg.org).
> The GPG signature (if present) will be attached as 'signature.asc'.
> For more information, seehttp://en.wikipedia.org/wiki/Pretty_Good_Privacy
>
> My public key is athttp://www.physics.drexel.edu/~wking/pubkey.txt
>
> application_pgp-signature_part
> < 1KVisualizzaScarica
ms
First of all, thanks a lot for your mail. This is the kind of excellent
feedback I've hoped when publishing the little thing.
> I'm a bit confused about the status of the project.
> Points for an active project:
> * Project published March, 2009.
> * Commits by separate authors as recently as November.
> Points against an active project:
> * Mailing list full of spam & otherwise quiet
> * None of the open High priority issues have seen recent action
> http://code.google.com/p/hooke/issues/detail?id=3
> http://code.google.com/p/hooke/issues/detail?id=10
> ...
>
> I expect the quiet is because it works for you guys, and you don't
> have time to spend tweaking, but I've been looking it over and have a
> few suggestions for future work (in order of increasing difficulty):
Your guesses are 90% right. The current software version, even if
simple, works mostly for most people (most pressing issue I get asked
about is updating a couple drivers, something that I should work on
these days). Also, I am currently working no more on SMFS (doing mostly
molecular dynamics and some SM-FRET nowadays), and this for sure doesn't
help.
The mailing list is unfortunately almost unused (people continue to
write me personally) and, yes, I've seen the spam on it but I thought
stupidly that, like with Gmail, Google could take care of it
automagically. It seems it is not the case :( -I've tried to report some
spam but with little effect, it seems.
Now, to your proposals:
>
> * Directory structure
>
> Dumping everything into one directory is ugly, confusing, and
> harder to package. I suggest something along the lines of
> hooke$ tree -d .
> .
> |-- conf
> |-- doc
> |-- hooke
> | |-- driver
> | |-- plugin
> | `-- test
> `-- mfp_igor_scripts
> Distutils/setuputils might have some say in this, but I don't know
> much about how they should be handling configuration files, etc.
>
This is something that would be absolutely necessary and I agree, but
dumping everything into one directory was, er, easier. Shame on me. I've
never investigated on how to do that however.
> * Plugin architecture
>
> The current plugins seem too tightly tied to their command-line
> interfaces. I've been working on Bugs Everywhere
> http://bugseverywhere.org
> on the side, and I recently reworked it's the command structure to a
> more class based one inspired by Bazaar's internal structure. I can
> send details if you want, but I think defining a plugin should look
> something like
[...]
Sounds really, really cool, but I am unsure to have understood how it
works in detail. For example, it seems that each class corresponds to a
single command, isn't it? If you have some more examples / documentation
on what it means, I'd be interested.
> * Driver architecture
>
> Plugins should list driver dependencies if they depend on driver
> extensions, e.g. picoforceDriver.deflection() and
> procplotsCommands.plotmanip_correct(). Hmm, what is plotmanip_correct
> for anyway? There doesn't seem to be anyway to access it...
I agree with your suggestion, but I don't get your question about
plotmanip_correct. Its function is to correct the distance for the
cantilever deflection.
> Is this the way you see Hooke going? How would you feel about me
> making these changes and emailing in a patch set? I'm tracking the
> SVN repository in Git, so I can publish my Git repo if you'd rather do
> things that way.
I've never used Git (I've heard a lot of good things about it, but for
my little things I've always found SVN fine). Given the magnitude of
changes in architecture, I have two suggestions:
- First of all, there is another guy, Rolf Schmidt, who is heavily
working on Hooke but still didn't release his branch. He's mostly
interested in building a good GUI interface, and the screenshots he sent
me were very cool. He probably would appreciate your ideas and maybe
help you. You should get in touch with him, if only to coordinate: email
is: rsch...@alcor.concordia.ca
- In the meantime, probably the best thing is to upload the thing as a
separate SVN branch, so that it can be developed and tested in parallel,
before re-merging with the mainline. I would be really happy to do that.
Once done, I'd give you commit privileges, and we can contine working on it.
Let me know what do you think. Thanks a lot!
Massimo
W. Trevor King
On Mon, Dec 28, 2009 at 01:50:31PM +0000, ms wrote:
> I thought stupidly that, like with Gmail, Google could take care of it
> automagically. It seems it is not the case :( -I've tried to report some
> spam but with little effect, it seems.
I'd never made the Gmail connection somehow :p, but yeah, that is odd...
> > * Directory structure
> > ...
> This is something that would be absolutely necessary ... I've
> never investigated on how to do that however.
I'm glad you agree, since I'm pretty much done with that part ;).
The problem with distutils/setuputils for complicated projects
(e.g. more than just python libraries and scripts), is that you
probably want a real packaging system (deb, rpm, ebuilds, ...), but
it's hard to do that in a distro/OS-agnostic way. If someone out
there has ideas, let me know...
> > * Plugin architecture
> ... I am unsure to have understood how it works in detail. For
> example, it seems that each class corresponds to a single command,
> isn't it?
Yes.
> If you have some more examples / documentation on what it means, I'd
> be interested.
Take a look at bzrlib for my inspiration. Here's a command definition:
http://bazaar.launchpad.net/%7Ebzr-pqm/bzr/bzr.dev/annotate/head%3A/bzrlib/builtins.py#L236
Parsing happens here:
http://bazaar.launchpad.net/%7Ebzr-pqm/bzr/bzr.dev/annotate/head%3A/bzrlib/commands.py#L712
The option class is defined here:
http://bazaar.launchpad.net/%7Ebzr-pqm/bzr/bzr.dev/annotate/head%3A/bzrlib/option.py#L133
And the commandline UI is defined here:
http://bazaar.launchpad.net/%7Ebzr-pqm/bzr/bzr.dev/annotate/head%3A/bzrlib/commands.py#L1074
I've got a simpler version of that going in my bugs-everywhere branch,
but it's not published yet. The basic idea is that your commands take
options and arguments, but you don't want to bind them to a specific
means of reading them in (e.g. parsing a string), so the commands just
list the options and arguments nicely, with associated information.
Any UI can grab the options and arguments for the particular command
and build some interface to read them in (e.g. construct an
OptionParser instance or build an apropriate GUI window).
> > * Driver architecture
> >
> > ... Hmm, what is plotmanip_correct for anyway? There doesn't seem
> > to be anyway to access it...
> ... I don't get your question about plotmanip_correct. Its function
> is to correct the distance for the cantilever deflection.
Ah, is that the
z_protein = z_piezo - force/cantilever_spring_constant
correction? I'm having trouble wrapping my head around that
particular piece of code...
Can you call it from the hooke commandline? Nothing uses it
internally:
$ grep -r plotmanip_correct .
./procplots.py: def plotmanip_correct(self, plot, current, customvalue=None):
./CHANGELOG: plotmanip_correct() works with new picoforce.py deflection output (see)
And there's no do_plotmanip_correct() or anything...
> - First of all, there is another guy, Rolf Schmidt, who is heavily
> working on Hooke but still didn't release his branch. He's mostly
> interested in building a good GUI interface, and the screenshots he sent
> me were very cool. He probably would appreciate your ideas and maybe
> help you. You should get in touch with him, if only to coordinate: email
> is: rsch...@alcor.concordia.ca
Ah, thanks, I'll send him a message...
> - In the meantime, probably the best thing is to upload the thing as a
> separate SVN branch, so that it can be developed and tested in parallel,
> before re-merging with the mainline. I would be really happy to do that.
> Once done, I'd give you commit privileges, and we can contine working on it.
The nice thing about the newer distributed versioning systems (Git,
Bzr, Hg, ...) is that they're really easy to publish, since all of
them that I've seen can clone over HTTP. Just push the repo to your
website and you're done :p. To avoid setting up an SVN server, I'll
toss up my Git repo and some tarballs once I get things up to speed...
git-svn can push to an SVN server too, so I should be able to merge
into the trunk once things settle down.
Cheers,
Trevor
--
This email may be signed or encrypted with GPG (http://www.gnupg.org).
The GPG signature (if present) will be attached as 'signature.asc'.
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
m.s.
> > I thought stupidly that, like with Gmail, Google could take care of it
> > automagically. It seems it is not the case :( -I've tried to report some
> > spam but with little effect, it seems.
>
> I'd never made the Gmail connection somehow :p, but yeah, that is odd...
I've put full moderation now. Annoying, but better than the spam.
> > > * Directory structure
> > > ...
> > This is something that would be absolutely necessary ... I've
> > never investigated on how to do that however.
>
> I'm glad you agree, since I'm pretty much done with that part ;).
>
> The problem with distutils/setuputils for complicated projects
> (e.g. more than just python libraries and scripts), is that you
> probably want a real packaging system (deb, rpm, ebuilds, ...), but
> it's hard to do that in a distro/OS-agnostic way. If someone out
> there has ideas, let me know...
I think that distutils is the best way to go. I used distutils, so
that's not the problem -it's the way hooke imports stuff etc. that I
was never sure how to do in a filesystem-agnostic way, but I admit
I've never put too much thought into it (well, let's say almost no
thought). Probably is dead-brain easy, but never looked into it.
> > > * Plugin architecture
>[...]
> I've got a simpler version of that going in my bugs-everywhere branch,
> but it's not published yet. The basic idea is that your commands take
> options and arguments, but you don't want to bind them to a specific
> means of reading them in (e.g. parsing a string), so the commands just
> list the options and arguments nicely, with associated information.
> Any UI can grab the options and arguments for the particular command
> and build some interface to read them in (e.g. construct an
> OptionParser instance or build an apropriate GUI window).
Yes, I understand the abstraction, it's the details that I wanted to
understand. But I'll look at the links.
> Ah, is that the
> z_protein = z_piezo - force/cantilever_spring_constant
> correction? I'm having trouble wrapping my head around that
> particular piece of code...
>
> Can you call it from the hooke commandline? Nothing uses it
> internally:
> $ grep -r plotmanip_correct .
> ./procplots.py: def plotmanip_correct(self, plot, current, customvalue=None):
> ./CHANGELOG: plotmanip_correct() works with new picoforce.py deflection output (see)
> And there's no do_plotmanip_correct() or anything...
Because there's no need to call it in the code. It is a "plot
manipulator" (that's how I call them), that is a function that
transparently gets a plot object in input and returns a plot object in
output. They are called in the order defined in the hooke.conf file,
and as such the function call is implicit in the loop that calls all
of them -see do_plot().
> Ah, thanks, I'll send him a message...
Absolutely do that.
> > - In the meantime, probably the best thing is to upload the thing as a
> > separate SVN branch, so that it can be developed and tested in parallel,
> > before re-merging with the mainline. I would be really happy to do that.
> > Once done, I'd give you commit privileges, and we can contine working on it.
>
> The nice thing about the newer distributed versioning systems (Git,
> Bzr, Hg, ...) is that they're really easy to publish, since all of
> them that I've seen can clone over HTTP. Just push the repo to your
> website and you're done :p. To avoid setting up an SVN server, I'll
> toss up my Git repo and some tarballs once I get things up to speed...
> git-svn can push to an SVN server too, so I should be able to merge
> into the trunk once things settle down.
You seem to know Python / CMS much much better than me, so I can't but
trust you. But you don't need a SVN server, I mean: I can give you
access to the hooke repository (which is SVN, don't know if it
supports Git) to publish your branch as *separate* for now, and then
we can happily merge it in /trunk when it looks possible. Can you do
that / do you want that?
cheers,
m.