missing/changed/uncompatible types in newer versions

[vaske maskinsen]

Hi Jmotif guys,

I recently tested the lumberyard-code (great work, j.p.!), because I
want to do some similar stuff. There was a precompiled jmotif jar
included and everything went fine. Now I tried exchanging the jar by
an up-to-date version (0.92) and found that I have to change most of
the imports (the locations in the package changed). However I cannot
find some of the Types, e.g.
edu.hawaii.jmotif.datatype.isax.index.AbstractNode. I
guess ...sax.trie.TrieNodeType is the replacement but I'm not sure and
I have to guess for some other types. I didn't find a changelog that
could direct me to name changes. If you could provide information
about the dependency between the old and new names (and locations), it
would be awesome.

Perhaps I can contribute to the project in the future if I stick to
using jmotif. Actually I am working on my phd-thesis on selforganizing
methods for data-intensive computing with application in energy data
(time-series) analysis.

Thanks and greets,
vaske

[Josh Patterson]

Vaske,
Nice to hear you liked Lumberyard and Jmotif --- a few thoughts:

- Lumberyard is highly experimental and some of the code could be a
lot better, particular around how I used the HBase client API (needs a
lot of improvement

- indexing a lot of timeseries data from mapreduce was a feature I
never got around to, but I found an example of someone doing b-tree
creation in MR so thats basically the same thing. just need to do it.
eventually. =D

- the iSAX code for doing a query has a bug in it that i havent
committed yet (I fell asleep). I noticed this last year and fixed it
in the Lumberyard code. Basically, when the search code gets to the
leaf node bucket, it was only looking for an exact match and not
finding the "best match after the exact match". I need to update that.

I dont get a ton of time to work on this project anymore, but if you
give me some ideas on how I can help, I'll dust off my compiler and
help out.

JP

--
Twitter: @jpatanooga
Solution Architect @ Cloudera
hadoop: http://www.cloudera.com

[Pavel Senin]

Hi Vaske:

Sorry to hear about troubles with code. It is my fault, hope you will
understand issues behind this.
Recently there were couple of very keen users of the library and
they both hit the same problem with
discords finding. The problem was in the core algorithm
implementation, and I had to dig through the whole thing. While I managed to
figure out what is going on and temporary fixed the issue, I
refactored bunch of code and API that was here.
As far as I understand, only discords were plagued by this bug. Which
means you can still use old version of the lib if you are not
discords-mining fellow.

It worth to mention, that while things work fine in the updated
version of the lib, there are performance issues and I will work on
the code in oncoming months (I plan). Which also could result in
changes too.

It is totally cool and awesome that you use JMotif in your work
on the thesis. We all will be really happy to share our experiences,
hear yours and if you like to contribute to the code - you always
welcome - just let me know if you want to get a committer account.

--
Mahalo, Pavel.

[Pavel Senin]

Oops. wait a second guys:

Do you mean that iSAX library doesn't work anymore? Whole thing is broken?

--
Mahalo, Pavel.

[Josh Patterson]

I did break a portion of it yes. I updated the bug in lumberyard and regrettably haven't fixed the isax search code. Apologies. Will look at it ASAP when I get home from this trip.

[vaske maskinsen]

Hi everyone,

thanks for your replies.
I will be busy writing conference papers until the end of the month, then I will see how I can go on with the code.
Thank you very much Josh for your willingness to help me with the lumberyard sources.

Greets,
Vaske.

[Pavel Senin]

Cool. By then I will be able to help you to refactor those things too, merging current JMotif with iSAX extension.

I fixed discords issues, now that part rocks too.

@Josh: I have checked, the iSAX branch in SVN has all old JMotif classes, so thing is not broken, only need to be merged with current trunk of JMotif.

Thank you guys!

--
Mahalo, Pavel.

[Josh Patterson]

The bug was related to the fact that the iSAX code was only doing
search for "exact search" per "bucket" and not doing a NN search in
that bucket, which is expected behavior per the iSAX paper. I updated
Lumberyard to reflect this for a talk and forgot to update iSAX
(Lumberyard reimpl's the node storage / search / insert stuff):

https://github.com/jpatanooga/Lumberyard/blob/master/src/tv/floe/lumberyard/hbase/isax/index/TerminalNodePersisted.java

method: ApproxSearch() -> SearchLocalInstnacesForClosestMatch( ... )

I need to update the iSAX stuff to reflect that.

I totally dropped the ball on this forgetting to update that. Its on
my list this week to work on. I need to dig into the branched JMotif
stuff to figure out how you want it laid out.


JP