TinkerPop3 Support

[Stephen Mallette]

I recently contacted Michael to see if I could be of any assistance in getting ClojureWerks support for the upcoming TinkerPop3 (http://tinkerpop.com/docs/current/) release.  We just released M1 last week, which was a version mostly for vendors who planned to implement the APIs, so that we could begin to gather feedback.  The impact of TinkerPop3 to the three ClojureWerk's projects (Archimedes, Ogre and Titanium) is reasonably severe as the changes to the TinkerPop API and philosophy were fairly major. These breaking changes were not made lightly and in many ways were made out of necessity for new features and corners we couldn't get out of in the TinkerPop2 codebase. I believe that one of the largest changes involves the project's packaging.  We no longer have the notion of Blueprints (wrapped by Archimedes), Gremlin (wrapped by Ogre) and other component projects...it is now all just "Gremlin".  The spirit of each of those components is still present, but they are all more simply represented and integrated as one thing.

So I think that begs the question of how the three related ClojureWerks projects will exist in relation to this change in TinkerPop3.  It would seem that Titanium is separated from this problem, in that it will simply depend upon the choices made around Archimedes and Ogre.  Titanium would technically not be affected by such things until Titan supports TinkerPop3 - at this point the plan is to support TinkerPop3 for Titan 1.0 which will likely be available at the beginning of 2015.  From that perspective, the Titanium roadmap is pretty clear.  Less clear to me is what happens with Archimedes and Ogre.  Should it all just become Ogre? or all just Archimedes? should there be a new project all together for TinkerPop3?

I don't really know the best answer to these questions given the ClojureWerks Community, so I post this information here to inform and gather feedback for what the best approach should be.  It is worth noting that we will continue to support TinkerPop2 for the foreseeable future. We don't expect to add major new features, but we do expect some releases along the 2.5.x line.  2.5.1 will release around the time 3.0.0 releases.

I look forward to hearing feedback on this and helping where I can to build out some slick support for TP3.  

Cheers!

Stephen

[Michael Klishin]

 On 21 August 2014 at 17:51:20, Stephen Mallette (spmal...@gmail.com) wrote:
> > Titanium would technically not be affected by such things until
> Titan supports TinkerPop3 - at this point the plan is to support
> TinkerPop3 for Titan 1.0 which will likely be available at the
> beginning of 2015. From that perspective, the Titanium roadmap
> is pretty clear.

So, is Titan 0.5 targeting TinkerPop 3? We'll have to upgrade to Titan 0.5
first but if there are major API changes, I suspect we should merge the
0.5 work in to master and work on adapting to the future TinkerPop API
there.

> Less clear to me is what happens with Archimedes
> and Ogre. Should it all just become Ogre? or all just Archimedes?
> should there be a new project all together for TinkerPop3?

They can be merged into Ogre. Or Archimedes. We can toss a coin ;)
--
@michaelklishin, github.com/michaelklishin

[Michael Klishin]

On 21 August 2014 at 17:51:20, Stephen Mallette (spmal...@gmail.com) wrote:

> > I don't really know the best answer to these questions given
> the ClojureWerks Community, so I post this information here
> to inform and gather feedback for what the best approach should
> be. It is worth noting that we will continue to support TinkerPop2
> for the foreseeable future. We don't expect to add major new features,
> but we do expect some releases along the 2.5.x line. 2.5.1 will
> release around the time 3.0.0 releases.
>
> I look forward to hearing feedback on this and helping where I
> can to build out some slick support for TP3.

Right, so this convinces me that we should target TP3 very soon,
after merging Zack and Ray's work into master.

It will be somewhat painful but maintaining 2 major API revisions
will be significantly more painful. Plus, we'll be able to work
closely with the TP team and give them some feedback, which means our
needs are more likely to be addressed. 
--
@michaelklishin, github.com/michaelklishin

[Michael Klishin]

 On 21 August 2014 at 18:10:23, Stephen Mallette (spmal...@gmail.com) wrote:
> > No. Titan 0.5 will not support TinkerPop3. The next major release
> of Titan will be 1.0 and that version will support TP3.

OK then, we'll merge the current work and investigate what should happen
to Ogre/Archimedes for TP3.

Ray, Zack: any objections to this plan?
--
@michaelklishin, github.com/michaelklishin

[Ray Miller]

On 21 August 2014 15:33, Michael Klishin <michael....@gmail.com> wrote:
> On 21 August 2014 at 18:10:23, Stephen Mallette (spmal...@gmail.com) wrote:
>> > No. Titan 0.5 will not support TinkerPop3. The next major release
>> of Titan will be 1.0 and that version will support TP3.
>
> OK then, we'll merge the current work and investigate what should happen
> to Ogre/Archimedes for TP3.
>
> Ray, Zack: any objections to this plan?

No, sounds like the way to go.

Ray.

[Zack Maril]

No objections here. I haven't been following T3 closely enough to make any sensible design chooses. 

-Zack

[Andrew Fitzgerald]

Ugh, let's just pretend I didn't post that other thread...

I'm a newish Clojure developer, but I'm interested in working on the TP3 port.

Any thoughts on where to start repo/branchwise?

It seems like Ogre's name now fits better, while archimedes seems like it makes more sense as a starting point code-wise.

-Fitz

[Michael Klishin]

On 18 September 2014 at 08:27:41, Andrew Fitzgerald (andrewcf...@gmail.com) wrote:
> Any thoughts on where to start repo/branchwise?
> It seems like Ogre's name now fits better, while archimedes seems
> like it makes more sense as a starting point code-wise.

All previously ongoing work is now merged into master.

If TP3 folds multiple projects into Gremlin, we should merge ours into Ogre.
Doing that may be a good starting point. 
--
@michaelklishin, github.com/michaelklishin

[Andrew Fitzgerald]

Should ogre currently be failing 6 of the transform tests?

Just want to make sure I'm not missing something in my setup.

Does someone with access want to create a branch for tinkerpop 3 to send pr's to?

[Ray Miller]

On 18 September 2014 07:04, Andrew Fitzgerald

<andrewcf...@gmail.com> wrote:
> Should ogre currently be failing 6 of the transform tests?
> Just want to make sure I'm not missing something in my setup.

It shouldn't, but it's a known issue. I haven't had a chance to look
into this yet. If you have time to look at the failing tests a pull
request to fix them would be very welcome.

> Does someone with access want to create a branch for tinkerpop 3 to send
> pr's to?

Could do, but I'm keen to get the next release against Blueprints 2.5
out before thinking about TP3. There are only 2 obstacles to this: the
failing tests mentioned in this thread; and the index issue I raised
in the MovieLens demo thread, where we need to rule out a Titanium bug
as the cause before releasing. Thinking about it, the Titanium issue
is a bit of a red herring, as it doesn't stop us from working on
Ogre, but I'd like to see the Ogre test suite passing against
Blueprints 2.5 before branching.

Ray.

[Stephen Mallette]

Just a note that TinkerPop 2.6.0 just released yesterday.  There were no serious changes there as compared to 2.5.0...mostly minor odds and ends.  I'll take a look at the failing tests in Ogre against 2.6.0 and see what I can fix.

Stephen

[Stephen Mallette]

I wasn't fully clear on the "development process" here so I forked Ogre and made changes there.  All the tests now pass in my fork and I've issued a PR:

https://github.com/clojurewerkz/ogre/pull/17

I also bumped to TinkerPop 2.6.0.  Hope that fixes everything up!

Stephen

[Ray Miller]

On 18 September 2014 19:17, Stephen Mallette <spmal...@gmail.com> wrote:
> I wasn't fully clear on the "development process" here so I forked Ogre and
> made changes there.

That's a fine process, and a word on this list so we don't duplicate
effort. If we build up momentum on the development front it might be
an idea to make better use of the GitHub issues - certainly that's
the way I'd like to handle bug reports and enhancement requests.

> All the tests now pass in my fork and I've issued a PR:

Many thanks. Merged.

> I also bumped to TinkerPop 2.6.0. Hope that fixes everything up!

Much better, but tests still failing due to a version/dependency issue
unrelated to your change. I'll see if I can sort that out.

Ray.

[Ray Miller]

On 18 September 2014 19:33, Ray Miller <r...@1729.org.uk> wrote:
> Much better, but tests still failing due to a version/dependency issue
> unrelated to your change. I'll see if I can sort that out.

We have a clean build on Travis-CI now. Problem was that we were
pulling in a version of Potemkin that needs Clojure 1.5 or better, to
tests against 1.4 were failing. I've dropped the version of Potemkin
to 0.3.3 (the most recent that works).

[Victor Su]

Hi Ray,

I've been working with Stephen on fixing ogre today.  I believe the tests still failing problem is 

in with the called to the reduced? function here:

https://github.com/ztellman/clj-tuple/blob/master/src/clj_tuple.clj#L317

which is referenced in the potemkin dependency

https://github.com/clojurewerkz/ogre/blob/master/project.clj

I believed reduced? was added in version 1.5, so it isn't defined in 1.4.

Perhaps you could remove the 1.4 build in project.clj?

-Victor

[Stephen Mallette]

Nice, Ray!  Glad that's good to go.  Plus we have latest TinkerPop release in there now too, so that's sweet too.  I'm not sure if we want to bump back to 2.5.0 for purpose of any planned release.  There's arguably more support for TinkerPop 2.5.0 than 2.6.0 at this time....might be good to have a stable release around both 2.5.0 and 2.6.0 to keep dependencies straight for folks.  Just a suggestion. 

Thinking on Victor's point, does Ogre need support of 1.4?  I'll admit i'm not really up on where Clojure stands on such things as it relates to "what's the cool version of clojure use".  Thoughts?

Stephen

[Ray Miller]

The test suite passes with both 2.5 and 2.6, so we can roll a release
with either (or both, as the Ogre version reflects the Bluperints
version).

We've talked here before about supporting Clojure 1.4. My view was no
need to break backwards compatibility if it's easy to maintain. So
we'll probably do our next release with 1.4 support.

Moving forward, I think we should focus on 1.5 and above, and the
TinkerPop3 release would be a good time for us to break 1.4
compatibility.

@Victor - thanks for your message, I think I got there just a few
minutes before seeing your reply.

Ray.

> --
> You received this message because you are subscribed to the Google Groups
> "Clojure Titanium" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure-titani...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Stephen Mallette]

What else is required to do the pair of releases?  clean up the readme perhaps?  anything more serious?

Thanks,

Stephen

[Ray Miller]

On 22 September 2014 17:18, Stephen Mallette <spmal...@gmail.com> wrote:

What else is required to do the pair of releases?  clean up the readme perhaps?  anything more serious?

I think for Archimedes and Ogre, it's just README and documentation fixes that are needed.

The Ogre docs, in particular, need to be updated to reflect the move into the clojurewerkz.ogre namespace. Apart from that, I think just a quick run through the examples to make sure they all work. I don't think Archimedes has a good body of documentation yet,

Ray.

 

[Stephen Mallette]

Do you want me to take a run at the documentation cleanup for Ogre?

[Victor Su]

I've opened a pull request with some fixes to the ogre documentation (ogre.docs) and makes it more 

consistent with the current code.  I also have a pull request for ogre itself that adds the into-list! function 

which is missing  from the API and is now documented in ogre.docs.

-Victor

[Michael Klishin]

 On 26 September 2014 at 15:59:43, Victor Su (vsu...@gmail.com) wrote:
> I've opened a pull request with some fixes to the ogre documentation
> (ogre.docs) and makes it more
> consistent with the current code. I also have a pull request for
> ogre itself that adds the into-list! function
> which is missing from the API and is now documented in ogre.docs.

Thank you!
--
@michaelklishin, github.com/michaelklishin

[Stephen Mallette]

I've updated the README for Ogre: https://github.com/clojurewerkz/ogre

With Victor's changes and mine complete, is it now possible to consider release of Ogre 2.5.0.0 and 2.6.0.0?  Anything else that needs to be done?  

Thanks,

Stephen

[Ray Miller]

Hi Stephen,

On 26 September 2014 15:17, Stephen Mallette <spmal...@gmail.com> wrote:

I've updated the README for Ogre: https://github.com/clojurewerkz/ogre

With Victor's changes and mine complete, is it now possible to consider release of Ogre 2.5.0.0 and 2.6.0.0?  Anything else that needs to be done?  

Thank you!

That's looking pretty good to me. I don't see anything blocking the release. Michael?

We could consider an Archimedes release too, just wondering if we should change the versioning scheme for archimedes also to reflect the blueprints version.

Ray.