Introduction

[sdmitry]

Hi!

I'm Dima, I'm working on mostly the backend stuff in beanstalkapp.com. We are a version control service provider and right now we support Git, Subversion and Mercurial repos. 

We've been working on rewriting some parts of our performance critical code in Clojure, in order to get access to JVM based libraries like JGit and SVNKit. We previously used Ruby bindings for VCS access, which are much slower and have their issues. I'm moving some stuff to Clojure not only for pure performance reasons, but also because I find Clojure to be a very promising language.

Right now the patch I posted to clj-jgit is an extraction of a read-only caching daemon that we use for caching incoming Git pushes to the database for faster access from UI. At this point our usage of clj-jgit is only for read-only operations that fetch various info from the repo. But we're probably going to need more stuff as the time goes by. Git is a big part of Beanstalk right now and it gets bigger everyday so we're definitely going to have increasing need for good Git bindings for Clojure. 

So this is for Beanstalk part. However I myself would be interested in implementing a good Git bindings for Clojure even for scenarios that we don't yet need in Beanstalk. 

Can we work together on a plan of what is missing from the clj-jgit right now and figure out some priorities? This can turn into a nice hobby project to hack on in the free time. Do you have an internal TODO of some sort?

Thanks!

Dima,

http://twitter.com/railsmonk

[semperos]

Dima,

Thanks again for contributing the work you've done on the read-only API.

To be honest, I've spent most of my free-time Clojure hacking on clj-webdriver, a wrapper for Selenium-WebDriver that I maintain almost exclusively. In contrast, clj-jgit hasn't received any planning, design, or a proper TODO list; I got as far as I needed for a hobby project several months back and stopped before attempting any kind of API or test coverage.

That said, I am interested in continuing to work on it, especially as my work on clj-webdriver slows down.

When it comes to wrapping a Java library in Clojure, I have a few priorities:

 * Provide low-level bindings that someone familiar with the original Java API would recognize

 * Hide The Java Object in plain sight, but hide it nonetheless

 * Make it functional (as in FP)

I think the first point is pretty clear. Except where it makes the Clojure code completely non-idiomatic, I like to have a low-level API with function names that are reminiscent of their Java counterparts and behave similarly.

The second point is a conclusion I came to in my work on clj-webdriver. I began by implementing an API that exposed the Java objects (basically just wrapping Java method calls with Clojure function calls). Over time, I replaced the important Java objects in the Selenium-WebDriver API with Clojure records and re-implemented the API using Clojure protocols.

I justified doing all this re-work by finding new features to enhance the API that I couldn't get without using Clojure data structures. Specifically, I've been able to add custom caching mechanisms to the API by having a Clojure record to work with.

However, the "in plain sight" aspect is that the Clojure records still hold their Java object counterpart as a field with consistent naming, so folks who need to drop down to the Java API can do so easily without sacrificing compasibility. This is especially important as the clj-jgit API remains unfinished.

The final point is part of the clear win with using Clojure; making it Functional. Hiding mutable Java objects in Clojure records is just a tiny step in making the API "feel" functional. Other opportunities include replacing Java "design patterns" with Clojure higher-order functions and making the API easy to REPL your way through.

These are my high-level thoughts. Now that there's someone (you) with a clear purpose and deadlines driving the development of the API, I'm going to take a step back and write up some design thoughts. I welcome you to do the same, and have shared a Google doc with you so that we can collaboratively edit a design document. For any length discussion, we can use this Google Group. If there's anyone else on your team that you'd like working on this, feel free to add them as collaborators to that document.

If the momentum on this keeps up this week, I'll go ahead and port the clj-jgit repository over to the clj-jgit organization I created and add you and your team as contributors on the repo.

-Daniel

[Dima Sabanin]

Hi Daniel,

I agree with your points on writing the wrapper for Java libraries in Clojure.

The work I did to this moment was more driven by a necessity, but I
think it would be worthwhile to build some kind of plan of covering
the basic and not-so-basic scenarios of using JGit API and work on
making sure that using clj-jgit for those scenarios results in a clean
and idiomatic Clojure code.

My concerns about having one-to-one match for JGit API are:

* Quite often JGit API has lots of tiny details that 99% of people are
not going to care about and if they do they can fall back to using
Java interop
* We have to be really careful about exposing mutable objects so
probably some tiny levels of API have to be combined in a more generic
calls encapsulating the mutable state

In general, I would love clj-jgit to be a simple API that would be
straightforward and easy to use, without exposing the unnecessary
internal implementation details onto developer. Something that people
can use to hack on a simple hobby project, but also something that
performs well in a high load production environment.

I'm going to write a small blog post about this on Beanstalk blog,
with a small story how we extracted code from our caching daemon into
clj-jgit and subversion-clj projects, so I was thinking that it would
be great if you could give me a final github link to the repo in the
organization already.

Thanks and keep up the great work!

> --
> You received this message because you are subscribed to the Google Groups
> "clj-jgit" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/clj-jgit/-/2D14lcb0rCAJ.
> To post to this group, send email to clj-...@googlegroups.com.
> To unsubscribe from this group, send email to
> clj-jgit+u...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/clj-jgit?hl=en.



--
Best regards,
Dima Sabanin
http://twitter.com/railsmonk

[semperos]

Dima,

I completely agree, regarding a hard one-to-one with the JGit API. I'm looking more for a reasonable parity with the API for our lowest-level code, which we will "hide" with a higher-level API for the 98% of cases. As part of the design, it's probably best to start with our high-level interfaces and make filling out the lower-level functions an ongoing task for JGit API coverage.

I began with the JGit Porcelain API for this reason, so as to cover the basic scenarios of day-to-day Git usage first. Figuring out how we want to express that API in Clojure and what, if any, lower-level parts we'd want to expose at the "porcelain" level might be a good place to start with our design discussions.

Thanks in advance for writing up your work on this in a blog post.

-Daniel