Development stalled?

[Eli Stevens (Gmail)]

Hello,

My company is using couchdb-python internally, and we have some
improvements we'd like to see to the project. Since the last official
release was 2 years ago, and it doesn't look like there has been a
code checkin since 2011, it seems like a safe assumption that
development on the project is pretty well stalled.

If that's not the case, can someone please speak up?

Assuming that it is, I'm wondering what the original maintainer and/or
community's reaction would be to my company forking the project,
putting it on github (that's what we use internally), and cleaning it
up enough to get a v0.9 released. We're not yet committed to this
course of action, we're just wanting to know if someone would be
bothered if we did (obv. the BSD license lets us, but we'd rather not
ruffle feathers).

We see that there are a small handful of forks on github already; are
any of the owners of those listening? What are your intentions?

Thanks,
Eli

[Alexander Shorin]

Hi Eli,

I'm actually in your situation: internally I've to use heavy
improvement couchdb-python package with patches from original issues,
other forks and those that I'd not publish yet -test driving first,
but seems I'd forgot about some of them.

But instead of creating new fork and continue with my own I'd like to
help original one. A lot of forks would only create a lot of mess for
devs.

So I'd like to join you in question "What's the status? What's next?".

Matt, Dirkjan, there are some devs that wanted to continue work with
couchdb-python. They don't like couchdbkit for some their own reasons.
They do want to contribute features, fixes, docs and more. We're even
have some motivation regardless of the fact that project getting
stalled. What we have to do to give it second breath?

--
,,,^..^,,,

> --
> You received this message because you are subscribed to the Google Groups "CouchDB-Python" group.
> To post to this group, send email to couchdb...@googlegroups.com.
> To unsubscribe from this group, send email to couchdb-pytho...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/couchdb-python?hl=en.
>

[Thomas Hommers]

Hi Eli, Alex, ... and the others who are listening,

for us it's the same situation.
I would rather prefer to work on the original together and create an active community around this then splitter it into a lot forks and everybody is working on it alone.

Github would be a good idea, we are using it too.

Who else is listening?

Thomas

[Dirkjan Ochtman]

Hi,

Thanks for writing in.

On Fri, Sep 7, 2012 at 7:34 PM, Eli Stevens (Gmail)
<wicke...@gmail.com> wrote:
> If that's not the case, can someone please speak up?

I don't speak for Matt, but I'm still using the project at work and
for my hobby projects, at least. It's true that I haven't been very
active, though I still have an effort lying around to pick apart some
of Alexander Shorin's patches to get a better query server going.

The ideal scenario, at least to me, is that forking isn't necessary
and we start moving the project again. I'd be happy to review some
patches, but I'm not happy to just be a merge monkey and take whatever
patches are offered -- I'm a bit critical as a reviewer. That said,
and to turn this around a little bit, I don't see any patches from Eli
Stevens or Thomas Hommers in either the issue tracker or the mailing
list (though the mailing list search seems a bit broken and the issue
tracker mostly just has GMail user names). If you actually submit your
patches, I'd be happy to take a look!

Here's some of what I'm looking for in a patch:

- Optimize for reviewing: small patches, one bit of fix/feature at a
time, refactor in a separate patch before fixing (if you're moving
code around or changing whitespace, do that in a separate patch).
- Good changeset messages: at most 80 characters on the first line,
although extended comments are encouraged after that.
- Comes with tests: preferably in the same patch as the fix for the
issue you're trying to fix. Of course, the test suite should run
cleanly after each patch.
- If you're adding a new feature, adding some documentation would be
much appreciated.

I think optimizing for reviewing is not just optimization for my time
now (though obviously that helps me) but also helps a lot along the
line when you're chasing through VCS history to find the why of some
change. I don't want to pick on Alexander Shorin here, but this is why
the new query server hasn't gone in, for example. If it was offered as
(a) some patches with tests for stuff we currently do right already,
(b) some patches with new features and their tests and (c) a patch to
refactor/change the design of the view server, each of those could be
reviewed and debated separately and forward progress became much
easier. As it is, I have to go through a very large patch and try to
imagine most of the issues you're trying to discuss (some of which,
such as performance, aren't easily revealed by source code changes
only).

If you think that's draconian, let me know. I know it can be a bit of
a burden, but I do believe it leads to better software in the end, and
that's important to me.

Cheers,

Dirkjan

[Alexander Shorin]

Hi Dirkjan!

> Here's some of what I'm looking for in a patch:
>
> - Optimize for reviewing: small patches, one bit of fix/feature at a
> time, refactor in a separate patch before fixing (if you're moving
> code around or changing whitespace, do that in a separate patch).
> - Good changeset messages: at most 80 characters on the first line,
> although extended comments are encouraged after that.
> - Comes with tests: preferably in the same patch as the fix for the
> issue you're trying to fix. Of course, the test suite should run
> cleanly after each patch.
> - If you're adding a new feature, adding some documentation would be
> much appreciated.

Very reasonable requirements. They needs to be added somewhere as an
instruction for contributors. I know they looks simple, intuitive and
like decencies, but it's easy to miss something, you know.

About query server ...

...I totally understand your position and didn't feel hurt about, but
it just grew from view.py patch to new subpackage. That's because I'd
wanted to make it more readable (tracking down nested functions with
overlapping namespace is hell for reviewer and developer), more easier
for testing (first tests was against QS interface like ruby one from
CouchDB repository, new are more functional not integral), more easier
for debugging python couchapps and so on.

In my couchdb-python clone this is about 150 commits that adds things,
fixes things, improves things and produces iterative evolution of
query server: one thing is to just to port JavaScript query server and
quite another one to use it in production - a lot of bugs was fixed at
the last stage.

I'm opened for you to any questions kind of "What's this? Why you did
that in this way? WTF this? WTF that?!" - just ask. Have I attach them
all to #143 issue or do I need to regroup them somehow? Any
suggestions would be nice - may be you had already told them, sorry
for forcing to repeat yourself.

--
,,,^..^,,,

[Eli Stevens (Gmail)]

On Sat, Sep 8, 2012 at 12:24 AM, Dirkjan Ochtman <dir...@ochtman.nl> wrote:
> The ideal scenario, at least to me, is that forking isn't necessary
> and we start moving the project again. I'd be happy to review some
> patches, but I'm not happy to just be a merge monkey and take whatever
> patches are offered -- I'm a bit critical as a reviewer. That said,
> and to turn this around a little bit, I don't see any patches from Eli
> Stevens or Thomas Hommers in either the issue tracker or the mailing
> list (though the mailing list search seems a bit broken and the issue
> tracker mostly just has GMail user names). If you actually submit your
> patches, I'd be happy to take a look!

While it's not a patch, the last feature request that I made ended on
a "not sure if it were to get accepted if I were to code it" note:

http://code.google.com/p/couchdb-python/issues/detail?id=205

That was around the same time that the project went silent. We
haven't run into much (beyond _replicator DB support) that we've
*needed* since then (and any features that we only would have liked
haven't been raised as issues, since the project's been so quiet).

I brought this whole thing up originally because we've found what
seems to be a bug where we're getting the wrong document back from the
DB when under high load. Basically, mydb['A'] returns a doc
{"_id":"B", ...} (we just found it last week, and haven't been able to
reproduce it in any sort of limited test case; only when running our
full stack while hammering couchdb with apache bench). While we don't
think it's happening for customers, it's a serious issue for us, since
when it does trigger, it causes errors in our product. We want to
make sure that however we go about solving the bug, the end result is
a stable, usable release of couchdb-python (and I should be clear,
we're not 100% certain it's in couchdb-python at all), and that we can
get that release within a reasonable timeframe (ie. weeks not months
after a working patch is provided).

I certainly don't have any inherent desire to fork the project, so the
original maintainers becoming active again sounds great.

> If you think that's draconian, let me know. I know it can be a bit of
> a burden, but I do believe it leads to better software in the end, and
> that's important to me.

I don't share the "rewrite history to make it pretty is best" opinion,
but it's not something I'd care to argue about. I do think that
github's pull requests make most of the review concerns a non-issue,
given how it shows the full diff vs. master/trunk/whatever, which
makes "one clear patch" equate "one clear pull request." The commit
history remains unchanged, though.

Github vs. bitbucket is similar; we chose github here at work because
we felt like it was the best choice for what we needed. I haven't had
a compelling reason to use bitbucket thus far, but I'm sure it would
work too. I'll add this note to the github thread, so it can get
discussed there.

Thanks,
Eli

[Dirkjan Ochtman]

On Sat, Sep 8, 2012 at 11:21 AM, Alexander Shorin <kxe...@gmail.com> wrote:
> In my couchdb-python clone this is about 150 commits that adds things,
> fixes things, improves things and produces iterative evolution of
> query server: one thing is to just to port JavaScript query server and
> quite another one to use it in production - a lot of bugs was fixed at
> the last stage.
>
> I'm opened for you to any questions kind of "What's this? Why you did
> that in this way? WTF this? WTF that?!" - just ask. Have I attach them
> all to #143 issue or do I need to regroup them somehow? Any
> suggestions would be nice - may be you had already told them, sorry
> for forcing to repeat yourself.

Thanks for understanding! As I outlined in my last email, something
like this would be perfect:

(a) some patches with tests for stuff we currently do right already

(one test or a small number of tests at a time)

(b) some patches with new features and their tests

(c) a patch to refactor/change the design of the view server

I.e. I'd like to get small changes almost like in your 150-commit
branch, but with just the material fixes (i.e. not your intermittent
progress on understanding the problem and architecture redesign).
AFAICT you've mimicked the design of the JavaScript query server, but
that change should be entirely separate from all the other bug fixes
and testing, so we can review and discuss it separately on its own
merits.

Cheers,

Dirkjan

[Alexander Shorin]

a) Ok, I've done porting ruby queryserver test script for testing it
through stdio interface, stripping some functionality like show/list
functions wouldn't be a problem. Checked up.

b) Possible, but useless since in current design it would be very hard
to test all things in way they should to be. Current view.py design as
my first versions of query server are well for integration tests (via
stdio), but hell for normal unittests - that's why I've split QS into
package and removed all cross function dependencies. For example, to
test `require` function I'll have to walk though all steps started
from sending ddoc as json string to query server, calling specific
ddoc function and still I'll test not internal behavior, but a
reaction on input data - it would be hard to split failure by reason
of recursive import deadlock from just a typo in ddoc function code.
All tests in actual version doesn't uses any IO interface, just pure
python function calls.
If this needed only for intermediate stage and it wouldn't require
stability at tricky edge cases, than ok.

c) Ok, this is not a problem.

Really, I also don't want to push all these 150 changesets into main
repo since there was a two or three complete things refactoring and
lot of stupid typo fixes. I've got a plan, will try to provide
meaningful patchset for this issue. Thanks for suggestions!

--
,,,^..^,,,

[Dirkjan Ochtman]

On Thu, Sep 20, 2012 at 7:17 PM, Alexander Shorin <kxe...@gmail.com> wrote:
> Really, I also don't want to push all these 150 changesets into main
> repo since there was a two or three complete things refactoring and
> lot of stupid typo fixes. I've got a plan, will try to provide
> meaningful patchset for this issue. Thanks for suggestions!

Thanks, sounds good! Looking forward to come out of this.

Would you agree we should probably not hold up 0.9 over this?

Cheers,

Dirkjan

[Alexander Shorin]

On Fri, Sep 21, 2012 at 12:32 AM, Dirkjan Ochtman <dir...@ochtman.nl> wrote:
> Would you agree we should probably not hold up 0.9 over this?

Sure, no problems. Let's keep something for next release(;

--
,,,^..^,,,