JAITools versioning and source repo

[Michael Bedward]

Hi Simone and all,

I'm moving this thread onto the list in case other contributors have
opinions....

For the benefit of others, there was thread some time ago (when we
moved form svn to git) about how to organise the source repo and do
releases. At the time no one had much opinion and I opted for
something basic. We are now having another go at this topic. Please
join in with your thoughts.

On 29 January 2013 06:15, Simone Giannecchini
<simone.gi...@geo-solutions.it> wrote:
> Well,
> my 0.02€. I would like to see something like:
>
> - master is open for development and new features. However to add new
> feature a small proposal page should be created. Not just go ahead and
> commit. We should ask an informal voting as well.
> Master right now should be 1.4-SNAPSHOT
> - we should have also a branch 1.3.X for bug fixing releases and we
> should allow backport of new features as long as there is a clear
> proposal and a plan with it. Of course everybody should be happy.
>
> What I don't see right now is a clear way to add new features and have
> them settle down before they become core.
>
> About how to do releases, the easier the better. I don't like things
> that are too complex. As long as changes must be documented with a
> proposal I believe we have alla the formalism that we need.
>
> Regards,
> Simone Giannecchini
> ==
> Our support, Your Success! Visit http://opensdi.geo-solutions.it for
> more information.
> ==
>
> Ing. Simone Giannecchini
> @simogeo
> Founder/Director
>
> GeoSolutions S.A.S.
> Via Poggio alle Viti 1187
> 55054 Massarosa (LU)
> Italy
> phone: +39 0584 962313
> fax: +39 0584 1660272
> mob: +39 333 8128928
>
> http://www.geo-solutions.it
> http://twitter.com/geosolutions_it
>
> -------------------------------------------------------
>
>
> On Mon, Jan 28, 2013 at 3:24 AM, Michael Bedward
> <michael...@gmail.com> wrote:
>> sorry - meant to cc this to Andrea as well...
>>
>> On 28 January 2013 13:23, Michael Bedward <michael...@gmail.com> wrote:
>>> Hi Simone,
>>>
>>> No doc about it but there's an old list thread where we were talking
>>> about it. At the time we were going to follow the GS/GT model where
>>> any new features meant a new major version number. But we've just
>>> broken backward compatibility for RangeLookup going from 1.3 to 1.3.1
>>> !
>>>
>>> I've cooled on applying GS/GT formal versioning for JAITools. It
>>> seems like a lot of trouble for such a small project and I doubt that
>>> anyone cares too much. Martin Davis has been publishing JTS 1.x
>>> versions for ever and I don't think it causes any confusion.
>>>
>>> How about in-between system ? Example: 1.4 will be a release with new
>>> features included whereas 1.4.1 would be a patch / maintenance release
>>> where we try a bit harder to keep backwards compatibility.
>>>
>>> Another thing to remember is that we're not keeping stable branches
>>> running in the git repo. The only permanent branches are master and
>>> releases. Tags identify commits that we deployed from on the releases
>>> branch.
>>>
>>> Summary: I don't have any strong opinions about how we do versioning
>>> other than I'd like it to be easy :)
>>>
>>> Michael
>>>
>>>
>>> On 28 January 2013 10:05, Simone Giannecchini
>>> <simone.gi...@geo-solutions.it> wrote:
>>>> Ciao Michael,
>>>> I am happy with the post.
>>>>
>>>> About adding new stuff, do we have a page summarising the rules for
>>>> new developments?
>>>> As an instance we have some replacements for the JAI operations like
>>>> scale and warp that do take into account a ROI. Let's say
>>>> we want to donate them, what should we do? I remember we discussed
>>>> this in the past but I cannot find any docs about it.
>>>>
>>>>
>>>> Regards,
>>>> Simone Giannecchini
>>>> ==
>>>> Our support, Your Success! Visit http://opensdi.geo-solutions.it for
>>>> more information.
>>>> ==
>>>>
>>>> Ing. Simone Giannecchini
>>>> @simogeo
>>>> Founder/Director
>>>>
>>>> GeoSolutions S.A.S.
>>>> Via Poggio alle Viti 1187
>>>> 55054 Massarosa (LU)
>>>> Italy
>>>> phone: +39 0584 962313
>>>> fax: +39 0584 1660272
>>>> mob: +39 333 8128928
>>>>
>>>> http://www.geo-solutions.it
>>>> http://twitter.com/geosolutions_it
>>>>

[Michael Bedward]

Hello all,

Simone, I'll see your 0.02€ with my AUS$0.02...

Does a project as small as this one require formal proposals and
voting ? A simpler alternative is just to be more diligent about
discussing current and proposed work on the list. While proposals can
serve as history and the basis for docs, they don't seem to be used
that way in GeoTools (for example).

Rather than formal proposals, perhaps we can just have a work-flow
with forks and pulls ? New features are developed on forks, discussed
here on the list, and then brought into the primary repo with a pull
request which would also serve as a historical record directly linked
to the sources. Pull requests were the main reason for moving from
Google to GitHub.

I take the point that we should not just commit new features to master
without discussion. I was guilty of that most recently with the new
Generate operation. Aside: I did seek opinions about it and received
no feedback, but I didn't wait for long enough. I'm happy to move the
Generate sources onto a topic branch pending discussion.

Formal voting requires allocating voting rights - a project committee
or similar. But I am the only regular contributor who doesn't work for
GeoSolutions. If we were voting on an issue that was critical for a
current GeoSolutions contract, for example, wouldn't the voting be
just going through the motions ? No criticism is meant by that, I'm
just trying to think it through with a mind to the practicalities.

Another factor to consider with formal voting is the project's history
to date. A quick look at the project stats in Ohloh shows that commits
from Simone, Andrea, Daniele and Moovida represent only about 6% of
total commits which have tended to arrive in bursts of activity with
long periods in between. Those commits were all incredibly valuable
for the project, but my point is that they have tended to be few in
number and well spaced in time. The same episodic involvement is
evident in list discussions, e.g. the previous thread on source repo
structure didn't attract much interest:

https://groups.google.com/d/topic/jai-tools/RiS72I7g9pQ/discussion

Now none of this is a big deal at the moment - people just come into
the loop when they have a need and the time to be involved given other
commitments. But that doesn't fit with a formal voting model which
assumes a quorum is always available.

Regarding the source repo structure - I chose the current option with
just a master and releases branch after finding that the stable
branches in the old svn repo were hardly used and just became another
source of work for no obvious advantage. That said, I don't think the
current structure works well. I found the last release to be too
fiddly and agree that having master sitting on 1.3-SNAPSHOT seems
confusing. So a bit more structure might actually lead to less work,
but I wouldn't be keen on a complex structure as used in large
projects.

Apologies if any of the above seems overly negative or dismissive. My
day job is bureaucratic hell at the moment and I'm grumpy.

Michael

[Simone Giannecchini]

Ciao Michael,
please, read below...

Regards,
Simone Giannecchini
==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for
more information.
==

Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

On Tue, Jan 29, 2013 at 3:07 AM, Michael Bedward
<michael...@gmail.com> wrote:
> Hello all,
>
> Simone, I'll see your 0.02€ with my AUS$0.02...
>
> Does a project as small as this one require formal proposals and
> voting ? A simpler alternative is just to be more diligent about
> discussing current and proposed work on the list. While proposals can
> serve as history and the basis for docs, they don't seem to be used
> that way in GeoTools (for example).
>

I don't agree with this view. I understand it does require some
formalism and overhead to sit down and write a proposal
but I found out that it pays out as it forces to gather your ideas and
structure them a little bit. Actually, I believe it is something you
should do
anyway regardless of the fact that this is required or not.

Long story short, I am strongly convinced that a proposal page is a
very good idea, I also agree that the formalism required to put it up
as well as that required to "vote" it should be minimal. I would
actually be happy with something like:

- proposal page
- work on a fork then create a pull request
- wait 1 week for feedback and try to incorporate it
- if there is strong negative feedback, merge after 1 week

I would not request complex voting or anything, I would just force
people (ourselves firsts) to sit down and think
while being able to move on quickly as required by circumstances
(forks are there for that reason IMHO)

> Rather than formal proposals, perhaps we can just have a work-flow
> with forks and pulls ? New features are developed on forks, discussed
> here on the list, and then brought into the primary repo with a pull
> request which would also serve as a historical record directly linked
> to the sources. Pull requests were the main reason for moving from
> Google to GitHub.
>

I love pull requests, but we can't simply wait on discussions on the
ML to have new features. The proposal page
does not replace discussions per se, but simply tries to make things
happen asynchronously which is very important
especially for company.
If I need to create something quickly, I would rather create a page
that describes what I need to do in order
to make possible for others to share theirh thoughts sooner VS later,
do my work on my fork and respect my deadlines.
Then I can decide that what I did was good and useful incorporate
feedback and prepare a pull request. Otherwise I can just throw things
away.

> I take the point that we should not just commit new features to master
> without discussion. I was guilty of that most recently with the new
> Generate operation. Aside: I did seek opinions about it and received
> no feedback, but I didn't wait for long enough. I'm happy to move the
> Generate sources onto a topic branch pending discussion.
>

Actually, I don't think you should :)
I read the emails you talked about and I did not fin anything to say
against it.
In a perfect world I should have taken some time and review the work
etc. etc. In the real world
I did not have time, hence i would prefer to have the operation in
have a small proposal page (actually a 1 small doc page would help as
well)
to check out when/if needed and move on. Aside from that, it would
have been probably better to have that work on master/trunk rather
than on a stable branch.

What happened with RangeLookup is how things should go. You did
implement something that was actually needed. I used it, I found some
bugs that I tried to fix, you improved my fixes and things progressed.
Was the operation ready and perfect the first time it was coded? No.
Is perfect now? Probably not, but things are progressing smoothly
anyway.

> Formal voting requires allocating voting rights - a project committee
> or similar. But I am the only regular contributor who doesn't work for
> GeoSolutions. If we were voting on an issue that was critical for a
> current GeoSolutions contract, for example, wouldn't the voting be
> just going through the motions ? No criticism is meant by that, I'm
> just trying to think it through with a mind to the practicalities.
>
> Another factor to consider with formal voting is the project's history
> to date. A quick look at the project stats in Ohloh shows that commits
> from Simone, Andrea, Daniele and Moovida represent only about 6% of
> total commits which have tended to arrive in bursts of activity with
> long periods in between. Those commits were all incredibly valuable
> for the project, but my point is that they have tended to be few in
> number and well spaced in time. The same episodic involvement is
> evident in list discussions, e.g. the previous thread on source repo
> structure didn't attract much interest:
>
> https://groups.google.com/d/topic/jai-tools/RiS72I7g9pQ/discussion
>
> Now none of this is a big deal at the moment - people just come into
> the loop when they have a need and the time to be involved given other
> commitments. But that doesn't fit with a formal voting model which
> assumes a quorum is always available.
>

I am not seeking a formal voting procedures, I am seeking a mean to
not have to check emails from past times
as I had to do with classified stats, due to out fault, but again, that's life.
Pushing things to JAI tools rather than keeping them hidden inside
other projects adds overehead to our work. We do it anyway because we
believe it is the best things to do in term of code reuse and
separation of concerns but we canno devote too many resources to it.
Therefore I believe that the best thing to do to avoid wha happened
with classified stats is to have a page together with the pull request
and the disussion.ì on the ML. If I don't have time to incorporate
feedback at least I don't have to go back and think hard to remember
why I did something.

That said, I don't see how this would add overhead or require formal
voting with a PSC and anything. Moreover it is not inteded to replace
eventual discussions on the ml.

> Regarding the source repo structure - I chose the current option with
> just a master and releases branch after finding that the stable
> branches in the old svn repo were hardly used and just became another
> source of work for no obvious advantage. That said, I don't think the
> current structure works well. I found the last release to be too
> fiddly and agree that having master sitting on 1.3-SNAPSHOT seems
> confusing. So a bit more structure might actually lead to less work,
> but I wouldn't be keen on a complex structure as used in large
> projects.
>

I don't think we need complexity. I would suggest what follows:
- master open for development (1.4-SNAPSHOT)
- stable for bugfixing (1.3.x now)

As soon as the curernt master become stable the old stable gets
abandoned. Having stable out does not mean doing any regular releases
or anything.
Right now we have 1.3.1, right? Ok, we will do 1.3.2 only if we have
some bugs that bother us. Otherwise next one would/could be directly
1.4.0 and so on.

My goal is to have a mean to implement new stuff (which also means
make errors, refactor etc..withouth intefering with stable stuff).

> Apologies if any of the above seems overly negative or dismissive. My
> day job is bureaucratic hell at the moment and I'm grumpy.
>

No worries, open discussions usually lead to better decisions.

> --
> You received this message because you are subscribed to the Google Groups "jai-tools" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jai-tools+...@googlegroups.com.
> To post to this group, send email to jai-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/jai-tools?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Michael Bedward]

Hi Simone,

No one else with any opinions on this it seems... Just your organised
mind vs my lazy anarchism :)

> Long story short, I am strongly convinced that a proposal page is a
> very good idea, I also agree that the formalism required to put it up
> as well as that required to "vote" it should be minimal. I would
> actually be happy with something like:
>
> - proposal page
> - work on a fork then create a pull request
> - wait 1 week for feedback and try to incorporate it

> - if there is [no] strong negative feedback, merge after 1 week

OK, so it's effectively veto rather than vote. I'm happy with that.

No doubt it's true that writing out requirements and proposed design
for each new feature is valuable.

> If I need to create something quickly, I would rather create a page
> that describes what I need to do in order
> to make possible for others to share theirh thoughts sooner VS later,
> do my work on my fork and respect my deadlines.
> Then I can decide that what I did was good and useful incorporate
> feedback and prepare a pull request. Otherwise I can just throw things
> away.

Cool - that works for me.

We could either have the proposal pages on GitHub (e.g. as rst files
which can be edited online or offline) or set up a wiki or similar at
jaitools.org. Any preference ?

Have you got a template in mind for proposal pages ?

> I don't think we need complexity. I would suggest what follows:
> - master open for development (1.4-SNAPSHOT)
> - stable for bugfixing (1.3.x now)
>
> As soon as the curernt master become stable the old stable gets
> abandoned. Having stable out does not mean doing any regular releases
> or anything.

Yep - sounds like minimal change == minimal work == good. But I just
want to check on the details...

For master, no change other than the poms being bumped up to 1.4-SNAPSHOT

For stable, we can either create a new branch each time we move from
1.n to 1.(n+1), or we could have a permanent stable branch effectively
taking the place of the current releases branch. I think I prefer the
latter option because it means we don't get an ever increasing list of
branches, most of which are abandoned. And since we're not talking
about doing "hot-fixes" or maintaining long-ago versions it should be
workable with just master and stable.

Thinking this through...

- Current releases branch is turned into "stable" and poms set to
1.3-SNAPSHOT. It already has tags for 1.3.0 and 1.3.1.

- We do some bug fixes and improvements, commit them to stable (as
well as master if appropriate), then do a release by:
1. set version to 1.3.2
2. deploy artifacts to staging repo
3. apply 1.3.2 tag to stable
4. set version back to 1.3-SNAPSHOT

- When we are ready to release 1.4.0 we do:
1. merge master into stable
2. set version to 1.4.0
3. deploy
4. apply 1.4.0 tag to stable
5. set version to 1.4-SNAPSHOT on stable
6. set version to 1.5-SNAPSHOT on master

I think that makes sense. Still only two permanent branches but much
less stuffing around and clearer division of stable vs development
than at present. Tags make it easy to go back to sources for any
released version.

Michael

[Simone Giannecchini]

On Wed, Jan 30, 2013 at 1:59 AM, Michael Bedward
<michael...@gmail.com> wrote:
> Hi Simone,
>
> No one else with any opinions on this it seems... Just your organised
> mind vs my lazy anarchism :)
>

I'll take that as a compliment :)

It is actually the first time someones uses the words "organized" and
"simone" in the same sentence without a negation.

>> Long story short, I am strongly convinced that a proposal page is a
>> very good idea, I also agree that the formalism required to put it up
>> as well as that required to "vote" it should be minimal. I would
>> actually be happy with something like:
>>
>> - proposal page
>> - work on a fork then create a pull request
>> - wait 1 week for feedback and try to incorporate it
>> - if there is [no] strong negative feedback, merge after 1 week
>
> OK, so it's effectively veto rather than vote. I'm happy with that.

cool. Convention over configuration.

>
> No doubt it's true that writing out requirements and proposed design
> for each new feature is valuable.
>
>> If I need to create something quickly, I would rather create a page
>> that describes what I need to do in order
>> to make possible for others to share theirh thoughts sooner VS later,
>> do my work on my fork and respect my deadlines.
>> Then I can decide that what I did was good and useful incorporate
>> feedback and prepare a pull request. Otherwise I can just throw things
>> away.
>
> Cool - that works for me.
>
> We could either have the proposal pages on GitHub (e.g. as rst files
> which can be edited online or offline) or set up a wiki or similar at
> jaitools.org. Any preference ?

I would go with github. As usual, the simpler, the better.

>
> Have you got a template in mind for proposal pages ?

Not really, but I can cook something simple

>
>> I don't think we need complexity. I would suggest what follows:
>> - master open for development (1.4-SNAPSHOT)
>> - stable for bugfixing (1.3.x now)
>>
>> As soon as the curernt master become stable the old stable gets
>> abandoned. Having stable out does not mean doing any regular releases
>> or anything.
>
> Yep - sounds like minimal change == minimal work == good. But I just
> want to check on the details...
>
> For master, no change other than the poms being bumped up to 1.4-SNAPSHOT
>

correct

> For stable, we can either create a new branch each time we move from
> 1.n to 1.(n+1), or we could have a permanent stable branch effectively
> taking the place of the current releases branch. I think I prefer the
> latter option because it means we don't get an ever increasing list of
> branches, most of which are abandoned. And since we're not talking
> about doing "hot-fixes" or maintaining long-ago versions it should be
> workable with just master and stable.
>

I agree with this approach. There is plenty of time in the future to
make things more complex.

> Thinking this through...
>
> - Current releases branch is turned into "stable" and poms set to
> 1.3-SNAPSHOT. It already has tags for 1.3.0 and 1.3.1.
>

k

> - We do some bug fixes and improvements, commit them to stable (as
> well as master if appropriate), then do a release by:
> 1. set version to 1.3.2
> 2. deploy artifacts to staging repo
> 3. apply 1.3.2 tag to stable
> 4. set version back to 1.3-SNAPSHOT

k

>
> - When we are ready to release 1.4.0 we do:
> 1. merge master into stable
> 2. set version to 1.4.0
> 3. deploy
> 4. apply 1.4.0 tag to stable
> 5. set version to 1.4-SNAPSHOT on stable
> 6. set version to 1.5-SNAPSHOT on master
>

k

> I think that makes sense. Still only two permanent branches but much
> less stuffing around and clearer division of stable vs development
> than at present. Tags make it easy to go back to sources for any
> released version.

that's the point. Minimized overhead, maximized flexibility.


>
> Michael