state of the git repo, master/development considered very awkward
80 views
Skip to first unread message
Greg Troxel
Feb 29, 2020, 3:57:43 PM2/29/20
to weewx-de...@googlegroups.com
I have been following the messages about 4.0 alpha/beta stuff with
interest, expecting to get around to testing some day.
I just did a git remote update, and was surprised to find that master,
which had previously been the stable branch, now contains 4.0beta
(loosely speaking). I must have missed the announcement of the big
merge from develop, which I consider to be of huge import. I spent a
bit of time digging through to find out if there were any changes to 3.8
since my last update, and concluded no.
I realize opinions differ, but I think the master/develop scheme is
awkward and difficult to deal with. Yes, I realize that new unstable
stuff goes on develop, and at some point develop is merged to master.
But, I'm trying to be on 3.8, and get bug fixes, and not jump to 4 yet,
until I can do a build/test and try the simulator. Part of this is that
I'm running NetBSD rather than Linux, which probably nobody has tested
on, and probably it is fine, but I am basically doing the
staging/qualify/upgrade normal IT change management thing, which I see
as well within normal for anyone who is trying hard to avoid trouble.
With the master/develop scheme, there are multiple serious problems:
I was trying to follow 3.8 stable, but if I "git pull", I jump
branches (even though we haven't had a release as far as I can tell).
If I want to follow 3.8 stable whether or not 4.0 is released, because
I want to choose when to jump branches, that's not just not automatic,
it's pretty difficult.
There is no mechanism to apply a bugfix to 3.8, as of the instant that
develop was merged to master. (Yes, I get it that a branch could be
created retrospectively. But the master/develop scheme militates
against that happening.)
In contrast, the stable/master scheme is much easier to deal with:
master has new code on it (exactly like develop is today)
each stable branch has a branch, e.g. "stable-3.8", and fix commits
are put on it. They are merged to master usually, just like master
commits are merged to develop usually (or cherry-picked from there).
when it's time to start a new stable for 4.0, a stable-4.0 is branched
off master.
people who want to switch from 3.8 to 4.0 do "git checkout
stable-4.0". People who aren't ready yet don't. There is no magic
and it is really obvious what one is on when typing 'git status'.
the downside is that people that pay zero attention and just git pull
remain on the stable branch they were on. (I actually think that's a
good thing.)
The stable-3.8 branch remains indefinitely as one that could be
updated with a fix, and can easily be accessed by name.
So this is a long way of saying that today the master/develop felt
extremely awkward, for no real overall benefit, and I'd like to suggest
changing to stable/master.
Now, I realize that a lot of people are using git to get code, when
those people do not necessarily understand git. I wonder if the
master/develop scheme's main point is that a naive checkout gets one the
most recent stable. So maybe getting that benefit without all the pain
of master/develop would be:
develop: new code codes here
stable-N: branched off develop for new branches, and each one gets bug
fixes
master: the highest stable-N is merged to master, so that master is
always the most recent version of the most recent stable. The only
point of master, and the only use, is people that do checkouts and
don't understand git or weewx well enough to choose a stable branch
explicitly. (IMHO, these people should be using distribution
packages, not git.)
Thanks for listening to me rant,
Greg
interest, expecting to get around to testing some day.
I just did a git remote update, and was surprised to find that master,
which had previously been the stable branch, now contains 4.0beta
(loosely speaking). I must have missed the announcement of the big
merge from develop, which I consider to be of huge import. I spent a
bit of time digging through to find out if there were any changes to 3.8
since my last update, and concluded no.
I realize opinions differ, but I think the master/develop scheme is
awkward and difficult to deal with. Yes, I realize that new unstable
stuff goes on develop, and at some point develop is merged to master.
But, I'm trying to be on 3.8, and get bug fixes, and not jump to 4 yet,
until I can do a build/test and try the simulator. Part of this is that
I'm running NetBSD rather than Linux, which probably nobody has tested
on, and probably it is fine, but I am basically doing the
staging/qualify/upgrade normal IT change management thing, which I see
as well within normal for anyone who is trying hard to avoid trouble.
With the master/develop scheme, there are multiple serious problems:
I was trying to follow 3.8 stable, but if I "git pull", I jump
branches (even though we haven't had a release as far as I can tell).
If I want to follow 3.8 stable whether or not 4.0 is released, because
I want to choose when to jump branches, that's not just not automatic,
it's pretty difficult.
There is no mechanism to apply a bugfix to 3.8, as of the instant that
develop was merged to master. (Yes, I get it that a branch could be
created retrospectively. But the master/develop scheme militates
against that happening.)
In contrast, the stable/master scheme is much easier to deal with:
master has new code on it (exactly like develop is today)
each stable branch has a branch, e.g. "stable-3.8", and fix commits
are put on it. They are merged to master usually, just like master
commits are merged to develop usually (or cherry-picked from there).
when it's time to start a new stable for 4.0, a stable-4.0 is branched
off master.
people who want to switch from 3.8 to 4.0 do "git checkout
stable-4.0". People who aren't ready yet don't. There is no magic
and it is really obvious what one is on when typing 'git status'.
the downside is that people that pay zero attention and just git pull
remain on the stable branch they were on. (I actually think that's a
good thing.)
The stable-3.8 branch remains indefinitely as one that could be
updated with a fix, and can easily be accessed by name.
So this is a long way of saying that today the master/develop felt
extremely awkward, for no real overall benefit, and I'd like to suggest
changing to stable/master.
Now, I realize that a lot of people are using git to get code, when
those people do not necessarily understand git. I wonder if the
master/develop scheme's main point is that a naive checkout gets one the
most recent stable. So maybe getting that benefit without all the pain
of master/develop would be:
develop: new code codes here
stable-N: branched off develop for new branches, and each one gets bug
fixes
master: the highest stable-N is merged to master, so that master is
always the most recent version of the most recent stable. The only
point of master, and the only use, is people that do checkouts and
don't understand git or weewx well enough to choose a stable branch
explicitly. (IMHO, these people should be using distribution
packages, not git.)
Thanks for listening to me rant,
Greg
Vince Skahan
Feb 29, 2020, 5:32:59 PM2/29/20
to weewx-development
Interesting post. Sounds like you want some kind of semi git-flow kind of thing except with long-lived minor-release extra level of integration branches.
I think weewx basically takes the position of "stick where you're at or get yourself current" in general, other than also pointing folks at individual commits they might hand-edit into place on their old sources (at their risk and pain) before things get released. But I would be surprised to ever see another 3.9.x release now that 4.0 is imminent, for example.
Can't you get the same effect by:
- git pull (get your repo current)
- git checkout -b v3.8.0 (create a local branch at the point in time you want to start with)
- git diff -b v3.9.2. (diff your working branch vs. some later point in time)
- and so on...
I worked a project that used git-flow. It was brutal. No way to effectively have multiple parallel lines of code that moved forward (ie, patch 3.8.x when current dev was on 3.9.x or the like). No matter what you do, you have the back merge problem if you want to lag current.
Seems to me that weewx's branching is a "one way....move forward..." kind of mindset. While there are posts saying "get this commit" for a reported bug, they 'tend' to be versus the current major.minor version, not one that might be a year+ behind. In your case, wouldn't you want to get to 3.latest regardless ?
Thomas Keffer
Feb 29, 2020, 11:29:04 PM2/29/20
to weewx-development
The merge was intended (and usually is) done just before release. It was our intention to do an immediate release, but we had some delays.
WeeWX may be a bit different from other software you are used to. We never support old releases. Either you upgrade, or you don't. There's nothing in between. So, the usual work flow is to do the merge into master, then release. There's no reason to have branches dedicated to maintaining older releases.
In this case, we did the merge, but then the development team was unable to finish. So, it has been languishing.
We do our best, but sometimes paying jobs interfere.
-tk
--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/rmiblphyuuz.fsf%40s1.lexort.com.
Greg Troxel
Mar 6, 2020, 12:30:05 PM3/6/20
to Vince Skahan, weewx-development
First, thanks to you and Thomas for replying; I was delaying on purpose
to see if anyone else commented happened and think about it.
Vince Skahan <vince...@gmail.com> writes:
> Interesting post. Sounds like you want some kind of semi git-flow kind of
> thing except with long-lived minor-release extra level of integration
> branches.
I was unaware of git-flow, but now that I read about it, I can tell you
that I am not wanting it.
In the git world (not the weewx world), people seem to think it's a big
deal which branch is master and which has some other name. I mostly
don't, except that I regard master as the natural place for development,
either straight on master or for landing feature branches.
> I think weewx basically takes the position of "*stick where you're at or
> get yourself current*" in general, other than also pointing folks at
updates. I see that as a completely separate step from keeping the
labels clear enough so that this could reasonably be done, which is what
I was trying to takl about.
> Can't you get the same effect by:
>
> - git pull (get your repo current)
> - git checkout -b v3.8.0 (create a local branch at the point in time you
> want to start with)
> - git diff -b v3.9.2. (diff your working branch vs. some later point in
> time)
> - and so on...
Yes, I can do something like that, but after pull master is in some
state on some branch, as opposed to the branch I know is stable and that
I want to stay on until I decide to jump forward. So it isn't as
simple as you make it out, because I have to read the history and figure
out when there were merges from develop, and create a branch there.
But me doing this is basically creating the release branches that I
think should exist in arrears, and it means everybody who wants to
basically take a configuration management approach to their upgrade
schedule has to do this. If a packaging system has packaged some
release and wants to cherry-pick security or critical bug fixes without
a full update, they have to figure this out too.
> I worked a project that used git-flow. It was brutal. No way to
> effectively have multiple parallel lines of code that moved forward (ie,
> patch 3.8.x when current dev was on 3.9.x or the like). No matter what you
> do, you have the back merge problem if you want to lag current.
Once you have named release branches, then anything on the branch
development is done on can be cherry-picked to each branch, if desired.
Yes, merges can be difficult, but I think that's intrinsic in supporting
release branches, and not about the branching model. I don't see how
git-flow makes this harder or easier, except that I find "master is a
nickanme for the latest release branch" not necessarily helpful.
> Seems to me that weewx's branching is a "one way....move forward..." kind
> of mindset. While there are posts saying "get this commit" for a reported
> bug, they 'tend' to be versus the current major.minor version, not one that
> might be a year+ behind. In your case, wouldn't you want to get to
> 3.latest regardless ?
I probably would want to follow 3.latest, but if I were on 3.8 I might
view 3.9 with some trepidation, vs just commits that were approved for
3.8. I think 4.0 is a much bigger deal than 3.(N+1). I certainly don't
mean to criticize it at all, but with python 3 accomodation I feel that
the risk of trouble is much higher than in say the 3.8->3.9 transition
(and the 4.0 telegraphs that I am not alone in that perception, I'd
say).
My point is not that people should want a particular release branch. It
is also not that that the weewx team should actively support older
release branches. It is merely that I think the branch tags should be
there in the repo so that if someone wants to stay on a branch, or find
out what's on a branch, etc., that they then actually be able to do
that, without first having to first back-construct the branches.
As things are, as I understand them, once things get merged from develop
to master, then the identity of master as representing the previous
release branch is lost, and within git there is no name for that branch.
My main point is that this just seems like an unnecessary loss of useful
information.
My second point is that as a user, I want to be able to say "update
along the release branch I have chosen to be on", which is very safe,
and to be able to choose when to say "jump to this other release
branch", which is something that I think needs even more backup and
perhaps testing, depending on one's level of production paranoia.
Right now, the only way to keep up with the release train is to follow
master, which is subject to sudden jumps to a new release branch with no
way within git to tell that this happened.
If development were on master, then it would be as simple as creating a
"stable-4-0" branch off master when the 4.0 release process (first of
alpha/beta/RC) starts. With development on develop and master as a
"this refers to the most recent stable branch" name, then I guess that
means fixes for stable-4-0 go on stable-4-0 or master and are merged to
the other. Perhaps just put on stable-4-0 and some bot has master track
that, and the bot is reprogrammed to track stable-4-1 when that day
comes. (I see this as merely an accomodation for people who want to
check out a repo and use what's on master as the most recent stable.)
The other thing would be just to put these stable branch tags down just
before merging from develop, so they only sort of exist in arrears, but
one can easily get back to the right place after updating if paying
attention.
This is of course not that big a deal; previously I was ok just
following the latest stable, but I am particularly scared of 4.0,
partly because I expect to be the first to run it on NetBSD.
to see if anyone else commented happened and think about it.
Vince Skahan <vince...@gmail.com> writes:
> Interesting post. Sounds like you want some kind of semi git-flow kind of
> thing except with long-lived minor-release extra level of integration
> branches.
that I am not wanting it.
In the git world (not the weewx world), people seem to think it's a big
deal which branch is master and which has some other name. I mostly
don't, except that I regard master as the natural place for development,
either straight on master or for landing feature branches.
> I think weewx basically takes the position of "*stick where you're at or
> get yourself current*" in general, other than also pointing folks at
> individual commits they might hand-edit into place on their old sources (at
> their risk and pain) before things get released. But I would be surprised
> to ever see another 3.9.x release now that 4.0 is imminent, for example.
I wasn't trying to suggest that old releases receive support and
> their risk and pain) before things get released. But I would be surprised
> to ever see another 3.9.x release now that 4.0 is imminent, for example.
updates. I see that as a completely separate step from keeping the
labels clear enough so that this could reasonably be done, which is what
I was trying to takl about.
> Can't you get the same effect by:
>
> - git checkout -b v3.8.0 (create a local branch at the point in time you
> want to start with)
> - git diff -b v3.9.2. (diff your working branch vs. some later point in
> time)
> - and so on...
Yes, I can do something like that, but after pull master is in some
state on some branch, as opposed to the branch I know is stable and that
I want to stay on until I decide to jump forward. So it isn't as
simple as you make it out, because I have to read the history and figure
out when there were merges from develop, and create a branch there.
But me doing this is basically creating the release branches that I
think should exist in arrears, and it means everybody who wants to
basically take a configuration management approach to their upgrade
schedule has to do this. If a packaging system has packaged some
release and wants to cherry-pick security or critical bug fixes without
a full update, they have to figure this out too.
> I worked a project that used git-flow. It was brutal. No way to
> effectively have multiple parallel lines of code that moved forward (ie,
> patch 3.8.x when current dev was on 3.9.x or the like). No matter what you
> do, you have the back merge problem if you want to lag current.
development is done on can be cherry-picked to each branch, if desired.
Yes, merges can be difficult, but I think that's intrinsic in supporting
release branches, and not about the branching model. I don't see how
git-flow makes this harder or easier, except that I find "master is a
nickanme for the latest release branch" not necessarily helpful.
> Seems to me that weewx's branching is a "one way....move forward..." kind
> of mindset. While there are posts saying "get this commit" for a reported
> bug, they 'tend' to be versus the current major.minor version, not one that
> might be a year+ behind. In your case, wouldn't you want to get to
> 3.latest regardless ?
view 3.9 with some trepidation, vs just commits that were approved for
3.8. I think 4.0 is a much bigger deal than 3.(N+1). I certainly don't
mean to criticize it at all, but with python 3 accomodation I feel that
the risk of trouble is much higher than in say the 3.8->3.9 transition
(and the 4.0 telegraphs that I am not alone in that perception, I'd
say).
My point is not that people should want a particular release branch. It
is also not that that the weewx team should actively support older
release branches. It is merely that I think the branch tags should be
there in the repo so that if someone wants to stay on a branch, or find
out what's on a branch, etc., that they then actually be able to do
that, without first having to first back-construct the branches.
As things are, as I understand them, once things get merged from develop
to master, then the identity of master as representing the previous
release branch is lost, and within git there is no name for that branch.
My main point is that this just seems like an unnecessary loss of useful
information.
My second point is that as a user, I want to be able to say "update
along the release branch I have chosen to be on", which is very safe,
and to be able to choose when to say "jump to this other release
branch", which is something that I think needs even more backup and
perhaps testing, depending on one's level of production paranoia.
Right now, the only way to keep up with the release train is to follow
master, which is subject to sudden jumps to a new release branch with no
way within git to tell that this happened.
If development were on master, then it would be as simple as creating a
"stable-4-0" branch off master when the 4.0 release process (first of
alpha/beta/RC) starts. With development on develop and master as a
"this refers to the most recent stable branch" name, then I guess that
means fixes for stable-4-0 go on stable-4-0 or master and are merged to
the other. Perhaps just put on stable-4-0 and some bot has master track
that, and the bot is reprogrammed to track stable-4-1 when that day
comes. (I see this as merely an accomodation for people who want to
check out a repo and use what's on master as the most recent stable.)
The other thing would be just to put these stable branch tags down just
before merging from develop, so they only sort of exist in arrears, but
one can easily get back to the right place after updating if paying
attention.
This is of course not that big a deal; previously I was ok just
following the latest stable, but I am particularly scared of 4.0,
partly because I expect to be the first to run it on NetBSD.
Greg Troxel
Mar 6, 2020, 12:35:02 PM3/6/20
to Thomas Keffer, weewx-development
Thomas Keffer <tke...@gmail.com> writes:
> The merge was intended (and usually is) done just before release. It was
> our intention to do an immediate release, but we had some delays.
>
> WeeWX may be a bit different from other software you are used to. We
> *never* support old releases. Either you upgrade, or you
> The merge was intended (and usually is) done just before release. It was
> our intention to do an immediate release, but we had some delays.
>
> WeeWX may be a bit different from other software you are used to. We
> don't. There's nothing in between. So, the usual work flow is to do
> the merge into master, then release. There's no reason to have
> branches dedicated to maintaining older releases.
I am used to the not supporting old releases concept. pkgsrc is like
> the merge into master, then release. There's no reason to have
> branches dedicated to maintaining older releases.
this; when a branch is released, then we immmediately cease maintenance
- even security fixes - on the old branch. I am merely not used to not
being able to tell with git names what is what, and not being able to
choose straightforwardly to stay on a branch.
I did not mean to suggest in the slightest that old releases should be
miintained. In my case I was thrown off because I was trying to make
sure I was up to date with all the old stable bits before contemplating
building/using what was about to be the new stable bits, and once I was
updated, it was quite difficult to find out if my old pointer was the
farthest along the old stable or not (it was).
I do wonder if any of the LTS Linux distributions have weewx packaged
and how they deal with backporting fixes. There may be only a few
individuals doing this, and few to no security issues needing
addressing.
Vince Skahan
Mar 6, 2020, 12:50:50 PM3/6/20
to weewx-development
Greg - still missing what you're looking for that you can't already do.
- git pull
- git checkout -b v3.8.2 (create a local branch off a release tag)
- go wild
If you want to install a particular release, you could alternately download the tarball from old_releases in the web page.
What's missing ?
Greg Troxel
Mar 6, 2020, 1:07:29 PM3/6/20
to Vince Skahan, weewx-development
Vince Skahan <vince...@gmail.com> writes:
> Greg - still missing what you're looking for that you can't already do.
>
> - git pull
> Greg - still missing what you're looking for that you can't already do.
>
> - git checkout -b v3.8.2 (create a local branch off a release tag)
Sorry, I misread that the first time, so my previous-message comments
about that didn't make sense.
That creates a branch off a release tag. It has seemed to me that fixes
go onto master after the release (which is a good thing).
On a repo where I have done a checkout to the commit that I want this
minute (what was on master before develop was merged; I've put my local
master pointer there), I just did:
$ git log --oneline v3.9.2..
a8834236 (HEAD -> master, tag: gdt_stable) LOCAL: change home prefix
0ecd4fed wmr200: Discard packets with 'interval' of zero (#472)
3eb97346 Fix engine to exit correctly on Terminate signal (#442)
c78e4b51 Documented commit https://github.com/weewx/weewx/commit/459ccb1309df776cfea9fc84549b76166cbeaf94 in change log
93d7d57a Now sorts using locale.strxfrm, so no longer necessary to convert to utf-8. CF: https://github.com/weewx/weewx/commit/e54b942557f2de32e2814509f7f4b2c2bfe1398e
80c612de Updated date in change log
20c8c3a3 don't mess with the permissions - just sync the files
and these are the commits that landed on what I'd call stable-3-9 that
are after v3.9.2. If there were a v3.9.3 (I am *not* complaining that
there isn't), then those commits would presumably be in it.
(That top commit is me just moving the install prefix from /home/weewx
to /usr/weewx; I expect to rebase them when updating.)
> If you want to install a particular release, you could alternately download
> the tarball from old_releases in the web page.
> What's missing ?
A way to "follow 3.9 stable" without me back-constructing the branch
tag.
So basically what's missing is a branch ref stable-3-9 which started out
pointing to master as soon as it became v3.9 time, is advanced along
commits on master intended for v3.9.next, and now points to 0ecd4fed.
Does that make sense (even if you don't think the cost of doing it is
worth the benefit, which I can see as arguable).
Vince Skahan
Mar 6, 2020, 1:31:18 PM3/6/20
to weewx-development
On Friday, March 6, 2020 at 10:07:29 AM UTC-8, Greg Troxel wrote:
Vince Skahan <vince...@gmail.com> writes:
> Greg - still missing what you're looking for that you can't already do.
> - git pull
> - git checkout -b v3.8.2 (create a local branch off a release tag)
That creates a branch off a release tag. It has seemed to me that fixes
go onto master after the release (which is a good thing).
They do. So ???
> What's missing ?
A way to "follow 3.9 stable" without me back-constructing the branch
tag.
I think you can look at 'master' as similar to the 'stream repo' concept in new RHEL/CentOS. It would contain the latest stable set of patches to the latest production version, pre-release of a new (tagged, bundled, packaged) version.
So master at any point in time is what the next release would look like, if it had to happen 'now'.
The other branches are (to me) stuff not so stable, possibly experimental, possibly never to see the light of day in an official release. In git-flow terminology they'd call them feature branches.
Bottom line - we can't assume 'master' is 'latest release' because it isn't. It's the 'to be NEXT release' as it exists at the current date.
So basically what's missing is a branch ref stable-3-9 which started out
pointing to master as soon as it became v3.9 time, is advanced along
commits on master intended for v3.9.next, and now points to 0ecd4fed.
Totally lost. There 'is' no stable N-1 branch. You can always get to what N-1 was by using its release tag.
Greg Troxel
Mar 6, 2020, 1:56:20 PM3/6/20
to Vince Skahan, weewx-development
Vince Skahan <vince...@gmail.com> writes:
>> So basically what's missing is a branch ref stable-3-9 which started out
>> pointing to master as soon as it became v3.9 time, is advanced along
>> commits on master intended for v3.9.next, and now points to 0ecd4fed.
>>
> Totally lost. There 'is' no stable N-1 branch. You can always get to what
> N-1 was by using its release tag.
Not sure what you mean by "totally lost".
>> So basically what's missing is a branch ref stable-3-9 which started out
>> pointing to master as soon as it became v3.9 time, is advanced along
>> commits on master intended for v3.9.next, and now points to 0ecd4fed.
>>
> Totally lost. There 'is' no stable N-1 branch. You can always get to what
> N-1 was by using its release tag.
Yes, I realize that there is no branch ref now. I am merely trying to
point out that logically there is a branch concept that one could have a
ref pointing to, and that I think it would be useful.
Vince Skahan
Mar 6, 2020, 2:07:19 PM3/6/20
to weewx-development
On Friday, March 6, 2020 at 10:56:20 AM UTC-8, Greg Troxel wrote:
Vince Skahan <vince...@gmail.com> writes:
>> So basically what's missing is a branch ref stable-3-9 which started out
>> pointing to master as soon as it became v3.9 time, is advanced along
>> commits on master intended for v3.9.next, and now points to 0ecd4fed.
>>
> Totally lost. There 'is' no stable N-1 branch. You can always get to what
> N-1 was by using its release tag.
Not sure what you mean by "totally lost".
Oh - I mean I'm totally lost :-)
Yes, I realize that there is no branch ref now. I am merely trying to
point out that logically there is a branch concept that one could have a
ref pointing to, and that I think it would be useful.
Sure - it's basically using branches vs. using tags.
Whatever Tom wants works for me.
Thomas Keffer
Mar 6, 2020, 4:41:50 PM3/6/20
to weewx-development
Busy here on packaging issues, but I just wanted to add that all releases are tagged. So, if you want v3.9.0 for some reason,
git checkout tags/v3.9.0
would do it. But, maybe you already knew that, in which case, my apologies.
--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/a8c62f5f-85f5-46c5-8849-8cc9d28c48e2%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages