Creating modular applications in the first place - that is exactly what
I thought about when finishing that email, and I do agree that doing
that is very common, and its quite well known how to do it. It could be
as easy as writing a few dll's/so's that link to their common
application core. The code for each of these would be kept in on
repository each, where git-flow is used for the branching. The
distribution to the client is done by individual scripts/build-steps
(one per client), which takes care of putting the right files into the
deliverable, according to the features the client has chosen. At the
end, this deliverable is is sent as a whole to each client, who fully
replaces his application with the new version. In such a system, you
wouldn't really track the inter-dll dependencies, as you know that if a
dll changes, all clients which use it need a new deliverable, and if the
core changes, every client needs to get a new deliverable - problem solved.
This kind of workflow usually works well for C++ and C programs.
Lets look at a program based on python, and lets assume that a package
is setup such that each module within it provides new functionality to
some framework it is built upon. Each of these modules adds a distinct
feature, that could be comparable to a separate dll that is being sold
separately. To keep things very simple, once again the developer just
keeps one development stream, and has deployment scripts which compose
the application for individual clients, be simply removing python
modules they didn't pay for, for instance. Whenever the developer has a
stable version he wants to share, he runs all his deployment scripts
which create (possibly byte-compiled) distributions. He puts these into
their own 'distribution git repositories', and ships the new version to
the client only if it actually changed ( the feature he worked on to
create a new release could have been completely removed in one clients
version, so for this one, it didn't change at all ).
In both examples, the knowledge about which client gets which features
is kept in some sort of script ( or let it be some database that the
build system uses to determine the feature set ).
Yet again, a system like that appears to work well enough to be usable,
and its relatively simple to implement and to use.
When reading this, I would come to exactly your conclusion: Why
complicate things in git-flow if we already have it working ? Git-flow
does the branching, the build-system cares about the distribution, et voilà.
Now lets have a look at the dependency graph formed by the commits in
our repositories. When using gitflow, we have multiple branches, which
represent distinct entities, like the development stream, feature
streams, hotfixes or streams of hotfixes, release streams, and the
stream containing the deliverable itself, master. Its great to have
that, and a lot of information is contained in there. Its dependency
information, and information about issues that arose ( hotfixes ),
features you added, and the time-relationships of all these.
At the current point, there is no software that would put that
information to good use. For us, currently, its just a bunch of commits
linked to each other, and all we really care about is the develop- and
the master-branch. But in fact this dependency graph contains the
dependencies of your code, as well the 'flow' of the development.
Realizing what actually happened to your code allows you to assemble and
combine it, while keeping complete track of said dependencies.
Whats even better is that we have all this in a single integrated
database, which is our git repository, and we can exactly tell what
features and fixes are in each development stream, because git-flow
understands them.
Lets add just one more example: You have a client who is in the middle
of a production, he uses software version 1, and he really wants to
stick with it as he is afraid that any change would have a
negative/unforseen impact on all the tools that depend on your software,
at version 1. Now he runs into a bug, and asks for help. You know this
bug is already fixed in version 1.5 of your software, but besides that,
many things changed.
If you didn't have git-flows new abilities, you would tell the client to
risk it, and use version 1.5 of your software, which might involve
relinking his tools against your library as many other things changed.
This essentially shifts the risk to the client, and he doesn't like that
at all. Also, the client had to run into the bug that you obviously
fixed a while ago in version 1.3 - as we know he didn't want to upgrade
to a new version anyway so he ignored all releases above 1.0.
With the new features, and provided the hotfix required to fix the bug
was applied at the point where it was introduced, git-flow could have
transferred the fix into all development streams that are affected by
it. You, as the developer of the software, are able to roll-out new
point releases of all affected releases for all your client, which just
add the bugfix but nothing more. You would have been the one to inform
the client about this bugfix, and would have made it available in
version 1.0.1, which is guaranteed to just fix a single bug, nothing else.
With the system I proposed for git-flow, something like backports and
feature-ports (i.e. the transfer of these into other development streams
) can be automated to a great extend, which is something I find very
valuable.
The service that you, as a software company can provide to the client,
is superior to that what the others can do, which gives you an edge over
the competition. You basically acknowledge that there are clients who
use an older version of the software, but using the new git-flow, you
can still support it with bugfixes, and even compatible features ( if
you like ).
What a great service :).
Cheers,
Sebastian
PS: I must admit that the point I make here is always based on the
assumption that you can find the spot in time ( i.e. the commit ) that
introduced a certain bug, so that you can fix it right there. If you
always apply hotfixes at the newest commit ( which is easiest ), even
git-flow could only assume that this hotfix really needs the latest
version of your software, in which it could only port/transfer it into
compatible development streams which are at the same 'level'.
Finding that spot in time could be done using bisection + a custom test
which you run whenever you have checked out a certain version of the
software. All this needs to be automated to be feasible, but ... it can
be automated, its all possible, someone just has to do it :).
PS: Yet another LONG email, can't help it, must be the topic ;).
Keeping it simple ( to the user ) is something I see likewise. To my mind, it is not necessary at all to change the interface of the user in the first version.<vincent>I read your e-mail top to bottom and tried to understand the ideas that you are proposing. Although it sounds pretty complicated, I think I know what you are trying to say here, but I think we have to make sure that it doesn't get any more complicated than what gitflow is right now. This doesn't mean however, that we shouldn't add new features or behaviour to gitflow.</vincent>
You see gitflow, in its first form as a necessary innovation, as you first formalised a good branching workflow when working with git. You call it a necessary evolutionary step, like the fish, which eventually came out of the water as reptiles. What I am proposing is to apply yet another evolutionary step, which doesn't make the Reptile a bird, but something like a crocodile which is faster and better than its predecessors, and ... it can bite ;). Before I go into details about this, I will pick up more of your statements.<vincent>Let me start by pointing out what gitflow is about and what I think should not be gitflow's responsibilities. The original blog post describing the workflow proposed a simple set of rules to formalise a uniform release management process. gitflow was simply the next evolutionary step to ease working with these rules, and to prevent users against accidentally skipping/forgetting steps.</vincent>
This is a very strong statement as you seriously doubt that the workflow is possibly achievable, and whether it is gitflow's turn to try. Gitflow formalised a way to work with branches, and the set of rules also keeps it as simple as possible. The resulting commit graph will be relatively easily read when watched in gitk even, and I believe such a design is good.<vincent>Although I can understand your proposal to let gitflow do more than "just that", I think it definitely helps to keep these core objectives clearly visible and to ask questions about whether or not this should (or even *can*) be gitflow's responsibility.</vincent>
A gut feeling is something very healthy - it prevents you from doing dangerous things, and definitely helps to keep things sane. When you tell me that I should write a fluid simulation solver, my gut-feeling would tell me that there is no way, and that I couldn't do it, there is no light at the end of the tunnel for me. This feeling is based on fear of something that I do not (yet) understand, so I have no idea how to possibly tackle this.<vincent>My gut feeling says this is something that gitflow was never developed for. I trust my gut feeling always when I'm developing software since it rarely lets me down, but I feel obligated to rationalise what exactly it is that causes it.</vincent>
<vincent>If I understood your proposal correctly, a good chunk of it is dedicated to something that I would call "feature management"�the act of managing what features land in what version of the software you are selling. Essentially, you have a piece of reusable software that you sell in different "flavours" to clients, where each flavour essentially is e core product and a fixed set of features/add-ons. The term "feature" is what causes the obscurity here, since you seem to approach a feature branch as if they are an integral set of commits that together implement a complete end-user feature (e.g. what a customer would call a feature). This is a false assumption!
If I understood your proposal correctly, a good chunk of it is dedicated to something that I would call "feature management"�the act of managing what features land in what version of the software you are selling. Essentially, you have a piece of reusable software that you sell in different "flavours" to clients, where each flavour essentially is e core product and a fixed set of features/add-ons. The term "feature" is what causes the obscurity here, since you seem to approach a feature branch as if they are an integral set of commits that together implement a complete end-user feature (e.g. what a customer would call a feature). This is a false assumption!
As Mark proposes correctly, there are better ways of dealing with this kind of managerial problem, but all variant of that require you to structure your source files in order to support this kind of management. Examples are feature switches [1], or using different Git repo's for each client, each "including" your core product using a Git submodule. Yet another one is restructuring your project using add-ons/ plugins, and making distributions for your customers by assembling packages to contain fixed sets of these plugins. Of course, you customers don't need to know anything about this implementation. If you know the Django framework, you know how it makes the application pluggable [2], which you can really learn a lot from.</vincent>
So I'd advise you to manage these changes in space (directories), not
time (branches). To make it a bit more visual: finishing feature
branches in gitflow is a bit like pissing into a swimming pool.
Filling up the swimming pool requires multiple pisses, possibly by
multiple people. But if you want to track what pisses are for which
customer, really the only way is to piss in another pool :-)
I like the idea, though, of gitflow being able to better deal with multiple remotes and to assist in getting a better understanding of what remotes are upfront/behind on what branches. I see a perfect fit for this in gitflow. I encourage you to take a look at the possibilities that the "feature pull" subcommand already offers [3], as this is currently the kind of remote-friendly commands that I'm targeting with the rewrite.
<vincent>If I understood your proposal correctly, a good chunk of it is dedicated to something that I would call "feature management"—the act of managing what features land in what version of the software you are selling. Essentially, you have a piece of reusable software that you sell in different "flavours" to clients, where each flavour essentially is e core product and a fixed set of features/add-ons. The term "feature" is what causes the obscurity here, since you seem to approach a feature branch as if they are an integral set of commits that together implement a complete end-user feature (e.g. what a customer would call a feature). This is a false assumption!If I understood your proposal correctly, a good chunk of it is dedicated to something that I would call "feature management"—the act of managing what features land in what version of the software you are selling. Essentially, you have a piece of reusable software that you sell in different "flavours" to clients, where each flavour essentially is e core product and a fixed set of features/add-ons. The term "feature" is what causes the obscurity here, since you seem to approach a feature branch as if they are an integral set of commits that together implement a complete end-user feature (e.g. what a customer would call a feature). This is a false assumption!
I think the list of premises that Ryan put in place is something that helps to scope the git-flow tool, too. In fact, premise 1 ("Your software project uses the same workflow that git flow is based on") is the most important one. But yet, it might be a bit unclear what that means exactly.
One of the key aspects of why this workflow has turned out to be "successful" is that it fits single-master projects perfectly. Branching off in terms of different distributions/flavours is something that the workflow never considered in the first place. Trying to map git-flow onto this workflow is what now causes the pain and confusion.
As long as you develop a single-flavour application, the workflow is well-defined. But if you intend to keep many supported master branches around, this fundamentally erodes the principles of git-flow. I'm not saying doing this is wrong in any way, I'm just saying that git-flow might not be the tool that is going to help you.
I still believe that VCSes are way better at tracking changes over time, rather than space. This is not a Git-specific issue. Therefore, if you manage to *build* flavours from the same, single-master source code, the workflow will fit your project fine again. It's the same space/time discussion.
I'm looking forward to your thoughts.
Cheers,
Vincent
The problem here is that such a system, and I am somewhat repeating
myself, still needs to be done, and I don't believe anyone will have the
time for it, especially when you consider that the current git-flow
branching model already suits the majority of the people.
Phil said: "Don't lose it. It is important, I think." - and this is
exactly what I will do (Phil's post was quite encouraging by the way :)
). I believe I was onto something there, and hopefully I will be able to
implement a first prototype at some point. Until then, I am sure the
first version if pygit-flow will be available already, and it will be
great !
To provide a playground for future endeavours, in that direction or in
others, I hereby introduce a new pet-project of mine: pgit (PeeGit -
https://github.com/Byron/pgit).
Its something like a python version of the git command-line tools, and
the most important reason for its existence is the improvement to
git-submodule's features that I have to implement to support the
distribution of one of my internal projects.
Maybe one day, it could host that 'first prototype' I was talking about,
but for now, I shall only add the advanced submodule handling, hopefully
by the end of the next week.
Cheers,
Sebastian
Hello Hartmut!
I’m aware of your gitflow fork.
My intent is not to rewrite gitflow in Python, but to implement a tool, alternative to gitflow.
It is similar to gitflow in the sense that it is git workflow automation tool too, and it also solves some problems mentioned in this thread.