At the risk of starting a religious or at least a preference war. I
sincerely call upon this community to standardize on distributed
repository development using Bazaar. My rationale is that we need to
embrace global development, this community has only limited resources,
it needs to maximize those resources, a distributed system like Bazaar
has features that svn does not and at least one significant source of
development for VistA has moved in that direction.
I am aware of arguments for and against svn, git, bazaar, and
mercurial. After considering them, my opinion is that this community
standardizes on Bazaar.
svn: easier, been around for awhile, some have already started using
it. Lacking important features like distributed development. Important
projects are moving away from it.
git: powerful, fast, harder to use, Clearhealth is using this.
bazaar: have data and use that appears to favor it for VistA
development. Has distributed development features that others such as
svn do not. Medsphere is using it.
mercurial: easier to use, python is now using it, no developers that I
know of in VistA are using it.
Ignacio, the repository is doing the job AND it is being administered by a volunteer. Seriously, do you think this is such an important issue that it warrents an uproar over this?
> At the risk of starting a religious or at least a preference war. I
> sincerely call upon this community to standardize on distributed
> repository development using Bazaar. My rationale is that we need to
> embrace global development, this community has only limited resources,
> it needs to maximize those resources, a distributed system like Bazaar
> has features that svn does not and at least one significant source of
> development for VistA has moved in that direction.
> I am aware of arguments for and against svn, git, bazaar, and
> mercurial. After considering them, my opinion is that this community
> standardizes on Bazaar.
> svn: easier, been around for awhile, some have already started using
> it. Lacking important features like distributed development. Important
> projects are moving away from it.
> git: powerful, fast, harder to use, Clearhealth is using this.
> bazaar: have data and use that appears to favor it for VistA
> development. Has distributed development features that others such as
> svn do not. Medsphere is using it.
> mercurial: easier to use, python is now using it, no developers that I
> know of in VistA are using it.
> Ignacio, the repository is doing the job AND it is being administered by a
> volunteer. Seriously, do you think this is such an important issue that it
> warrents an uproar over this?
> On Monday 22 June 2009, I, Valdes wrote:
> > Hi,
> > At the risk of starting a religious or at least a preference war. I
> > sincerely call upon this community to standardize on distributed
> > repository development using Bazaar. My rationale is that we need to
> > embrace global development, this community has only limited resources,
> > it needs to maximize those resources, a distributed system like Bazaar
> > has features that svn does not and at least one significant source of
> > development for VistA has moved in that direction.
> > I am aware of arguments for and against svn, git, bazaar, and
> > mercurial. After considering them, my opinion is that this community
> > standardizes on Bazaar.
> > svn: easier, been around for awhile, some have already started using
> > it. Lacking important features like distributed development. Important
> > projects are moving away from it.
> > git: powerful, fast, harder to use, Clearhealth is using this.
> > bazaar: have data and use that appears to favor it for VistA
> > development. Has distributed development features that others such as
> > svn do not. Medsphere is using it.
> > mercurial: easier to use, python is now using it, no developers that I
> > know of in VistA are using it.
While I appreciate the kind words, I think the choice of revision control system by any particular VistA distribution is of little consequence at the moment because we can't take advantage of automated merging right now. None of the tools out there understand VistA structures, so it's impossible to communicate a menu option or an RPC in a changeset. I believe there are some people working on a solution to this problem, but I'm not sure how far along they are. Until we have a way to communicate VistA structures in changesets, we're mostly using SCMs as a way to snapshot a VistA database at various points in time, and all the existing tools do that.
That said, there may be some value in standardizing on a system now, so that we only have to build one set of tools that can communicate VistA structures. And, once the tools are built, we're already all using the same system. Another argument for standardizing on a single tool now is that there's less to learn for newcomers to the community.
We (Medsphere) chose bazaar because it's modern, distributed, and extensible. It is easy of use, flexible (has great support for multiple workflows, see http://bazaar-vcs.org/Workflows), and integrates with Launchpad.net. Launchpad provides bug tracking, repository browsing, code review, blueprints, and support for gettext translations and is very conducive to distributed community development.
On Mon, 2009-06-22 at 08:22 -0700, I, Valdes wrote: > Hi,
> At the risk of starting a religious or at least a preference war. I > sincerely call upon this community to standardize on distributed > repository development using Bazaar. My rationale is that we need to > embrace global development, this community has only limited resources, > it needs to maximize those resources, a distributed system like Bazaar > has features that svn does not and at least one significant source of > development for VistA has moved in that direction.
> I am aware of arguments for and against svn, git, bazaar, and > mercurial. After considering them, my opinion is that this community > standardizes on Bazaar.
> svn: easier, been around for awhile, some have already started using > it. Lacking important features like distributed development. Important > projects are moving away from it. > git: powerful, fast, harder to use, Clearhealth is using this. > bazaar: have data and use that appears to favor it for VistA > development. Has distributed development features that others such as > svn do not. Medsphere is using it. > mercurial: easier to use, python is now using it, no developers that I > know of in VistA are using it.
Jonathan Tai wrote: > That said, there may be some value in standardizing on a system now, so > that we only have to build one set of tools that can communicate VistA > structures. And, once the tools are built, we're already all using the > same system.
In that context choosing the same version control system in advance would save a lot of work.
> ... Another argument for standardizing on a single tool now is > that there's less to learn for newcomers to the community.
Lowering the barriers to usage or participation is very important. Using one tool instead of two or three helps in that regard.
Precisely. I would further say that I believe that this community is
becoming more international. I strongly suspect, without actual data,
that development for such an increasingly global community is better
served by a distributed system like Bazaar. Thank you for your
comments. -- IV
On Jun 22, 12:50 pm, Jonathan Tai <jon....@medsphere.com> wrote:
> That said, there may be some value in standardizing on a system now, so
> that we only have to build one set of tools that can communicate VistA
> structures. And, once the tools are built, we're already all using the
> same system. Another argument for standardizing on a single tool now is
> that there's less to learn for newcomers to the community.
> We (Medsphere) chose bazaar because it's modern, distributed, and
> extensible. It is easy of use, flexible (has great support for multiple
> workflows, seehttp://bazaar-vcs.org/Workflows), and integrates with
> Launchpad.net. Launchpad provides bug tracking, repository browsing,
> code review, blueprints, and support for gettext translations and is
> very conducive to distributed community development.
Could someone chime in with a note about how to address the progress on the
"VistA automated merging using version control" problem? Links are fine...
This has been a huge problem for a while. I think what Ignacio is suggesting
is that we choose a version control context from which we plan on addressing
this larger problem. Is there a generalized method that will work for any
version control system? or should we choose one to solve this problem?
On Mon, Jun 22, 2009 at 12:50 PM, Jonathan Tai <jon....@medsphere.com>wrote:
> While I appreciate the kind words, I think the choice of revision
> control system by any particular VistA distribution is of little
> consequence at the moment because we can't take advantage of automated
> merging right now. None of the tools out there understand VistA
> structures, so it's impossible to communicate a menu option or an RPC in
> a changeset. I believe there are some people working on a solution to
> this problem, but I'm not sure how far along they are. Until we have a
> way to communicate VistA structures in changesets, we're mostly using
> SCMs as a way to snapshot a VistA database at various points in time,
> and all the existing tools do that.
> That said, there may be some value in standardizing on a system now, so
> that we only have to build one set of tools that can communicate VistA
> structures. And, once the tools are built, we're already all using the
> same system. Another argument for standardizing on a single tool now is
> that there's less to learn for newcomers to the community.
> We (Medsphere) chose bazaar because it's modern, distributed, and
> extensible. It is easy of use, flexible (has great support for multiple
> workflows, see http://bazaar-vcs.org/Workflows), and integrates with
> Launchpad.net. Launchpad provides bug tracking, repository browsing,
> code review, blueprints, and support for gettext translations and is
> very conducive to distributed community development.
> - Jon
> On Mon, 2009-06-22 at 08:22 -0700, I, Valdes wrote:
> > Hi,
> > At the risk of starting a religious or at least a preference war. I
> > sincerely call upon this community to standardize on distributed
> > repository development using Bazaar. My rationale is that we need to
> > embrace global development, this community has only limited resources,
> > it needs to maximize those resources, a distributed system like Bazaar
> > has features that svn does not and at least one significant source of
> > development for VistA has moved in that direction.
> > I am aware of arguments for and against svn, git, bazaar, and
> > mercurial. After considering them, my opinion is that this community
> > standardizes on Bazaar.
> > svn: easier, been around for awhile, some have already started using
> > it. Lacking important features like distributed development. Important
> > projects are moving away from it.
> > git: powerful, fast, harder to use, Clearhealth is using this.
> > bazaar: have data and use that appears to favor it for VistA
> > development. Has distributed development features that others such as
> > svn do not. Medsphere is using it.
> > mercurial: easier to use, python is now using it, no developers that I
> > know of in VistA are using it.
> > I refer you to the informed work of Medsphere's Jonathan Tai for data
> > on why this is good for VistA specifically:
On Jun 22, 11:10 am, Nancy Anthracite <nanthrac...@earthlink.net>
wrote:
> Ignacio, the repository is doing the job AND it is being administered by a
> volunteer. Seriously, do you think this is such an important issue that it
> warrants an uproar over this?
I think so. Aren't we playing to win? Shouldn't we seek every
advantage possible against increasingly long odds by using the
sharpest, best tools that facilitate development?
fred trotter wrote:
> Could someone chime in with a note about how to address the progress on the
> "VistA automated merging using version control" problem? Links are fine...
Or, what are some of the show-stoppers as well as must-have features
for version control?
On Tue, 2009-06-30 at 03:51 -0700, Lars Noodén wrote: > fred trotter wrote: > > Could someone chime in with a note about how to address the progress on the > > "VistA automated merging using version control" problem? Links are fine...
> Or, what are some of the show-stoppers as well as must-have features > for version control?
For VistA development to really take off, we need distributed revision control and automated merging.
Take Linux for example. Linux uses a distributed revision control system called git. Linus has a branch of Linux that is the "official" branch. This is called the "mainline". Everyone can make a copy of this branch and do their own development. So far, it sounds just like centralized revision control, but the difference between centralized and distributed is that you can commit to your own private branch. You don't need permission from anyone. With a centralized system like Subversion, you have to have commit access to the repository, so you start by submitting one-off patches here and there until you gain enough karma to be entrusted with commit access. This doesn't scale. More on this later...
Getting back to the example, when you have done some development and are satisfied with it, you can publish your branch in a public location. You can then bring the branch to the official maintainer's attention, and request that he "merge" or "pull" your branch. This is another critical difference - with distributed version control, changes are *pulled*, not *pushed*. So the first time you request a merge, the maintainer can inspect your code closely. He may suggest changes. You can do those in your own branch (and each change is tracked because you can commit to your own branch) and re-submit your request. With a centralized system, you'd be sending a patch. If you're a frequent contributor, the maintainer may review your code less closely because he trusts you more, but the review process is built in.
After a little re-working, the maintainer finally decides your code is worthy for merging. With automated merging, even if that maintainer has made changes to other parts of the codebase since you cloned his branch, when the maintainer merges your branch, your changes are brought in automatically. If other people have branched off *your* branch and those changes have been merged already, most distributed systems are smart enough to figure that out and only merge the unmerged revisions. Subversion has gotten a lot better at this recently, but in earlier versions, it was *terrible* at this.
Another important point is that each branch is equal (at least on a technical level) - the maintainer's branch is only special because it's the maintainer's branch and is recognized by the community as the official branch. If the maintainer were to become disinterested in the software or otherwise unable to maintain it, or should you want to take the software in a different direction, your branch could just as easily become *the* branch. Getting back to the Linux example, each subsystem maintainer has his own tree - there are trees dedicated to filesystems, to memory managers, drivers, etc. There are also trees dedicated to stability, so they branch off an official Linux release and then only merge critical bugfixes from that point on. If you wanted a more stable version of Linux, you may follow that branch instead of the official mainline branch.
So in the VistA world - imagine several distributions like WorldVistA, OpenVista, FOIA VistA, vxVistA, etc. Each would have their own mainline. We could have package branches - so the FileMan group might have their own branch that's distro-independent. All the distros would merge changes from the FileMan group to maintain a "shared core", while each distribution does their own application work. Provided that the code/data structures are compatible and the licenses are compatible, the distributions could merge changes across - so WorldVista could merge in the server-side OpenVista CIS patches to allow CIS to run against WorldVista. And most importantly, when OpenVista updates those patches, WorldVista could just merge the new changes. If automated merging worked properly, this would be very seamless and easy - no re-porting required.
In fact, you could take it a step further. Each implementing site could choose to have (and maintain) their own branch. So a site might branch off of WorldVistA's mainline, make a few local mods, then merge new changes from WorldVista as time goes on. And if WorldVistA liked those local changes, they could merge in the other direction - the local changes are accepted back into the main distribution. For an all-volunteer distribution like WorldVistA, this may be the main source of their changes.
So why don't we do all of this now? First of all, some of us are still using centralized revision control. That's what Ignacio was trying to address by starting this thread. Fixing this is the easy part - switching from a centralized model to a distributed model is fairly simple because the centralized model is just a subset of the distributed model. It's like having just one branch that everyone commits to.
The second thing we need is automated merging. What we need to do is come up with a way to express VistA structures in a way that existing revision control tools can understand. Right now, a menu is stored in FileMan, which is stored as globals in a binary database file. A tool designed to handle source code can't handle that. It doesn't know how to merge two multi-gigabyte binary blobs to make a meaningful new version. We could export the database as plain text (e.g., %GO or ZWRITE format), but even that is not enough to allow automated merging because a menu might have a certain IEN at one site but another IEN at another site. The existing tools have no idea what's important and needs to be merged and what is site-specific and can be discarded. What we need is an abstract representation of a menu, similar to what KIDS has. If we can present these VistA structures to an existing tool, we can leverage the power of these existing tools to do development as a community more efficiently.
From talking to various folks at the VistA Community Meetings, some thought has gone into how to do #2, but I haven't seen anything workable yet. For now, I agree with Ignacio that we should do what we can, i.e., #1. Standardize on a distributed revision control tool so we can build the tools for #2 around it.
> On Tue, 2009-06-30 at 03:51 -0700, Lars Noodén wrote:
> > fred trotter wrote:
> > > Could someone chime in with a note about how to address the progress on the
> > > "VistA automated merging using version control" problem? Links are fine...
> > Or, what are some of the show-stoppers as well as must-have features
> > for version control?
> For VistA development to really take off, we need distributed revision
> control and automated merging.
> Take Linux for example. Linux uses a distributed revision control
> system called git. Linus has a branch of Linux that is the "official"
> branch. This is called the "mainline". Everyone can make a copy of
> this branch and do their own development. So far, it sounds just like
> centralized revision control, but the difference between centralized and
> distributed is that you can commit to your own private branch. You
> don't need permission from anyone. With a centralized system like
> Subversion, you have to have commit access to the repository, so you
> start by submitting one-off patches here and there until you gain enough
> karma to be entrusted with commit access. This doesn't scale. More on
> this later...
> Getting back to the example, when you have done some development and are
> satisfied with it, you can publish your branch in a public location.
> You can then bring the branch to the official maintainer's attention,
> and request that he "merge" or "pull" your branch. This is another
> critical difference - with distributed version control, changes are
> *pulled*, not *pushed*. So the first time you request a merge, the
> maintainer can inspect your code closely. He may suggest changes. You
> can do those in your own branch (and each change is tracked because you
> can commit to your own branch) and re-submit your request. With a
> centralized system, you'd be sending a patch. If you're a frequent
> contributor, the maintainer may review your code less closely because he
> trusts you more, but the review process is built in.
> After a little re-working, the maintainer finally decides your code is
> worthy for merging. With automated merging, even if that maintainer has
> made changes to other parts of the codebase since you cloned his branch,
> when the maintainer merges your branch, your changes are brought in
> automatically. If other people have branched off *your* branch and
> those changes have been merged already, most distributed systems are
> smart enough to figure that out and only merge the unmerged revisions.
> Subversion has gotten a lot better at this recently, but in earlier
> versions, it was *terrible* at this.
> Another important point is that each branch is equal (at least on a
> technical level) - the maintainer's branch is only special because it's
> the maintainer's branch and is recognized by the community as the
> official branch. If the maintainer were to become disinterested in the
> software or otherwise unable to maintain it, or should you want to take
> the software in a different direction, your branch could just as easily
> become *the* branch. Getting back to the Linux example, each subsystem
> maintainer has his own tree - there are trees dedicated to filesystems,
> to memory managers, drivers, etc. There are also trees dedicated to
> stability, so they branch off an official Linux release and then only
> merge critical bugfixes from that point on. If you wanted a more stable
> version of Linux, you may follow that branch instead of the official
> mainline branch.
> So in the VistA world - imagine several distributions like WorldVistA,
> OpenVista, FOIA VistA, vxVistA, etc. Each would have their own
> mainline. We could have package branches - so the FileMan group might
> have their own branch that's distro-independent. All the distros would
> merge changes from the FileMan group to maintain a "shared core", while
> each distribution does their own application work. Provided that the
> code/data structures are compatible and the licenses are compatible, the
> distributions could merge changes across - so WorldVista could merge in
> the server-side OpenVista CIS patches to allow CIS to run against
> WorldVista. And most importantly, when OpenVista updates those patches,
> WorldVista could just merge the new changes. If automated merging
> worked properly, this would be very seamless and easy - no re-porting
> required.
> In fact, you could take it a step further. Each implementing site could
> choose to have (and maintain) their own branch. So a site might branch
> off of WorldVistA's mainline, make a few local mods, then merge new
> changes from WorldVista as time goes on. And if WorldVistA liked those
> local changes, they could merge in the other direction - the local
> changes are accepted back into the main distribution. For an
> all-volunteer distribution like WorldVistA, this may be the main source
> of their changes.
> So why don't we do all of this now? First of all, some of us are still
> using centralized revision control. That's what Ignacio was trying to
> address by starting this thread. Fixing this is the easy part -
> switching from a centralized model to a distributed model is fairly
> simple because the centralized model is just a subset of the distributed
> model. It's like having just one branch that everyone commits to.
> The second thing we need is automated merging. What we need to do is
> come up with a way to express VistA structures in a way that existing
> revision control tools can understand. Right now, a menu is stored in
> FileMan, which is stored as globals in a binary database file. A tool
> designed to handle source code can't handle that. It doesn't know how
> to merge two multi-gigabyte binary blobs to make a meaningful new
> version. We could export the database as plain text (e.g., %GO or
> ZWRITE format), but even that is not enough to allow automated merging
> because a menu might have a certain IEN at one site but another IEN at
> another site. The existing tools have no idea what's important and
> needs to be merged and what is site-specific and can be discarded. What
> we need is an abstract representation of a menu, similar to what KIDS
> has. If we can present these VistA structures to an existing tool, we
> can leverage the power of these existing tools to do development as a
> community more efficiently.
> From talking to various folks at the VistA Community Meetings, some
> thought has gone into how to do #2, but I haven't seen anything workable
> yet. For now, I agree with Ignacio that we should do what we can, i.e.,
> #1. Standardize on a distributed revision control tool so we can build
> the tools for #2 around it.
FYI, Brian Behlendorf discussed this some in his presentation at the NHIN Connect conference today and I believe the video tape will be posted, probably on the ConnectOpenSource.org site.
> On Tue, 2009-06-30 at 03:51 -0700, Lars Noodén wrote: > > fred trotter wrote: > > > Could someone chime in with a note about how to address the progress on > > > the "VistA automated merging using version control" problem? Links are > > > fine...
> > Or, what are some of the show-stoppers as well as must-have features > > for version control?
> For VistA development to really take off, we need distributed revision > control and automated merging.
> Take Linux for example. Linux uses a distributed revision control > system called git. Linus has a branch of Linux that is the "official" > branch. This is called the "mainline". Everyone can make a copy of > this branch and do their own development. So far, it sounds just like > centralized revision control, but the difference between centralized and > distributed is that you can commit to your own private branch. You > don't need permission from anyone. With a centralized system like > Subversion, you have to have commit access to the repository, so you > start by submitting one-off patches here and there until you gain enough > karma to be entrusted with commit access. This doesn't scale. More on > this later...
> Getting back to the example, when you have done some development and are > satisfied with it, you can publish your branch in a public location. > You can then bring the branch to the official maintainer's attention, > and request that he "merge" or "pull" your branch. This is another > critical difference - with distributed version control, changes are > *pulled*, not *pushed*. So the first time you request a merge, the > maintainer can inspect your code closely. He may suggest changes. You > can do those in your own branch (and each change is tracked because you > can commit to your own branch) and re-submit your request. With a > centralized system, you'd be sending a patch. If you're a frequent > contributor, the maintainer may review your code less closely because he > trusts you more, but the review process is built in.
> After a little re-working, the maintainer finally decides your code is > worthy for merging. With automated merging, even if that maintainer has > made changes to other parts of the codebase since you cloned his branch, > when the maintainer merges your branch, your changes are brought in > automatically. If other people have branched off *your* branch and > those changes have been merged already, most distributed systems are > smart enough to figure that out and only merge the unmerged revisions. > Subversion has gotten a lot better at this recently, but in earlier > versions, it was *terrible* at this.
> Another important point is that each branch is equal (at least on a > technical level) - the maintainer's branch is only special because it's > the maintainer's branch and is recognized by the community as the > official branch. If the maintainer were to become disinterested in the > software or otherwise unable to maintain it, or should you want to take > the software in a different direction, your branch could just as easily > become *the* branch. Getting back to the Linux example, each subsystem > maintainer has his own tree - there are trees dedicated to filesystems, > to memory managers, drivers, etc. There are also trees dedicated to > stability, so they branch off an official Linux release and then only > merge critical bugfixes from that point on. If you wanted a more stable > version of Linux, you may follow that branch instead of the official > mainline branch.
> So in the VistA world - imagine several distributions like WorldVistA, > OpenVista, FOIA VistA, vxVistA, etc. Each would have their own > mainline. We could have package branches - so the FileMan group might > have their own branch that's distro-independent. All the distros would > merge changes from the FileMan group to maintain a "shared core", while > each distribution does their own application work. Provided that the > code/data structures are compatible and the licenses are compatible, the > distributions could merge changes across - so WorldVista could merge in > the server-side OpenVista CIS patches to allow CIS to run against > WorldVista. And most importantly, when OpenVista updates those patches, > WorldVista could just merge the new changes. If automated merging > worked properly, this would be very seamless and easy - no re-porting > required.
> In fact, you could take it a step further. Each implementing site could > choose to have (and maintain) their own branch. So a site might branch > off of WorldVistA's mainline, make a few local mods, then merge new > changes from WorldVista as time goes on. And if WorldVistA liked those > local changes, they could merge in the other direction - the local > changes are accepted back into the main distribution. For an > all-volunteer distribution like WorldVistA, this may be the main source > of their changes.
> So why don't we do all of this now? First of all, some of us are still > using centralized revision control. That's what Ignacio was trying to > address by starting this thread. Fixing this is the easy part - > switching from a centralized model to a distributed model is fairly > simple because the centralized model is just a subset of the distributed > model. It's like having just one branch that everyone commits to.
> The second thing we need is automated merging. What we need to do is > come up with a way to express VistA structures in a way that existing > revision control tools can understand. Right now, a menu is stored in > FileMan, which is stored as globals in a binary database file. A tool > designed to handle source code can't handle that. It doesn't know how > to merge two multi-gigabyte binary blobs to make a meaningful new > version. We could export the database as plain text (e.g., %GO or > ZWRITE format), but even that is not enough to allow automated merging > because a menu might have a certain IEN at one site but another IEN at > another site. The existing tools have no idea what's important and > needs to be merged and what is site-specific and can be discarded. What > we need is an abstract representation of a menu, similar to what KIDS > has. If we can present these VistA structures to an existing tool, we > can leverage the power of these existing tools to do development as a > community more efficiently.
> From talking to various folks at the VistA Community Meetings, some > thought has gone into how to do #2, but I haven't seen anything workable > yet. For now, I agree with Ignacio that we should do what we can, i.e., > #1. Standardize on a distributed revision control tool so we can build > the tools for #2 around it.
Nancy Anthracite wrote: > FYI, Brian Behlendorf discussed this some in his presentation at the NHIN > Connect conference today and I believe the video tape will be posted, probably > on the ConnectOpenSource.org site.
Thanks. I took a quick look at the site and the sessions are listed by title of the presentation / session.
If anyone happen to know when the video will be posted, please be so kind as to post the URL.
Regards -Lars
PS. A site about open source might improve availability of its content by following established best practices in regards to web server operating systems.
The links currently present on the site produce only error messages, a situation I myself find common to sites that attempt the particular brand currently present at connectopensource.org.
The only comment I would add is that the major systems are (I understand) able to move trees back & forth. So, although standardization is nice, it is not required. If my understanding is incorrect, I guess we need to standardize.
Regards
-- Bhaskar
--------------------------
Sent from my BlackBerry Wireless Handheld
----- Original Message -----
From: Hardhats@googlegroups.com <Hardhats@googlegroups.com>
To: hardhats@googlegroups.com <hardhats@googlegroups.com>
Sent: Tue Jun 30 17:40:43 2009
Subject: [Hardhats] Re: Proposal to standardize on Bazaar versus Mercurial versus Git versus SVN
On Tue, 2009-06-30 at 03:51 -0700, Lars Noodén wrote:
> fred trotter wrote:
> > Could someone chime in with a note about how to address the progress on the
> > "VistA automated merging using version control" problem? Links are fine...
> Or, what are some of the show-stoppers as well as must-have features
> for version control?
For VistA development to really take off, we need distributed revision
control and automated merging.
Take Linux for example. Linux uses a distributed revision control
system called git. Linus has a branch of Linux that is the "official"
branch. This is called the "mainline". Everyone can make a copy of
this branch and do their own development. So far, it sounds just like
centralized revision control, but the difference between centralized and
distributed is that you can commit to your own private branch. You
don't need permission from anyone. With a centralized system like
Subversion, you have to have commit access to the repository, so you
start by submitting one-off patches here and there until you gain enough
karma to be entrusted with commit access. This doesn't scale. More on
this later...
Getting back to the example, when you have done some development and are
satisfied with it, you can publish your branch in a public location.
You can then bring the branch to the official maintainer's attention,
and request that he "merge" or "pull" your branch. This is another
critical difference - with distributed version control, changes are
*pulled*, not *pushed*. So the first time you request a merge, the
maintainer can inspect your code closely. He may suggest changes. You
can do those in your own branch (and each change is tracked because you
can commit to your own branch) and re-submit your request. With a
centralized system, you'd be sending a patch. If you're a frequent
contributor, the maintainer may review your code less closely because he
trusts you more, but the review process is built in.
After a little re-working, the maintainer finally decides your code is
worthy for merging. With automated merging, even if that maintainer has
made changes to other parts of the codebase since you cloned his branch,
when the maintainer merges your branch, your changes are brought in
automatically. If other people have branched off *your* branch and
those changes have been merged already, most distributed systems are
smart enough to figure that out and only merge the unmerged revisions.
Subversion has gotten a lot better at this recently, but in earlier
versions, it was *terrible* at this.
Another important point is that each branch is equal (at least on a
technical level) - the maintainer's branch is only special because it's
the maintainer's branch and is recognized by the community as the
official branch. If the maintainer were to become disinterested in the
software or otherwise unable to maintain it, or should you want to take
the software in a different direction, your branch could just as easily
become *the* branch. Getting back to the Linux example, each subsystem
maintainer has his own tree - there are trees dedicated to filesystems,
to memory managers, drivers, etc. There are also trees dedicated to
stability, so they branch off an official Linux release and then only
merge critical bugfixes from that point on. If you wanted a more stable
version of Linux, you may follow that branch instead of the official
mainline branch.
So in the VistA world - imagine several distributions like WorldVistA,
OpenVista, FOIA VistA, vxVistA, etc. Each would have their own
mainline. We could have package branches - so the FileMan group might
have their own branch that's distro-independent. All the distros would
merge changes from the FileMan group to maintain a "shared core", while
each distribution does their own application work. Provided that the
code/data structures are compatible and the licenses are compatible, the
distributions could merge changes across - so WorldVista could merge in
the server-side OpenVista CIS patches to allow CIS to run against
WorldVista. And most importantly, when OpenVista updates those patches,
WorldVista could just merge the new changes. If automated merging
worked properly, this would be very seamless and easy - no re-porting
required.
In fact, you could take it a step further. Each implementing site could
choose to have (and maintain) their own branch. So a site might branch
off of WorldVistA's mainline, make a few local mods, then merge new
changes from WorldVista as time goes on. And if WorldVistA liked those
local changes, they could merge in the other direction - the local
changes are accepted back into the main distribution. For an
all-volunteer distribution like WorldVistA, this may be the main source
of their changes.
So why don't we do all of this now? First of all, some of us are still
using centralized revision control. That's what Ignacio was trying to
address by starting this thread. Fixing this is the easy part -
switching from a centralized model to a distributed model is fairly
simple because the centralized model is just a subset of the distributed
model. It's like having just one branch that everyone commits to.
The second thing we need is automated merging. What we need to do is
come up with a way to express VistA structures in a way that existing
revision control tools can understand. Right now, a menu is stored in
FileMan, which is stored as globals in a binary database file. A tool
designed to handle source code can't handle that. It doesn't know how
to merge two multi-gigabyte binary blobs to make a meaningful new
version. We could export the database as plain text (e.g., %GO or
ZWRITE format), but even that is not enough to allow automated merging
because a menu might have a certain IEN at one site but another IEN at
another site. The existing tools have no idea what's important and
needs to be merged and what is site-specific and can be discarded. What
we need is an abstract representation of a menu, similar to what KIDS
has. If we can present these VistA structures to an existing tool, we
can leverage the power of these existing tools to do development as a
community more efficiently.
From talking to various folks at the VistA Community Meetings, some
thought has gone into how to do #2, but I haven't seen anything workable
yet. For now, I agree with Ignacio that we should do what we can, i.e.,
#1. Standardize on a distributed revision control tool so we can build
the tools for #2 around it.
- Jon
_____________
The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. _____________
On Wed, 2009-07-01 at 01:05 -0700, Bhaskar, KS wrote: > Right on, Jon.
> The only comment I would add is that the major systems are (I > understand) able to move trees back & forth.
> So, although standardization is nice, it is not required. If my > understanding is incorrect, I guess we need to standardize.
You mean between systems? Like merging a git tree into a bzr branch?
While there are some tools to do this, they typically geared toward repository conversion (not daily use) and the conversion is imperfect. And using multiple tools would mean that we'd have to write any VistA-structure-exposing tools in a version-control-tool-agnostic way. (As opposed to say, a bzr plugin.)
And regardless of the technical details, there's still the learning curve for newcomers to the community.
--------------------------
Sent from my BlackBerry Wireless Handheld
----- Original Message -----
From: Hardhats@googlegroups.com <Hardhats@googlegroups.com>
To: hardhats@googlegroups.com <hardhats@googlegroups.com>
Sent: Wed Jul 01 10:06:49 2009
Subject: [Hardhats] Re: Proposal to standardize on Bazaar versus Mercurial versus Git versus SVN
On Wed, 2009-07-01 at 01:05 -0700, Bhaskar, KS wrote:
> Right on, Jon.
> The only comment I would add is that the major systems are (I
> understand) able to move trees back & forth.
> So, although standardization is nice, it is not required. If my
> understanding is incorrect, I guess we need to standardize.
You mean between systems? Like merging a git tree into a bzr branch?
While there are some tools to do this, they typically geared toward
repository conversion (not daily use) and the conversion is imperfect.
And using multiple tools would mean that we'd have to write any
VistA-structure-exposing tools in a version-control-tool-agnostic way.
(As opposed to say, a bzr plugin.)
And regardless of the technical details, there's still the learning
curve for newcomers to the community.
I think it would be best to standardize.
- Jon
_____________
The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. _____________
While Standardization is preferred, it should not necessarily be a
barrier to innovation and expansion. The needs of the community change
easily with the calendar, location, and the local politics. VistA was
developed to be modified in the field by the people who are the subject
matter experts, the people at the point of care. The low cost of these
configurations is such that new innovations can be tried in actual
clinical trials without using the production machine.
If done correctly, the applications can be enhanced and modified to
reflect the emerging reality and still provide the expected standard
responses. The two approaches are not necessarily in opposition to each
other. They, standards and innovation, can be orthogonal to each other
with only a minimum of impact on each other, but there needs to be
standards which can be crafted to make the process of innovation clean
and unobtrusive (right back to your point, Jon). Think of the network
standards behind a firewall, 192.168.x.x is the range used by local
machines to assign as needed. We can do the same thing with naming and
numbering conventions (standards) in VistA so that loading new
innovations can be done in these local name and number spaces, but
eventually moved over into the general release (and new name and number
spaces) if the functionality is identified as desirable.
> --------------------------
> Sent from my BlackBerry Wireless Handheld
> ----- Original Message -----
> From: Hardhats@googlegroups.com <Hardhats@googlegroups.com>
> To: hardhats@googlegroups.com <hardhats@googlegroups.com>
> Sent: Wed Jul 01 10:06:49 2009
> Subject: [Hardhats] Re: Proposal to standardize on Bazaar versus Mercurial
> versus Git versus SVN
> On Wed, 2009-07-01 at 01:05 -0700, Bhaskar, KS wrote:
>> Right on, Jon.
>> The only comment I would add is that the major systems are (I
>> understand) able to move trees back & forth.
>> So, although standardization is nice, it is not required. If my
>> understanding is incorrect, I guess we need to standardize.
> You mean between systems? Like merging a git tree into a bzr branch?
> While there are some tools to do this, they typically geared toward
> repository conversion (not daily use) and the conversion is imperfect.
> And using multiple tools would mean that we'd have to write any
> VistA-structure-exposing tools in a version-control-tool-agnostic way.
> (As opposed to say, a bzr plugin.)
> And regardless of the technical details, there's still the learning
> curve for newcomers to the community.
> I think it would be best to standardize.
> - Jon
> _____________
> The information contained in this message is proprietary and/or
> confidential. If you are not the
> intended recipient, please: (i) delete the message and all copies; (ii) do
> not disclose,
> distribute or use the message in any manner; and (iii) notify the sender
> immediately. In addition,
> please be aware that any message addressed to our domain is subject to
> archiving and review by
> persons other than the intended recipient. Thank you.
> _____________
On Tue, Jun 30, 2009 at 5:40 PM, Jonathan Tai<jon....@medsphere.com> wrote: > On Tue, 2009-06-30 at 03:51 -0700, Lars Noodén wrote: >> fred trotter wrote: >> > Could someone chime in with a note about how to address the progress on the >> > "VistA automated merging using version control" problem? Links are fine...
>> Or, what are some of the show-stoppers as well as must-have features >> for version control?
> For VistA development to really take off, we need distributed revision > control and automated merging.
> Take Linux for example. Linux uses a distributed revision control > system called git. Linus has a branch of Linux that is the "official" > branch. This is called the "mainline". Everyone can make a copy of > this branch and do their own development. So far, it sounds just like > centralized revision control, but the difference between centralized and > distributed is that you can commit to your own private branch. You > don't need permission from anyone. With a centralized system like > Subversion, you have to have commit access to the repository, so you > start by submitting one-off patches here and there until you gain enough > karma to be entrusted with commit access. This doesn't scale. More on > this later...
> Getting back to the example, when you have done some development and are > satisfied with it, you can publish your branch in a public location. > You can then bring the branch to the official maintainer's attention, > and request that he "merge" or "pull" your branch. This is another > critical difference - with distributed version control, changes are > *pulled*, not *pushed*. So the first time you request a merge, the > maintainer can inspect your code closely. He may suggest changes. You > can do those in your own branch (and each change is tracked because you > can commit to your own branch) and re-submit your request. With a > centralized system, you'd be sending a patch. If you're a frequent > contributor, the maintainer may review your code less closely because he > trusts you more, but the review process is built in.
> After a little re-working, the maintainer finally decides your code is > worthy for merging. With automated merging, even if that maintainer has > made changes to other parts of the codebase since you cloned his branch, > when the maintainer merges your branch, your changes are brought in > automatically. If other people have branched off *your* branch and > those changes have been merged already, most distributed systems are > smart enough to figure that out and only merge the unmerged revisions. > Subversion has gotten a lot better at this recently, but in earlier > versions, it was *terrible* at this.
> Another important point is that each branch is equal (at least on a > technical level) - the maintainer's branch is only special because it's > the maintainer's branch and is recognized by the community as the > official branch. If the maintainer were to become disinterested in the > software or otherwise unable to maintain it, or should you want to take > the software in a different direction, your branch could just as easily > become *the* branch. Getting back to the Linux example, each subsystem > maintainer has his own tree - there are trees dedicated to filesystems, > to memory managers, drivers, etc. There are also trees dedicated to > stability, so they branch off an official Linux release and then only > merge critical bugfixes from that point on. If you wanted a more stable > version of Linux, you may follow that branch instead of the official > mainline branch.
> So in the VistA world - imagine several distributions like WorldVistA, > OpenVista, FOIA VistA, vxVistA, etc. Each would have their own > mainline. We could have package branches - so the FileMan group might > have their own branch that's distro-independent. All the distros would > merge changes from the FileMan group to maintain a "shared core", while > each distribution does their own application work. Provided that the > code/data structures are compatible and the licenses are compatible, the > distributions could merge changes across - so WorldVista could merge in > the server-side OpenVista CIS patches to allow CIS to run against > WorldVista. And most importantly, when OpenVista updates those patches, > WorldVista could just merge the new changes. If automated merging > worked properly, this would be very seamless and easy - no re-porting > required.
> In fact, you could take it a step further. Each implementing site could > choose to have (and maintain) their own branch. So a site might branch > off of WorldVistA's mainline, make a few local mods, then merge new > changes from WorldVista as time goes on. And if WorldVistA liked those > local changes, they could merge in the other direction - the local > changes are accepted back into the main distribution. For an > all-volunteer distribution like WorldVistA, this may be the main source > of their changes.
> So why don't we do all of this now? First of all, some of us are still > using centralized revision control. That's what Ignacio was trying to > address by starting this thread. Fixing this is the easy part - > switching from a centralized model to a distributed model is fairly > simple because the centralized model is just a subset of the distributed > model. It's like having just one branch that everyone commits to.
> The second thing we need is automated merging. What we need to do is > come up with a way to express VistA structures in a way that existing > revision control tools can understand. Right now, a menu is stored in > FileMan, which is stored as globals in a binary database file. A tool > designed to handle source code can't handle that. It doesn't know how > to merge two multi-gigabyte binary blobs to make a meaningful new > version. We could export the database as plain text (e.g., %GO or > ZWRITE format), but even that is not enough to allow automated merging > because a menu might have a certain IEN at one site but another IEN at > another site. The existing tools have no idea what's important and > needs to be merged and what is site-specific and can be discarded. What > we need is an abstract representation of a menu, similar to what KIDS > has. If we can present these VistA structures to an existing tool, we > can leverage the power of these existing tools to do development as a > community more efficiently.
> From talking to various folks at the VistA Community Meetings, some > thought has gone into how to do #2, but I haven't seen anything workable > yet. For now, I agree with Ignacio that we should do what we can, i.e., > #1. Standardize on a distributed revision control tool so we can build > the tools for #2 around it.
