Git tools for Thor

[Mike Potjer]

Hi All,

A few months ago I created a class containing some Git utilities to make it easier to work with Git in a VFP development environment, especially with VFP's binary files.  I created some Thor tools to allow me to run them from the Thor repository.  I was speaking about Git at our user group, and decided to polish up these tools a bit for a more general distribution through the Thor repository.  For anyone interested in checking them out, I've made the code publicly available on Github: https://github.com/mikepotjer/vfp-git-utils, and on BitBucket: https://bitbucket.org/mikepotjer/vfp-git-utils.  More detail is found in the ReadMe file included with the code.

I hope this is as useful to others as it has been for me.

Mike Potjer

[Eric Selje]

Very cool, Mike. They should be on VFPx for maximum exposure I think. (they can still reside on your github account if you do that - see the way Matt has his GoFish source code).

Eric

--
You received this message because you are subscribed to the Google Groups "Thor, the Tool Manager for FoxPro" group.
To unsubscribe from this group and stop receiving emails from it, send an email to FoxProThor+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mike Potjer]

Thanks Eric.

Since I designed it to operate out of the Thor repository, and it is dependent on Thor and FoxBin2Prg, I'm not sure if it warrants a separate VFPx project.  Plus there's the question of whether this would be considered a "competing product" to the new VFPx project "Bin 2 Text extension" (though I don't think we're doing quite the same things).

I appreciate your concern for making others aware of this utility.  I figured making it publicly available on GitHub and posting a message here would be a good start.  Interestingly enough, I just did a Google search on just the 2 keywords "vfp" and "git", and my GitHub repository came up #2 on the list.  So it has that going for it. :-)

[Fernando D. Bozzo]

Hey, Mike!

Nice work. I will add a reference to it on VFPx home page index, so others can find it if navigating VFPx :D

Best regards!

[Fernando D. Bozzo]

Done! Look at the bottom of https://vfpx.codeplex.com/, under "Other Open Source VFP Projects"

:D

[Mike Potjer]

Thanks Fernando!

[Eric Selje]

Wow, learn something new every day. I wasn't aware of the "Other Open Source VFP Projects" section of VFPX. Very cool.

Now what you need to do Mike is set up a Google Alert so whenever someone posts a question to a forum, like StackExchange, about using Git with VFP you'll get an alert and can steer 'em your way.

Eric

[Mike Potjer]

FYI, I made a couple minor documentation updates to this project, which have been pushed to both GitHub and BitBucket.

1. I updated the readme file for the project to include a link to a Getting Started with Git guide that I've been working on, which is now shared on my Google Drive.
   
2. All the Git utilities tools now have a "Tool home page" link to the project on GitHub.  The tool home page link is available when viewing the tools in the Thor Tool Launcher, or on the Tool Definitions page in Thor Configuration.

[Jerry Stager]

Hey Mike, could you please provide me or point me to basic steps of using these Git Utilities? We've been using VFP with VSS for about 15 years and need to switch to a more modern source control and it looks like this is the best approach. I'm just getting up to speed on Git and just not totally sure what the steps are to using all of this together. 

For example, I have been playing around with a test and I'm not sure if I am on the right track. When you complete changes to a binary file do you then

1. use the "Prepare for Git commit" in your new VFP Git Utilities
   
2. in Git commit the .sc2|.vc2 text file generated by FoxBin2Prg
3. do a Git pull to see if there are any merge/conflicts needed to be resolved
4. regenerate the binaries from the .sc2|.vc2 text file using FoxBin2Prg (not sure how to do this step)
5. and then push the binaries and text versions of them up to Git

Am I understanding the process correctly? Any comments or more details you can provide on these steps would be great.

Thanks for all your time (as well as Fenando's) in making a option like this available to all of us!

Jerry 

[Mike Potjer]

Hi Jerry,

1. use the "Prepare for Git commit" in your new VFP Git Utilities

Yes

2. in Git commit the .sc2|.vc2 text file generated by FoxBin2Prg

Yes (along with the binaries, which I assume you know based on question #5)

3. do a Git pull to see if there are any merge/conflicts needed to be resolved

If you know that a git pull is what you want here, then yes.  Other options include doing a git fetch, then either merge or rebase, depending on your circumstances and/or preferences.

4. regenerate the binaries from the .sc2|.vc2 text file using FoxBin2Prg (not sure how to do this step)

You will only need to regenerate binaries for sc2|vc2 files that have conflicts which you resolved by merging the changes in the text file.  I believe you know that based on your question, but just want to clarify.  Anyway, an easy way to handle that is to create a Windows "Send to" shortcut for FoxBin2Prg.  There is a Thor tool for that under Applications > FoxBin2Prg > Create/Edit FoxBin2Prg "SendTo" Shortcuts.  I recommend creating the 2-Way Conversion shortcut.  Then simply navigate to the sc2|vc2 text file in Windows Explorer (or other file managers), right-click on the file, and select Send to > FoxBin2Prg - 2-Way Conversion.  If you are in the middle of a merge or rebase, then you will likely have one set of binary files that is "staged", and the other is "unstaged".  The binaries created by FoxBin2Prg will be the unstaged set.  Simply stage them for commit, replacing the set that was previously staged.  Once all such conflicts are resolved, you can complete the commit.

5. and then push the binaries and text versions of them up to Git

Yes

I used VSS at least as long as you have, and I think that once you get used to using a DVCS like Git or Mercurial, you'll never want to go back (I know I don't!).  VFP's binary files make any version control system a challenge, but FoxBin2Prg virtually eliminates that pain.  Hope this helps!

Mike

[Jerry S]

Thanks Mike, this has been really helpful and I was able to complete a merge of a binary in a test project. There's still a lot more work of figuring out how Git totally works and how to structure our repository but at least I know we are heading in the right direction. Thanks!

[Mike Potjer]

Glad I could help.  I had to rearrange a few things to better fit Git's repository model, but on the whole it went pretty smoothly, and I prefer the end result to the previous work environment.

[Jerry S]

Hey Mike, just getting back to running some tests on this... I've noticed that if I add a new form or class, run the "Prepare For Git Commit" it isn't generating the sc2|vc2 files right away. Eventually after doing some unknown steps (perhaps it is when I add another form or class) it generates the sc2|vc2 files for the first form|class. 

Am I missing a step?

Thanks,

Jerry 

[Mike Potjer]

Hi Jerry,

 

For new binaries, the tool doesn’t run FoxBin2Prg until the binary file is staged.  Don’t know what UI you are using, but staged and unstaged files are clearly distinguished in SourceTree and Git GUI.  I think TortoiseGit refers to unstaged new files as “unversioned”.

Anyway, thanks for calling this to my attention.  I’m so used to that behavior that I didn’t think to document it in the ReadMe file on the tools page.

 

Mike

[Mike Potjer]

Hi Jerry,

Thanks again for your question.  Last week I updated the ReadMe to (hopefully) be clearer on the prepare process.  As I was updating the documentation, I realized that I could improve the prepare process.  The update I made available this morning is the fruit of that.

Mike

[Brian V]

On Friday, July 31, 2015 at 7:39:58 PM UTC-4, Mike Potjer wrote:

Hi Jerry,

...

2. in Git commit the .sc2|.vc2 text file generated by FoxBin2Prg

Yes (along with the binaries, which I assume you know based on question #5)

 Hi Mike,

I am wondering why you need to include the binaries if FoxBin2Prg works in both directions. I was actually evaluating FoxBin2Prg along with Git (& Hg) Tools for Thor before I realized that your tool utilises FoxBin2Prg. But in my research I didn't come across anything that said FoxBin2Prg required or recommended the storage of both the binaries and the generated text files.

The other tool I've been looking at is Vfpscm. Do you know anything about it and how it might compare to your tool? (apologies if this is an inappropriate question to post here)

Thanks,

Brian

[Mike Potjer]

Hi Brian,

You are correct that you don't need to store binaries if you are using FoxBin2Prg,  I guess it's mostly historical that I chose to store binaries, since I had been doing that for 15-16 years of using SourceSafe.  Doug Hennig's session at SWFox this year convinced me that I should consider storing just the text files.  Coincidentally, I am planning on using this long weekend to work on an enhancement to my tool to support repositories which only store the binaries, or even a mixture of just some binaries with other files only stored as text.  I can let you know when I have something to test, if you are interested.  Fortunately, you don't have to wait for me to complete that if you plan to use FoxBin2Prg.  You can start using FoxBin2Prg today, and add my Git/Hg tool later.

I'm not familiar with VFPSCM, but it looks like it serves the function as FoxBin2Prg, but using XML as the text format.  Personally, I would continue to use FoxBin2Prg.  Fernando created a text format that is much closer to normal VFP syntax, making it easier to read and edit than XML.  FoxBin2Prg's integration with Thor and other features make it the best tool out there for 2-way conversion of VFP binaries, IMO.

Hope this helps.

Mike

[Brian V]

You've been a huge help. Thank you.

[Mike Potjer]

You're welcome.

BTW, I noticed in my response I said I would be working on an enhancement "to support repositories which only store binaries".  What I MEANT to say was that enhancement would support repositories which only store the TEXT files for the VFP binaries.  Hope that wasn't too confusing to anyone who saw it.

[Eric Selje]

One argument for storing the binaries is if you're using a continuous integration system (like Jenkins, e.g.) and you're using the AutomatedBuild project to build the project, if the binaries are in source control then there's one less step for Jenkins to do (deserialize), which makes the process go faster.

Eric

--
You received this message because you are subscribed to the Google Groups "Thor, the Tool Manager for FoxPro" group.

To unsubscribe from this group and stop receiving emails from it, send an email to FoxProThor+unsubscribe@googlegroups.com.

[Jim Nelson]

On the same line of thinking:

I work with another developer.  We store both binaries and text files.  If we did not store binaries, I would have to re-create the binaries each time I pulled any of her changes (and vice versa).  

--

Jim Nelson

(805) 498-9195 (preferred)
(720) 837-3536 (cell)

www.kongcompany.com

[Mike Potjer]

Thanks Jim and Eric.  Those are good reasons why the Git/Hg will continue to support repositories that store binaries+text, in addition to the text-only support I'm working on.  You will also be able to have any combination of the two in the same repository, if you have reason to keep binaries+text for some files, and text-only for the rest.

Jim, after a git checkout - which I believe is "update to" in Mercurial - or after merge or pull, there is basically a one-click process in the Git tools that will regenerate those binaries for you, in addition to some other tasks that typically need to be done.  I'm not saying you have to switch to text-only, but I'm making that process even simpler than using FoxBin2Prg directly.  If you use a timestamp file in your repository (which this tool also maintains for you), the process can be much faster than using FoxBin2Prg alone, because the tool can check which files are part of the repository, and check the timestamp file, to determine exactly which binaries need to be regenerated.

To unsubscribe from this group and stop receiving emails from it, send an email to FoxProThor+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Thor, the Tool Manager for FoxPro" group.

To unsubscribe from this group and stop receiving emails from it, send an email to FoxProThor+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Fernando D. Bozzo]

Hi Mike,

Just want to add something of my experience about dealing with timetamps, when they are convenient to use and when they aren't.

As you know, the old scctext(x).prg (SourceSafe days:) did have this timestamp support to regenerate the text files automatically when the timestamp of the binary is newer than the text file, and did not regenerate it if the timestamp of the binary is same (or lower?) than the text, which gives a great performance when dealing with the files of a project.

But this scheme is OK when you always do modify the binaries exclusively... which is not the case with a DVCS/SCM tool when working with real 2-or-more parallel modifications of the same binaries, on which who commands is the text file and not the binary, because in this scenario, when you merge 2 branches, you sometimes need to manually merge same changes, while the DVCS/SCM tool can make some automatic decisions on other parts of the code depending of the change detection algorithm it uses, so finally the binaries must be regenerated from the text files (in a merge: always)

The thing to consider about timestamps is that you can freely change not only the binary, but the text too (ie: massive changes with an external editor), so there are at least one important use case that could happen:

- If you do some real changes at text file then open the binary just to look something, almost always VFP will change the timestamp of the binary for the sole action of opening the binary, so when you regenerate by timestamps comparison you loose all your real text changes


You can ask: "Why someone should modify a text file?"

Well, the thing is that when you feel confident changing the text files as if where PRG code, some changes are more comfortable/easy to make at the text than at the binary (and some people use this capability extensively!), because in a DVCS world when working with concurrent changes (2 or more devs) the text file is a first class citizen.

But if you only work just modifying the binaries and do not thing about working with the text files directly (except when merging), then you can safely base your regenerations on the timestamps.

Talking about timestamps, FoxBin2Prg have a CFG setting to work this way (as scctext did), which is disabled by default because what I've explained:

* OptimizeByFilestamp: 0        && Optimize file regeneration depending on file timestamp

but can be uncommented and turned on if neccesary:

OptimizeByFilestamp: 1        && Optimize file regeneration depending on file timestamp


Fernando.-

[Mike Potjer]

Hi Fernando,

Your point is well taken, but I'm afraid I created some confusion by bringing up the issue of timestamps with the Git/Hg tools.  I'm not talking about the timestamps inside a VFP binary file.  When you run the "prepare for commit" process in the Git/Hg tool, one of the tasks it performs by default is to generate or update a text file containing a list of the Windows modification dates for all files in the repository.  This timestamp text file would be committed to the repository along with all the code files.  The screenshot below shows a sample of the timestamp text file.  By default the file is called .gittimestamp for Git, and .hgtimestamp for Mercurial.  I use a combination of Git or Mercurial and this timestamp file to determine whether to call FoxBin2Prg.

I will explain how part of the "post-checkout" process of the Git/Hg tool works to illustrate how this can be faster than using FoxBin2Prg alone.  After the developer does a Git checkout or an Hg update (or a pull or merge in either DVCS), they would run the "post-checkout" tool.  If the developer only tracks FoxBin2Prg text files in the repository, the Git/Hg tool will perform the steps below.  I will use Git and a .VCX class library in my example, but what I describe works the same for Mercurial and all the other VFP binary file types.

• My tool calls Git to get a list of the files tracked by the repository.
• For a .VC2 file tracked by the repository, lookup the .VC2 file in the .gittimestamp text file, and get the datetime value for the .VC2 file.
• Change the Windows modification datetime of the .VC2 file to match the datetime value from the .gittimestamp file.
• Check if the corresponding .VCX and .VCT file exist in the same folder as the .VC2 file.
  
• If one or both of the binary files do NOT exist:
• Call FoxBin2Prg to regenerate the binary files.
• Change the Windows modification datetime of the VCX and .VCT files to match the datetime value of the .VC2 file.
• If both binary files exist:
• Check the Windows modification datetime of the .VCX and .VCT files.
• If the modification datetimes of BOTH files exactly match the datetime of the .VC2 file, we're done.  It's reasonable to assume that those 2 files have not been touched.
• If one or both of the modification datetimes do NOT match the datetime of the .VC2 file:
• Call FoxBin2Prg to regenerate the binary files.
• Change the Windows modification datetime of the VCX and .VCT files to match the datetime value of the .VC2 file.

The "post-checkout" tool also supports scenarios where a timestamp text file is not being used, or where VFP binaries are tracked in the repository.  In addition, it performs other tasks not related to FoxBin2Prg.  There is a separate "prepare for commit" tool that handles tasks related to getting your VFP files ready for a commit to the DVCS, including calls to FoxBin2Prg.

I hope that is clearer.

Mike

[Fernando D. Bozzo]

Thanks Mike, very clear explanation!

You received this message because you are subscribed to a topic in the Google Groups "Thor, the Tool Manager for FoxPro" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/FoxProThor/k67BkZJoasU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to FoxProThor+unsubscribe@googlegroups.com.