Roadmap discussion

[Ivan Danilov]

Let's write here some info about further development of git-tfs in general to coordinate efforts.

Important things I could think of right now are:

• Handling TFS branches.
• Speedup of fetching and checking in. Native TFS client receives versions much faster, especially evident on a large volume of small files or renaming/moving (~1 min in TeamExplorer against 10-20 min in git-tfs in my experience with ~2500 files renamed).
• Refactoring of git-tfs internals. During development it became messy at here and there.
• Bugfixing, eliminating unanticipated exceptions etc
  

The question is what do you think is critical right now. My understanding is that refactoring would touch many aspects so we should implement critical things before we start it.
Thoughts?

[matt]

I had started a list a couple months ago.

https://github.com/spraints/git-tfs/wiki/Wishlist

I like the stuff on your list, too. The TFS branch thing is something
I don't have a lot of interest in doing, but I can see that it would
be valuable.

The internals of git-tfs could stand a good refactoring.

My best general idea for addressing performance issues is to swap out
the current interface(s) to git (git# and Process.Start) with libgit2.

Also, it's worth pointing out that I no longer work at a TFS shop, and
I no longer have access to a Windows machine (sweet freedom!), so my
contributions going forward are likely to be more advice and review
than code.

--
Matt

On Jul 19, 11:29 am, Ivan Danilov <van.dani...@gmail.com> wrote:
> Let's write here some info about further development of git-tfs in general
> to coordinate efforts.
>
> Important things I could think of right now are:
>

>    - Handling TFS branches.
>    - Speedup of fetching and checking in. Native TFS client receives

>    versions much faster, especially evident on a large volume of small files or
>    renaming/moving (~1 min in TeamExplorer against 10-20 min in git-tfs in my
>    experience with ~2500 files renamed).

>    - Refactoring of git-tfs internals. During development it became messy at
>    here and there.
>    - Bugfixing, eliminating unanticipated exceptions etc

[Ivan Danilov]

I saw your wishlist and that thread is something like try to figure
out what v1.0 means (your first point) :)

And btw it is not clear for me what do you mean by 'Config params to
support a more concise notation for TFS server URLs and/or a default
server URL'. Could you explain please?

[Matt Burke]

> And btw it is not clear for me what do you mean by 'Config params to
> support a more concise notation for TFS server URLs and/or a default
> server URL'. Could you explain please?

I would like to replace this:

git tfs clone http://tfs:8080/tfs/DefaultCollection $/myproject

with this:

git tfs clone tfs $/myproject

The first notation should always be supported, but the second would be
very convenient. I could see the implementation for the second
notation using either the TFS server registry that VS maintains or git
config variables (e.g. tfs-server.tfs.url) or both. With git vars, it
would be nice to also store username and/or password (e.g.
tfs-server.tfs.username).

--
Matt

[Ivan Danilov]

I see. It seems not hard to do at all.

> --
> You received this message because you are subscribed to the Google Groups "git-tfs-dev" group.
> To post to this group, send email to git-t...@googlegroups.com.
> To unsubscribe from this group, send email to git-tfs-dev...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/git-tfs-dev?hl=en.
>
>

--

Yours sincerely,

Ivan Danilov
 Senior Developer
 Tel. +380937632900

[Jimit]

+1 for handling TFS branches. That's my number 1 pain point right now. I think a lot of ppl would also like to be able handle branching/merging in TFS rather than git.

[Ivan Danilov]

After unshelve and autotag branches are my top priority.

I'm thinking though what route should we take. Probably we should announce 0.12 version first as Dan suggested.
Then... I'd like to try. If branches would not require heavy changes in the codebase - I'll probably add support for them first and will work on refactorings afterwards. Otherwise refactorings will go first because there're many risks to break existing functionality which would be not good obviously :)

[Matt Burke]

Sounds good. Doing 0.12 before going down the branch road is a good
idea. Is everything ready for 0.12?

> --
> You received this message because you are subscribed to the Google Groups
> "git-tfs-dev" group.

> To view this discussion on the web visit
> https://groups.google.com/d/msg/git-tfs-dev/-/dspLE9rv6UoJ.

[Dan Rigby]

If we're worried about stability, could we release a 0.12 build and label it as a test/alpha build? 

Not sure how many downstream consumers git-tfs has, but it might enable us to get more testing and feedback on 0.12 before offering it as a general release.

I've just built the 0.12 sources today and I'm going to use them and see if I encounter any issues (using TFS 2010). 

[Dan Rigby]

Some initial thoughts:

• Unshelve does not appear to be working properly for me. I undeshelved a shelveset from another user into a new branch, but rather then applying the changes to the files it wiped out the contents of the changed files
  
  
• The command help could use a description field above usage that explains what the command does, For example, what does rcheckin do? I can see how to use it, but I don't actually know what it does. It would also be helpful to include a few example usages of the command at the end of the command help for common scenarios, e.g. "git tfs unshelve -u username new_branchname"
  
  
• There's a line break missing between the description for the -V and the -m options for the rcheckin command help.
  
  
• When I fail to specify a branch for the unshelve command, the error message just tells me it's expecting 2 arguments. Perhaps if no branch is specified we should just unshelve onto the present branch? Or maybe we can improve the error message here.

I'm going to try and take a look at some of these when I get home, but I'm not presently in a situation where I can examine the code.

Thanks.

[Matt Burke]

> Unshelve does not appear to be working properly for me. I undeshelved a
> shelveset from another user into a new branch, but rather then applying the
> changes to the files it wiped out the contents of the changed files

As in, the files were deleted? empty? have the exact contents as
represented in the shelveset?

> The command help could use a description field above usage that explains
> what the command does, For example, what does rcheckin do? I can see how to
> use it, but I don't actually know what it does. It would also be helpful to
> include a few example usages of the command at the end of the command help
> for common scenarios, e.g. "git tfs unshelve -u username new_branchname"

A description in the help would be good. I would leave examples for
the wiki. I started documenting the individual commands, but there is
definitely more to add. https://github.com/spraints/git-tfs/wiki

> There's a line break missing between the description for the -V and the -m
> options for the rcheckin command help.

Hopefully this will be resolved by switching to a different option
parser. (Probably NDesk.Options, as discussed here
https://github.com/spraints/git-tfs/issues/53)

> When I fail to specify a branch for the unshelve command, the error message
> just tells me it's expecting 2 arguments. Perhaps if no branch is specified
> we should just unshelve onto the present branch? Or maybe we can improve the
> error message here.

A better error message would be best. Unshelving to the current branch
is ok if the user asks for it explicitly, but because unshelving is
going to produce the desired outcome basically by luck only, I'm
reluctant to have it default to the current branch. Another option
that would be cool is a 'don't commit' option, similar to cherry-pick.

> I'm going to try and take a look at some of these when I get home, but I'm
> not presently in a situation where I can examine the code.

Awesome. It's always good to get more input.

[Ivan Danilov]

Thanks for feedback. I have a suspicion that unshelve problems were caused by one of the latest commits. I've commented that [here](https://github.com/spraints/git-tfs/commit/98ee703bc04950c09fa1924561e552dfb31d1643)

Could you try it from [eae8616](https://github.com/spraints/git-tfs/commit/eae86164d979c3c596d9e4e6259447a37dfd8361) commit please?

[Ivan Danilov]

I can confirm. Now I have the same symptoms - files are just clear. Will try to resolve right now. I definitely was working several commits ago as I did one review with it :)

[Ivan Danilov]

> The command help could use a description field above usage that explains
> what the command does, For example, what does rcheckin do? I can see how to
> use it, but I don't actually know what it does. It would also be helpful to
> include a few example usages of the command at the end of the command help
> for common scenarios, e.g. "git tfs unshelve -u username new_branchname"

A description in the help would be good. I would leave examples for
the wiki. I started documenting the individual commands, but there is
definitely more to add. https://github.com/spraints/git-tfs/wiki

We could take the best of both worlds. As msysgit launches browser with documentation on the command - so could we. Something like --helpdetailed.
As for rcheckin - I've described it in details in the wiki here.

> When I fail to specify a branch for the unshelve command, the error message
> just tells me it's expecting 2 arguments. Perhaps if no branch is specified
> we should just unshelve onto the present branch? Or maybe we can improve the
> error message here.

A better error message would be best. Unshelving to the current branch
is ok if the user asks for it explicitly, but because unshelving is
going to produce the desired outcome basically by luck only, I'm
reluctant to have it default to the current branch. Another option
that would be cool is a 'don't commit' option, similar to cherry-pick.

I've already have more clear error message in local commits. Will push it when fix issue with empty files.

As for unshelve to current branch - I'm in Matt's camp... if you in master branch - moving it with some doubtful changes is not good. Maybe later as option... it is not a priority now.

[Ivan Danilov]

I have to go now. I still think that TemporaryFile is the source of
problems, so pushed
https://github.com/spraints/git-tfs/tree/unshelve-issues branch. It
has somewhat cleared error messages and reverted TemporaryFile back to
MemoryStream.

I'll try to find out what was caused #55 and update the thread.

P.S. Btw, I think we are discussing unshelve problems in the wrong thread :)

> --
> You received this message because you are subscribed to the Google Groups
> "git-tfs-dev" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/git-tfs-dev/-/eX8__EG-Lq0J.

[Dan Rigby]

I actually really like the idea of unshelve having the option to not create a commit and just modifying the current files. It should probably force a commit/discard first of any uncommitted changes like merge/pull does. This would support the workflow I want to use it for really well (code reviews). 

[Ivan Danilov]

Dan, note my complaints about inability of TFS to store shelvesets properly in sibling topic. So, shelve can be used for review, but it is poor man's choice. Though if part of your team is working with "pure" TFS - you have no other choice unfortunately. But I'll try.

[Matt Burke]

One thing that might be possible, but i have no idea how difficult to
implement, would be to treat each change in the shelveset as a
mini-cherry-pick... In other words, we could (theoretically) take each
change, get its base version and the updated contents, generate a
patch for it, then apply the patch to the current version. I'm not
sure how easy it would be to deal with renames, but I think git does
sometimes. An easier approach might be to guess at a base repo
version, based on the base versions from the shelveset, and use that
for the point where the branch gets made.

Of course, I could be just imagining that tfs stores base versions for
changed items in a shelveset.

> --
> You received this message because you are subscribed to the Google Groups
> "git-tfs-dev" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/git-tfs-dev/-/kOWkaSZGnsoJ.

[Ivan Danilov]

For changed items it really does. There're even two methods -
DownloadShelvedFile() and DownloadBaseFile() in the PendingChange.
It is a good suggestion. But not as a hotfix to 0.12 IMO.

--

[Matt Burke]

Definitely not for 0.12. It would be a pretty significant change.

[Ivan Danilov]

I'm almost finished now with shelve-list command (separated it from
unshelve) which will support sorting. I think it could still go to
0.12.

[Ivan Danilov]

Btw, could you review my latest changes in unshelve-issues branch
please? I've reworked TemporaryFile completely. If you're ok with
these changes - I think master should take this route (force push will
be ok IMO). I didn't push it myself so that you can review and maybe
you'll have something to say about it :)

[Ivan Danilov]

Just pushed https://github.com/spraints/git-tfs/commit/e7d31522c6b30b6044b14ed6bff54a92844291f3
New command 'shelve-list' introduced and 'unshelve -l' taken away. Usage is as follows:
    -s, --sort
        (Type: Value required, Value Type:[String])
        How to sort resulting shelves.

    -u, --user
        (Type: Value required, Value Type:[String])
        Shelveset owner (default is the current user; 'all' means all users)

    -f, --full
        (Type: Flag, Value Type:[Boolean])
        Use detailed format.

...ooops, forgot to mention in usage possible values for -s. They are: date, owner, name, comment (only one is supported now).

If -f is specified you'll see something like following:
Name   : my-shelve
Owner  : IDANILOV-SERVER\Administrator
Date   : 03.08.2011 13:30
Comment: comment to shelve

And such block would be for each shelve found.

Will push addendum to usage description now...

[Dan Rigby]

These changes are awesome. 

I tested the unshelve fix earlier today (filestream -> memorystream) and was able to successfully unshelve a couple of different changesets. 

As far as my team goes, I'm presently the only one out of a 20 man dev team using git, everyone else is using pure tfs.

I think I'm slowly winning mind share though (several of the other devs want to try and get started with it). (-:

In our case, we're actually part of a much larger project of about 120 developers worldwide and the tfs server is located in a different country (canada -> us) which we connect to over vpn (slow!). 

The speed improvements and the ability to work when the vpn or tfs server are down are tremendous.

The one frustrating thing left with my workflow is that every time I open one of our solutions I have to immediately go to Visual Studio options and switch the source control provider back to none (or git) as it picks up the SCC bindings in the solution and flips the source control provider to TFS automatically. I haven't figured out a solution for that just yet.

Thanks!

--
You received this message because you are subscribed to the Google Groups "git-tfs-dev" group.

To view this discussion on the web visit https://groups.google.com/d/msg/git-tfs-dev/-/sv1Sjv1ygf0J.

[Ivan Danilov]

Wow, you're just describing my own situation! Except that our command is somewhat smaller.

As for SCC bindings... I did nothing with them. Just went offline and that's all. Alternatively you could just remove TFS settings entirely and specify everything in git's config. Sometimes it asks if I want to go online, but 2010 seems to be more ignorant of it (or maybe it is one of extensions turned that vexing dialog off?).

Btw, did you setup central git repo with updates by time? :)
I typed `git tfs fetch` very long ago))) `git fetch` is enough almost always as my server has fetching TFS every 5 min. Very handy.

[Ivan Danilov]

Dan, I've noticed that you're participating in git# community. I'm curious how the things are going there? I don't see much of activity there recently, and we have suggestions to move from it to something else (also bugs like #42 seems to belong to git# also). What is your opinion on that?

[Ivan Danilov]

I would like to replace this:

    git tfs clone http://tfs:8080/tfs/DefaultCollection $/myproject

with this:

    git tfs clone tfs $/myproject

Done a few minutes ago here. I've implemented checking if passed thing is well-formed absolute Uri. If not - git-tfs tries to recognize it as one of the TFS servers registered in the system.
Note two things:
1) it is working for 2010 studio only now
2) I did investigation of my VS2010 instance with procmon.exe (from SysInternals suite) and found out that it looks at HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\10.0\TeamFoundation\Instances to get list of TFS servers so I implemented precisely the same. I didn't found official docs where these settings can be found so it is possible someone has them elsewhere. By the way it was the reason I didn't implemented such thing for 2008.

So, if someone have VS2008 in use - please write me registry path where you have these settings and I will add the same support for 2008. Or alternatively make it yourselves, there's just three lines need to be changed :)

[Ivan Danilov]

Dan, I've noticed that you're participating in git# community. I'm curious how the things are going there? I don't see much of activity there recently, and we have suggestions to move from it to something else (also bugs like #42 seems to belong to git# also). What is your opinion on that?

Oops. I was inattentive. Just noticed sentence "GitSharp development is currently on hold because we believe that the libgit2 project and its C# bindings libgit2sharp are far more promising to work on. "
Question waived :)

[Matt Burke]

One question... did you try setting the server Name to the short name
to see if the TeamProjectCollection class would do the lookup for you?
I have another project that uses the DomainProjectPicker to pop up a
UI to choose a server, and I think it returns a TeamFoundationServer
object with the Name set to the short name.

> --
> You received this message because you are subscribed to the Google Groups
> "git-tfs-dev" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/git-tfs-dev/-/P-28RjBn1HoJ.

[Ivan Danilov]

TfsTeamProjectCollection has Uri as first parameter or
RegisteredProjectCollection. And it seems the latter is what we need.
No docs situation relieved by Reflector/ILSpy :)
There's 'public static class RegisteredTfsConnections' that do almost
same thing that I did. Will use it, thanks for suggestion Matt.

[Ivan Danilov]

Done in https://github.com/spraints/git-tfs/commit/740c8fc1af9ed088a1dbf68a90e064fffdaec2ca
Still I don't have libs for 2008 so it is still doesn't support the thing. I'll fix it tomorrow from office (there's 2008 installed).

[Ivan Danilov]

I've renamed master to master-old and marked newest version as v0-12-rc after restoring compatibility with VS2008.
To v0-12-rc branch only bugfixes will be committed.

Also I've checked that VS2008 version can understand tfs instance name instead of URL, so no changes seems to be necessary there.

Old branch was not deleted only for historical reasons (and because someone could forked it). It will be deleted in several weeks, so please do not start your work from there.

[Ivan Danilov]

Version 0.12.0 tagged.

[Matt Burke]

I'm not sure why master and master-old are different. It looks like
you basically did the same work on master that I did on master-old, so
I'll go ahead and delete master-old. In the future, I'd prefer to
avoid force-pushing any branches (other than topic branches) on github
so that we don't screw anyone up down stream.

Also, I'd like to rename the v0-12-rc branch back to 0-12-stable, and
use it like you said: just bug fixes, and tagged point releases for
0.12 (e.g. 0.12.1). Then, when master is ready for a release, we can
make it 0.13 or so. I'll bump the version on it now just to
differentiate it.

> --
> You received this message because you are subscribed to the Google Groups
> "git-tfs-dev" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/git-tfs-dev/-/gWCNSpoyohkJ.

[Ivan Danilov]

master is different because I didn't use TemporaryFile class and your latest fixes there but rewritten it from scratch as TemporaryFileStream class. Plus other features down the line.

[Dan Rigby]

I haven't tried the central report, but sounds like an interesting idea. I just do a pull periodically and handle any merge conflicts. I'm pretty isolated in my use of git. For your central repo, where are you running it and how did you setup the automatic pulls? 

In terms of SCC bindings, I set the registry setting that tells to Visual Studio to not bother to ask the server if it's online and only try to go online when I tell it to so I don't get the prompt when opening the solution. The reason it changing the SCC is bothersome to me is when I have the solution open in VS with TFS set as the SCC, I experience weird perf problems. 

In my ideal world I could just tell Visual Studio to just ignore the bindings and stay set to none.

--
You received this message because you are subscribed to the Google Groups "git-tfs-dev" group.

To view this discussion on the web visit https://groups.google.com/d/msg/git-tfs-dev/-/tpZVFDpSkhEJ.

[Dan Rigby]

I wouldn't call myself active, my main contribution to the project was setting up the irc channel, getting the twitter account, and porting some unit tests. 

Prior to the libgit2 announcement, development on gitsharp had slowed anyway mostly because ngit made large parts of the project irrelevant since it was an automated port of jgit while git# was a manual one. 

The CLI work is still relevant, but I think libgit2 is really the way forward and consensus seems to be building behind it.

Also, sorry for the slow replies. I've been on vacation for the last week in Montreal. Such a lovely city.

On Sat, Aug 6, 2011 at 2:31 PM, Ivan Danilov <van.d...@gmail.com> wrote:

Dan, I've noticed that you're participating in git# community. I'm curious how the things are going there? I don't see much of activity there recently, and we have suggestions to move from it to something else (also bugs like #42 seems to belong to git# also). What is your opinion on that?

--
You received this message because you are subscribed to the Google Groups "git-tfs-dev" group.

To view this discussion on the web visit https://groups.google.com/d/msg/git-tfs-dev/-/vPiqQ-XKe00J.

[Ivan Danilov]

I set up bare repository at our project server. Currently we use just shared folder (it is easier, you only have to map server's share to some local drive letter), but it is not complex to get normal git via OpenSSH too. Besides that shared "project.git" folder I've made normal git repository (non-shared) and named it "project-working". With Windows Task Manager it is easy to set up periodical (we have every 5 minutes) script execution, which always stays at master and runs "git tfs pull". If something was pulled - the script runs "git push origin master" to central repository. Effectively thus way we have master branch in central repository mirroring TFS state.

When I need updates on my dev box - usually "git fetch" is enough.
When I need to push something to TFS - most of the time I use "rcheckin", but from a script also. The script executes "git tfs rcheckin" first, and if something was checked in - "git push central HEAD:master" afterwards (just to minimize round-trips to TFS).

The only problem I saw so far is... suppose you have chain of commits locally, and some of them are merge commits. And you want to rcheckin them all at once preserving merges (rcheckin tries to preserve them) - e.g. you want detailed history of your feature branch stays in git while TFS receives only squashed commit - then it is fairly possible that periodic updates on central repository will try to pull new commit from TFS before you finished your rcheckin. TFS doesn't know anything about this changeset being merge-commit, so it pulls straight history line. And then you have to resolve the situation manually, which could be tiresome, especially if someone caught up the chance to sync with central repository while it was in incorrect state :(
The solution could be to make "pull script" know who works with git-tfs and ignore their changesets to TFS assuming that these should find their way to git central repository themselves, but it is somewhat fragile also.

[Jimit]

Is support for git submodules anywhere on the radar?

[Ivan Danilov]

Not from me at least. I'm still planning to do branches support. Sorry for delay - tight schedules with main employment...
What will be here after branches - I don't know at all :)

[Jimit]

Support for TFS branches is definitely the more urgent need, in my opinion - submodules would just be a nice to have.