Out Of Memory error

[Valeriy Koval]

Hi,
git-tfs 0.27, very big TFS repo (many years, 900k changesets, many branches)
I'm trying to clone a single branch. It's very big  (50Gb).
To decrease  amount of data I use:
1. init with gitignore to skip most of the folders in the root to work with few folders only
2. fetch for last chageset - 1
In this case I'm getting Out Of Memory error:
...
2018-01-18 18:42:08.0892 [Debug] get [C8XXXXX]$/XXXXXXXXXX/XX/XXXXXX.zip
2018-01-18 18:44:07.4450 [Info] 1 objects created...
2018-01-18 18:50:17.2748 [Debug] TFS Workspace 'git-tfs-a5907ff8-2bce-4386-873b-fa63302cdf9c;Valeriy Koval' was removed.
2018-01-18 18:50:17.2748 [Debug] Cleaning...
2018-01-18 18:53:24.2724 [Debug] System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> LibGit2Sharp.LibGit2SharpException: Out of memory
   at LibGit2Sharp.Core.Ensure.HandleError(Int32 result)
   at LibGit2Sharp.Core.Proxy.git_object_lookup(RepositoryHandle repo, ObjectId id, GitObjectType type)
   at LibGit2Sharp.Repository.LookupInternal(ObjectId id, GitObjectType type, FilePath knownPath)
   at LibGit2Sharp.Repository.Lookup(ObjectId id, ObjectType type)
   at LibGit2Sharp.RepositoryExtensions.Lookup[T](IRepository repository, ObjectId id)
   at LibGit2Sharp.ObjectDatabase.CreateBlob(String path)
   at LibGit2Sharp.TreeEntryDefinition.<>c__DisplayClassc.<TransientBlobFrom>b__a(ObjectDatabase odb)
   at LibGit2Sharp.TreeDefinition.Build(Repository repository)
   at LibGit2Sharp.TreeDefinition.WrapAllTreeDefinitions(Repository repository)
   at LibGit2Sharp.TreeDefinition.Build(Repository repository)
   at LibGit2Sharp.TreeDefinition.WrapAllTreeDefinitions(Repository repository)
   at LibGit2Sharp.TreeDefinition.Build(Repository repository)
   at LibGit2Sharp.TreeDefinition.WrapAllTreeDefinitions(Repository repository)
   at LibGit2Sharp.TreeDefinition.Build(Repository repository)
   at LibGit2Sharp.TreeDefinition.WrapAllTreeDefinitions(Repository repository)
   at LibGit2Sharp.TreeDefinition.Build(Repository repository)
   at LibGit2Sharp.ObjectDatabase.CreateTree(TreeDefinition treeDefinition)
   at Sep.Git.Tfs.Core.GitTreeBuilder.GetTree()
   at Sep.Git.Tfs.Core.GitTfsRemote.<>c__DisplayClass137_0.<CopyTree>b__0(ITfsWorkspace workspace)
   at Sep.Git.Tfs.VsCommon.TfsHelperBase.WithWorkspace(String localDirectory, IGitTfsRemote remote, TfsChangesetInfo versionToFetch, Action`1 action)
   at Sep.Git.Tfs.Core.GitTfsRemote.WithWorkspace(TfsChangesetInfo parentChangeset, Action`1 action)
   at Sep.Git.Tfs.Core.GitTfsRemote.CopyTree(String lastCommit, ITfsChangeset changeset)
   at Sep.Git.Tfs.Core.GitTfsRemote.quickFetch(ITfsChangeset changeset)
   at Sep.Git.Tfs.Core.GitTfsRemote.QuickFetch(Int32 changesetId)
   at Sep.Git.Tfs.Commands.Fetch.DoFetch(IGitTfsRemote remote, Boolean stopOnFailMergeCommit)
   at Sep.Git.Tfs.Commands.Fetch.FetchRemote(Boolean stopOnFailMergeCommit, IGitTfsRemote remote)
   at Sep.Git.Tfs.Commands.Fetch.Run(Boolean stopOnFailMergeCommit, String[] args)
   at Sep.Git.Tfs.Commands.Fetch.Run()
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Sep.Git.Tfs.Util.GitTfsCommandRunner.Run(GitTfsCommand command, IList`1 args)
   at Sep.Git.Tfs.GitTfs.Main(GitTfsCommand command, IList`1 unparsedArgs)
   at Sep.Git.Tfs.GitTfs.Run(IList`1 args)
   at Sep.Git.Tfs.Program.MainCore(String[] args)
   at Sep.Git.Tfs.Program.Main(String[] args)
2018-01-18 18:53:24.2724 [Error] Out of memory

Can someone help me with it OR just explain the proper way how to do what I described above?

Details:

TFS folder structure:

$/<branch_name>/PROJECT
|________folderA
|________folderB
|________folderC

gitignore to work with folderB only:
/folderA/
/folderC/

Commands:
1. git tfs init -d <a href="http://tfs:8080/tfs/collection" $/<branch_name>/PROJECT --username=<email> --password="pwd" PRJ_IN_GIT --gitignore="c:\git-tfs\gitignore"
2. cd PRJ_IN_GIT ; git tfs fetch --batch-size=100 --from=9xxxxx


As result I want to see folderB in PRJ_IN_GIT:
PRJ_IN_GIT
|________folderB

[Philippe Miossec]

Did you test just the released version?
Because I have made a change (not yet release) to circumvent OutOfMemory crashes.
You could download the dev version here: https://ci.appveyor.com/project/pmiossec/git-tfs-v2qcm/build/0.27.55.638/artifacts

Also, if you have problems of memory, prefer small batch (even of size 1. ` --batch-size=1`)

And, I don't think it's an easy thing to migrate such an history.

Perhaps you should consider keeping the history on TFVC as an archive and migrate just a small history (count in days or weeks depending on your needs).

git tfs clone -d http://tfs:8080/tfs/collection $/<branch_name>/PROJECT --username=<email> --password="pwd" PRJ_IN_GIT --gitignore="c:\git-tfs\gitignore" --batch-size=1 --from=CHANGESET_ID


I hope it will help.

[Valeriy Koval]

Thank you for response.

It does work with 0.28 build.