updatemeta: correctly handle empty metadata
The group you are posting to is a
Usenet group . Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
From:
Jun Fang <junf... @fb.com>
Date: Mon, 23 Jul 2012 14:20:20 -0700
Local: Mon, Jul 23 2012 5:20 pm
Subject: [PATCH] updatemeta: correctly handle empty metadata
# HG changeset patch
# User Jun Fang <junf... @fb.com>
# Date 1343077239 25200
# Node ID 3a31f765153dbbb599c042feab5b644935703378
# Parent 761a8713450164a5754adff55223d6f08e37cfab
updatemeta: correctly handle empty metadata
When the repo metadata is empty (just created/cloned but not populated yet),
there is no lastpulled file or the revmap doesn't have a last entry with hash.
Currently "hg svn updatemeta" would crash due to unexpected exception.
See issue reported at https://bitbucket.org/durin42/hgsubversion/issue/356/updatemeta-crash...
The fix is to check the existence of lastpulled file and the hash entry in
revmap, as part of "hg svn updatemeta" command. When they are not present,
do a rebuildmetadata.
diff --git a/hgsubversion/svncommands.py b/hgsubversion/svncommands.py
--- a/hgsubversion/svncommands.py
+++ b/hgsubversion/svncommands.py
@@ -64,12 +64,19 @@
if partial:
try:
youngestpath = os.path.join(svnmetadir, 'lastpulled')
- youngest = int(util.load_string(youngestpath).strip())
- sofar = list(maps.RevMap.readmapfile(repo))
- lasthash = sofar[-1].split(' ', 2)[1]
- startrev = repo[lasthash].rev() + 1
- branchinfo = pickle.load(open(os.path.join(svnmetadir,
- 'branch_info')))
+ foundpartialinfo = False
+ if os.path.exists(youngestpath):
+ youngest = int(util.load_string(youngestpath).strip())
+ sofar = list(maps.RevMap.readmapfile(repo))
+ if sofar and len(sofar[-1].split(' ', 2)) > 1:
+ lasthash = sofar[-1].split(' ', 2)[1]
+ startrev = repo[lasthash].rev() + 1
+ branchinfo = pickle.load(open(os.path.join(svnmetadir,
+ 'branch_info')))
+ foundpartialinfo = True
+ if not foundpartialinfo:
+ ui.status('missing some metadata -- doing a full rebuild\n')
+ partial = False
except IOError, err:
if err.errno != errno.ENOENT:
raise
diff --git a/tests/test_rebuildmeta.py b/tests/test_rebuildmeta.py
--- a/tests/test_rebuildmeta.py
+++ b/tests/test_rebuildmeta.py
@@ -45,6 +45,20 @@
origchildren = getattr(context.changectx, 'children')
extensions.wrapfunction(context.changectx, 'children', failfn)
+ # test updatemeta on an empty repo
+ try:
+ svncommands.updatemeta(u, dest,
+ args=[test_util.fileurl(repo_path +
+ subdir), ])
+ finally:
+ # remove the wrapper
+ context.changectx.children = origchildren
+
+ self._run_assertions(name, stupid, single, src, dest, u)
+
+ # insert a wrapper that prevents calling changectx.children()
+ extensions.wrapfunction(context.changectx, 'children', failfn)
+
try:
svncommands.rebuildmeta(u, dest,
args=[test_util.fileurl(repo_path +
You must
Sign in before you can post messages.
You do not have the permission required to post.
From:
Augie Fackler <r... @durin42.com>
Date: Thu, 26 Jul 2012 18:06:47 -0500
Local: Thurs, Jul 26 2012 7:06 pm
Subject: Re: [PATCH] updatemeta: correctly handle empty metadata
On Jul 23, 2012, at 4:20 PM, Jun Fang wrote:
> # HG changeset patch
> # User Jun Fang <junf
... @fb.com>
> # Date 1343077239 25200
> # Node ID 3a31f765153dbbb599c042feab5b644935703378
> # Parent 761a8713450164a5754adff55223d6f08e37cfab
> updatemeta: correctly handle empty metadata
Implementation looks reasonable, tests are still concerning me.
> + # test updatemeta on an empty repo
> + try:
> + svncommands.updatemeta(u, dest,
> + args=[test_util.fileurl(repo_path +
> + subdir), ])
> + finally:
> + # remove the wrapper
> + context.changectx.children = origchildren
> +
> + self._run_assertions(name, stupid, single, src, dest, u)
> +
> + # insert a wrapper that prevents calling changectx.children()
> + extensions.wrapfunction(context.changectx, 'children', failfn)
> +
Sorry I missed this before: doesn't this rebuild dest, so that the following test assertions are vacuously succeeding? I think it might be best to have some duplicated test method code here so these two cases are obviously tested and working independently.
> try:
> svncommands.rebuildmeta(u, dest,
> args=[test_util.fileurl(repo_path +
You must
Sign in before you can post messages.
You do not have the permission required to post.
From:
Augie Fackler <r... @durin42.com>
Date: Mon, 6 Aug 2012 12:48:26 -0500
Local: Mon, Aug 6 2012 1:48 pm
Subject: Re: [PATCH] updatemeta: correctly handle empty metadata
Can you start a new thread using 'hg email' so the patch is inline?
Thanks,
Augie
On Mon, Aug 6, 2012 at 12:46 PM, Jun Fang <junf
... @fb.com> wrote:
> Hi Augie,
> I created a new unit test, test_updatemeta.py, to test the updatemeta
> command specifically.
> Here is the new change.
> Please take a look.
> Thanks,
> Jun
> On 7/26/12 4:06 PM, "Augie Fackler" <r... @durin42.com> wrote:
>>On Jul 23, 2012, at 4:20 PM, Jun Fang wrote:
>>> # HG changeset patch
>>> # User Jun Fang <junf... @fb.com>
>>> # Date 1343077239 25200
>>> # Node ID 3a31f765153dbbb599c042feab5b644935703378
>>> # Parent 761a8713450164a5754adff55223d6f08e37cfab
>>> updatemeta: correctly handle empty metadata
>>Implementation looks reasonable, tests are still concerning me.
>>> + # test updatemeta on an empty repo
>>> + try:
>>> + svncommands.updatemeta(u, dest,
>>> + args=[test_util.fileurl(repo_path +
>>> + subdir), ])
>>> + finally:
>>> + # remove the wrapper
>>> + context.changectx.children = origchildren
>>> +
>>> + self._run_assertions(name, stupid, single, src, dest, u)
>>> +
>>> + # insert a wrapper that prevents calling changectx.children()
>>> + extensions.wrapfunction(context.changectx, 'children', failfn)
>>> +
>>Sorry I missed this before: doesn't this rebuild dest, so that the
>>following test assertions are vacuously succeeding? I think it might be
>>best to have some duplicated test method code here so these two cases are
>>obviously tested and working independently.
>>> try:
>>> svncommands.rebuildmeta(u, dest,
>>> args=[test_util.fileurl(repo_path +
You must
Sign in before you can post messages.
You do not have the permission required to post.
From:
Jun Fang <junf... @fb.com>
Date: Mon, 6 Aug 2012 17:46:31 +0000
Local: Mon, Aug 6 2012 1:46 pm
Subject: Re: [PATCH] updatemeta: correctly handle empty metadata
Hi Augie,
I created a new unit test, test_updatemeta.py, to test the updatemeta
command specifically.
Here is the new change.
Please take a look.
Thanks,
Jun
On 7/26/12 4:06 PM, "Augie Fackler" <r... @durin42.com> wrote:
>On Jul 23, 2012, at 4:20 PM, Jun Fang wrote:
>> # HG changeset patch
>> # User Jun Fang <junf... @fb.com>
>> # Date 1343077239 25200
>> # Node ID 3a31f765153dbbb599c042feab5b644935703378
>> # Parent 761a8713450164a5754adff55223d6f08e37cfab
>> updatemeta: correctly handle empty metadata
>Implementation looks reasonable, tests are still concerning me.
>> + # test updatemeta on an empty repo
>> + try:
>> + svncommands.updatemeta(u, dest,
>> + args=[test_util.fileurl(repo_path +
>> + subdir), ])
>> + finally:
>> + # remove the wrapper
>> + context.changectx.children = origchildren
>> +
>> + self._run_assertions(name, stupid, single, src, dest, u)
>> +
>> + # insert a wrapper that prevents calling changectx.children()
>> + extensions.wrapfunction(context.changectx, 'children', failfn)
>> +
>Sorry I missed this before: doesn't this rebuild dest, so that the
>following test assertions are vacuously succeeding? I think it might be
>best to have some duplicated test method code here so these two cases are
>obviously tested and working independently.
>> try:
>> svncommands.rebuildmeta(u, dest,
>> args=[test_util.fileurl(repo_path +
You must
Sign in before you can post messages.
You do not have the permission required to post.
From:
Jun Fang <junf... @fb.com>
Date: Mon, 06 Aug 2012 10:50:59 -0700
Local: Mon, Aug 6 2012 1:50 pm
Subject: [PATCH] updatemeta: correctly handle empty metadata
# HG changeset patch
# User Jun Fang <junf... @fb.com>
# Date 1344274551 25200
# Node ID 6b7ac659c85574bb819650848af501c95b825baa
# Parent 8faa91951bb10b580cbfc2ad065cd26aae47c618
updatemeta: correctly handle empty metadata
When the repo metadata is empty (just created/cloned but not populated yet),
there is no lastpulled file or the revmap doesn't have a last entry with hash.
Currently "hg svn updatemeta" would crash due to unexpected exception.
See issue reported at https://bitbucket.org/durin42/hgsubversion/issue/356/updatemeta-crash...
The fix is to check the existence of lastpulled file and the hash entry in
revmap, as part of "hg svn updatemeta" command. When they are not present,
do a rebuildmetadata.
Also added a new unit test, test_updatemeta.py.
diff --git a/hgsubversion/svncommands.py b/hgsubversion/svncommands.py
--- a/hgsubversion/svncommands.py
+++ b/hgsubversion/svncommands.py
@@ -64,12 +64,19 @@
if partial:
try:
youngestpath = os.path.join(svnmetadir, 'lastpulled')
- youngest = int(util.load_string(youngestpath).strip())
- sofar = list(maps.RevMap.readmapfile(repo))
- lasthash = sofar[-1].split(' ', 2)[1]
- startrev = repo[lasthash].rev() + 1
- branchinfo = pickle.load(open(os.path.join(svnmetadir,
- 'branch_info')))
+ foundpartialinfo = False
+ if os.path.exists(youngestpath):
+ youngest = int(util.load_string(youngestpath).strip())
+ sofar = list(maps.RevMap.readmapfile(repo))
+ if sofar and len(sofar[-1].split(' ', 2)) > 1:
+ lasthash = sofar[-1].split(' ', 2)[1]
+ startrev = repo[lasthash].rev() + 1
+ branchinfo = pickle.load(open(os.path.join(svnmetadir,
+ 'branch_info')))
+ foundpartialinfo = True
+ if not foundpartialinfo:
+ ui.status('missing some metadata -- doing a full rebuild\n')
+ partial = False
except IOError, err:
if err.errno != errno.ENOENT:
raise
diff --git a/tests/run.py b/tests/run.py
--- a/tests/run.py
+++ b/tests/run.py
@@ -32,6 +32,7 @@
import test_template_keywords
import test_utility_commands
import test_unaffected_core
+ import test_updatemeta
import test_urls
sys.path.append(os.path.dirname(__file__))
diff --git a/tests/test_updatemeta.py b/tests/test_updatemeta.py
new file mode 100644
--- /dev/null
+++ b/tests/test_updatemeta.py
@@ -0,0 +1,81 @@
+import test_util
+
+import os
+import pickle
+import unittest
+import test_rebuildmeta
+
+from mercurial import context
+from mercurial import extensions
+from mercurial import hg
+from mercurial import ui
+
+from hgsubversion import svncommands
+from hgsubversion import svnmeta
+
+
+
+def _do_case(self, name, stupid, single):
+ subdir = test_util.subdir.get(name, '')
+ layout = 'auto'
+ if single:
+ layout = 'single'
+ repo, repo_path = self.load_and_fetch(name, subdir=subdir, stupid=stupid,
+ layout=layout)
+ assert len(self.repo) > 0
+ wc2_path = self.wc_path + '_clone'
+ u = ui.ui()
+ src, dest = test_util.hgclone(u, self.wc_path, wc2_path, update=False)
+ src = getattr(src, 'local', lambda: src)()
+ dest = getattr(dest, 'local', lambda: dest)()
+
+ # insert a wrapper that prevents calling changectx.children()
+ def failfn(orig, ctx):
+ self.fail('calling %s is forbidden; it can cause massive slowdowns '
+ 'when rebuilding large repositories' % orig)
+
+ origchildren = getattr(context.changectx, 'children')
+ extensions.wrapfunction(context.changectx, 'children', failfn)
+
+ # test updatemeta on an empty repo
+ try:
+ svncommands.updatemeta(u, dest,
+ args=[test_util.fileurl(repo_path +
+ subdir), ])
+ finally:
+ # remove the wrapper
+ context.changectx.children = origchildren
+
+ self._run_assertions(name, stupid, single, src, dest, u)
+
+
+def _run_assertions(self, name, stupid, single, src, dest, u):
+ test_rebuildmeta._run_assertions(self, name, stupid, single, src, dest, u)
+
+
+skip = set([
+ 'project_root_not_repo_root.svndump',
+ 'corrupt.svndump',
+])
+
+attrs = {'_do_case': _do_case,
+ '_run_assertions': _run_assertions,
+ }
+for case in [f for f in os.listdir(test_util.FIXTURES) if f.endswith('.svndump')]:
+ # this fixture results in an empty repository, don't use it
+ if case in skip:
+ continue
+ bname = 'test_' + case[:-len('.svndump')]
+ attrs[bname] = test_rebuildmeta.buildmethod(case, bname, False, False)
+ name = bname + '_stupid'
+ attrs[name] = test_rebuildmeta.buildmethod(case, name, True, False)
+ name = bname + '_single'
+ attrs[name] = test_rebuildmeta.buildmethod(case, name, False, True)
+
+UpdateMetaTests = type('UpdateMetaTests', (test_util.TestBase,), attrs)
+
+
+def suite():
+ all_tests = [unittest.TestLoader().loadTestsFromTestCase(UpdateMetaTests),
+ ]
+ return unittest.TestSuite(all_tests)
You must
Sign in before you can post messages.
You do not have the permission required to post.
From:
Jun Fang <junf... @fb.com>
Date: Tue, 14 Aug 2012 20:56:05 +0000
Local: Tues, Aug 14 2012 4:56 pm
Subject: Re: [PATCH] updatemeta: correctly handle empty metadata
Hi Augie,
Did you get a chance to take a look? Please let me know if there is still
issue with the unit test.
Thanks,
Jun
On 8/6/12 10:50 AM, "Jun Fang" <junf... @fb.com> wrote:
># HG changeset patch
># User Jun Fang <junf
... @fb.com>
># Date 1344274551 25200
># Node ID 6b7ac659c85574bb819650848af501c95b825baa
># Parent 8faa91951bb10b580cbfc2ad065cd26aae47c618
>updatemeta: correctly handle empty metadata
>When the repo metadata is empty (just created/cloned but not populated
>yet),
>there is no lastpulled file or the revmap doesn't have a last entry with
>hash.
>Currently "hg svn updatemeta" would crash due to unexpected exception.
>See issue reported at
>https://bitbucket.org/durin42/hgsubversion/issue/356/updatemeta-crash...
>th-traceback-if-there
>The fix is to check the existence of lastpulled file and the hash entry in
>revmap, as part of "hg svn updatemeta" command. When they are not present,
>do a rebuildmetadata.
>Also added a new unit test, test_updatemeta.py.
>diff --git a/hgsubversion/svncommands.py b/hgsubversion/svncommands.py
>--- a/hgsubversion/svncommands.py
>+++ b/hgsubversion/svncommands.py
>@@ -64,12 +64,19 @@
> if partial:
> try:
> youngestpath = os.path.join(svnmetadir, 'lastpulled')
>- youngest = int(util.load_string(youngestpath).strip())
>- sofar = list(maps.RevMap.readmapfile(repo))
>- lasthash = sofar[-1].split(' ', 2)[1]
>- startrev = repo[lasthash].rev() + 1
>- branchinfo = pickle.load(open(os.path.join(svnmetadir,
>- 'branch_info')))
>+ foundpartialinfo = False
>+ if os.path.exists(youngestpath):
>+ youngest = int(util.load_string(youngestpath).strip())
>+ sofar = list(maps.RevMap.readmapfile(repo))
>+ if sofar and len(sofar[-1].split(' ', 2)) > 1:
>+ lasthash = sofar[-1].split(' ', 2)[1]
>+ startrev = repo[lasthash].rev() + 1
>+ branchinfo =
>pickle.load(open(os.path.join(svnmetadir,
>+ >'branch_info')))
>+ foundpartialinfo = True
>+ if not foundpartialinfo:
>+ ui.status('missing some metadata -- doing a full
>rebuild\n')
>+ partial = False
> except IOError, err:
> if err.errno != errno.ENOENT:
> raise
>diff --git a/tests/run.py b/tests/run.py
>--- a/tests/run.py
>+++ b/tests/run.py
>@@ -32,6 +32,7 @@
> import test_template_keywords
> import test_utility_commands
> import test_unaffected_core
>+ import test_updatemeta
> import test_urls
> sys.path.append(os.path.dirname(__file__))
>diff --git a/tests/test_updatemeta.py b/tests/test_updatemeta.py
>new file mode 100644
>--- /dev/null
>+++ b/tests/test_updatemeta.py
>@@ -0,0 +1,81 @@
>+import test_util
>+
>+import os
>+import pickle
>+import unittest
>+import test_rebuildmeta
>+
>+from mercurial import context
>+from mercurial import extensions
>+from mercurial import hg
>+from mercurial import ui
>+
>+from hgsubversion import svncommands
>+from hgsubversion import svnmeta
>+
>+
>+
>+def _do_case(self, name, stupid, single):
>+ subdir = test_util.subdir.get(name, '')
>+ layout = 'auto'
>+ if single:
>+ layout = 'single'
>+ repo, repo_path = self.load_and_fetch(name, subdir=subdir,
>stupid=stupid,
>+ layout=layout)
>+ assert len(self.repo) > 0
>+ wc2_path = self.wc_path + '_clone'
>+ u = ui.ui()
>+ src, dest = test_util.hgclone(u, self.wc_path, wc2_path,
>update=False)
>+ src = getattr(src, 'local', lambda: src)()
>+ dest = getattr(dest, 'local', lambda: dest)()
>+
>+ # insert a wrapper that prevents calling changectx.children()
>+ def failfn(orig, ctx):
>+ self.fail('calling %s is forbidden; it can cause massive
>slowdowns '
>+ 'when rebuilding large repositories' % orig)
>+
>+ origchildren = getattr(context.changectx, 'children')
>+ extensions.wrapfunction(context.changectx, 'children', failfn)
>+
>+ # test updatemeta on an empty repo
>+ try:
>+ svncommands.updatemeta(u, dest,
>+ args=[test_util.fileurl(repo_path +
>+ subdir), ])
>+ finally:
>+ # remove the wrapper
>+ context.changectx.children = origchildren
>+
>+ self._run_assertions(name, stupid, single, src, dest, u)
>+
>+
>+def _run_assertions(self, name, stupid, single, src, dest, u):
>+ test_rebuildmeta._run_assertions(self, name, stupid, single, src,
>dest, u)
>+
>+
>+skip = set([
>+ 'project_root_not_repo_root.svndump',
>+ 'corrupt.svndump',
>+])
>+
>+attrs = {'_do_case': _do_case,
>+ '_run_assertions': _run_assertions,
>+ }
>+for case in [f for f in os.listdir(test_util.FIXTURES) if
>f.endswith('.svndump')]:
>+ # this fixture results in an empty repository, don't use it
>+ if case in skip:
>+ continue
>+ bname = 'test_' + case[:-len('.svndump')]
>+ attrs[bname] = test_rebuildmeta.buildmethod(case, bname, False,
>False)
>+ name = bname + '_stupid'
>+ attrs[name] = test_rebuildmeta.buildmethod(case, name, True, False)
>+ name = bname + '_single'
>+ attrs[name] = test_rebuildmeta.buildmethod(case, name, False, True)
>+
>+UpdateMetaTests = type('UpdateMetaTests', (test_util.TestBase,), attrs)
>+
>+
>+def suite():
>+ all_tests = >[unittest.TestLoader().loadTestsFromTestCase(UpdateMetaTests),
>+ ]
>+ return unittest.TestSuite(all_tests)
You must
Sign in before you can post messages.
You do not have the permission required to post.
From:
Augie Fackler <r... @durin42.com>
Date: Tue, 14 Aug 2012 16:15:29 -0500
Local: Tues, Aug 14 2012 5:15 pm
Subject: Re: [PATCH] updatemeta: correctly handle empty metadata
backlogged, should have time friday. sorry about the wait.
On Aug 14, 2012, at 3:56 PM, Jun Fang wrote:
> Hi Augie,
> Did you get a chance to take a look? Please let me know if there is still
> issue with the unit test.
> Thanks,
> Jun
> On 8/6/12 10:50 AM, "Jun Fang" <junf... @fb.com> wrote:
>> # HG changeset patch
>> # User Jun Fang <junf... @fb.com>
>> # Date 1344274551 25200
>> # Node ID 6b7ac659c85574bb819650848af501c95b825baa
>> # Parent 8faa91951bb10b580cbfc2ad065cd26aae47c618
>> updatemeta: correctly handle empty metadata
>> When the repo metadata is empty (just created/cloned but not populated
>> yet),
>> there is no lastpulled file or the revmap doesn't have a last entry with
>> hash.
>> Currently "hg svn updatemeta" would crash due to unexpected exception.
>> See issue reported at
>> https://bitbucket.org/durin42/hgsubversion/issue/356/updatemeta-crash...
>> th-traceback-if-there
>> The fix is to check the existence of lastpulled file and the hash entry in
>> revmap, as part of "hg svn updatemeta" command. When they are not present,
>> do a rebuildmetadata.
>> Also added a new unit test, test_updatemeta.py.
>> diff --git a/hgsubversion/svncommands.py b/hgsubversion/svncommands.py
>> --- a/hgsubversion/svncommands.py
>> +++ b/hgsubversion/svncommands.py
>> @@ -64,12 +64,19 @@
>> if partial:
>> try:
>> youngestpath = os.path.join(svnmetadir, 'lastpulled')
>> - youngest = int(util.load_string(youngestpath).strip())
>> - sofar = list(maps.RevMap.readmapfile(repo))
>> - lasthash = sofar[-1].split(' ', 2)[1]
>> - startrev = repo[lasthash].rev() + 1
>> - branchinfo = pickle.load(open(os.path.join(svnmetadir,
>> - 'branch_info')))
>> + foundpartialinfo = False
>> + if os.path.exists(youngestpath):
>> + youngest = int(util.load_string(youngestpath).strip())
>> + sofar = list(maps.RevMap.readmapfile(repo))
>> + if sofar and len(sofar[-1].split(' ', 2)) > 1:
>> + lasthash = sofar[-1].split(' ', 2)[1]
>> + startrev = repo[lasthash].rev() + 1
>> + branchinfo =
>> pickle.load(open(os.path.join(svnmetadir,
>> + >> 'branch_info')))
>> + foundpartialinfo = True
>> + if not foundpartialinfo:
>> + ui.status('missing some metadata -- doing a full
>> rebuild\n')
>> + partial = False
>> except IOError, err:
>> if err.errno != errno.ENOENT:
>> raise
>> diff --git a/tests/run.py b/tests/run.py
>> --- a/tests/run.py
>> +++ b/tests/run.py
>> @@ -32,6 +32,7 @@
>> import test_template_keywords
>> import test_utility_commands
>> import test_unaffected_core
>> + import test_updatemeta
>> import test_urls
>> sys.path.append(os.path.dirname(__file__))
>> diff --git a/tests/test_updatemeta.py b/tests/test_updatemeta.py
>> new file mode 100644
>> --- /dev/null
>> +++ b/tests/test_updatemeta.py
>> @@ -0,0 +1,81 @@
>> +import test_util
>> +
>> +import os
>> +import pickle
>> +import unittest
>> +import test_rebuildmeta
>> +
>> +from mercurial import context
>> +from mercurial import extensions
>> +from mercurial import hg
>> +from mercurial import ui
>> +
>> +from hgsubversion import svncommands
>> +from hgsubversion import svnmeta
>> +
>> +
>> +
>> +def _do_case(self, name, stupid, single):
>> + subdir = test_util.subdir.get(name, '')
>> + layout = 'auto'
>> + if single:
>> + layout = 'single'
>> + repo, repo_path = self.load_and_fetch(name, subdir=subdir,
>> stupid=stupid,
>> + layout=layout)
>> + assert len(self.repo) > 0
>> + wc2_path = self.wc_path + '_clone'
>> + u = ui.ui()
>> + src, dest = test_util.hgclone(u, self.wc_path, wc2_path,
>> update=False)
>> + src = getattr(src, 'local', lambda: src)()
>> + dest = getattr(dest, 'local', lambda: dest)()
>> +
>> + # insert a wrapper that prevents calling changectx.children()
>> + def failfn(orig, ctx):
>> + self.fail('calling %s is forbidden; it can cause massive
>> slowdowns '
>> + 'when rebuilding large repositories' % orig)
>> +
>> + origchildren = getattr(context.changectx, 'children')
>> + extensions.wrapfunction(context.changectx, 'children', failfn)
>> +
>> + # test updatemeta on an empty repo
>> + try:
>> + svncommands.updatemeta(u, dest,
>> + args=[test_util.fileurl(repo_path +
>> + subdir), ])
>> + finally:
>> + # remove the wrapper
>> + context.changectx.children = origchildren
>> +
>> + self._run_assertions(name, stupid, single, src, dest, u)
>> +
>> +
>> +def _run_assertions(self, name, stupid, single, src, dest, u):
>> + test_rebuildmeta._run_assertions(self, name, stupid, single, src,
>> dest, u)
>> +
>> +
>> +skip = set([
>> + 'project_root_not_repo_root.svndump',
>> + 'corrupt.svndump',
>> +])
>> +
>> +attrs = {'_do_case': _do_case,
>> + '_run_assertions': _run_assertions,
>> + }
>> +for case in [f for f in os.listdir(test_util.FIXTURES) if
>> f.endswith('.svndump')]:
>> + # this fixture results in an empty repository, don't use it
>> + if case in skip:
>> + continue
>> + bname = 'test_' + case[:-len('.svndump')]
>> + attrs[bname] = test_rebuildmeta.buildmethod(case, bname, False,
>> False)
>> + name = bname + '_stupid'
>> + attrs[name] = test_rebuildmeta.buildmethod(case, name, True, False)
>> + name = bname + '_single'
>> + attrs[name] = test_rebuildmeta.buildmethod(case, name, False, True)
>> +
>> +UpdateMetaTests = type('UpdateMetaTests', (test_util.TestBase,), attrs)
>> +
>> +
>> +def suite():
>> + all_tests = >> [unittest.TestLoader().loadTestsFromTestCase(UpdateMetaTests),
>> + ]
>> + return unittest.TestSuite(all_tests)
You must
Sign in before you can post messages.
You do not have the permission required to post.
From:
Augie Fackler <r... @durin42.com>
Date: Tue, 28 Aug 2012 08:09:26 -0500
Local: Tues, Aug 28 2012 9:09 am
Subject: Re: [PATCH] updatemeta: correctly handle empty metadata
queued, thanks
On Aug 6, 2012, at 12:50 PM, Jun Fang wrote:
> # HG changeset patch
> # User Jun Fang <junf
... @fb.com>
> # Date 1344274551 25200
> # Node ID 6b7ac659c85574bb819650848af501c95b825baa
> # Parent 8faa91951bb10b580cbfc2ad065cd26aae47c618
> updatemeta: correctly handle empty metadata
> When the repo metadata is empty (just created/cloned but not populated yet),
> there is no lastpulled file or the revmap doesn't have a last entry with hash.
> Currently "hg svn updatemeta" would crash due to unexpected exception.
> See issue reported at https://bitbucket.org/durin42/hgsubversion/issue/356/updatemeta-crash...
> The fix is to check the existence of lastpulled file and the hash entry in
> revmap, as part of "hg svn updatemeta" command. When they are not present,
> do a rebuildmetadata.
> Also added a new unit test, test_updatemeta.py.
> diff --git a/hgsubversion/svncommands.py b/hgsubversion/svncommands.py
> --- a/hgsubversion/svncommands.py
> +++ b/hgsubversion/svncommands.py
> @@ -64,12 +64,19 @@
> if partial:
> try:
> youngestpath = os.path.join(svnmetadir, 'lastpulled')
> - youngest = int(util.load_string(youngestpath).strip())
> - sofar = list(maps.RevMap.readmapfile(repo))
> - lasthash = sofar[-1].split(' ', 2)[1]
> - startrev = repo[lasthash].rev() + 1
> - branchinfo = pickle.load(open(os.path.join(svnmetadir,
> - 'branch_info')))
> + foundpartialinfo = False
> + if os.path.exists(youngestpath):
> + youngest = int(util.load_string(youngestpath).strip())
> + sofar = list(maps.RevMap.readmapfile(repo))
> + if sofar and len(sofar[-1].split(' ', 2)) > 1:
> + lasthash = sofar[-1].split(' ', 2)[1]
> + startrev = repo[lasthash].rev() + 1
> + branchinfo = pickle.load(open(os.path.join(svnmetadir,
> + 'branch_info')))
> + foundpartialinfo = True
> + if not foundpartialinfo:
> + ui.status('missing some metadata -- doing a full rebuild\n')
> + partial = False
> except IOError, err:
> if err.errno != errno.ENOENT:
> raise
> diff --git a/tests/run.py b/tests/run.py
> --- a/tests/run.py
> +++ b/tests/run.py
> @@ -32,6 +32,7 @@
> import test_template_keywords
> import test_utility_commands
> import test_unaffected_core
> + import test_updatemeta
> import test_urls
> sys.path.append(os.path.dirname(__file__))
> diff --git a/tests/test_updatemeta.py b/tests/test_updatemeta.py
> new file mode 100644
> --- /dev/null
> +++ b/tests/test_updatemeta.py
> @@ -0,0 +1,81 @@
> +import test_util
> +
> +import os
> +import pickle
> +import unittest
> +import test_rebuildmeta
> +
> +from mercurial import context
> +from mercurial import extensions
> +from mercurial import hg
> +from mercurial import ui
> +
> +from hgsubversion import svncommands
> +from hgsubversion import svnmeta
> +
> +
> +
> +def _do_case(self, name, stupid, single):
> + subdir = test_util.subdir.get(name, '')
> + layout = 'auto'
> + if single:
> + layout = 'single'
> + repo, repo_path = self.load_and_fetch(name, subdir=subdir, stupid=stupid,
> + layout=layout)
> + assert len(self.repo) > 0
> + wc2_path = self.wc_path + '_clone'
> + u = ui.ui()
> + src, dest = test_util.hgclone(u, self.wc_path, wc2_path, update=False)
> + src = getattr(src, 'local', lambda: src)()
> + dest = getattr(dest, 'local', lambda: dest)()
> +
> + # insert a wrapper that prevents calling changectx.children()
> + def failfn(orig, ctx):
> + self.fail('calling %s is forbidden; it can cause massive slowdowns '
> + 'when rebuilding large repositories' % orig)
> +
> + origchildren = getattr(context.changectx, 'children')
> + extensions.wrapfunction(context.changectx, 'children', failfn)
> +
> + # test updatemeta on an empty repo
> + try:
> + svncommands.updatemeta(u, dest,
> + args=[test_util.fileurl(repo_path +
> + subdir), ])
> + finally:
> + # remove the wrapper
> + context.changectx.children = origchildren
> +
> + self._run_assertions(name, stupid, single, src, dest, u)
> +
> +
> +def _run_assertions(self, name, stupid, single, src, dest, u):
> + test_rebuildmeta._run_assertions(self, name, stupid, single, src, dest, u)
> +
> +
> +skip = set([
> + 'project_root_not_repo_root.svndump',
> + 'corrupt.svndump',
> +])
> +
> +attrs = {'_do_case': _do_case,
> + '_run_assertions': _run_assertions,
> + }
> +for case in [f for f in os.listdir(test_util.FIXTURES) if f.endswith('.svndump')]:
> + # this fixture results in an empty repository, don't use it
> + if case in skip:
> + continue
> + bname = 'test_' + case[:-len('.svndump')]
> + attrs[bname] = test_rebuildmeta.buildmethod(case, bname, False, False)
> + name = bname + '_stupid'
> + attrs[name] = test_rebuildmeta.buildmethod(case, name, True, False)
> + name = bname + '_single'
> + attrs[name] = test_rebuildmeta.buildmethod(case, name, False, True)
> +
> +UpdateMetaTests = type('UpdateMetaTests', (test_util.TestBase,), attrs)
> +
> +
> +def suite():
> + all_tests = [unittest.TestLoader().loadTestsFromTestCase(UpdateMetaTests),
> + ]
> + return unittest.TestSuite(all_tests)
> -- > You received this message because you are subscribed to the Google Groups "hgsubversion" group.
> To post to this group, send email to hgsubversion@googlegroups.com.
> To unsubscribe from this group, send email to hgsubversion+unsubscribe@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/hgsubversion?hl=en .
You must
Sign in before you can post messages.
You do not have the permission required to post.
From:
Jun Fang <junf... @fb.com>
Date: Tue, 28 Aug 2012 16:08:36 +0000
Local: Tues, Aug 28 2012 12:08 pm
Subject: Re: [PATCH] updatemeta: correctly handle empty metadata
Thanks!
On 8/28/12 6:09 AM, "Augie Fackler" <r... @durin42.com> wrote:
>queued, thanks
>On Aug 6, 2012, at 12:50 PM, Jun Fang wrote:
>> # HG changeset patch
>> # User Jun Fang <junf... @fb.com>
>> # Date 1344274551 25200
>> # Node ID 6b7ac659c85574bb819650848af501c95b825baa
>> # Parent 8faa91951bb10b580cbfc2ad065cd26aae47c618
>> updatemeta: correctly handle empty metadata
>> When the repo metadata is empty (just created/cloned but not populated
>>yet),
>> there is no lastpulled file or the revmap doesn't have a last entry
>>with hash.
>> Currently "hg svn updatemeta" would crash due to unexpected exception.
>> See issue reported at
>>https://bitbucket.org/durin42/hgsubversion/issue/356/updatemeta-crash...
>>ith-traceback-if-there
>> The fix is to check the existence of lastpulled file and the hash entry
>>in
>> revmap, as part of "hg svn updatemeta" command. When they are not
>>present,
>> do a rebuildmetadata.
>> Also added a new unit test, test_updatemeta.py.
>> diff --git a/hgsubversion/svncommands.py b/hgsubversion/svncommands.py
>> --- a/hgsubversion/svncommands.py
>> +++ b/hgsubversion/svncommands.py
>> @@ -64,12 +64,19 @@
>> if partial:
>> try:
>> youngestpath = os.path.join(svnmetadir, 'lastpulled')
>> - youngest = int(util.load_string(youngestpath).strip())
>> - sofar = list(maps.RevMap.readmapfile(repo))
>> - lasthash = sofar[-1].split(' ', 2)[1]
>> - startrev = repo[lasthash].rev() + 1
>> - branchinfo = pickle.load(open(os.path.join(svnmetadir,
>> - 'branch_info')))
>> + foundpartialinfo = False
>> + if os.path.exists(youngestpath):
>> + youngest = int(util.load_string(youngestpath).strip())
>> + sofar = list(maps.RevMap.readmapfile(repo))
>> + if sofar and len(sofar[-1].split(' ', 2)) > 1:
>> + lasthash = sofar[-1].split(' ', 2)[1]
>> + startrev = repo[lasthash].rev() + 1
>> + branchinfo =
>>pickle.load(open(os.path.join(svnmetadir,
>> + >>'branch_info')))
>> + foundpartialinfo = True
>> + if not foundpartialinfo:
>> + ui.status('missing some metadata -- doing a full
>>rebuild\n')
>> + partial = False
>> except IOError, err:
>> if err.errno != errno.ENOENT:
>> raise
>> diff --git a/tests/run.py b/tests/run.py
>> --- a/tests/run.py
>> +++ b/tests/run.py
>> @@ -32,6 +32,7 @@
>> import test_template_keywords
>> import test_utility_commands
>> import test_unaffected_core
>> + import test_updatemeta
>> import test_urls
>> sys.path.append(os.path.dirname(__file__))
>> diff --git a/tests/test_updatemeta.py b/tests/test_updatemeta.py
>> new file mode 100644
>> --- /dev/null
>> +++ b/tests/test_updatemeta.py
>> @@ -0,0 +1,81 @@
>> +import test_util
>> +
>> +import os
>> +import pickle
>> +import unittest
>> +import test_rebuildmeta
>> +
>> +from mercurial import context
>> +from mercurial import extensions
>> +from mercurial import hg
>> +from mercurial import ui
>> +
>> +from hgsubversion import svncommands
>> +from hgsubversion import svnmeta
>> +
>> +
>> +
>> +def _do_case(self, name, stupid, single):
>> + subdir = test_util.subdir.get(name, '')
>> + layout = 'auto'
>> + if single:
>> + layout = 'single'
>> + repo, repo_path = self.load_and_fetch(name, subdir=subdir,
>>stupid=stupid,
>> + layout=layout)
>> + assert len(self.repo) > 0
>> + wc2_path = self.wc_path + '_clone'
>> + u = ui.ui()
>> + src, dest = test_util.hgclone(u, self.wc_path, wc2_path,
>>update=False)
>> + src = getattr(src, 'local', lambda: src)()
>> + dest = getattr(dest, 'local', lambda: dest)()
>> +
>> + # insert a wrapper that prevents calling changectx.children()
>> + def failfn(orig, ctx):
>> + self.fail('calling %s is forbidden; it can cause massive
>>slowdowns '
>> + 'when rebuilding large repositories' % orig)
>> +
>> + origchildren = getattr(context.changectx, 'children')
>> + extensions.wrapfunction(context.changectx, 'children', failfn)
>> +
>> + # test updatemeta on an empty repo
>> + try:
>> + svncommands.updatemeta(u, dest,
>> + args=[test_util.fileurl(repo_path +
>> + subdir), ])
>> + finally:
>> + # remove the wrapper
>> + context.changectx.children = origchildren
>> +
>> + self._run_assertions(name, stupid, single, src, dest, u)
>> +
>> +
>> +def _run_assertions(self, name, stupid, single, src, dest, u):
>> + test_rebuildmeta._run_assertions(self, name, stupid, single, src,
>>dest, u)
>> +
>> +
>> +skip = set([
>> + 'project_root_not_repo_root.svndump',
>> + 'corrupt.svndump',
>> +])
>> +
>> +attrs = {'_do_case': _do_case,
>> + '_run_assertions': _run_assertions,
>> + }
>> +for case in [f for f in os.listdir(test_util.FIXTURES) if
>>f.endswith('.svndump')]:
>> + # this fixture results in an empty repository, don't use it
>> + if case in skip:
>> + continue
>> + bname = 'test_' + case[:-len('.svndump')]
>> + attrs[bname] = test_rebuildmeta.buildmethod(case, bname, False,
>>False)
>> + name = bname + '_stupid'
>> + attrs[name] = test_rebuildmeta.buildmethod(case, name, True, False)
>> + name = bname + '_single'
>> + attrs[name] = test_rebuildmeta.buildmethod(case, name, False, True)
>> +
>> +UpdateMetaTests = type('UpdateMetaTests', (test_util.TestBase,), attrs)
>> +
>> +
>> +def suite():
>> + all_tests =
>>[unittest.TestLoader().loadTestsFromTestCase(UpdateMetaTests),
>> + ]
>> + return unittest.TestSuite(all_tests)
>> -- >> You received this message because you are subscribed to the Google
>>Groups "hgsubversion" group.
>> To post to this group, send email to hgsubversion@googlegroups.com.
>> To unsubscribe from this group, send email to
>>hgsubversion+unsubscribe@googlegroups.com.
>> For more options, visit this group at
>>http://groups.google.com/group/hgsubversion?hl=en .
You must
Sign in before you can post messages.
You do not have the permission required to post.