How to solve svnadmin load "File already exists" error?
Dennis Jones
I used rsvndump to dump a remote repository (because the server is running
an old version of SVN from before support for 'replay' was added). Now I am
trying to load the dump file into my local repository (VisualSVN 2.1.9) and
I'm getting the following error:
* adding path : trunk/SomePath/SomeFile.zip ...svnadmin: File already
exists: filesystem 'E:\Repositories\Archive\db', transaction '3186-2gi',
path 'trunk/SomePath/SomeFile.zip'
Interestingly, it looks like it fails at revision 3186, but a glance at the
output shows:
------- Committed revision 3186 >>>
<<< Started new transaction, based on original revision 3187
So why is the transaction for 3186 still there?
This is my second attempt at doing this, and I got the same error (in the
same place) the first time. The dump file is almost 5GB in size and the
dump takes about 3 days to complete, so I'm not keen on doing it again, and
I doubt it will help anyway.
Of course, I started with an empty repository on the local server, so it's
not like it already had data in it. So how can I get around this?
Dennis
Dennis Jones
- Dennis
"Dennis Jones" <djo...@grassvalleysoftware.com> wrote in message
news:j1shsm$lbu$1...@dough.gmane.org...
Konstantin Kolinko
>> I used rsvndump
What is that? You mean you performed the dump remotely using svnrdump
utility from some beta of svn 1.7 ? Does it work without replay
support on the server? (Isn't this utility essentially the same as
svnsync?)
I would suggest to compare what actually is in that revision in the
old and new repository. Do they match, or anything differs?
When you performed the dump, do you have read access to all paths in
the original repository?
Last time when I saw a message similar to yours was about a year ago,
when I tried to commit revision that had complicated moves and renames
/ replaces in it. The svn client (Tortoise) failed to commit it with a
similar message, but when I rerun the operation it committed
successfully. I still do not know what the cause was -- maybe some
ordering between operations. (I am using HTTPS access protocol + neon
+ it was some version of 1.6 or 1.5 several months after release),
Best regards,
Konstantin Kolinko
Dennis Jones
"Konstantin Kolinko" <knst.k...@gmail.com> wrote in message
news:CABzHfVmZ1Acxa5Vp2SBKcTyE...@mail.gmail.com...
>> I used rsvndump
> What is that? You mean you performed the dump remotely using svnrdump
> utility from some beta of svn 1.7 ? Does it work without replay
> support on the server? (Isn't this utility essentially the same as
> svnsync?)
No, I used "rsvndump", a third-party utility that I happened upon when
googling for a way to dump a repository from an old server version <1.4. I
did try svnrdump from the 1.7 beta, but it (obviously) did not work because
it requires the server to support replay.
> I would suggest to compare what actually is in that revision in the
> old and new repository. Do they match, or anything differs?
There is no new repository. I am trying to create a new, clean repository
from the old server (of which I only have remote access).
>
> When you performed the dump, do you have read access to all paths in
> the original repository?
Yes.
> Last time when I saw a message similar to yours was about a year ago,
> when I tried to commit revision that had complicated moves and renames
> / replaces in it. The svn client (Tortoise) failed to commit it with a
> similar message, but when I rerun the operation it committed
> successfully. I still do not know what the cause was -- maybe some
> ordering between operations. (I am using HTTPS access protocol + neon
> + it was some version of 1.6 or 1.5 several months after release),
I've tried multiple times to re-run the operation. It always fails at the
same place.
- Dennis
Les Mikesell
>
>> I would suggest to compare what actually is in that revision in the
>> old and new repository. Do they match, or anything differs?
>
> There is no new repository. I am trying to create a new, clean repository
> from the old server (of which I only have remote access).
Your new, clean repository should have the contents up to the commit
that failed.
But, it might be easier to use svnsync to do this.
--
Les Mikesell
lesmi...@gmail.com
Stephen Butler
On Aug 19, 2011, at 18:45 , Les Mikesell wrote:
> On 8/19/2011 11:20 AM, Dennis Jones wrote:
>
>>
>>> I would suggest to compare what actually is in that revision in the
>>> old and new repository. Do they match, or anything differs?
>>
>> There is no new repository. I am trying to create a new, clean repository
>> from the old server (of which I only have remote access).
>
> Your new, clean repository should have the contents up to the commit that failed.
Dennis,
You could compare the histories of the problematic file (and its parent
directories) in the two repositories. I'll bet there's some moves and
replacements there, as Konstantin suggests.
Does rsvndump support moved/replaced items? I'm sure it's not
trivial.
>
> But, it might be easier to use svnsync to do this.
According to the original post, the old repository doesn't support svnsync.
Regards,
Steve
--
Stephen Butler | Senior Consultant
elego Software Solutions GmbH
Gustav-Meyer-Allee 25 | 13355 Berlin | Germany
tel: +49 30 2345 8696 | mobile: +49 163 25 45 015
fax: +49 30 2345 8695 | http://www.elegosoft.com
Geschäftsführer: Olaf Wagner | Sitz der Gesellschaft: Berlin
Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194
Dennis Jones
"Stephen Butler" <sbu...@elego.de> wrote in message
news:E19FC591-322D-4775...@elego.de...
>On Aug 19, 2011, at 18:45 , Les Mikesell wrote:
>
>> On 8/19/2011 11:20 AM, Dennis Jones wrote:
>>
>>>
>>>> I would suggest to compare what actually is in that revision in the
>>>> old and new repository. Do they match, or anything differs?
>>>
>>> There is no new repository. I am trying to create a new, clean
>>> repository
>>> from the old server (of which I only have remote access).
>>
>> Your new, clean repository should have the contents up to the commit that
>> failed.
Okay, then I will take a look and report back.
>You could compare the histories of the problematic file (and its parent
>directories) in the two repositories. I'll bet there's some moves and
>replacements there, as Konstantin suggests.
>
>Does rsvndump support moved/replaced items? I'm sure it's not
>trivial.
I would assume so. I believe there are plenty of moves/branches/tags that
occured prior to the revision where the error occurred.
>> But, it might be easier to use svnsync to do this.
>
>According to the original post, the old repository doesn't support svnsync.
That is correct. The server is pre 1.4.
- Dennis
Dennis Jones
>
> "Stephen Butler" <sbu...@elego.de> wrote in message
> news:E19FC591-322D-4775...@elego.de...
>
>>On Aug 19, 2011, at 18:45 , Les Mikesell wrote:
>>
>>> On 8/19/2011 11:20 AM, Dennis Jones wrote:
>>>
>>>>
>>>>> I would suggest to compare what actually is in that revision in the
>>>>> old and new repository. Do they match, or anything differs?
>>>>
>>>> There is no new repository. I am trying to create a new, clean
>>>> repository
>>>> from the old server (of which I only have remote access).
>>>
>>> Your new, clean repository should have the contents up to the commit
>>> that failed.
>
> Okay, then I will take a look and report back.
Revision 3186 was committed successfully. It was a delete of a folder. The
next revision, 3187 is the one that failed, it is just a normal bunch of
modified files.
I cannot re-run the svnadmin load, because at this point, the repository has
already been created, and svnadmin load expects an empty repository. I wish
I could start the load from a specific revision, but I don't see a way to do
that. Is there?
- Dennis
Daniel Shahaf
> I cannot re-run the svnadmin load, because at this point, the repository has
> already been created, and svnadmin load expects an empty repository.
svnadmin doesn't require an empty repository; svnsync does.
> I wish I could start the load from a specific revision, but I don't
> see a way to do that. Is there?
Delete the first N revision records from the dumpfile?
Load the dumpfile into a new repository and then run
'svnadmin dump -r' on the new repository?
Engebakken Geir
Geir
Note : All inquiries regarding Subversion, MKS and general Development servers should be directed to "EDB SourceControl System"
Engebakken Geir
I tried to edit a 5 Gb file on a Linuxserver we have with vi, it took some 30 minutes to open the file but it did work!
The format is quite simple : a revision starts with the string :
Revision-number:
Just delete all the lines until the next
Revision-number:
And the revision is removed from the dump. I am not quite sure if you need to renumber the revisions afterwards though, but I don't think so.
There is a problem though if what you delete is referenced later on in the dump, then the load will fail.
Geir
Note : All inquiries regarding Subversion, MKS and general Development servers should be directed to "EDB SourceControl System"
> -----Original Message-----
> From: Dennis Jones [mailto:djo...@grassvalleysoftware.com]
> Sent: 21. august 2011 22:12
> To: Engebakken Geir
> Subject: Re: How to solve svnadmin load "File already exists" error?
>
> ----- Original Message -----
> From: "Engebakken Geir" <geir.en...@edb.com>
> To: "Daniel Shahaf" <d...@daniel.shahaf.name>; "Dennis Jones"
> <djo...@grassvalleysoftware.com>
> Cc: <us...@subversion.apache.org>
> Sent: Sunday, August 21, 2011 12:22 PM
> Subject: RE: How to solve svnadmin load "File already exists" error?
>
>
> Isnt it possible to just remove the N records from the dump file and
> restart
> the svnadmin load? I seem to think I have tried that with success
> before.
>
> You are the second person to suggest this. There are two problems with
> that
> approach, both of which *can* be solved, but will take some effort:
>
> 1) The file is 5GB. I haven't been able to find an editor that can
> handle a
> file that big. I can solve this by splitting the file into smaller
> pieces
> and editing the piece containing the bad revision and splicing them
> back
> together afterwards. Just a pain to do.
>
> 2) I don't understand the file format. What are the beginning and end
> markers for a particular revision?
>
> - Dennis
Dennis Jones
> 30 minutes to open the file but it did work!
>
> The format is quite simple : a revision starts with the string :
>
> Revision-number:
>
> Just delete all the lines until the next
>
> Revision-number:
>
>
> And the revision is removed from the dump. I am not quite sure if you need
> to renumber the revisions afterwards though, but I
> don't think so.
>
> There is a problem though if what you delete is referenced later on in the
> dump, then the load will fail.
Unfortunately, this did not work. I was able to split the file into
smaller, editable pieces, cut out the bad revision, and paste it all back
together. However, when I attempted to start loading at the revision
following the bad one, I got checksum mismatches. So I tried again with a
new repository and tried to load a new dump file with everything in it
except the bad revision (so it started from 0 again), and this time it
failer even earlier than before, but again with a checksum mismatch. Ugh.
Subversion is a great tool overall, but getting this particular process to
work has been utterly frustrating and ridiculously slow, not to mention
error-prone, with no real explanation for the failing behaviors.
- Dennis
Les Mikesell
>
>>
>> And the revision is removed from the dump. I am not quite sure if you need
>> to renumber the revisions afterwards though, but I
>> don't think so.
>>
>> There is a problem though if what you delete is referenced later on in the
>> dump, then the load will fail.
>
>
> Unfortunately, this did not work. I was able to split the file into
> smaller, editable pieces, cut out the bad revision, and paste it all back
> together. However, when I attempted to start loading at the revision
> following the bad one, I got checksum mismatches. So I tried again with a
> new repository and tried to load a new dump file with everything in it
> except the bad revision (so it started from 0 again), and this time it
> failer even earlier than before, but again with a checksum mismatch. Ugh.
> Subversion is a great tool overall, but getting this particular process to
> work has been utterly frustrating and ridiculously slow, not to mention
> error-prone, with no real explanation for the failing behaviors.
Is there any chance of getting the original repository to upgrade or to
get a file-level copy so you could do a server upgrade on the copy? It
is always painful to try to work around bugs that have long been fixed.
--
Les Mikesell
lesmi...@gmail.com
Konstantin Kolinko
>> I tried to edit a 5 Gb file on a Linuxserver we have with vi, it took some
>> 30 minutes to open the file but it did work!
>>
>> The format is quite simple : a revision starts with the string :
>>
>> Revision-number:
>>
>> Just delete all the lines until the next
>>
>> Revision-number:
>>
>>
>> And the revision is removed from the dump. I am not quite sure if you need
>> to renumber the revisions afterwards though, but I
>> don't think so.
>>
>> There is a problem though if what you delete is referenced later on in the
>> dump, then the load will fail.
>
>
> Unfortunately, this did not work. I was able to split the file into
> smaller, editable pieces, cut out the bad revision, and paste it all back
> together. However, when I attempted to start loading at the revision
> following the bad one, I got checksum mismatches.
Apparently you cannot edit the dump file with vi.
It is mentioned somewhere in the SVN Book that the dump just looks
like a textual file but actually it is binary. It explicitly does not
recommend editing it in a text editor. (So the problem with checksums
that you are facing is documented).
Can you generate a dump file for a range of revisions? Looking at [1]
you can specify a range of revisions to be dumped by rsvndump tool.
[1] http://rsvndump.sourceforge.net/
Have you tried to compare old and new repository at revision 3186.
E.g. do a checkout and compare whether files and folders all exist and
are the same?
Especially the files and folders touched by r3187.
Best regards,
Konstantin Kolinko