Malformed dumpfile header & Premature end of dumpstream in VSS2SVN migration

[RRC]

Hi All,

I have been working on vss2svn conversion.I am a VSS database of 20GB,
which i want to convert into SVN repository.What i am doing:

1> I have converted that VSS database into a dumpfile of 35 GB using
vss2svn binary.It is
having around 55,000 revisions. When i am doing "svnadmin load"
command, it is stopping at
17369 revision. The message on command prompt is "no such
revision".
2> After that i have divided the dumpfile into 14 parts of 2GB each.My
plan was to convert each
part into repository, then merge each repository to make a final
repository. But at the end of
first repository making , it is giving me error "Premature end of
dumpstream". After that when i
start my second part dumpfile conversion,it is giving me error
"malformed dumpfile header".
3> I have done a lot of googling and still i am doing it,but didn't
get any useful solution.
4> Please suggest me some solution and suugest if any good tool is
there for VSS migration.

Regards,
Ravi Chobey

[Toby Johnson]

On Tue, Sep 14, 2010 at 7:09 AM, RRC <ravi....@gmail.com> wrote:

> Hi All,
>
> I have been working on vss2svn conversion.I am a VSS database of 20GB,
> which i want to convert into SVN repository.What i am doing:
>
> 1> I have converted that VSS database into a dumpfile of 35 GB using
> vss2svn binary.It is
>    having  around 55,000 revisions. When i am doing "svnadmin load"
> command, it is stopping at
>    17369 revision. The message on command prompt is "no such
> revision".
>

This error message sounds like it is trying to copy or move from a source
file which doesn't exist. I would suggest trying to dig deeper into this and
maybe remove the offending operation manually from the dumpfile.

> 2> After that i have divided the dumpfile into 14 parts of 2GB each.My
> plan was to convert each
>    part into repository, then merge each repository to make a final
> repository. But at the end of
>    first repository making , it is giving me error "Premature end of
> dumpstream". After that when i
>    start my second part dumpfile conversion,it is giving me error
> "malformed dumpfile header".
>

Dividing the dumpfile is probably a good idea to make it easier to work with
but it sounds like you didn't quite do it correctly. I suggest looking into
the following:
http://code.google.com/p/vss2svn/wiki/FixingTheDumpfile

In particular take a look at SvnDumpTool from
http://svn.borg.ch/svndumptool/

> 3> I have done a lot of googling and still i am doing it,but didn't
> get any useful solution.
> 4> Please suggest me some solution and suugest if any good tool is
> there for VSS migration.
>

Unfortunately as you're finding out every large VSS database has a certain
degree of corruption which no tool may be able to fix. You should ask
yourself or your client whether converting VSS it SVN is really worthwhile.
Many times it's simply better to place the existing VSS repo into read-only
mode and either import all your latest code into SVN or start SVN from
scratch. In fact it may be a great opportunity to refactor your repository
layout if it's currently in an unwieldy state (which many large VSS
repositories are).

It may sound drastic to "start from scratch" but after a short while of
working in SVN you'll find that you need to go back to VSS very seldom. If
you still decide that converting the repo is what you want then it's
probably going to involve some manual editing of the dumpfile because as I
said every case is unique and will probably have issues others haven't seen.

Good luck,
Toby

[RRC]

Hi Toby,

Great to see your reply.I was knowing that you have written this tool.
I am two more questions, please help me:

1> Why the size of dumpfile for 20 GB database is 35GB?
2> Why the size of SVN repository is less, in comparison to the VSS?
Is there any compression of data during
conversion?


Thanks,
Ravi Chobey

On Sep 14, 6:37 pm, Toby Johnson <t...@etjohnson.us> wrote:

> On Tue, Sep 14, 2010 at 7:09 AM, RRC <ravi.cho...@gmail.com> wrote:
> > Hi All,
>
> > I have been working on vss2svn conversion.I am a VSS database of 20GB,
> > which i want to convert into SVN repository.What i am doing:
>
> > 1> I have converted that VSS database into a dumpfile of 35 GB using
> > vss2svn binary.It is
> >    having  around 55,000 revisions. When i am doing "svnadmin load"
> > command, it is stopping at
> >    17369 revision. The message on command prompt is "no such
> > revision".
>
> This error message sounds like it is trying to copy or move from a source
> file which doesn't exist. I would suggest trying to dig deeper into this and
> maybe remove the offending operation manually from the dumpfile.
>
> > 2> After that i have divided the dumpfile into 14 parts of 2GB each.My
> > plan was to convert each
> >    part into repository, then merge each repository to make a final
> > repository. But at the end of
> >    first repository making , it is giving me error "Premature end of
> > dumpstream". After that when i
> >    start my second part dumpfile conversion,it is giving me error
> > "malformed dumpfile header".
>
> Dividing the dumpfile is probably a good idea to make it easier to work with
> but it sounds like you didn't quite do it correctly. I suggest looking into
> the following:http://code.google.com/p/vss2svn/wiki/FixingTheDumpfile
>

> In particular take a look at SvnDumpTool fromhttp://svn.borg.ch/svndumptool/

[Toby Johnson]

On Wed, Sep 15, 2010 at 9:00 AM, RRC <ravi....@gmail.com> wrote:

1>  Why the size of dumpfile for 20 GB database is 35GB?

The dumpfile is in plain text format; even binary data in the repository is serialized so it's not surprising that it would take up quite a bit more room.

2>  Why the size of SVN repository is less, in comparison to the VSS?
Is there any compression of data during
     conversion?

I'm not as familiar with the actual storage model of SVN but it doesn't surprise me that it's quite a bit more efficient, and I'm sure it uses some type of compression. I'm sure there's much more info available at the Subversion website if you're interested but I wouldn't worry too much about the disk space of VSS vs. dumpfile vs. SVN.

Toby

[RRC]

Hi Toby,

As i am halting at 17369 revision, i have edited the dumpfile, but
still i am not able to move ahead. One more thing i want to tell you,
i have used "hjsplit" software from freebytes.com for dividing that
dumpfile, as i have tried "svndumptool split"
for didviding the revision into parts, and i was not able to do it.
Can u guide me, how to use that split command "svndumptool split
1-17500" , is it the right format?

I want to do like this from "Fixing the dumpfile" link:

Using svndumptool (see version note below), svndumptool.py split the
failing revision out of the main dumpfile such that you get A.dump
(revisions before bad rev), B.dump (the bad revision) and C.dump
(revisions after bad rev).

Please guide me.I will be thankful for your efforts.

Thanks and Regards,
Ravi

On Sep 15, 6:47 pm, Toby Johnson <t...@etjohnson.us> wrote:

[Toby Johnson]

On Wed, Sep 15, 2010 at 10:19 AM, RRC <ravi....@gmail.com> wrote:

Hi Toby,

As i am halting at 17369 revision, i have edited the dumpfile, but
still i am not able to move ahead. One more thing i want to tell you,
i have used "hjsplit" software from freebytes.com for dividing that
dumpfile, as i have tried "svndumptool split"
for didviding the revision into parts, and i was not able to do it.
Can u guide me, how to use that split command "svndumptool split
1-17500" , is it the right format?

I want to do like this from "Fixing the dumpfile" link:

Using svndumptool (see version note below), svndumptool.py split the
failing revision out of the main dumpfile such that you get A.dump
(revisions before bad rev), B.dump (the bad revision) and C.dump
(revisions after bad rev).

Sorry I'm really not able to help much more, I have never actually used the dumpfile splitting tools linked there and I don't even have VSS installed on any computers anymore as it's been several years since I had to perform a conversion. If you are able to figure out which specific item(s) in VSS are giving you problems, it may help to remove just those parts from the dumpfile using svndumpfilter:

http://svnbook.red-bean.com/en/1.5/svn.ref.svndumpfilter.html