Hello Petr,
Thank you for the details provided, please find my comments below.
> I tried experimentally to convert our SVN repository with subgit. It is q=
uite a large repository, almost 80.000 revisions and some branches (althoug=
h just spinning off, not merging back) and tags (and, alas, with a number o=
f binaries, too). Therefore I limited the conversion options to the bare mi=
nimum which I was able to find out:
> - svn.minimalRevision =3D 79800 (there are about 100 more revisions)
> - branches and tags set to skip all, but few selected ones
...
> The problem is that running subgit install again then seems to do all ove=
r. It shows a message that it resumes, but the log then shows quite suspici=
ous messages (like DELETE_PATH 'trunk') and the files that were already fet=
ched before are fetched fetched according to the log again. Perhaps I just =
don't understand the log, however, the process takes so long and requires r=
estarts that it looks wrong.
Note that with svn.minimalRevision =3D 79800, SubGit has to fetch from
scratch those few branches you've specified in config file. That means
SubGit tries to fetch trunk@79800, branches/branch1@79800, ...,
branches/branchN@79800 and this is basically fetching one single
revision r79800. When SubGit hits network related error, it retries
fetching problematic revision from scratch which means SubGit
re-fetches all those branches again. I think that's the behavior
you're observing.
This 'DELETE_PATH trunk' line is a trick SubGit uses to fetch all the
directories and files within 'trunk' folder. Basically, we make SVN
server think that 'trunk' is missing in our working copy, so the
server sends the whole tree back to us.
> - the directories with most of the binaries are set to be skipped with sv=
n.skip
Unfortunately, svn.skip option doesn't prevent SubGit from fetching
those files as they come within one checkout request. SubGit just
skips those files from being added to Git repository, but network
overhead is still there.
> - translate.eols =3D false
Indeed, this option should help in this case.
It would help us to see the actual error message, could you send it to us?
In any case you should workaround the network problems in order to
complete the import. There two viable strategies here:
1. You can use svnsync or svnrdump in order to copy your Subversion
repository to local machine and then perform Git-SVN conversion on
that copied repository. Hopefully, copying Subversion repository to
your local machine doesn't lead to the same network errors and there
are no heavy client-side operations involved into the process.
If you'd need to maintain bi-directional mirror with remote SVN
repository, there's a way to switch svn.url option in SubGit
configuration, so you can perform initial import with local repository
and keep resulted Git repository in sync with remote SVN repository.
2. You can also adjust SubGit configuration in order to prevent
network timeout problems:
a) edit <REPO>/subgit/config
[svn]
...
connectTimeout =3D 300
readTimeout =3D 600
Default connect timeout is 30 seconds and default read timeout is 60
seconds, increasing those values may workaround the problem you hit.
b) edit <REPO>/subgit/config
[svn]
...
httpSpooling =3D true
If you're using HTTP(s) protocol, enabling httpSpooling is a very
powerful technique to resolve network related problems. Once spooling
is enabled, SubGit first reads SVN server response and dumps all the
data to local files and only then it processes this data to convert
SVN revisions to Git commits.
Please let us know if any of the strategies above work in your environment.
Regards,
Semyon Vadishev,
TMate Software,
http://subgit.com/ git-svn import & mirror
On 11 December 2014 at 16:46, Dolezal, Petr <
petr.d...@atos.net> wrote:
>
> Hello,
>
> I tried experimentally to convert our SVN repository with subgit. It is q=
uite a large repository, almost 80.000 revisions and some branches (althoug=
h just spinning off, not merging back) and tags (and, alas, with a number o=
f binaries, too). Therefore I limited the conversion options to the bare mi=
nimum which I was able to find out:
>
> - svn.minimalRevision =3D 79800 (there are about 100 more revisions)
> - branches and tags set to skip all, but few selected ones
> - the directories with most of the binaries are set to be skipped with sv=
n.skip
> - translate.eols =3D false
>
> After subgit install, the process looks proceeding fine. Then it gets stu=
ck after few hours or even (after an hour or so) stops with an error messag=
e about connection timeout. This is not very suprising because our network =
infrastructure is not very willing to keep too long transfers.
>
> The problem is that running subgit install again then seems to do all ove=
r. It shows a message that it resumes, but the log then shows quite suspici=
ous messages (like DELETE_PATH 'trunk') and the files that were already fet=
ched before are fetched fetched according to the log again. Perhaps I just =
don't understand the log, however, the process takes so long and requires r=
estarts that it looks wrong.
>
> Here is a part of the log after a restart:
>
> [2014-12-05 16:16:37.092][client-install][1] Command name: install.
> [2014-12-05 16:16:37.092][client-install][1] Command argument: dxa.git.
> [2014-12-05 16:16:37.093][client-install][1] Initialized console logger.
> [2014-12-05 16:16:37.093][client-install][1] SubGit version 2.0.3 ('Patri=
ck') build #2796
> [2014-12-05 16:16:37.108][client-install][1] Initialized file logger, log=
s directory is: 'null'.
> [2014-12-05 16:16:37.108][client-install][1] SubGit version 2.0.3 ('Patri=
ck') build #2796
> [2014-12-05 16:16:37.311][client-install][1] Using Subversion configurati=
on and credentials cache at: C:\Users\U2354978\AppData\Roaming\Subversion
> [2014-12-05 16:16:38.807][client-install][1] Pid file does not exist: W:\=
work\dxa.git\subgit\daemon.pid
> [2014-12-05 16:16:38.808][client-install][1] Daemon info pid=3D-1; port=
=3D-1; address=3D; user=3Dnull from file 'W:\work\dxa.git\subgit\daemon.pid=
'
> [2014-12-05 16:16:38.808][client-install][1] Pid file 'W:\work\dxa.git\su=
bgit\daemon.pid' deleted.
> [2014-12-05 16:16:38.809][client-install][1] Trying to obtain file lock o=
n 'W:\work\dxa.git\subgit\daemon.lock'.
> [2014-12-05 16:16:38.811][client-install][1] Succeeded attempt to obtain =
file lock on 'W:\work\dxa.git\subgit\daemon.lock'
> [2014-12-05 16:16:38.815][client-install][1] Pid file does not exist: W:\=
work\dxa.git\subgit\daemon.pid
> [2014-12-05 16:16:38.815][client-install][1] Daemon info pid=3D-1; port=
=3D-1; address=3D; user=3Dnull from file 'W:\work\dxa.git\subgit\daemon.pid=
'
> [2014-12-05 16:16:38.846][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key
core.section.id
> [2014-12-05 16:16:38.846][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key
core.section.id
> [2014-12-05 16:16:38.846][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key
core.section.id
> [2014-12-05 16:16:38.847][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key
core.section.id
> [2014-12-05 16:16:38.847][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key
core.section.id
> [2014-12-05 16:16:38.847][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key
core.section.id
> [2014-12-05 16:16:38.848][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key
core.section.id
> [2014-12-05 16:16:38.849][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key svn.branches
> [2014-12-05 16:16:38.849][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key svn.tags
> [2014-12-05 16:16:38.850][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key svn.shelves
> [2014-12-05 16:16:38.852][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key svn.skip
> [2014-12-05 16:16:38.852][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key
auth.section.id
> [2014-12-05 16:16:38.852][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key
auth.section.id
> [2014-12-05 16:16:38.852][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key
auth.section.id
> [2014-12-05 16:16:38.852][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key auth.useDefaultSubversionConfigura=
tionDirectory
> [2014-12-05 16:16:38.853][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key
auth.section.id
> [2014-12-05 16:16:38.853][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key auth.subversionConfigurationDirect=
ory
> [2014-12-05 16:16:38.853][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key
daemon.section.id
> [2014-12-05 16:16:38.853][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key
daemon.section.id
> [2014-12-05 16:16:38.853][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key
daemon.section.id
> [2014-12-05 16:16:38.854][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key
daemon.section.id
> [2014-12-05 16:16:38.854][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key
translate.section.id
> [2014-12-05 16:16:38.854][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key translate
> [2014-12-05 16:16:38.854][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key
translate.section.id
> [2014-12-05 16:16:38.854][client-install][1] Can't find resource for bund=
le java.util.PropertyResourceBundle, key translate.eols
> [2014-12-05 16:16:38.873][client-install][1] Pid file 'W:\work\dxa.git\su=
bgit\daemon.pid' deleted.
> [2014-12-05 16:16:38.873][client-install][1] Trying to release file lock =
on 'W:\work\dxa.git\subgit\daemon.lock'.
> [2014-12-05 16:16:38.873][client-install][1] Released file lock on 'W:\wo=
rk\dxa.git\subgit\daemon.lock'
> [2014-12-05 16:16:38.874][client-install][1] Closed random access file 'W=
:\work\dxa.git\subgit\daemon.lock'
> [2014-12-05 16:16:39.117][client-install][1] Created installation report:
> ErrorReport[description=3D'Push rejected: SubGit installation is in progr=
ess.
> Pushes will be allowed as soon as SubGit installation is completed.', war=
ning=3Dtrue, git enabled=3Dfalse, svn enabled=3Dtrue]
> [2014-12-05 16:16:39.118][client-install][1] Using Subversion configurati=
on and credentials cache at: C:\Users\U2354978\AppData\Roaming\Subversion
> [2014-12-05 16:16:40.057][client-install][1] Translating Subversion revis=
ions to Git commits...
> [2014-12-05 16:16:40.057][client-install][1] Subversion revisions: 79861.
> [2014-12-05 16:16:40.070][client-install][1] Using Subversion configurati=
on and credentials cache at: C:\Users\U2354978\AppData\Roaming\Subversion
> [2014-12-05 16:16:41.074][client-install][1] Using Subversion configurati=
on and credentials cache at: C:\Users\U2354978\AppData\Roaming\Subversion
> [2014-12-05 16:16:41.208][client-install][1] SubGit temporary blocking ho=
oks installed.
> [2014-12-05 16:16:41.209][client-install][1] Translating 'W:\work\dxa.git=
'.
> [2014-12-05 16:16:41.209][client-install][1] Sync started for git locatio=
n 'default'
> [2014-12-05 16:16:41.224][client-install][1] Checking remote '
http://iamd=
ev.iam.my-it-solutions.net/svn/dirxaccess'.
> [2014-12-05 16:16:42.327][client-install][1] Revisions: <fetched=3D-1; la=
st=3D79861>
> [2014-12-05 16:16:42.328][client-install][1] There are unfetched revision=
s in Subversion repository.
> [2014-12-05 16:16:42.396][client-install][1] Fetching
http://iamdev.iam.m=
y-it-solutions.net/svn/dirxaccess up to revision=3DHEAD
> [2014-12-05 16:16:47.254][client-install][1] SET_PATH '' 79800 not empty =
depth=3Dinfinity
> [2014-12-05 16:16:47.255][client-install][1] DELETE_PATH 'trunk'
> [2014-12-05 16:16:47.677][client-install][1] fetching: branch =3D refs/sv=
n/root/trunk, revision =3D 79800, receivedFileCount=3D0
>
> Thanks for any help in advance!
> P.
>
>
> Atos IT Solutions and Services, s.r.o. - CEO: Ing. Emil Fito=C5=A1 - regi=
stered in the Commercial Register of the Municipal Court in Prague, Sec. C,=
File 8954, Registered office: Doudlebsk=C3=A1 1699/5, 140 00 Prague 4, Cze=
ch Republic, I=C4=8C: 44851391, DI=C4=8C: CZ44851391, Bank connection: UniC=
redit Bank Czech Republic a.s., Na P=C5=99=C3=ADkop=C4=9B 858/20, 113 80 Pr=
aha 1, Acc. Nr. CZK: 1001885001/2700, IBAN CZK - CZ4627000000001001885001; =
Acc. Nr. EUR: 1001885095/2700, IBAN EUR - CZ3027000000001001885095
> Tato zpr=C3=A1va m=C3=A1 pouze informativn=C3=AD charakter, kter=C3=BD vy=
ch=C3=A1z=C3=AD z podklad=C5=AF, kter=C3=A9 byly odes=C3=ADlateli p=C5=99ed=
=C3=A1ny, nebo zasl=C3=A1ny. Obsah t=C3=A9to zpr=C3=A1vy odes=C3=ADlatele n=
ezavazuje, pokud to v n=C3=AD nen=C3=AD v=C3=BDslovn=C4=9B uvedeno a odes=
=C3=ADlatel nem=C3=A1 v =C3=BAmyslu na z=C3=A1klad=C4=9B t=C3=A9to zpr=C3=
=A1vy uzav=C5=99=C3=ADt smlouvu, p=C5=99ijmout nab=C3=ADdku, potvrdit uzav=
=C5=99en=C3=AD smlouvy ani nezakl=C3=A1d=C3=A1 p=C5=99edsmluvn=C3=AD odpov=
=C4=9Bdnost jej=C3=ADho odes=C3=ADlatele, leda=C5=BEe je odes=C3=ADlatelem =
ve zpr=C3=A1v=C4=9B uvedeno v=C3=BDslovn=C4=9B jinak.Tato zpr=C3=A1va je ur=
=C4=8Dena pouze pro osobn=C3=AD a d=C5=AFv=C4=9Brn=C3=A9 u=C5=BEit=C3=AD os=
obou (osobami) uvedenou (uveden=C3=BDmi) v=C3=BD=C5=A1e. Nejste-li osobou, =
kter=C3=A9 je tato zpr=C3=A1va ur=C4=8Dena, upozor=C5=88ujeme V=C3=A1s, =C5=
=BEe jak=C3=A9koli =C5=A1=C3=AD=C5=99en=C3=AD, distribuce =C4=8Di kop=C3=AD=
rov=C3=A1n=C3=AD t=C3=A9to zpr=C3=A1vy je zak=C3=A1z=C3=A1no. Jestli=C5=BEe=
tuto zpr=C3=A1vu omylem obdr=C5=BE=C3=ADte, pros=C3=ADme, oznamte tuto sku=
te=C4=8Dnost odes=C3=ADlateli a vyma=C5=BEte ji ze sv=C3=A9ho syst=C3=A9mu.
> The sole purpose of this message is to provide information based on mater=
ials that were given or sent to the sender. The content of the message does=
not place its sender under any obligation, unless expressly stated otherwi=
se, and the sender has no intention to conclude a contract, accept an offer=
or confirm the conclusion of a contract on the basis of this message nor s=
hall the message give rise to pre-contractual liability of the sender, unle=
ss it expressly states otherwise. The message is only intended for private,=
confidential use by a person (persons) mentioned above. If you are not the=
intended recipient of this message, you are hereby notified that any disse=
mination, distribution or copying thereof is prohibited. If you have receiv=
ed the message in error, please, inform the sender of the fact and delete i=
t from your system.
> _______________________________________________
> subgit-users mailing list
>
subgit...@subgit.com
>
http://lists.tmatesoft.com/mailman/listinfo/subgit-users