Spaces in source paths

[ChrisD]

We have a project in which the source paths contain spaces, and they are
getting messed up during source server prep, apparently by ssindex. The
folder name includes "12.0 Inline 1" but it's getting stored in the PDBs as
"12.020Inline201". Clearly the %'s are getting dropped from
"12.0%20Inline%201". As a result the SVN commands to download the files don't
work.

Does anyone know what might be causing this and how to fix it?

TIA,
Chris

[pat styles [microsoft]]

Hello Chris.

I don't know anything about Subversion, but I did some checking. The
general source server scripts work fine with directories that have spaces in
them. And glancing at svn.pm, I didn't see any problems that stand out to
me.

One thing did stand out to me. In your email, you mention the errors in
processing %20 which seem to be replacing the space character. There is
nothing in Perl or in our scripts that would make such a replacement. Is
this something you might know about? It seems to be the root of your
problem.

.pat styles [microsoft]

"ChrisD" <Chr...@discussions.microsoft.com> wrote in message
news:D70C6D8D-7A54-46E0...@microsoft.com...

[ChrisD]

No, this is something that is occurring while symstore is running. I guess
I'm going to have to do some debugging and see where the 20's are coming
from. And I'm not much of a Perl guy. :)

Thanks,
Chris

"pat styles [microsoft]" <pat.s...@microsoft.com> wrote in message
news:357A77AD-D1CE-4DC0...@microsoft.com...

[ChrisD]

No, this is something that is occurring while ssindex is running. I guess

I'm going to have to do some debugging and see where the 20's are coming
from. And I'm not much of a Perl guy. :)

Thanks,
Chris

"pat styles [microsoft]" <pat.s...@microsoft.com> wrote in message
news:357A77AD-D1CE-4DC0...@microsoft.com...

[ChrisD]

(I don't know why the previous message posted twice. I had written "symstore"
instead of "ssindex" and edited it before sending, but somehow both versions
got posted.)

I don't know enough Perl to really understand what's going on, but svn.pm is
getting $FileRelative values with the %20's in GatherFileInformation, and at
some point the percent signs are disappearing, leaving just the "20"s. I
added a little loop to replace all occurrences of %20 with spaces before the
file is added to FILE_LOOKUP_TABLE, and it now works correctly.

[pat styles [microsoft]]

Hello Chris.

You can see that the $FileRelative values are fed by the following
command...

if ( ! open($hProcess, "$$self{'SVNCMD'} info -R \"$SourceRoot\"|") ) {

That means that the %20 sequences *seem* to be generated by the SVN.EXE
command. I do not have a copy of this program to test with, but you do.
Could you go to the root of your enlistment and simply type "svn.exe info -R
>foo.txt" and examine foo.txt for the %20 sequences. If this doesn't work,
how about trying it without the file redirection to foo.txt?

If you get the results I expect, then I think we need to implement your
changes in the svn.pm that ships with the debugger. I'll try to track down
the guy that wrote it and see if he has any comments.

.pat styles [microsoft]

"ChrisD" <Chr...@discussions.microsoft.com> wrote in message

news:508BBD72-3B7B-456A...@microsoft.com...

[ChrisD]

Hi Pat,

Yes, SVN is returning the %20s, so a path like "foo bar" is returned as
"foo%20bar".

As I said, I don't know beans about Perl, but doesn't the percent sign mean
something? If so, my guess would be that some kind of processing of the
contents of $FileRelative is removing them.

The little fix I implemented does work, but as always it would be better to
know what's wrong. :)

Thanks,
Chris

[P.]

Hi Pat

Just an FYI, I know we have the same problems using P4 when there are spaces
in the file or directory names.

P.

"pat styles [microsoft]" <pat.s...@microsoft.com> wrote in message
news:357A77AD-D1CE-4DC0...@microsoft.com...

[pat styles [microsoft]]

Hello Chris.

From my perspective, it would appear that the problem is with Subversion. I
can't understand why it would return path information that is not consistent
with with the operating system it is running on.

That said, would you be willing to make your change available to the
community? If so, please attach it to an email and send it to
pat.s...@microsoft.com.

.pat styles [microsoft]

"ChrisD" <Chr...@discussions.microsoft.com> wrote in message

news:E091ACDC-D740-4AC1...@microsoft.com...

[pat styles [microsoft]]

Interesting. Could you give me more details similar to what Chris provided?
We have made test runs here using directory paths with spaces and we have
not experienced this.

.pat styles [microsoft]

"P." <al...@newsgroup.nospam> wrote in message
news:C978F49C-A789-41EA...@microsoft.com...

[pat styles [microsoft]]

Chris and P.

Just to be sure here, are you running these scripts under cmd.exe or some
other shell?

.pat styles [microsoft]

"ChrisD" <Chr...@discussions.microsoft.com> wrote in message

news:E091ACDC-D740-4AC1...@microsoft.com...

[ChrisD]

Yes, we're running under cmd.exe, Vista SP1.

[ChrisD]

Hi Pat,

I would have to differ with your assessment. Since the source control
system is on an HTTP server, the file locations are URLs, and the use of %20
in the URLs would seem to me to be perfectly valid.

I'd be happy to share my "fix", but I don't think it's the right fix. If we
can agree that having the %20's in the URLs is valid, then the problem isn't
really in SVN, svn.pm, or, it turns out, even in ssindex. Fixing it in
svn.pm takes care of SVN users, but the problem could just crop up again
if/when someone adds support for some other SCS.

The reason why ssindex is off the hook is this: I found with a hex editor
that the URLs are, in fact, correctly stored in the modified PDBs. I find
data like this:

x:\projects\whatever.cpp*http://svnsrv/svn*dev/12.0%20Inline%201/whatever.cpp*9195

That's correct; the source is in the folder "http://svnsrv/svn/dev/12.0
Inline 1".

BUT, when I run srctool on the same PDB, this is what I see as the extract
command:

cmd /c svn.exe cat "http://svnsrv/svn/dev/12.0201/whatever.cpp@9195"

Note that the path is completely bollixed up (worse than I had previously
realized): "12.0%20Inline%201" has become "12.0201". That is what symstore
puts on the symbol server.

Does symstore use srctool, or maybe some common DLL? It certainly looks to
me like that is where the problem really is.

FWIW, here's the code I added to svn.pm. It follows the closing brace at
line 203, which is commented "# End loop through the file details" (I had to
dust off my emergency copy of the Perl black book, so this is probably Perl
most horrible):

if ($FileRevision) {
my $ix = index($FileRelative, "%20");
while ($ix != -1) {
substr($FileRelative, $ix, 3) = " ";
$ix = index($FileRelative, "%20");

[ChrisD]

Hi Pat,

I've just found that pdbstr does show the paths contained in the PDBs
correctly. So the problem is definitely in srctool or some DLL that it uses.

[pat styles [microsoft]]

Now I think I understand your situation.

Please email me the pdb in question.

.pat styles [microsoft]

"ChrisD" <Chr...@discussions.microsoft.com> wrote in message

news:D16FED88-32D7-4C02...@microsoft.com...

[pat styles [microsoft]]

Okay. So I forgot that % is a reserved character used by the Source Server
dll to indicate variable names and functions. You can see this mentioned in
srcsrv.doc. %% is a mechanism used to indicate %. What happens when you
change the script to put in %%20 where %20 used to be? This should fix
things including helping the debugger to extract the file as well as
srctool.exe.

Please tell me how this works for you.

.pat styles [microsoft]

"pat styles [microsoft]" <pat.s...@microsoft.com> wrote in message

news:801EDA8D-0A62-46CC...@microsoft.com...

[P.]

Pat, sure, I'll get the details when I return to the office in a week.

If I recall correctly, the problem occurred when the source was being
retrieved by WinDbg from P4 resulting in the P4 commands failing because of
missing double quotes around paths with spaces.

Happy holidays
P.

[ChrisD]

I think this works. srctool does show correct paths after the %'s are
doubled. I can't be absolutely (100%%) certain that it's all OK because
windbg seems to be coming up with the source too quickly for it to be coming
from a remote server. I cleared out the local cache folder; is the source
cached anywhere else?

In any event, to implement this, I replaced the code I sent in an earlier
reply with the following. (For our purposes the fix is in svn.pm, but I do
think that the permanent fix should be in ssindex.cmd so that it would work
for any SCS.)

if ($FileRelative) {
$FileRelative =~ s/%/%%/g;
}

As I said, I don't really speak perl, so if this isn't the right way to
replace all occurrences of % with %%, someone please let me know.

[pat styles [microsoft]]

Hello.

I would be very interested if you can reproduce this. But please make sure
that the pdbs were source indexed using the latest tools and that the
debugger you are using to extract the source is also the latest. I ask
because I have just run some tests here and found that our bits are indeed
putting spaces around the extraction path. I believe that earlier versions
of the source server technology did not do this.

.pat styles [microsoft]

"P." <al...@newsgroup.nospam> wrote in message

news:8C89ED42-5C9C-488E...@microsoft.com...

[P.]

Hi Pat

Here is an example of a path with problems (actual names have been changed
to remain anon):
“//depot/Aaa_Bbbb_Cccc/Aaa_Bbbb/trunk/src/AaaBbb/Engines/Cccc/Aaa Bbb Ccc/”.
Note the spaces in the last path.

I have not re-tested with 6.10.3.233, but using 6.9.3.113 and earlier, we
had to change srcsrv.ini to include:
P4_EXTRACT_CMD=p4.exe -p %fnvar%(%var2%) print -o %srcsrvtrg% -q
"//%var3%#%var4%"
Note the double quotes around the -q arguments.

I just checked the 6.10.3.233 script and the p4.pm script does not double
quote the path:
push(@stream, "P4_EXTRACT_CMD=p4.exe -p %fnvar%(%var2%) print ".
"-o %srcsrvtrg% ".
"-q //%var3%#%var4%");

Unfortunately it is not feasible to regenerate source info for old builds,
so we have to correct the behavior on the debugging clients.

If you observe a different behavior, then maybe the version of p4.exe or of
Pearl makes a difference?

Regards
P.

"pat styles [microsoft]" <pat.s...@microsoft.com> wrote in message

news:779E0572-E5FE-44A9...@microsoft.com...

[pat styles [microsoft]]

I think I got confused when you specified the problem concerned spaces
around the path. Furthermore I think the tester here did not actually try
to extract a file, but instead simply examined the stream. Regardless, I
will work on getting a fix. Thanks for your help.

.pat styles [microsoft]

"P." <al...@newsgroup.nospam> wrote in message

news:#Coxx5ea...@TK2MSFTNGP03.phx.gbl...

[P.]

Thanks Pat.

"pat styles [microsoft]" <pa...@microsoft.com> wrote in message
news:2999B22F-097D-46C2...@microsoft.com...