post-review error "Unable to parse diff header:"

[Amit Mokal]

Hi

I would like to thank you for developing review board.

I am using review board with perforce.
When I use the post-review script I get the error
"Unable to parse diff header:"

Running the script with debug set to True gives
-----------------------------------------------------------------------------
$ python post-review.py 52004
>>> svn info
>>> p4 info
>>> repository info: Path: milan.engdlp.symantec.com:1666, Base path: None, Supports changesets: True
>>> Repository info 'Path: milan.engdlp.symantec.com:1666, Base path: None, Supports changesets: True'
>>> Generating diff for changenum 52004
>>> p4 describe -s 52004
>>> Processing edit of //depot/Vontu9/dev/native/src/endpoint/Agent/DetectionCore/ExecutionGraphStatePool.cpp
>>> Writing "//depot/Vontu9/dev/native/src/endpoint/Agent/DetectionCore/ExecutionGraphStatePool.cpp#2" to "/c/DOCUME~1/ADMINI~1/LOCALS~1/Temp/tmpkWJo
L"
>>> p4 print -q //depot/Vontu9/dev/native/src/endpoint/Agent/DetectionCore/ExecutionGraphStatePool.cpp#2
>>> p4 where //depot/Vontu9/dev/native/src/endpoint/Agent/DetectionCore/ExecutionGraphStatePool.cpp
>>> diff -urNp /c/DOCUME~1/ADMINI~1/LOCALS~1/Temp/tmpkWJoCL C:/dev/sources\Vontu9\dev\native\src\endpoint\Agent\DetectionCore\ExecutionGraphStatePool
cpp

+++ C:/dev/sources\Vontu9\dev\native\src\endpoint\Agent\DetectionCore
\ExecutionGraphStatePool.cpp

Unable to parse diff header: +++ C:/dev/sources\Vontu9\dev\native\src
\endpoint\Agent\DetectionCore\ExecutionGraphStatePool.cpp
----------------------------------------------------------------------------

My environment is cygwin. I am using
python 2.5.1
diffutils 2.8.7

If I copy paste the diff command to the command line and save the
output to a file I actually see a timestamp. However when the script
executes the command the time stamp is missing.

Thanks
Amit

[Christian Hammond]

Interesting. It could be that there's assumptions being made that confuse post-review when running inside the cygwin shell instead of a DOS prompt on Windows. The file paths look very inconsistent. One file is UNIX-style (/foo/bar), whereas the other is a mix of Windows (C:\...) and then partly UNIX. It's probably due to where we're getting these paths and how we're concatenating them.

Can you run the same command in a DOS box and see what happens?

Christian

--
Christian Hammond - chi...@chipx86.com
VMware, Inc.

[AndyP]

I am attempting to upgrade to the latest-and-greatest and am now
running into this exact same problem.

post-review under cygwin (under DOS).

It looks like the "p4 where" command is returning a DOS style path,
and then if you look in a redirected differences output file, all the
letters are scrunched together with the '\' characters missing.

--- /cygdrive/c/DOCUME~1/APETER~1/LOCALS~1/Temp/Device.cpp 2008-08-12
10:52:53.549697400 -0700
+++ c:companyproductcoresrcwin32desktopDevice.cpp 1969-12-31
16:00:00.000000000 -0800


... and of course, the +++ file does not exist.

On Oct 9, 5:55 pm, "Christian Hammond" <chip...@chipx86.com> wrote:
> Interesting. It could be that there's assumptions being made that confuse
> post-review when running inside the cygwin shell instead of a DOS prompt on
> Windows. The file paths look very inconsistent. One file is UNIX-style
> (/foo/bar), whereas the other is a mix of Windows (C:\...) and then partly
> UNIX. It's probably due to where we're getting these paths and how we're
> concatenating them.
>
> Can you run the same command in a DOS box and see what happens?
>
> Christian
>
> --

> Christian Hammond - chip...@chipx86.com

> VMware, Inc.
>
>
>
> On Thu, Oct 9, 2008 at 5:45 PM, Amit Mokal <amitmo...@gmail.com> wrote:
>
> > Hi
>
> > I would like to thank you for developing review board.
>
> > I am using review board with  perforce.
> > When I use the post-review script I get the error
> > "Unable to parse diff header:"
>
> > Running the script with debug set to True gives
>

> > ---------------------------------------------------------------------------­--

> > $ python post-review.py 52004
> > >>> svn info
> > >>> p4 info
> > >>> repository info: Path: milan.engdlp.symantec.com:1666, Base path:
> > None, Supports changesets: True
> > >>> Repository info 'Path: milan.engdlp.symantec.com:1666, Base path:
> > None, Supports changesets: True'
> > >>> Generating diff for changenum 52004
> > >>> p4 describe -s 52004
> > >>> Processing edit of

> > //depot/Vontu9/dev/native/src/endpoint/Agent/DetectionCore/ExecutionGraphSt­atePool.cpp
> > >>> Writing
> > "//depot/Vontu9/dev/native/src/endpoint/Agent/DetectionCore/ExecutionGraphS­tatePool.cpp#2"

> > to "/c/DOCUME~1/ADMINI~1/LOCALS~1/Temp/tmpkWJo
> > L"
> > >>> p4 print -q

> > //depot/Vontu9/dev/native/src/endpoint/Agent/DetectionCore/ExecutionGraphSt­atePool.cpp#2
> > >>> p4 where
> > //depot/Vontu9/dev/native/src/endpoint/Agent/DetectionCore/ExecutionGraphSt­atePool.cpp
> > >>> diff -urNp /c/DOCUME~1/ADMINI~1/LOCALS~1/Temp/tmpkWJoCL
> > C:/dev/sources\Vontu9\dev\native\src\endpoint\Agent\DetectionCore\Execution­GraphStatePool

> > cpp
>
> > +++ C:/dev/sources\Vontu9\dev\native\src\endpoint\Agent\DetectionCore
> > \ExecutionGraphStatePool.cpp
>
> > Unable to parse diff header: +++ C:/dev/sources\Vontu9\dev\native\src
> > \endpoint\Agent\DetectionCore\ExecutionGraphStatePool.cpp
>

> > ---------------------------------------------------------------------------­-

>
> > My environment is cygwin. I am using
> > python 2.5.1
> > diffutils 2.8.7
>
> > If I copy paste the diff command to the command line and save the
> > output to a file I actually see a timestamp. However when the script
> > executes the command the time stamp is missing.
>
> > Thanks

> > Amit- Hide quoted text -
>
> - Show quoted text -

[AndyP]

I'm stuck.

I did a difference between the 0.7 version I was using (a patched
version from a while ago) and the new pristine 0.8 version. The diff
command line is different, so I figured I'd just use the older
version. The execute subroutine is pretty significantly different too
though, so I wasn't able to complete this effort. I don't know python
enough to do anything substantial.

I'd really like to use the latest and greatest (unpatched), but what I
downloaded doesn't work under cygwin.

---

# new version of diff:
diff_cmd = ["diff", "-urNp", old_file, new_file]

# old version of diff (0.7)
diff_cmd = "diff -urNp \"%s\" \"%s\"" % (old_file, new_file)


Whatever these differences mean, the v0.8 execute command is unable to
handle the v0.7 diff command.

[Christian Hammond]

Maybe I missed this part in your response, but did you try running post-review in a standard DOS box with command.com?

Christian

--
Christian Hammond - chi...@chipx86.com
VMware, Inc.

[AndyP]

Nope. all the python stuff is under cygwin - doesn't run (without
some serious setup issues) under dos. Note that this used to work
with the previous version of post-review (v0.7)... under cygwin.

[Joshua Slominski]

I am also getting this error my exact error is "Unable to parse diff header: : No such file or directory"

 

I put in some more debug functions and this is what i get:

 

>>> Repository info 'Path: isis.hoem.ad.harman.com:1666, Base path: None, Supports changesets: True'
>>> Generating diff for changenum 163531
>>> p4 describe -s 163531
>>> making tempfile
>>> fd == 6
>>> tmpfile == d:\docume~1\jslomi~1\locals~1\temp\tmpodiyp_
>>> making tempfile
>>> fd == 6
>>> tmpfile == d:\docume~1\jslomi~1\locals~1\temp\tmpnfuhoi
>>> making tempfile
>>> fd == 6
>>> tmpfile == d:\docume~1\jslomi~1\locals~1\temp\tmpahrqhd
>>> Processing edit of //DC_Refresh/Development/Main/HMIController/cocos/hmi/common/app/private/CHMIAppImpl.cpp
>>> Writing "//DC_Refresh/Development/Main/HMIController/cocos/hmi/common/app/private/CHMIAppImpl.cpp#13" to "d:\docume~1\jslomi~1\locals~1\temp\tmpnfuhoi"
>>> p4 print -q //DC_Refresh/Development/Main/HMIController/cocos/hmi/common/app/private/CHMIAppImpl.cpp#13
>>> f.name = d:\docume~1\jslomi~1\locals~1\temp\tmpnfuhoi
>>> p4 where //DC_Refresh/Development/Main/HMIController/cocos/hmi/common/app/private/CHMIAppImpl.cpp
>>> diff -urNpbBw d:\docume~1\jslomi~1\locals~1\temp\tmpnfuhoi D:/NTG4/Refresh\cocos\hmi\common\app\private\CHMIAppImpl.cpp

I think the problem is that the temp file is never being created but i don't know why.

[Joshua Slominski]

I fixed mine (sort of) in r1512 on line 1274, the shell option in subprocess.Popen() was changed from True to False.  When I changed it back to True it started working agian.  I don't know what the intent was so i don't know if i fixed it properly.  Does anyone have any insigts on why this would fix my problem?

[AndyP]

That didn't work for me out of the box. The two Popen() statements
are different than the version that I had (v0.7 that worked), but when
I changed it to True I got a different error:

"The current directory does not contain a checkout from a supported
source code repository."


Looking at this further, it appears that the only real difference is
the way the execute command is created:

v0.7
data = execute('p4 info', ignore_errors=True)

v0.8
data = execute(["p4", "info"], ignore_errors=True)

On Oct 15, 12:45 pm, "Joshua Slominski" <hoohaa8...@gmail.com> wrote:

[Joshua Slominski]

I seen that error when i forgot to put in my review board URL in post-review.  Also be sure that post-review is using the correct SCM tool.  It defaults to SVN if it can't find anything else.

[AndyP]

Well I have a .reviewboardrc with valid stuff in it. I modified (just
now) the script too with the same data but it didn't work.

I really suspect that that the 'execute' command isn't working right.

Note that it gets past that if I change those Popen calls from 'True'
back to 'False'.

If they are 'False' it can't find the repository, and if it is 'True'
it can't execute the diff (per the original posts in this thread).

Thanks,

> > On Oct 15, 12:45 pm, "Joshua Slominski" <hoohaa8...@gmail.com> wrote:- Hide quoted text -

[Joshua Slominski]

Can you post the debug output?

Sent from my iPhone

[Florian Föbel]

Hi,

I just got the same error after upgrading post-review. What I found out is that the subprocess.stdout that is read for the diff command contains a newline (\n) right after the second filename in the diff header. Kind of weird...

No fix or explanation found yet.

Regards,
Florian

2008/10/16 Joshua Slominski <hooha...@gmail.com>

[tom]

Hi,

Has this been resolved?

If not, this is to do with the fact that the newline character(s) are
in the name of the local file extracted:

here's a diff of what I did to fix the issue as a quick solution:

#
# Repository URL.
@@ -963,6 +963,7 @@
else:
die("Unknown change type '%s' for %s" % (changetype,
depot_path))

+ new_file = new_file.rstrip('\r\n');

diff_cmd = ["diff", "-urNp", old_file, new_file]

# Diff returns "1" if differences were found.


However, I got an internal server error when I tried a test post:

Unable to access http://localhost:8000/api/json/reviewrequests/new/.
The host path may be invalid
HTTP Error 500: INTERNAL SERVER ERROR

logs:

[28/Oct/2008 05:59:48] "POST /api/json/reviewrequests/new/ HTTP/1.1"
500 96260


On Oct 16, 8:54 am, "Florian Föbel" <arenaweb...@googlemail.com>
wrote:

> Hi,
>
> I just got the same error after upgrading post-review. What I found out is
> that the subprocess.stdout that is read for the diff command contains a
> newline (\n) right after the second filename in the diff header. Kind of
> weird...
>
> No fix or explanation found yet.
>
> Regards,
> Florian
>

> 2008/10/16 Joshua Slominski <hoohaa8...@gmail.com>

>
>
>
> > Can you post the debug output?
>
> > Sent from my iPhone
>

[Christian Hammond]

If you run with --debug, you should be able to see more error info.

Christian

--
Christian Hammond - chi...@chipx86.com
VMware, Inc.

[13Strider]

>>> Repository info 'Path: salmon.test.local:1999, Base path: None, Supports changesets: True'
>>> Generating diff for changenum 121473
>>> p4 describe -s 121473
>>> Processing edit of //data/cmn/ini/common.ini
>>> Writing "//data/cmn/ini/common.ini#1" to "c:\users\sam1\appdata\local\temp\tmpliutrf"
>>> p4 print -q //data/cmn/ini/common.ini#1
>>> p4 where //data/cmn/ini/common.ini
>>> diff -urNp c:\users\sam1\appdata\local\temp\tmpliutrf c:\data\cmn\ini\common.ini

Unable to parse diff header: : Invalid argument

On Oct 28, 4:46 pm, "Christian Hammond" <chip...@chipx86.com> wrote:
> If you run with --debug, you should be able to see more error info.
>
> Christian
>
> --

> Christian Hammond - chip...@chipx86.com
> VMware, Inc.

>
> On Tue, Oct 28, 2008 at 5:00 AM, tom <december1...@gmail.com> wrote:
>
> > Hi,
>
> > Has this been resolved?
>
> > If not, this is to do with the fact that the newline character(s) are
> > in the name of the local file extracted:
>
> > here's a diff of what I did to fix the issue as a quick solution:
>
> >  #
> >  # Repository URL.
> > @@ -963,6 +963,7 @@
> >             else:
> >                 die("Unknown change type '%s' for %s" % (changetype,
> > depot_path))
>
> > +            new_file = new_file.rstrip('\r\n');
> >              diff_cmd = ["diff", "-urNp", old_file, new_file]
> >              # Diff returns "1" if differences were found.
>
> > However, I got an internal server error when I tried a test post:
>

> > Unable to accesshttp://localhost:8000/api/json/reviewrequests/new/.

[tom]

Hi Christian,


Thanks for the tip - the 500 is because of an exception to do with a
licensing issue with p4:

Traceback:
File "/usr/lib/python2.5/site-packages/django/core/handlers/base.py"
in get_response
86. response = callback(request, *callback_args,
**callback_kwargs)
File "/usr/lib/python2.5/site-packages/django/views/decorators/
cache.py" in _wrapped_view_func
44. response = view_func(request, *args, **kwargs)
...
...
35. changeset =
repository.get_scmtool().get_changeset(changenum)
File "/home/guest/dist/reviewboard/scmtools/perforce.py" in
get_changeset
57. changeset = self.p4.run_describe(&#39;-s&#39;,
str(changesetid))
File "/usr/lib/python2.5/site-packages/P4.py" in &lt;lambda&gt;
215. f = lambda *args: self.run(cmd, *args)
File "/usr/lib/python2.5/site-packages/P4.py" in run
260. return P4API.P4Adapter.run(self, *self.__flatten(args))

Exception Type: P4Exception at /api/json/reviewrequests/new/
Exception Value: [P4#run] Errors during command execution( &quot;p4
describe -s 23710&quot; )

[Error]: Can&#39;t create a new user - over license quota.
Try deleting old users with &#39;user -d&#39;.
License count: 55 users used of 55 licensed.

The reason for this was that I needed to "source ~/.p4c" from the
shell, so that it could set the valid $P4USER, before then running ./
manage.py runserver

On Oct 28, 7:46 pm, "Christian Hammond" <chip...@chipx86.com> wrote:
> If you run with --debug, you should be able to see more error info.
>
> Christian
>
> --

> Christian Hammond - chip...@chipx86.com
> VMware, Inc.

>
> On Tue, Oct 28, 2008 at 5:00 AM, tom <december1...@gmail.com> wrote:
>
> > Hi,
>
> > Has this been resolved?
>
> > If not, this is to do with the fact that the newline character(s) are
> > in the name of the local file extracted:
>
> > here's a diff of what I did to fix the issue as a quick solution:
>
> >  #
> >  # Repository URL.
> > @@ -963,6 +963,7 @@
> >             else:
> >                 die("Unknown change type '%s' for %s" % (changetype,
> > depot_path))
>
> > +            new_file = new_file.rstrip('\r\n');
> >              diff_cmd = ["diff", "-urNp", old_file, new_file]
> >              # Diff returns "1" if differences were found.
>
> > However, I got an internal server error when I tried a test post:
>

> > Unable to accesshttp://localhost:8000/api/json/reviewrequests/new/.

[13Strider]

Have you fixed this?

On Oct 28, 9:00 am, tom <december1...@gmail.com> wrote:
> Hi,
>
> Has this been resolved?
>
> If not, this is to do with the fact that the newline character(s) are
> in the name of the local file extracted:
>

> here's adiffof what I did to fix the issue as a quick solution:

>
>  #
>  # Repository URL.
> @@ -963,6 +963,7 @@
>              else:
>                  die("Unknown change type '%s' for %s" % (changetype,
> depot_path))
>
> +            new_file = new_file.rstrip('\r\n');
>              diff_cmd = ["diff", "-urNp", old_file, new_file]

>              #Diffreturns "1" if differences were found.

>
> However, I got an internal server error when I tried a test post:
>

> Unableto accesshttp://localhost:8000/api/json/reviewrequests/new/.

> The host path may be invalid
> HTTP Error 500: INTERNAL SERVER ERROR
>
> logs:
>
> [28/Oct/2008 05:59:48] "POST /api/json/reviewrequests/new/ HTTP/1.1"
> 500 96260
>
> On Oct 16, 8:54 am, "Florian Föbel" <arenaweb...@googlemail.com>
> wrote:
>
> > Hi,
>
> > I just got the same error after upgrading post-review. What I found out is

> > that the subprocess.stdout that is read for thediffcommand contains a
> > newline (\n) right after the second filename in thediffheader. Kind of

> > weird...
>
> > No fix or explanation found yet.
>
> > Regards,
> > Florian
>
> > 2008/10/16 Joshua Slominski <hoohaa8...@gmail.com>
>
> > > Can you post the debug output?
>
> > > Sent from my iPhone
>
> > > On Oct 15, 2008, at 6:00 PM, AndyP <apsoftw...@gmail.com> wrote:
>
> > > > Well I have a .reviewboardrc with valid stuff in it.  I modified (just
> > > > now) the script too with the same data but it didn't work.
>
> > > > I really suspect that that the 'execute' command isn't working right.
>
> > > > Note that it gets past that if I change those Popen calls from 'True'
> > > > back to 'False'.
>
> > > > If they are 'False' it can't find the repository, and if it is 'True'

> > > > it can't execute thediff(per the original posts in this thread).