Not able to update diff with umlaut characters

51 views
Skip to first unread message

Vinayak Awasare

unread,
Mar 19, 2020, 2:26:43 AM3/19/20
to Review Board Community
Hi 

Following are contents of my diff which contains umlaut characters in diff file path.


--- //Bcmtools_S/CDE2020-DEV/Tools/Test/Kächele/Resources.fr.resx       //Bcmtools_S/CDE2020-DEV/Tools/Test/Kächele/Resources.fr.resx#1
+++ //Bcmtools_S/CDE2020-DEV/Tools/Test/Kächele/Resources.fr.resx       2020-03-18 12:15:41
@@ -3,7 +3,7 @@ <?xml version="1.0" encoding="utf-8"?
   <!--
     Microsoft ResX Schema

-    Version 2.0
+    Version 1.0

     The primary goals of this format is to allow a simple XML format
     that is mostly human readable. The generation and parsing of the
--- //Bcmtools_S/CDE2020-DEV/Tools/Test/Resources.fr.resx       //Bcmtools_S/CDE2020-DEV/Tools/Test/Resources.fr.resx#1
+++ //Bcmtools_S/CDE2020-DEV/Tools/Test/Resources.fr.resx       2020-03-18 12:13:42
@@ -3,7 +3,7 @@ <?xml version="1.0" encoding="utf-8"?
   <!--
     Microsoft ResX Schema

-    Version 2.0
+    Version 1.0

     The primary goals of this format is to allow a simple XML format
     that is mostly human readable. The generation and parsing of the

Starting new HTTP connection (1): server
http://server:80 "POST /mscrb/api/review-requests/257382/diffs/ HTTP/1.1" 400 126
-----------------response----------------
<Response [400]>

David Trowbridge

unread,
Mar 19, 2020, 2:28:23 AM3/19/20
to revie...@googlegroups.com
Hi,

A couple questions:

1. It looks like you’re using some custom tool to post this. Can you detail what your workflow looks like?

2. Can you get (or ask your server administrator to get) the error from the server log?

David

--
Supercharge your Review Board with Power Pack: https://www.reviewboard.org/powerpack/
Want us to host Review Board for you? Check out RBCommons: https://rbcommons.com/
Happy user? Let us know! https://www.reviewboard.org/users/
---
You received this message because you are subscribed to the Google Groups "Review Board Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/reviewboard/c85e15c6-2ac0-4ae9-a794-e7210df6534f%40googlegroups.com.

Vinayak Awasare

unread,
Mar 19, 2020, 4:03:45 PM3/19/20
to Review Board Community
Thanks for Quick Response 


1. It looks like you’re using some custom tool to post this. Can you detail what your workflow looks like?
Ans -> Following is workflow of my rbtools client

i) Connect with perforce and get changelist info and file list
For example file list : ['//Depo_Name/Branch/Tools/Test/Kächele/Resources.fr.resx']

ii) Compute diff of workspace file to old revision file using diff.exe given by reviewboard 
For example :

Command to compute diff :
D:\\<path>\\bin\\diff.exe -urNp C:\\<path>\\vawasar_g1vqthpt D:\\Workspace\\Perforce\\vawasar_IND-AWASARE_2020_dev1\\Tools\\Kächele\\Resources.fr.resx']

Result of above command:
--- C:\Users\vawasar\AppData\Local\Temp\MSC_RBTools_v3\vawasar_g1vqthpt 2020-03-19 16:32:16.112210800 +0530
+++ D:\Workspace\Perforce\vawasar_IND-AWASARE_2020_dev1\Tools\Test\Kächele\Resources.fr.resx    2020-03-18 12:15:41.970299800 +0530
@@ -3,7 +3,7 @@ <?xml version="1.0" encoding="utf-8"?
   <!--
Microsoft ResX Schema

-    Version 2.0
+    Version 1.0

The primary goals of this format is to allow a simple XML format
that is mostly human readable. The generation and parsing of the


iii) Create Review Request :
Making HTTP GET request to http://host-name/mscrb/api/repositories/
Cached response for HTTP GET http://host-name/mscrb/api/repositories/ expired and was modified

Using perforce repository on reviewboard to fetch clist info : pf-sa-1666

Cached response for HTTP GET http://host-name/mscrb/api/review-requests/ expired and was modified
Making HTTP POST request to http://host-name/mscrb/api/review-requests/

Review request '257565' created.


iv) Modify diff and upload diff to Reviewboard server

diff_contents = '''
--- //Bcmtools_S/CDE2020-DEV/Tools/Test/Kächele/Resources.fr.resx       //Bcmtools_S/CDE2020-DEV/Tools/Test/Kächele/Resources.fr.resx#1
+++ //Bcmtools_S/CDE2020-DEV/Tools/Test/Kächele/Resources.fr.resx       2020-03-18 12:15:41
@@ -3,7 +3,7 @@ <?xml version="1.0" encoding="utf-8"?
   <!--
Microsoft ResX Schema

-    Version 2.0
+    Version 1.0

The primary goals of this format is to allow a simple XML format
that is mostly human readable. The generation and parsing of the

'''
Added cache entry for HTTP GET request to http://host-name/mscrb/api/review-requests/257616/diffs/

Error while posting review request with message :
Remote end closed connection without response
Traceback (most recent call last):
  File "rb.py", line 443, in _run
rbtools_client.post_review_request(review_request, diff_contents, codeline_name, summary, description, myjobs, options_info, existing_reviewers_people, existing_reviewers_group, description_text_type=description_text_type)
  File "D:\Workspace\Perforce\vawasar_IND-AWASARE_2020_dev1\Tools\RBTools\source\rbtools_wrapper.py", line 143, in post_review_request
review_request.get_diffs().upload_diff(diff_contents)
  File "D:\Python36\lib\site-packages\rbtools\api\decorators.py", line 27, in request_method
*args, **kwargs)
  File "D:\Python36\lib\site-packages\rbtools\api\transport\sync.py", line 77, in execute_request_method
return self._execute_request(request)
  File "D:\Python36\lib\site-packages\rbtools\api\transport\sync.py", line 86, in _execute_request
rsp = self.server.make_request(request)
  File "D:\Python36\lib\site-packages\rbtools\api\request.py", line 685, in make_request
request.url, body, headers, request.method))
  File "D:\Python36\lib\site-packages\rbtools\api\cache.py", line 209, in make_request
return self.urlopen(request)
  File "D:\Python36\lib\urllib\request.py", line 223, in urlopen
return opener.open(url, data, timeout)
  File "D:\Python36\lib\urllib\request.py", line 526, in open
response = self._open(req, data)
  File "D:\Python36\lib\urllib\request.py", line 544, in _open
'_open', req)
  File "D:\Python36\lib\urllib\request.py", line 504, in _call_chain
result = func(*args)
  File "D:\Python36\lib\urllib\request.py", line 1346, in http_open
return self.do_open(http.client.HTTPConnection, req)
  File "D:\Python36\lib\urllib\request.py", line 1321, in do_open
r = h.getresponse()
  File "D:\Python36\lib\http\client.py", line 1331, in getresponse
response.begin()
  File "D:\Python36\lib\http\client.py", line 297, in begin
version, status, reason = self._read_status()
  File "D:\Python36\lib\http\client.py", line 266, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
Added cache entry for HTTP GET request to http://host-name/mscrb/api/review-requests/257616/



2. Can you get (or ask your server administrator to get) the error from the server log?
Ans ->
There is no any error in server log.


Please help !




On Thursday, March 19, 2020 at 11:58:23 AM UTC+5:30, David Trowbridge wrote:
Hi,

A couple questions:

1. It looks like you’re using some custom tool to post this. Can you detail what your workflow looks like?

2. Can you get (or ask your server administrator to get) the error from the server log?

David
To unsubscribe from this group and stop receiving emails from it, send an email to revie...@googlegroups.com.

Vinayak Awasare

unread,
Mar 21, 2020, 12:19:21 AM3/21/20
to Review Board Community
diff_contents = '''
--- //Bcmtools_S/CDE2020-DEV/Tools/Test/Kächele/Resources.fr.resx       //Bcmtools_S/CDE2020-DEV/Tools/Test/Kächele/Resources.fr.resx#1
+++ //Bcmtools_S/CDE2020-DEV/Tools/Test/Kächele/Resources.fr.resx       2020-03-18 12:15:41
@@ -3,7 +3,7 @@ <?xml version="1.0" encoding="utf-8"?
   <!--
 Microsoft ResX Schema

-    Version 2.0
+    Version 1.0

 The primary goals of this format is to allow a simple XML format
 that is mostly human readable. The generation and parsing of the

'''
As you can see there is a umlaut character in diff , which cause above failure...

Please help for same

Christian Hammond

unread,
Mar 24, 2020, 8:29:57 PM3/24/20
to revie...@googlegroups.com
This *might* be due to a bug that will be fixed in RBTools (planning for this week). Keep an eye out on this mailing list and I'll post when that's out.

What version of Review Board is being run there?

Christian

To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/reviewboard/4619e3bf-50a4-467b-b857-af98a0e222b3%40googlegroups.com.


--
Christian Hammond
President/CEO of Beanbag
Makers of Review Board

Vinayak Awasare

unread,
Mar 25, 2020, 3:37:42 AM3/25/20
to Review Board Community
Thanks Christian for quick response.

We are using ReviewBoard version : 3.0.15 

Please inform me once this issue will be fixed, i will be waiting for reply.

Regards,
Vinayak

On Wednesday, March 25, 2020 at 5:59:57 AM UTC+5:30, Christian Hammond wrote:
This *might* be due to a bug that will be fixed in RBTools (planning for this week). Keep an eye out on this mailing list and I'll post when that's out.

What version of Review Board is being run there?

Christian

Vinayak Awasare

unread,
May 8, 2020, 2:54:08 AM5/8/20
to Review Board Community
Hi Christian,

Did you get a chance to look at this issue ?
We are still facing issues in dealing with special characters like umlaut.

Really appreciate your help.

Regards,
Vinayak

Christian Hammond

unread,
May 8, 2020, 5:02:50 AM5/8/20
to revie...@googlegroups.com
Can you show me an RBTools debug log using the new release?

Christian


To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/reviewboard/926f904a-aa86-4ee2-b4e1-6e25a83b6142%40googlegroups.com.
--

Vinayak Awasare

unread,
May 8, 2020, 5:50:15 AM5/8/20
to Review Board Community
Hi Christian,

Thanks for quick response.

My RBTools version :
RBTools                          1.0.3
ReviewBoard server       3.0.15 

My diff : in this diff "Kächele" word contains umlaut character

--- //Bcmtools_S/CDE2020-DEV-VA/Tools/Test/Kächele/Resources.fr.resx    //Bcmtools_S/CDE2020-DEV-VA/Tools/Test/Kächele/Resources.fr.resx#1
+++ //Bcmtools_S/CDE2020-DEV-VA/Tools/Test/Kächele/Resources.fr.resx    2020-05-07 23:04:13
@@ -3,7 +3,7 @@ <?xml version="1.0" encoding="utf-8"?
   <!--
     Microsoft ResX Schema

-    Version 2.0
+    Version 1.0

     The primary goals of this format is to allow a simple XML format
     that is mostly human readable. The generation and parsing of the


 Error while posting review request with message :
 Remote end closed connection without response
Traceback (most recent call last):
  File "rb.py", line 441, in _run
    rbtools_client.post_review_request(review_request, diff_contents, codeline_name, summary, description, myjobs, options_info, existing_reviewers_people, existing_reviewers_group, description_text_type=description_text_type)
  File "D:\Workspace\Perforce\rbtools_wrapper.py", line 143, in post_review_request

Really appreciate your help.

Regards,
Vinayak
On Friday, May 8, 2020 at 2:32:50 PM UTC+5:30, Christian Hammond wrote:
Can you show me an RBTools debug log using the new release?

Christian

Christian Hammond

unread,
May 8, 2020, 11:50:18 PM5/8/20
to revie...@googlegroups.com
Thanks, Vinayak,

Would you mind outputting the diff to a file and zipping it in some form, and then sending it here? That’ll help preserve encodings, which will be necessary to take a closer look.

I can’t promise I can get you a resolution right away, as I’m extremely swamped right now, but if we can find a clear flaw in some part of RBTools or Review Board that we can get into the next release, we’ll definitely do that.

Christian


To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/reviewboard/a770abba-c258-47de-a803-733ba35c5595%40googlegroups.com.

Vinayak Awasare

unread,
May 9, 2020, 4:51:13 AM5/9/20
to Review Board Community
Hi Christian,

Thanks for reply.

Please find attached diff zip file.


Following is my code which is uploading diff

            f = open("D:\\gen_diff.diff", mode="r")
            diff_contents = f.read()
            f.close()
            review_request.get_diffs().upload_diff(diff_contents)

Following is the error which occurred at my end :

 Error while posting review request with message :
 Remote end closed connection without response
Traceback (most recent call last):
  File "rb.py", line 441, in _run
    rbtools_client.post_review_request(review_request, diff_contents, codeline_name, summary, description, myjobs, options_info, existing_reviewers_people, existing_reviewers_group, description_text_type=description_text_type)
  File "D:\Workspace\Perforce\vawasar_IND-AWASARE_CDE2020-DEV-VA_7046\Tools\RBTools\source\rbtools_wrapper.py", line 149, in post_review_request
    review_request.get_diffs().upload_diff(diff_contents.encode("UTF-8"))

On Saturday, May 9, 2020 at 9:20:18 AM UTC+5:30, Christian Hammond wrote:
Thanks, Vinayak,

Would you mind outputting the diff to a file and zipping it in some form, and then sending it here? That’ll help preserve encodings, which will be necessary to take a closer look.

I can’t promise I can get you a resolution right away, as I’m extremely swamped right now, but if we can find a clear flaw in some part of RBTools or Review Board that we can get into the next release, we’ll definitely do that.

Christian

gen_diff.zip

Christian Hammond

unread,
May 18, 2020, 12:10:27 AM5/18/20
to revie...@googlegroups.com
So a couple of issues that I'm seeing (and it may not be limited to these):

1) Your diff file doesn't appear to be UTF-8. There's some other encoding in the filenames. You might have to figure out the situation there and deal with it on your end. Encodings and diffs and filenames are a giant mess that you may have to figure out your own applicable solution to.
2) Your script appears to be thinking it's a Unicode string that it can encode to UTF-8 (judging by the error output — that doesn't really match your example code). Instead, the file contents should be binary, and you should ensure that on Windows by using a file mode of 'rb' instead of 'r' when opening it, and then passing that directly.

Since this is a custom integration, I can't spend a lot of time helping to debug this here. We do offer more hands-on assistance with custom integrations of this level as part of a Premium Support contract, which I can talk to you more about.

Christian

To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/reviewboard/fab187ac-40c0-4b0d-86e0-ee8af5ae6d49%40googlegroups.com.

Vinayak Awasare

unread,
May 18, 2020, 2:38:18 AM5/18/20
to Review Board Community
Hi Christian,

Thanks for reply.

We will check issue from our side once again.


Regards,
Vinayak

On Monday, May 18, 2020 at 9:40:27 AM UTC+5:30, Christian Hammond wrote:
So a couple of issues that I'm seeing (and it may not be limited to these):

1) Your diff file doesn't appear to be UTF-8. There's some other encoding in the filenames. You might have to figure out the situation there and deal with it on your end. Encodings and diffs and filenames are a giant mess that you may have to figure out your own applicable solution to.
2) Your script appears to be thinking it's a Unicode string that it can encode to UTF-8 (judging by the error output — that doesn't really match your example code). Instead, the file contents should be binary, and you should ensure that on Windows by using a file mode of 'rb' instead of 'r' when opening it, and then passing that directly.

Since this is a custom integration, I can't spend a lot of time helping to debug this here. We do offer more hands-on assistance with custom integrations of this level as part of a Premium Support contract, which I can talk to you more about.

Christian

Reply all
Reply to author
Forward
0 new messages