Issue 1088 in include-what-you-use: iwyu tool.py will fail if output contains non-ascii character

[notifi...@include-what-you-use.org]

New issue 1088 by kkpattern: iwyu_tool.py will fail if output contains non-ascii character
https://github.com/include-what-you-use/include-what-you-use/issues/1088

Currently, iwyu_tool.py will try to use utf-8 to [decode](https://github.com/include-what-you-use/include-what-you-use/blob/master/iwyu_tool.py#L261) include-what-you-use output. However, later when iwyu_tool.py try to [print](https://github.com/include-what-you-use/include-what-you-use/blob/master/iwyu_tool.py#L369) the output it forgets to encode the output back. If the source code contains non-ascii characters and happens to appear in the output(for our case, a line with non-ascii character comments appears in a compiler warning), the iwyu_tool.py will fail due to unicode error.

The fix is fairly simple, convert the output back to utf-8 encoding before print. If this fix is ok I'd like to create a PR later.

[notifi...@include-what-you-use.org]

Comment #1 on issue 1088 by kimgr: iwyu_tool.py will fail if output contains non-ascii character
https://github.com/include-what-you-use/include-what-you-use/issues/1088

That would be great, thanks! Any chance to produce a testcase for it, or is printing hard to test here?

[notifi...@include-what-you-use.org]

Comment #2 on issue 1088 by kkpattern: iwyu_tool.py will fail if output contains non-ascii character
https://github.com/include-what-you-use/include-what-you-use/issues/1088

I haven’t look carefully at iwyu_tool_test.py. But I think it wouldn’t be hard. I will give it a try.

[notifi...@include-what-you-use.org]

Comment #3 on issue 1088 by kkpattern: iwyu_tool.py will fail if output contains non-ascii character
https://github.com/include-what-you-use/include-what-you-use/issues/1088

After some more tests, I found that this only happens if the running python version is python2. Python3 seems to handle the output correctly. Since python2 is no longer supported, is this still needed to be fixed?

[notifi...@include-what-you-use.org]

Comment #4 on issue 1088 by kimgr: iwyu_tool.py will fail if output contains non-ascii character
https://github.com/include-what-you-use/include-what-you-use/issues/1088

@kkpattern Thanks for root-causing! I think we can leave python2 behind, but I won't work actively to remove support for it. I think it's fine to leave this behavior as-is for python2, but I'd be happy to take a simple fix if you have it :)