Jira (PDB-135) Improve warning message for invalid UTF-8 byte sequences

[Robin Bowes (JIRA)]

Robin Bowes commented on PDB-135

Re: Improve warning message for invalid UTF-8 byte sequences Is anyone actively looking at this? Add Comment

This message was sent by Atlassian JIRA (v6.3.7#6337-sha1:2ed701e)

[Matthias (JIRA)]

Matthias commented on PDB-135

Re: Improve warning message for invalid UTF-8 byte sequences

I took a look at this for my manifests. In my catalog, I had two issues: one was a utf-8 arrow in a comment of a config file, the other was a default favicon.ico that was distributed to newly created vhosts. So, I did not have real problems with this one, but it took me a while to determine this. I will meditate a little about this, but I think it is possible to give better info that just this correct but unhelpful warning. My idea so far is to extend the terminus so that details are logged into a file. I am leaning towards only keeping the last occurence, since puppet may run a lot and the output may include the better part of a catalog as json. Maybe it is sensible to keep the last occurence of each host. Either way, I would like to have this controllable from some config and note the output-file as another Puppet-warning. I will open a Pull-Request on github in the next days, hopefully.

Add Comment

This message was sent by Atlassian JIRA (v6.4.5#64020-sha1:78acd6c)

[Ryan Senior (JIRA)]

Ryan Senior updated an issue

PuppetDB / PDB-135

Improve warning message for invalid UTF-8 byte sequences

Change By: Ryan Senior Sprint: PuppetDB 2015-09-09

Add Comment

[Matthias Viehweger (JIRA)]

Matthias Viehweger commented on PDB-135

Re: Improve warning message for invalid UTF-8 byte sequences should I try to reissue my PR to support you work? This way, my previous research would not be wasted.

Add Comment

[Ryan Senior (JIRA)]

Ryan Senior assigned an issue to Ryan Senior

PuppetDB / PDB-135

Improve warning message for invalid UTF-8 byte sequences

Change By: Ryan Senior Assignee: Ryan Senior

Add Comment

[Susan McNerney (JIRA)]

Susan McNerney updated an issue

PuppetDB / PDB-135 Improve warning message for invalid UTF-8 byte sequences

Change By: Susan McNerney Sprint: PuppetDB 2015-09-09 , PuppetDB 2015-09-23

Add Comment

This message was sent by Atlassian JIRA (v6.4.11#64026-sha1:78f6ec4)

[Kurt Wall (JIRA)]

Kurt Wall updated an issue

PuppetDB / PDB-135 Improve warning message for invalid UTF-8 byte sequences

Change By: Kurt Wall QA Status: Reviewed

Add Comment

[Kurt Wall (JIRA)]

Kurt Wall updated an issue

PuppetDB / PDB-135 Improve warning message for invalid UTF-8 byte sequences

Change By: Kurt Wall QA Risk Assessment: Low

Add Comment

[Kurt Wall (JIRA)]

Kurt Wall updated an issue

PuppetDB / PDB-135 Improve warning message for invalid UTF-8 byte sequences

Change By: Kurt Wall QA Risk Assessment Reason: Enhancement

Add Comment

[Ryan Senior (JIRA)]

Ryan Senior commented on PDB-135

Re: Improve warning message for invalid UTF-8 byte sequences Matthias Viehweger I have a PR up on this here. While digging into the problem, I found that we were generating that error message more times than we needed to. In the case of the arrow character (i.e. valid UTF-8, just not valid ASCII) we should be able to encode that properly and send it to PuppetDB. With my PR that should be happening. In the case of real binary data (your favicon example) we can not. In my PR, I sniff out those binary bytes, display the warning (now including the kind of command and the node it came from) and if you have debug logging enabled, it gives you some context around the binary data (i.e. 100 characters before the unencodable data, 100 characters after). Would be great if you had some time to test out the fix once it gets merged in.

Add Comment

[Ryan Senior (JIRA)]

Ryan Senior assigned an issue to Ryan Senior

PuppetDB / PDB-135

Improve warning message for invalid UTF-8 byte sequences

Change By: Ryan Senior Assignee: Ryan Senior

Add Comment

[Matthias Viehweger (JIRA)]

Matthias Viehweger commented on PDB-135

Re: Improve warning message for invalid UTF-8 byte sequences It is always interesting to see other devs solve problems in a different way. Just from looking at code, your version seems more integrated (and less duck-tapey) than mine. Also, the context generation is more precise. I will try this out next week. Since we still use puppetdb 2.3, I hope that just changing the terminus is enough.

Add Comment

[Kenneth Barber (JIRA)]

Kenneth Barber updated an issue

PuppetDB / PDB-135

Improve warning message for invalid UTF-8 byte sequences

Change By: Kenneth Barber Fix Version/s: PDB 3.2.0

Add Comment

[Olivier Hanesse (JIRA)]

Olivier Hanesse commented on PDB-135

Re: Improve warning message for invalid UTF-8 byte sequences Hello Everyone, I think, there is a regression with PR #1640. Indeed, using puppetdb-termini 3.2 (which includes this PR), puppetserver is "hanging" when sending catalog to PuppetDB. Catalog Compilation is working. My catalog contains a binary file (a kerberos keytab in fact), that is the cause of the hanging. If I use the "char_encoding.rb" file from version 3.1, sending catalog to puppetdb is working again. Do you want me to open a new ticket ?

Add Comment

This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc)

[Kenneth Barber (JIRA)]

Kenneth Barber commented on PDB-135

Re: Improve warning message for invalid UTF-8 byte sequences

Olivier Hanesse if you can open a new bug on the problematic version, that would be optimal. That way we can focus on the particular problem, this current ticket description and problems it solves are pretty generic. We'll probably want to see the binary file also, so we can replicate it, if thats at all possible, plus any exception thrown in the terminus that you might see.

Add Comment

[Olivier Hanesse (JIRA)]

Olivier Hanesse commented on PDB-135

Re: Improve warning message for invalid UTF-8 byte sequences

Kenneth Barber Done. I've just created PDB-2256

Add Comment

[Claudia Petty (Jira)]

Claudia Petty updated an issue

PuppetDB / PDB-135

Improve warning message for invalid UTF-8 byte sequences

Change By: Claudia Petty Labels: new-feature redmine

Add Comment

This message was sent by Atlassian Jira (v8.20.21#820021-sha1:38274c8)