Updated manifest, now get Failed to apply catalog: "\xF8\xDD" on UTF-16LE on Windows

[jmp242]

I'm using puppet 5.3.3 and had been using a previous version of my module for a long time. Now I added another package to my management and started getting an odd error:
Failed to apply catalog: "\xF8\xDD" on UTF-16LE

The actual underlying chocolatey packages install correctly. Any ideas what this error means? What I need to debug? I use Gepetto to author the edits and check into SVN... It almost sounds like a file format issue, but I don't know in Gepetto how to change / fix this.

[程伟]

I don't what happened. But I think you may try 'puppet agent' command with '--debug' or '--trace' to find what's going on? 

在 2018年3月27日星期二 UTC+8上午3:58:14，jmp242写道：

[David Schmitt]

Hi,

try an advanced text editor like https://code.visualstudio.com/ or https://notepad-plus-plus.org/ which displays the encoding of the files. That way you can identify and fix the one that is off (not in UTF-8). VSCode is also maintained and has a puppet plugin.

Other than that, 程伟 had it right: try enabling the --debug and --trace options on either your agent and/or your server to get a hint which file's encoding is wrong.

Cheers, David

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/1776d069-6506-4d59-acdf-8309e16af250%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Cheers, David

https://twitter.com/dev_el_ops

[jcbollinger]

The immediate problem appears to be that Puppet is complaining about invalid character data, presumably in the catalog itself.  It is attempting to interpret a piece of data as if it were text encoded according to UTF-16LE, but the data contain an invalid byte sequence with respect to that encoding (in particular, it looks like there is an unpaired surrogate).

Clearly something has gone wrong, more likely in your manifests than in Puppet itself, but we don't have enough information to pinpoint what that is.  As others have suggested, obtaining the --debug output from an agent run would probably help clarify.  You could also consider searching your manifests (on the master) for the two-byte sequence Puppet complains about (represented as \xF8\xDD) or for the 3-byte UTF-8 encoding of that surrogate code point (\xED\xB7\xB8).  Perhaps look for the byte-swapped version, too: \xDD\xF8 and \xEF\xA3\x9D.

John

[Henrik Lindberg]

You may find what you need here:
https://stackoverflow.com/questions/3751791/how-to-change-default-text-file-encoding-in-eclipse

Best,
- henrik

>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com

> <mailto:puppet-users...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/1776d069-6506-4d59-acdf-8309e16af250%40googlegroups.com

> <https://groups.google.com/d/msgid/puppet-users/1776d069-6506-4d59-acdf-8309e16af250%40googlegroups.com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout.


--

Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/

[jmp242]

I just got back to this. I tried re-installing the Windows 10 VM, tried a different hostgroup in Foreman, seems like it's a catalog problem of some sort.
puppet agent --trace gives no output for me.

puppet agent --debug around the error gives:

Debug: Prefetching chocolatey resources for package
Debug: Executing: 'C:\ProgramData\chocolatey\bin\choco.exe pin list'
Debug: Executing 'C:\ProgramData\chocolatey\bin\choco.exe list -lo -r'
Debug: Executing 'C:\ProgramData\chocolatey\bin\choco.exe upgrade --noop chocolatey.extension -r'
Debug: Class[Yum]: The container Stage[main] will propagate my refresh event
Debug: Prefetching windows resources for package
Debug: Unable to convert value "{\x00B\x003\x00A\x00F\x00C\x006\x000\x008\x00-\x00D\x008\x001\x001\x00-\x000\x000\x000\x003\x00-\x004\x000\x005\x000\x00-\x002\x001\x00F\x00B\x002\x005\x00B\x004\x008\x00D\x006\x00E\x00}\x00\x00\x00\x00\x00{\xEEs\vj\x93\x00\x00H\x14t\x00\x18.t\x00d\x00l\x00l\x00\x00\x00~\xEEr\x0Fb\x93\x00\b\xD0\xC3\xF2w`8t\x00\x00 \xD9u\b \xD9ul~\xD8uX\xDC\x8Bu\x00\x00\x00\x00\x00\x000\x000$t\x00\x02\x00\x00\x00v\xEEr\ag\x93\x00\f?\x0E\x00\x00\t\x04\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00(:t\x00\xB0:t\x00@\xCDs\x00 \xCCs\x00(Kt\x00hMt\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00(;t\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xE8\x03\x00\x00\x00\x00\x00\x00\xD8Dt\x00\x06\x03\x00\x00".force_encoding("UTF-16LE") to encoding UTF-8 due to #<Encoding::InvalidByteSequenceError: "X\xDC" on UTF-16LE>
Debug: Storing state
Debug: Stored state in 0.08 seconds
Error: Failed to apply catalog: "X\xDC" on UTF-16LE

Notepad++ shows the manifests used here (as far as I'm able to tell) are both UTF-8...

[jcbollinger]

On Wednesday, April 18, 2018 at 2:49:37 PM UTC-5, jmp242 wrote:

I just got back to this. I tried re-installing the Windows 10 VM, tried a different hostgroup in Foreman, seems like it's a catalog problem of some sort.
puppet agent --trace gives no output for me.

puppet agent --debug around the error gives:

Debug: Prefetching chocolatey resources for package
Debug: Executing: 'C:\ProgramData\chocolatey\bin\choco.exe pin list'
Debug: Executing 'C:\ProgramData\chocolatey\bin\choco.exe list -lo -r'
Debug: Executing 'C:\ProgramData\chocolatey\bin\choco.exe upgrade --noop chocolatey.extension -r'
Debug: Class[Yum]: The container Stage[main] will propagate my refresh event
Debug: Prefetching windows resources for package
Debug: Unable to convert value "{\x00B\x003\x00A\x00F\x00C\x006\x000\x008\x00-\x00D\x008\x001\x001\x00-\x000\x000\x000\x003\x00-\x004\x000\x005\x000\x00-\x002\x001\x00F\x00B\x002\x005\x00B\x004\x008\x00D\x006\x00E\x00}\x00\x00\x00\x00\x00{\xEEs\vj\x93\x00\x00H\x14t\x00\x18.t\x00d\x00l\x00l\x00\x00\x00~\xEEr\x0Fb\x93\x00\b\xD0\xC3\xF2w`8t\x00\x00 \xD9u\b \xD9ul~\xD8uX\xDC\x8Bu\x00\x00\x00\x00\x00\x000\x000$t\x00\x02\x00\x00\x00v\xEEr\ag\x93\x00\f?\x0E\x00\x00\t\x04\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00(:t\x00\xB0:t\x00@\xCDs\x00 \xCCs\x00(Kt\x00hMt\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00(;t\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xE8\x03\x00\x00\x00\x00\x00\x00\xD8Dt\x00\x06\x03\x00\x00".force_encoding("UTF-16LE") to encoding UTF-8 due to #<Encoding::InvalidByteSequenceError: "X\xDC" on UTF-16LE>
Debug: Storing state
Debug: Stored state in 0.08 seconds
Error: Failed to apply catalog: "X\xDC" on UTF-16LE

Notepad++ shows the manifests used here (as far as I'm able to tell) are both UTF-8... 

When I said the problem is more likely to be in the manifests than in Puppet itself, I didn't mean specifically that the manifests might be encoded wrongly, although there are potential issues of that kind.  That Notepad++ thinks they are encoded in UTF-8 is a relevant detail, but not indicative of anything one way or the other.

The debug messages you presented are more useful, though.  Supposing that you haven't cut anything out, it looks like you have a chunk of binary data that Puppet is for some reason trying to interpret as text.  The embedded sequences of multiple consecutive 0 bytes are a dead giveaway that it's binary.  I'm uncertain whether Puppet is expecting from context that the byte sequence is UTF-16-encoded text, or whether it is guessing based on the content, but the byte sequence overall does not conform to any of the UTFs, and it does not look like text in any of ASCII-compatible single-byte encoding, either.

Since this appears to be happening while Puppet is pre-fetching resources, I am forced to revise my opinion that the problem is somewhere among your manifests.  Your manifests are probably not responsible for this, except inasmuch as they declare at least one Package resource that explicitly or implicitly uses the default 'windows' provider.  That's a flaw only if you intended to do differently.

Inasmuch as the problem appears to occur in the scope of prefetching 'windows' packages, it is probably right to attribute it to Puppet.  But Puppet's 'windows' package provider generally works as advertised, so it seems likely that something about the current state of the target machine is contributing.

John