gsutil rsync not working due to encoding

[Arvind Stutzen]

When trying to execute below command faced error related to encoding. 

sudo gsutil -m -d rsync -r  /folder gs://bucket

The error logs are provided below. As there are more than a million files, How to find the filename which has wrong encoding?

header: Vary: Origin

header: Vary: X-Origin

header: Content-Type: application/json; charset=UTF-8

header: Expires: Sun, 17 Jun 2018 10:14:42 GMT

header: Date: Sun, 17 Jun 2018 10:14:42 GMT

header: Cache-Control: private, max-age=0, must-revalidate, no-transform

header: Content-Length: 226

header: Server: UploadServer

INFO 0617 10:14:42.335708 base_api.py] Response of type Objects: <Objects

 items: [<Object

 acl: []

 crc32c: u'AAAAAA=='

 generation: 1529229799889295

 md5Hash: u'1B2M2Y8AsgTpgAmY7PhCfg=='

 name: u'sbkp/plesk-vm10-vm/'

 size: 0

 timeCreated: datetime.datetime(2018, 6, 17, 10, 3, 19, 887000, tzinfo=<apitools.base.protorpclite.util.TimeZoneOffset object at 0x7fd00dbe94d0>)>]

 prefixes: []>

At source listing 10000...

At source listing 20000...

Caught non-retryable exception while listing file:///mnt/disks/exdata/: 'ascii' codec can't encode character u'\uf0a7' in position 86: ordinal not in range(128)

DEBUG: Exception stack trace:

    Traceback (most recent call last):

      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 567, in _RunNamedCommandAndHandleExceptions

        user_project=user_project)

      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/command_runner.py", line 319, in RunNamedCommand

        return_code = command_inst.RunCommand()

      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/commands/rsync.py", line 1412, in RunCommand

        diff_iterator = _DiffIterator(self, src_url, dst_url)

      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/commands/rsync.py", line 829, in __init__

        raise CommandException('Caught non-retryable exception - aborting rsync')

    CommandException: CommandException: Caught non-retryable exception - aborting rsync

    

CommandException: Caught non-retryable exception - aborting rsync

[George (Cloud Platform Support)]

This is a known issue, and developers are currently working towards a fix. Till that time, a workaround would consist in searching your file names for the character indicated in the error, and eventually renaming the file with ASCII characters.