Cron job using rsync

[Thomas Johnson]

I can run this command on my instance using web console;

gsutil rsync -d -r /my-path gs://my-bucket

But when I try on my remote ssh terminal I get this error;

root@instance-2: gsutil rsync -d -r /my-path gs://my-bucket
Building synchronization state...
INFO 0923 12:48:48.572446 multistore_file.py] Error decoding credential, skipping
    Traceback (most recent call last):
      File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/oauth2client/oauth2client/multistore_file.py", line 381, in _refresh_data_cache
        (key, credential) = self._decode_credential_from_json(cred_entry)
      File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/oauth2client/oauth2client/multistore_file.py", line 400, in _decode_credential_from_json
        credential = Credentials.new_from_json(json.dumps(cred_entry['credential']))
      File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/oauth2client/oauth2client/client.py", line 292, in new_from_json
        return from_json(s)
      File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/apitools/apitools/base/py/credentials_lib.py", line 356, in from_json
        data['token_expiry'], oauth2client.client.EXPIRY_FORMAT)
    TypeError: must be string, not None
    Caught non-retryable exception while listing gs://pacwill-database-backup/: Could not reach metadata service: Not Found
    At source listing 10000...
    At source listing 20000...
    At source listing 30000...
    At source listing 40000...
    CommandException: Caught non-retryable exception - aborting rsync

[Thomas Johnson]

I solved this by switching the user to the default CGE one that is created when the project is created. Root on the VM does not have privileges to run gsutil commands it seems.

On Sunday, 24 September 2017 08:27:20 UTC-3, Thomas Johnson wrote:

I can run this command on my instance using web console;

gsutil rsync -d -r /my-path gs://my-bucket

But when I try on my remote ssh terminal I get this error;

root@instance-2: gsutil rsync -d -r /my-path gs://my-bucket
Building synchronization state...
INFO 0923 12:48:48.572446 multistore_file.py] Error decoding credential, skipping
    Traceback (most recent call last):
      File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/oauth2client/oauth2client/multistore_file.py", line 381, in _refresh_data_cache
        (key, credential) = self._decode_credential_from_json(cred_entry)
      File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/oauth2client/oauth2client/multistore_file.py", line 400, in _decode_credential_from_json
        credential = Credentials.new_from_json(json.dumps(cred_entry['credential']))
      File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/oauth2client/oauth2client/client.py", line 292, in new_from_json
        return from_json(s)
      File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/apitools/apitools/base/py/credentials_lib.py", line 356, in from_json
        data['token_expiry'], oauth2client.client.EXPIRY_FORMAT)
    TypeError: must be string, not None

    Caught non-retryable exception while listing gs://my-bucket/: Could not reach metadata service: Not Found

[Kamran (Google Cloud Support)]

Hello Thomas,

gsutil and gcloud command tools do not use Linux users (like root) to communicate with Google API servers, but they use credentials of Google User Accounts or Service Accounts to authenticate with the API servers. In order to use a user account to authorize gcloud / gsutil to access GCP, you can authenticate with server by running gcloud auth login command. To authorize access to GCP using a service account, gcloud auth activate-service-account command can be used to add and activate the service account. Please visit this article for more information about  service accounts.

You can run gcloud auth list command to find out what account is active and being used by gsutil / gcloud tools.