Google Cloud Bucket connection from Kubernetes deployment using Storage API

[AB]

We have a java application running in Kubernetes cluster deployment. We're using Google Cloud Bucket as storage. We were using Java Files.move method to move files from our Persistent Volume Claim (PVC) to the storage bucket:

    import java.nio.file.Files;

    

    Files.move(source, target, StandardCopyOption.REPLACE_EXISTING)

But we're getting poor write performance. So we tried exploring [Google Cloud Storage API][1] to move files from our PVC to bucket.

    try {

    log.info("before getService");

    

    Storage storage = StorageOptions.newBuilder()

    .setCredentials(GoogleCredentials.create(aToken)).build()

    .getService();

        // aToken is the access token of the service account

   

    log.info("after getService");

    } catch (Exception e) {

    log.error("Error while creating storage object - ", e);

    }

But only "before getService" is getting logged. And nothing happens after that. No exception is thrown. The process gets stuck in getService()

The same application works on local deployment with Google Storage Bucket, but is not working on Kubernetes deployment.

[Mehdi]

Hello,

I'm having the exact same issue and can't figure out why. Did you solve your problem and found a way to get past this?

Thanks,

Mehdi

[Nathan Fisher]

I’m successfully using GCS with kubernetes and instance level oauth permissions in a GKE cluster.

Note there are basically 3 ways you can authenticate with GCP resources;

- gcloud client.
- instance level authorisation.
- service account JSON file with minimal access.

For a kubernetes cluster the last option placed in a k8s secret or similar mapped into the container is probably the best approach. You’ll need to set an environment variable GOOGLE_APPLICATION_CREDENTIALS pointing to the path in the container. I would suggest testing it locally first in a docker container so you can easily isolate any issues. If you do on your dev machine directly ensure you’re logged out as the sdk will iterate through the various methods.

--
You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-use...@googlegroups.com.
To post to this group, send email to kubernet...@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

--

- sent from my mobile

[Nathan Fisher]

Related documentation;

https://cloud.google.com/docs/authentication/production#auth-cloud-implicit-java

Despite having used instance authentication for my application we want to move away from it to something more fine grained using service accounts. Instance level authorisation means every container that gets scheduled on the node is granted that access, which you may or may not want, depending on how your cluster is configured.