Guide for using just CURL to GET & PUT data to a collection that requires authentication

[Patrick Mulrooney]

I just worked through the method for using just CURL ( and grep & sed ) to GET & PUT data to a Globus collection that requires authentication and wanted to share in case anyone finds it useful.

We wanted to be able to provide folks with these commands, mostly for pulling data, and not require them to set up a Globus account. We did this using the developer service account functionality.

This tutorial assumes a few things...
- You are running Globus server 5
- You have the ability to add permissions to the share

First, you need to set up the service account, to do that visit: https://app.globus.org/settings/developers

1. The first step is to create a new project.
   1. You can create multiple service accounts for each project so name it something like 'Service Account Key Project'.
2. Once you have that setup you should click 'Add an App'.
3. That will take you to a list of app options, select 'Register a service account or application credential for automation'
   1. Give the app a descriptive name and create it
4. You should now see your app ( / service account ) details
   1. Take note of the 'Client UUID', for this example I will use 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
5. Click 'Add Client Secret'
   1. Give it a name that will help you remember where the secret has been used.
   2. Record the secret, it will not be provided again. For this example, I will use 'yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy=' when referring to the secret
6. Go to your collection and then permissions.
7. Select 'Add Permissions'.
   1. Select the desired path, leave 'Share With' as 'user - share with specific individuals', and set the permissions (write required if doing a PUT).
   2. Under 'Username or Email' paste in the Client UUID from step 4 with '@clients.auth.globus.org' appended, e.g. xxxxxxxx-xxxx-xxxx...@clients.auth.globus.org
      1. No results will be found, that is fine click 'add'. When you do you will see the app name you set up in step 4.
   3. Click 'Add Permission' and you are all good to go.
      

Once that is all done you now need to do the two-step process for pulling the data. For this example, I am going to use 'zzzzzzzz-zzzz-zzzz-zzzz-zzzzzzzzzzzz' as the collection UUID and 'g-wwwwww.wwwwww.wwww.data.globus.org' as the collection 'Domain'. You can get these two values from the collection overview page.

In order to GET/PUT the data you first need to authenticate with the app Client ID (without the @clients.auth.globus.org) & client secret (from step 5) in order to get a 'bearer' token. If you run this command without the sed & grep command you will see it returns some JSON output that includes a bearer token in the 'access_token' field. There are better methods than the sed & grep combination to get the 'access_token' field. The token is good for 2 days.

$ BEARER=$(curl -s -X POST -d "grant_type=client_credentials" -d "scope=https://auth.globus.org/scopes/zzzzzzzz-zzzz-zzzz-zzzz-zzzzzzzzzzzz/https" -H 'Content-Type: application/x-www-form-urlencoded' -u "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy=" "https://auth.globus.org/v2/oauth2/token"|grep -o '"access_token":"[^"]*"'|sed 's/.*://'|sed 's/"//g')

Then in order to do a GET you just need to:

$ curl -s -H "Authorization: Bearer ${BEARER}" "https://g-wwwwww.wwwwww.wwww.data.globus.org/test.jpg"

To do a PUT just a few small changes. If you want to put the file you are uploading into a sub directory it will either need to exist already or you will need to create it before trying to write the file there. It will not create a directory that does not exist.

$ curl -s -X PUT --data-binary @test.jpg -H "X-Requested-With: XMLHttpRequest" -H "Authorization: Bearer ${BEARER}" "https://g-wwwwww.wwwwww.wwww.data.globus.org/test.jpg"

Hope this helps someone.

[Yuriy Halytskyy]

Cheers, that was quite helpful. I had a similar script using python SDK but this will be very useful for our users that don't use python.