Globus use-case assistance

[Corey Threatt]

I'm looking for assistance in how best to execute this use case in the Globus API.

I have created a mapped collection that I want to be able to create multiple guest collections, via the API script, to be shared with specified end clients email to access. 

Currently I am trying to run the Guest Collection Creation Script from github:

import globus_sdk
from globus_sdk import scopes

# constants
endpoint_hostname = "https://this.is.mine.data.globus.org"
endpoint_id = "1234678-1234-1234-1234-123456789"
mapped_collection_id = " 1234678-1234-1234-1234-123456789  "
storage_gateway_id = " 1234678-1234-1234-1234-123456789"

# client credentials
# This client identity must have the needed permissions to create a guest
# collection on the mapped collection, and a valid mapping to a local account
# on the storage gateway that matches the local_username
# If using user tokens, the user must be the one with the correct permissions
# and identity mapping.
client_id = " 1234678-1234-1234-1234-123456789  "
client_secret = "abcdefghijklmnopqrstuvwxyz="
local_username = "this-user"

# The scope the client will need, note that primary scope is for the endpoint,
# but it has a dependency on the mapped collection's data_access scope
scope = scopes.GCSEndpointScopeBuilder(endpoint_id).make_mutable("manage_collections")
scope.add_dependency(scopes.GCSCollectionScopeBuilder(mapped_collection_id).data_access)

# Build a GCSClient to act as the client by using a ClientCredentialsAuthorizor
confidential_client = globus_sdk.ConfidentialAppAuthClient(
    client_id=client_id, client_secret=client_secret
)
authorizer = globus_sdk.ClientCredentialsAuthorizer(confidential_client, scopes=scope)
client = globus_sdk.GCSClient(endpoint_hostname, authorizer=authorizer)

# The identity creating the guest collection must have a user credential on
# the mapped collection.
# Note that this call is connector specific. Most connectors will require
# connector specific policies to be passed here, but POSIX does not.
credential_document = globus_sdk.UserCredentialDocument(
    storage_gateway_id=storage_gateway_id,
    identity_id=client_id,
    username=local_username,
)
client.create_user_credential(credential_document)

# Create the collection
collection_document = globus_sdk.GuestCollectionDocument(
    public="True",
    collection_base_path="/",
    display_name="guest_collection",
    mapped_collection_id=mapped_collection_id,
)
response = client.create_collection(collection_document)
guest_collection_id = response["id"]
print(f"guest collection {guest_collection_id} created")

But it returns this error: globus_sdk.services.auth.errors.AuthAPIError: ('POST', 'https://auth.globus.org/v2/oauth2/token', 'Basic', 401, 'UNAUTHORIZED', 'Basic auth failed')

I have the app setup to 'Use effective identity (ID token + userinfo)' and provided the accurate IDs for the Endpoint, Mapped_Collection, Storage_Gateway, App/Client_ID, App/Client_Secret and an authorized local_username. So I'm not sure what I am missing in the execution process for authorizing the application?

Also is there an existing use case for this type of application that I may have missed in the group discussions?

[Stephen Rosen]

Hi Corey,

Almost any time you see references in Globus to Basic Auth, as in the "Basic auth failed"  part of that error message, it indicates the use of client credentials.

Without the full trace I'm guessing, but the most likely explanation is that your client credentials are somehow malformed, and that the initial token request made via the confidential client is what's failing.

The simplest way to isolate this and confirm that your client credentials are correct and working is to use them for an explicit token callout, as in

    # the default scopes are fine for the purposes of testing credentials

    confidential_client.oauth2_client_credentials_tokens()

( see also: https://globus-sdk-python.readthedocs.io/en/stable/examples/client_credentials.html )

If that doesn't work, it means the client credentials aren't valid. If it does work, then please share the full traceback so that we can get a better idea of what's going wrong.

One rare, but not-impossible, case we have seen is that some firewalls silently rewrite requests in flight. If identical code works from outside of a network, but not inside of the network, it may mean that you need to talk with your network admins about what kind of controls they have on your network.

As a final thought and note, the examples found in the globus-sdk docs are considered part of the globus-sdk project (even though they aren't part of the python package). We consider bugs found in those examples as globus-sdk bugs. If you find bugs in an example, please consider reporting them on the globus-sdk repo.

Thanks,

-Stephen

[Threatt, Corey]

Hello Stephen, first thank so much for your fast response.

 

I tried the method you suggested, and I was able to get a successful return using the Client ID attached to the Endpoint. Now when I run the Create Guest script it returns a different error:

 

ValueError: Attempting get new access token for client credentials authorizer didn't return exactly one token. Ensure scopes urn:globus:auth:scope:<ENDPOINT ID>:manage_collections[https://auth.globus.org/scopes/<MAPPED COLLECTION ID>/data_access] are for only one resource server.

 

To clarify, Client ID and the Endpoint ID ‘should’ be different values for the guest collection script, correct?

 

Because the explicit call has only worked so far when I have used the ENDPOINT ID for the CLIENT_ID variable.

 

From the documentation, I’m still confused on were the Web information/Client ID for Application and Secret created for application in the containing project are mapped to the physical server? Is there a local server person/application ID that needs to be created to be mapped to the application?

 

 

From: Stephen Rosen <sir...@globus.org>
Sent: Thursday, August 11, 2022 10:35 AM
To: Threatt, Corey <thre...@missouri.edu>
Cc: Discuss <dis...@globus.org>
Subject: Re: [Globus Discuss] Globus use-case assistance

 

WARNING: This message has originated from an External Source. This may be a phishing expedition that can result in unauthorized access to our IT System. Please use proper judgment and caution when opening attachments, clicking links, or responding to this email.

[Stephen Rosen]

On Thu, Aug 11, 2022 at 4:01 PM Threatt, Corey <thre...@missouri.edu> wrote:

Hello Stephen, first thank so much for your fast response.

 

I tried the method you suggested, and I was able to get a successful return using the Client ID attached to the Endpoint. Now when I run the Create Guest script it returns a different error:

 

ValueError: Attempting get new access token for client credentials authorizer didn't return exactly one token. Ensure scopes urn:globus:auth:scope:<ENDPOINT ID>:manage_collections[https://auth.globus.org/scopes/<MAPPED COLLECTION ID>/data_access] are for only one resource server.

Okay, that's very surprising to me! I'm going to need to discuss this one with our team to see if we have everything lined up correctly.

The error comes from the ClientCredentialsAuthorizer, which is meant to get and refresh access tokens as needed for client-credentials authenticated access.

It insists on only seeing one token in the response because any number of additional tokens becomes ambiguous.

The part I'm missing is that the scope string you've shared looks like it should map to a single token. This is what I'll check with our team about -- perhaps there's a subtlety in this case which I've missed.

 

To clarify, Client ID and the Endpoint ID ‘should’ be different values for the guest collection script, correct? 

 

Because the explicit call has only worked so far when I have used the ENDPOINT ID for the CLIENT_ID variable.

They can (and probably should) be separate. If it works when you use the endpoint ID, then the secret in your script is a secret belonging to your endpoint, not your other client.

A client represents "an application". If that's our mental model, then arguably any time you write a script you should ensure it has new and distinct credentials.

Usually we can fudge this quite a bit -- for example, I have some sets of testing clients whose credentials I keep on hand, but without those credentials being associated with any particular codebase.

The one place I'd sound a note of caution is that when an application "talks to itself", there are some slight oddities that can show up.

If you use the endpoint ID and secret to communicate with that same endpoint, there are some actions which you will be allowed to do as a client, like token verification, which only the server is allowed to do.

It's a bit like sending mail to yourself, with your own return address -- it ought to work in theory, but I wouldn't necessarily advise it.

 

From the documentation, I’m still confused on were the Web information/Client ID for Application and Secret created for application in the containing project are mapped to the physical server? Is there a local server person/application ID that needs to be created to be mapped to the application?

I'm not totally clear on whether you mean this as a general question, or specifically about Globus Connect Server v5 (GCSv5).

If you create credentials for a GCSv5, then those credentials are used when you run server setup, and at that point they are stored on the physical machine where you ran setup. You may note that you need those credentials to add nodes to your GCS (`globus-connect-server node setup`), as every node for the endpoint -- meaning, each physical machine in a clustered deployment -- needs those credentials.

In the more general case, there are no rules or restrictions on how or where the credentials for an application are stored.

You can build your own applications in a similar manner to GCS, where credentials are in some sensible way associated with physical machines, you may associate them in some way with user accounts (e.g. stored in some canonical path under $HOME), or you may have other more sophisticated usages.

I hope that helps, but if not I'm happy to try to clarify further. Knowing what parts of the documentation you're currently reading may help me understand.

Best,

-Stephen

[Corey Threatt]

So to address the first issue. I'm not sure I set the application up properly in the developers page to authenticate. So I set up the application under the same project as the GCS server I'm working with. I could see how I may have misunderstood the tutorial site but is the app dropdown does not allow to create a secret for a 'public client'

Also I wanted to know if there is a best practice you would recommend for this use case:

I work with the research groups of an institution that shares their data with clients outside the organization regularly. I am working to develop the GCS that would interface with the groups Nextflow pipeline their data is processed through. So that when data is ready Nextflow can communicate with the GCS to create a Guest Collection for the data and share a specific data set with a specified individual. Is there a configuration or perhaps a feature I am overlooking that could achieve this through the CLI, GCS or sdk? 

[Corey Threatt]

So I discovered my issue with the application registration. The application was registered as a native application and trying to work through a confidential client.

Now I have run into an error that seems to be indicating an issue with role permissions.

globus_sdk.services.gcs.errors.GCSAPIError: ('POST', 'https://764e0.8540.data.globus.org/user_credentials', 'Bearer', 403, 'Error', '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">\n<html><head>\n<title>403 Forbidden</title>\n</head><body>\n<h1>Forbidden</h1>\n<p>You don\'t have permission to access this resource.</p>\n</body></html>\n', None)

I confirmed that the application ID has the administrator role on the endpoint and mapped collection. Also per the documentation I have mapped the application ID to a local user that is allowed through the storage-gateway. Is there a step I am missing to confirm the applications access to the mapped collection?