globus-connect-server login localhost
Please authenticate with Globus here:
------------------------------------
https://auth.globus.org/v2/oauth2/authorize?client_id=blahblahblah&prompt=login
------------------------------------
Enter
the resulting Authorization Code here:
2. Can I reuse of the client ids and secrets? It would be nice if we could just delete a vm with the Globus endpoints and globus connect servers on them and recreate them just using the client id and secrets we had before. This would also delete the deployement-key.json but I'm not sure it is needed if everything is being recreated anyway or is this a problem?
Best
export CLIENT_ID_USERNAME=4d6e9126-f428-4dd9...@clients.auth.globus.org$ globus-connect-server endpoint role create administrator $CLIENT_ID_USERNAMERole ID: ef8a7108-d917-11ec-b37e-fdd01edbf245$ globus-connect-server endpoint role listRole ID | Role | Principal------------------------------------ | ------------- | ------------------------------------------------------------62dd115a-10c9-11ec-a018-811dd7c5dbfa | administrator | jaso...@globus.orgef8a7108-d917-11ec-b37e-fdd01edbf245 | administrator | 4d6e9126-f428-4dd9...@clients.auth.globus.orgfc9ab067-5ce3-4815-bfed-59c6770b3ad3 | owner | jaso...@globus.org
#!/usr/bin/env python3
import globus_sdk
# Substitute your values here:
ENDPOINT_ID = "ENDPOINT_ID"
GCS_MANAGER_FQDN = "GCS_MANAGER_FQDN"
CLIENT_ID = "YOUR_APP_CLIENT_ID"
CLIENT_ID_USERNAME=CLIENT_ID + "@clients.auth.globus.org"
CLIENT_SECRET = "YOUR_APP_CLIENT_SECRET"
#
# We need an access token with the 'manage_collections' scope in order
# to interact with the GCS Manager API.
#
# The authorizer manages our access token for the scopes we request
authorizer = globus_sdk.ClientCredentialsAuthorizer(
# The ConfidentialAppAuthClient authenticates us to Globus Auth
globus_sdk.ConfidentialAppAuthClient(
CLIENT_ID,
CLIENT_SECRET
),
f"urn:globus:auth:scope:{ENDPOINT_ID}:manage_collections"
)
# The access token is stored in authorizer.access_token
access_token = authorizer.access_token
#
# We'll need a GCS Client
# https://globus-sdk-python.readthedocs.io/en/stable/services/gcs.html
#
gcs_client = globus_sdk.GCSClient(GCS_MANAGER_FQDN, environment='sandbox', authorizer=authorizer)
#
# Create a storage gateway. The SDK GCSClient doesn't currently have a member function for
# creating storage gateway, so we'll make the POST call according to the GCS API docs.
# https://docs.globus.org/globus-connect-server/v5.4/api/openapi_Storage_Gateways/#postStorageGateway
#
gateway_doc = {
'DATA_TYPE': 'storage_gateway#1.1.0',
'display_name': 'My Unique Storage Gateway Display Name',
# POSIX Connector ID
'connector_id': '145812c8-decc-41f1-83cf-bb2a85a2a70b',
# Set whichever domain you want to allow data access on the mapped collection. In this case,
# the client will be able to access the mapped collection.
'allowed_domains': ['clients.auth.globus.org'],
# We only have a single domain so we aren't required to supply an identity_mapping, however,
# I want to make sure this is the only client that maps _and_ I want to be able to map to a
# more useful local username than the CLIENT_ID.
'identity_mappings': [{
'DATA_TYPE': 'expression_identity_mapping#1.0.0',
'mappings': [{
'source': '{username}',
'match': CLIENT_ID_USERNAME,
'output': 'ci_client',
}]
}],
'policies': {'DATA_TYPE': 'posix_storage_policies#1.0.0'}
}
# Returns globus_sdk.response.GlobusHTTPResponse
resp = gcs_client.post('/storage_gateways', data=gateway_doc)
gateway_id = resp.data['data'][0]['id']
#
# Create a mapped collection on the storage gateway. This is supported by the SDK.
# https://globus-sdk-python.readthedocs.io/en/stable/services/gcs.html#globus_sdk.GCSClient.create_collection
# Returns UnpackingGCSResponse
# Collections doc reference: https://docs.globus.org/globus-connect-server/v5.4/api/schemas/Mapped_Collection_schema/
collection_doc = {
'DATA_TYPE': 'collection#1.5.0',
'collection_type': 'mapped',
'display_name': 'My Client-Created Mapped Collection Display Name',
'storage_gateway_id': gateway_id,
'public': True,
'collection_base_path': '/',
}
resp = gcs_client.create_collection(collection_doc)
collection_id = resp.data['id']
$ globus-connect-server storage-gateway listDisplay Name | ID | Connector | High Assurance | MFA-------------------------------------- | ------------------------------------ | --------- | -------------- | -----My Unique Storage Gateway Display Name | 8d038f24-2e10-4f52-9308-58a9d068e944 | POSIX | False | False$ globus-connect-server storage-gateway show 8d038f24-2e10-4f52-9308-58a9d068e944Display Name: My Unique Storage Gateway Display NameID: 8d038f24-2e10-4f52-9308-58a9d068e944Connector: POSIXHigh Assurance: FalseAuthentication Timeout: 15840Multi-factor Authentication: FalseAllowed Domains: ['clients.auth.globus.org'](venv) [centos@(gcs dev 1) client_admin]$ globus-connect-server collection listID | Display Name | Owner | Collection Type | Storage Gateway ID------------------------------------ | ------------------------------------------------ | ------------------------------------------------------------ | --------------- | ------------------------------------c458e931-3b73-4798-9729-43f1a4de3870 | My Client-Created Mapped Collection Display Name | 4d6e9126-f428-4dd9...@clients.auth.globus.org | mapped | 8d038f24-2e10-4f52-9308-58a9d068e944$ globus-connect-server collection show c458e931-3b73-4798-9729-43f1a4de3870Display Name: My Client-Created Mapped Collection Display NameID: c458e931-3b73-4798-9729-43f1a4de3870Collection Type: mappedStorage Gateway ID: 8d038f24-2e10-4f52-9308-58a9d068e944Connector: POSIXAllow Guest Collections: FalseDisable Anonymous Writes: FalseHigh Assurance: FalseAuthentication Timeout: 15840Multi-factor Authentication: FalseTLSFTP URL: tlsftp://m-fe434a.1008a.8540.sandbox2.zones.dnsteam.globuscs.info:443Force Encryption: FalsePublic: TrueContact E-mail: jaso...@globus.org$ globus-connect-server collection role list c458e931-3b73-4798-9729-43f1a4de3870Role ID | Collection ID | Role | Principal------------------------------------ | ------------------------------------ | ------------- | ------------------------------------------------------------9653a0c0-d924-11ec-b37e-fdd01edbf245 | c458e931-3b73-4798-9729-43f1a4de3870 | administrator | 4d6e9126-f428-4dd9...@clients.auth.globus.org
# globus-connect-server node setup --import-node <node_config> --deployment-key <deployment-key> --client-id <client_id>
I am unsure at this point if running the globus-connect-server setup command must be run on the same machine where a node must also be deployed. I am hoping not.
Follow up questions, I have been able to create the project and client successfully, I'm having a hard time understanding however, how to go about using the python SDK and the Globus API to set up a Globus Connect Server endpoint and node. For instance, the python SDK has commands such as:
tc.create_endpoint(ep_data)
tc.add_endpoint_server(endpoint_id, server_data)
Doesn't look like using subprocess with 'globus-connect-server endpoint setup' will work, even with GCS_CLIENT_ID and GCS_CLIENT_SECRET I'm still required to login via a browser link.
I am running into an
additional problem when trying to automate standing up a Globus Connect Server.
Currently, we are making use of guest collections and anonymous access as part
of our default configuration, this requires a managed endpoint. The steps I
have been able to achieve thus far are.
1. Generate
deployment key by running globus-connect-server endpoint setup, this requires
interactive user input as it triggers the authorization code oauth flow, which
is fine because it is a one-time occurrence. The following steps however need
to be automated in a non-interactive manner which is all done using a
confidential client with a secret.
2. Next, I
can run my GCS container by passing in the deployment key, this also works
fine.
3. The next step is to set up the storage gateways and collections etc.
This is where I hit a snag. Using a confidential client it doesn't look like I can make my endpoint a managed endpoint, there does not seem to be a way to give a confidential client the scope to do this, or if there is I was unable to find out how. If there is that would solve my problem. The alternative would be to set the subscription in step 1 where I have used my user credentials as part of the initial manual ouath2 authentication . However, it looks like running globus-connect-server endpoint set-subscription-id is failing in this step because no nodes are running, because the first node is only set up after the first container is launched.
globus-connect-server endpoint update --subscription-id "blahblah"
Error contacting blahblah.data.globus.org
Error resolving blahblah.data.globus.org
This may be because the endpoint is deleted, it is not deployed on any
nodes, or your DNS resolver is misconfigured.