Some notes on implications of last meeting

[arlarge]

Wrote these up in a Google Doc: https://docs.google.com/document/d/1AUfwROP6wQiVnpaKDP3y3A548IyP_pRYgjU1j0HQ1n8/edit?usp=sharing

[jcope]

I hit some hotkey and submitted halfway through writing - sorry for the duped post :)

As far as how this would guide the API design, I think we'd be looking at probably 11 APIs in total.  That sounds like a lot, but I think most of them would be very small individually.  We'll need to add APIs for Identity and Policy operations.  Assuming the automation follows the least privilege model that we're currently using, this means we'll have both user facing and driver facing APIs, as well as an admin API to enable some administrative control over the processes.

APIs here are designated by "user," "driver," and "admin."  User is a namespaced API, in the app's namespace.  Driver and Admin are cluster-scoped.

Bucket handling (3):

The largest APIs as they type the protocols discretely, and must satisfy the varying informational needs of each.  That gives us a user (i.e. "claim"), driver (cluster-scoped), and admin (*Class) API.

• Bucket(user): requests that a new storage instance be provisioned, or that an existing storage instance been exposed to the app.  
• BucketContent(driver): transmits info needed for creating and connecting to a bucket to the code that handles these processes
• BucketClass(admin): contains driver specific parameters.  Referenced by a user for bucket creation, contains driver specific parameters that an admin can use to control aspects of bucket creation

Identity handling (5):

This breaks down into 2 categories: key minting and identity binding (kube service accounts to cloud provider identities/roles).

Key Provisioning:  This is a common operation and doesn't require much information to accomplish.  We're not assigning permissions in this phase, so it's a really basic request, and common for probably all cloud and BYO object stores.  Assuming again the least privileged automation, we'll need 3 APIs:

• IdentityKeyRequest(user): At it's simplest, this should have 3 fields:
• IdentityKeySecretName: the secret created in a user's namespace containing access credentials
• IdentityKeyName: the name of the driver side API object that is bound to this request.
• IdentityClassName: the name of the admin facing API
• IdentityKey(driver): contains info the driver will need to create a service identity and expose the credentials to the user's namespace.  This may be a new user or a new key for an existing user account - it's up to the driver authors.
  
• IdentityClass(admin): contains driver specific parameters and designates the provisioner.

IdentityBinding:  The approach to this varies between cloud providers.  Google and AWS don't handle this similarly.  I couldn't find anywhere in the Azure docs that describes support for kube service account integration.  AKS does integrate with Azure Active Directory for user authentication, which would fall under Key Provisioning.

• GCP Workload Identity: A GCP SA is bound to a Kube SA through a "workload identity pool".  When an app in GKS calls out to the GCP interface, the Pod's SA is mapped to a GCP SA, and the action is executed using the GCP SA's credentials.  So in this case, we have a SA to SA relationship, with GCP SA's controlled by GCP IAM Roles.
• AWS Service Account Integration:  AWS short circuits their model by associating a K8s SA directly to an AWS IAM Role.  This means that unlike Google, where kube identity is mirrored to GCP identity, AWS just assigns policies directly to EKS identities.

        I'm a stuck on how to design an API around both models.  An IdentityBinding API would be exclusive to GCP.  AWS's service account role model could be incorporated into the Policy API (below), but would also be exclusive.

Policy Handling (3):
These will also have to be representative of each protocol's set of polices.  That said, there's lots of overlap between the protocols in terms of the actions they control (i.e. read, write, list, set acls, etc).  An intersection set of policies + an arbitrary strings array for 

• AccessPolicyRequest(user): the user requests that a specified identity be given access to a specified bucket with a set of polices.  Minimum fields should be:
• IdentityKeyRequest: either a credential/user or ...
• ServiceAccountName: a k8s service account.
• BucketRequest: the user's bucket 
• AccessPolicy(driver): contains information necessary for a driver to generate a policy that provides access for a user/SA to a bucket.
• AccessPolicyClass(admin): designates the provisioner and provides a set of driver specific parameters an admin can set to control policy provisioning

Limitations:

- Policies bind users to buckets, so both have to exist before the policy can be created

[jcope]

Thinking a little more on it, it may be that we couple SA integration with Policy APIs.  

In GCP, this would mean generating GCP SAs in the background before binding them with k8s SAs, then applying the policies.  

In AWS, it would create or bind existing roles to a k8s SA.  This would mean the policy request API would have a field for specifying k8s SA, which should be mutually exclusive with another field for specifying a user/credential set.

e.g.

kind: AccessPolicyRequest

spec:

  cloudProviderIdentityBinding: # service account name

  identityKeyRequestName:  # name of the identity request which IDs the user or key name

  bucketName:  # bucket API object for which the policy applies