s3ql and AWS assume role
CHEVALIER Philippe
Hello,
Maybe the answer is already out there, but I didn't find it.
Is there a way to make s3ql commands use an assumed role to access the s3 bucket?
Basically, my s3 buckets can be accessed only with a specific IAM role, so I either use a profile, or temporary credentials using aws sts.
Example:
aws sts assume-role --role-arn arn:aws:iam::123456789:role/s3admin --role-session-name s3backup --profile default
Which gives me back temporary credentials with aws key, aws secret and aws_token.
I tried this with awscli and it works perfectly if I update the .aws/credentials file with aws_access_key_id, aws_secret_access_key and aws_session_token
All 3 are mandatory, or aws s3 commands don't work.
I originally thought I could just create a authinfo file with these informations so s3ql can access my bucket under a temporary identity, but there's no parameter for the token. Without it, I get a 403 denied.
I can't even start to make a mkfs.s3ql.
After reading the aws doc (https://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html) , the headers must contain x-amz-security-token
Which s3ql doesn't support?
Any idea?
Alex Krush
Daniel Jagszent
S3QL cannot handle STS generated temporary credentials. If I understand https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html#API_AssumeRole_RequestParameters correctly they would only allow you to mount the file system for a maximum of 12 hours. There might be use cases where that's enough (which?). I do not think that support for temporary credentials will get into S3QL without handling the token expiration gracefully (how?). Besides this road block: If you want this feature, you probably need to find someone (maybe yourself) who implements it (see https://github.com/s3ql/s3ql/issues/265#issuecomment-1004689564 )
Implementing basic support for the x-amz-security-token shouldn't be that hard if you have some Python knowledge. It's basically changing one line and adding two new lines of code:
- Add a new backend option for the security token https://github.com/s3ql/s3ql/blob/19493c29990e849ebaa261cb0549bbda4d7819ab/src/s3ql/backends/s3.py#L39
- Optionally use this backend option to add the x-amz-security-token header in https://github.com/s3ql/s3ql/blob/19493c29990e849ebaa261cb0549bbda4d7819ab/src/s3ql/backends/s3.py#L166-L229 (see https://github.com/s3ql/s3ql/blob/19493c29990e849ebaa261cb0549bbda4d7819ab/src/s3ql/backends/s3.py#L88-L89 for how to access the backend options in this method)
- Document the new backend option in this file
https://github.com/s3ql/s3ql/blob/master/rst/backends.rst#amazon-s3