credentials within aws.xml are committed in the clear to git?

[bitsofinfo]

Is this expected?

https://docs.craftercms.org/en/3.1/site-administrators/studio/aws-profiles-configuration.html?highlight=aws

i.e. AWS secrets end up in the Git origin source in the clear? 

https://github.com/bitsofinfo/craftercms-test1/blob/master/config/studio/aws/aws.xml#L45-L52

[Sumer Jabri]

Yes, it is until we roll out the symmetric cipher for parameters (we're working on that now).

[bitsofinfo]

Whats the spec of how that will work? A way to wrap or annotated elements to be system encrypted? Where would the keys be managed for this kind of thing? You guys def need a way to securely these config credentials at rest, especially being git based and stuff being pushed around all over the place. 

[Sumer Jabri]

The design right now is to pass the keys via environment vars on the servers and have that flow to the containers. Parameters are encrypted first via CLI with the same key by sys-admin, and the params are decorated indicating they're encrypted while in config/git. Modules (like Studio), will decrypt using the key provided via env in RAM and used. Parameters won't be stored unencrypted at rest, and keys are managed by admins on servers or some other mechanism depending on their rules.

We also added support for a number of envs (not fully documented yet), that allows you to have a full set of configs per environment (in this case, an environment means Dev, QA, Staging, Load Testing, Production -- typically). Each of which can have its own encrypted params and inject with different crypto keys.

Finally, as a stretch goal in the near term, we may provide a tool in Site Config (Studio > Site > Site Config) that lets you generate an encrypted parameter for convenience. This API behind this will need to have a rate limiter as you can imagine.

--sumer

[bitsofinfo]

I'd avoid sending anything sensitive in ENV vars when in an orchestrated environment (unless its a limited time/use based token), instead use any "secrets" capability of the underlying platform, most orchestrators have this kind of thing. Or just externalize this shared config/secrets in something like vault or consul and even further avoid any need for persistent volumes etc.

parameters being....? element values? or the entire file? When I bring up (aws.xml) for example in studio, you would dynamically decrypt the values for presentation/editing by the user, then subsequently mutate the raw xml that the user submits and replace the element values?

[Sumer Jabri]

Please find inline:

On Thursday, December 12, 2019 at 11:33:26 AM UTC-5, bitsofinfo wrote:

I'd avoid sending anything sensitive in ENV vars when in an orchestrated environment (unless its a limited time/use based token), instead use any "secrets" capability of the underlying platform, most orchestrators have this kind of thing. Or just externalize this shared config/secrets in something like vault or consul and even further avoid any need for persistent volumes etc.

Can you give examples we can follow for this?

We want the keys passed into the modules securely so the modules can do things like dip into S3 to index a file etc.

We must also support non-orchestrated deployments...

One other thought was proposed by customers is to use machine-based role authentication, but that's AWS specific and doesn't work well with AWS S3 client. We're trying to remain friendly to on-prem and want to also support Azure and GCP, etc. Also, that doesn't answer the Box/CMIS/WebDAV/etc. remote repos that we currently support as well.

 

parameters being....? element values? or the entire file? When I bring up (aws.xml) for example in studio, you would dynamically decrypt the values for presentation/editing by the user, then subsequently mutate the raw xml that the user submits and replace the element values?

The element values. For instance, the AWS S3 access tokens. Studio's front-end/end-user would only see the encrypted values. What's stored in Git is only the encrypted values.

The decryption happens when:

- Deployer pulls a file from S3/Box/CMIS for indexing or deployment

- Engine reads from S3/Box/CMIS

- Studio pushes a file to S3/Box/CMIS

Decryption only happens on the backend during access to the remote repositories.

Users of the UI can only _encrypt_ when we add that UI I mentioned. Otherwise, it's the CLI tool.

A good example we're trying to follow is `travis-ci`, you can see their tokens in `.travis.yml` (they also use a CLI tool).

[Sumer Jabri]

My team reminded me of this:

Crafter Engine supports this today: https://docs.craftercms.org/en/3.1/site-administrators/engine/engine-site-configuration.html#encrypted-configuration-values but not Studio and Deployer.

You can inject keys via env right now like this:

${env:AWS_ACCESS_KEY}

By doing the above, it won't be stored at rest.

Having said the above, the plan is to push forward with encrypted parameters.

--sumer

[bitsofinfo]

Some ways I've seen it done

1) Store (at a min) all secrets in something like aws secrets manager, hashicorp vault or other. When an application boots up it using a tool like consul-template or just a vendor specific SDK to authenticate against the "secrets" store's APIs, and pull down the secrets it needs on boot (render configs into templates or just hold in mem). The app obviously needs one "bootstrap secret" to be able to auth against the secret store on boot, this can be generated at deploy time w/ a limited life (time/uses) and provided to the application via another secure mechanism such as a kubernetes secret via the filesystem or docker secrets (swarm), etc etc. (i.e. it can be used In this case to boot the app you are only passing one bootstrap token/secret that it can then use to get everything else it needs. (the app also needs to renew this token if you want to be able to horizontally scale etc). You can also use this mechanism to "template" all of your configs, and source even the non-secret configs from another external source, making your deployable artifacts truly environment agnostic; 

2) Use a library/framework like Spring Cloud Config to source down all configuration from another git repo. It behaves similarly in that when the app boots it requires a secret to establish a connection to a discovery service, that tells it the location of its configs, it then uses oauth2 to talk to a config service that feeds it its configs (including secrets crypted at rest and decrypted on the fly). This kind of thing might be a better approach w/ crafter as it appears you already use spring/java etc.

My answers are biased towards orchestrated and being cloud agnostic, but be adapted outside of containers. 

Maybe crafter too tightly couples some of the configs that drive the underlying authoring/delivery engine itself (i,e. like the aws configs), from the raw end-user site content store itself. Might be good to keep this stuff totally separate to then can have diff security around stuff that might hold "secrets" (like this aws.xml) for example, vs raw end-user content that is under a different class of data.

Yes just having a syntax that indicates a value is encrypted is sufficient and leave it up to the operator to encrypt values put into XML configs (i.e. <value>secret:XXXXXXX</value>)

[Sumer Jabri]

Encryption mechanics added to Crafter Studio and can now encrypt parameters in regular configuration files as well as a general encryption tool in the main menu. This will flow into the upcoming release 3.1.5.

--sumer