Using environment variables in Crossbar config.json

141 views
Skip to first unread message

Jimmy Jia

unread,
Nov 18, 2014, 8:12:55 PM11/18/14
to autob...@googlegroups.com
I'm looking to set up a system where I run my Crossbar node and my database in separate Docker containers. As part of Docker container linking, the port I need to use to access the exposed database port is defined as an environment variable.

I see that in the pg:publisher template, if I want to use the PostgreSQLDatabasePublisher, I need to specify the database information in the config.json file. Is there a way to directly use the environment variable here? I could write a script that wraps "crossbar run" and sets the relevant values, but this seems like it should be a not uncommon use case.

Tobias Oberstein

unread,
Nov 20, 2014, 5:18:35 AM11/20/14
to autob...@googlegroups.com
Hi Jimmy,

Am 19.11.2014 02:12, schrieb Jimmy Jia:
> I'm looking to set up a system where I run my Crossbar node and my
> database in separate Docker containers. As part of Docker container
> linking, the port I need to use to access the exposed database port is
> defined as an environment variable.

What about the other necessary DB connection info like user and password?

In particular: exposing the DB password in a env var seems like a
security issue ...

>
> I see that in the pg:publisher template, if I want to use the
> PostgreSQLDatabasePublisher, I need to specify the database information
> in the config.json file. Is there a way to directly use the environment
> variable here? I could write a script that wraps "crossbar run" and sets
> the relevant values, but this seems like it should be a not uncommon use
> case.

How did you find out about PostgreSQLDatabasePublisher? This is brand
new and we didn't announce it yet;)

Anyway, I can see you point:

https://github.com/crossbario/crossbar/issues/173

Cheers,
/Tobias

>
> --
> You received this message because you are subscribed to the Google
> Groups "Autobahn" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to autobahnws+...@googlegroups.com
> <mailto:autobahnws+...@googlegroups.com>.
> To post to this group, send email to autob...@googlegroups.com
> <mailto:autob...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/autobahnws/e5c1cc32-c880-49e6-8dfd-47ca0bb7e973%40googlegroups.com
> <https://groups.google.com/d/msgid/autobahnws/e5c1cc32-c880-49e6-8dfd-47ca0bb7e973%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

Jimmy Jia

unread,
Nov 20, 2014, 10:52:02 AM11/20/14
to autob...@googlegroups.com
I did mean username and password as well. The environment variable thing is a bit weird, but it's a very common pattern for working in PaaS systems. For example, both 12 Factor (opinionated PaaS style guide from Heroku co-founder) and this random StackOverflow question regarding Docker suggest using environment variables for credentials:


Right now I'm using a wrapper around "crossbar start" that replaces values in the config of type {{VARIABLE}} with os.getenv('VARIABLE'), because I'm using this to configure credentials for other services. I can send in a PR if you think this is a reasonable approach, but I couldn't find a contributor style guide, only the legal contributor agreement bit.

I found out about the publisher by searching for "postgres" in the crossbar repo. Your documentation is great, but I've gotten pretty used to looking at code when I want to figure out how to use something.

Tobias Oberstein

unread,
Nov 21, 2014, 1:15:58 PM11/21/14
to autob...@googlegroups.com
alright, implemented:
https://github.com/crossbario/crossbar/commit/b2c898761a4cf0d8946a0714bc5b96d2a114a7e5
> > an email to autobahnws+...@googlegroups.com <javascript:>
> > <mailto:autobahnws+...@googlegroups.com <javascript:>>.
> > To post to this group, send email to autob...@googlegroups.com
> <javascript:>
> > <mailto:autob...@googlegroups.com <javascript:>>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/autobahnws/e5c1cc32-c880-49e6-8dfd-47ca0bb7e973%40googlegroups.com
> <https://groups.google.com/d/msgid/autobahnws/e5c1cc32-c880-49e6-8dfd-47ca0bb7e973%40googlegroups.com>
>
> >
> <https://groups.google.com/d/msgid/autobahnws/e5c1cc32-c880-49e6-8dfd-47ca0bb7e973%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/autobahnws/e5c1cc32-c880-49e6-8dfd-47ca0bb7e973%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> > For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Autobahn" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to autobahnws+...@googlegroups.com
> <mailto:autobahnws+...@googlegroups.com>.
> To post to this group, send email to autob...@googlegroups.com
> <mailto:autob...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/autobahnws/2410503d-761e-4b97-9210-bc255cc56163%40googlegroups.com
> <https://groups.google.com/d/msgid/autobahnws/2410503d-761e-4b97-9210-bc255cc56163%40googlegroups.com?utm_medium=email&utm_source=footer>.

Jimmy Jia

unread,
Nov 21, 2014, 1:42:38 PM11/21/14
to autob...@googlegroups.com
Thanks!

This implies that the idiomatic way to configure components that need to read settings from the environment is to have special config values that tell the component itself to go check the environment, as opposed to building this into Crossbar's config loading.

Am I interpreting that correctly?
>      > <mailto:autobahnws+unsub...@googlegroups.com <javascript:>>.
>      > To post to this group, send email to autob...@googlegroups.com
>     <javascript:>
>      > <mailto:autob...@googlegroups.com <javascript:>>.
>      > To view this discussion on the web visit
>      >
>     https://groups.google.com/d/msgid/autobahnws/e5c1cc32-c880-49e6-8dfd-47ca0bb7e973%40googlegroups.com
>     <https://groups.google.com/d/msgid/autobahnws/e5c1cc32-c880-49e6-8dfd-47ca0bb7e973%40googlegroups.com>
>
>      >
>     <https://groups.google.com/d/msgid/autobahnws/e5c1cc32-c880-49e6-8dfd-47ca0bb7e973%40googlegroups.com?utm_medium=email&utm_source=footer
>     <https://groups.google.com/d/msgid/autobahnws/e5c1cc32-c880-49e6-8dfd-47ca0bb7e973%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
>      > For more options, visit https://groups.google.com/d/optout
>     <https://groups.google.com/d/optout>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Autobahn" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to autobahnws+...@googlegroups.com

Tobias Oberstein

unread,
Nov 21, 2014, 2:07:26 PM11/21/14
to autobahnws

Yes.

Though I was (and still am) thinking about the alternative you suggest. Probably I will move it into CB core.

Also: this special value magic .. anothwr alt.

Sent from Mobile (Google Nexus 5)

To unsubscribe from this group and stop receiving emails from it, send an email to autobahnws+...@googlegroups.com.
To post to this group, send email to autob...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/autobahnws/049032ed-75ae-465a-b5b2-788f18068cd9%40googlegroups.com.

Jimmy Jia

unread,
Nov 21, 2014, 2:32:28 PM11/21/14
to autob...@googlegroups.com
FWIW this is what I'm doing right now (though it only works with string values for the moment). This lets me specify a value as e.g. "{{$DATABASE_USERNAME}}" and get that resolved from the environment.

START_TOKEN = '{{$'
END_TOKEN = '}}'


# -----------------------------------------------------------------------------

def inject_environment(config):
    if isinstance(config, dict):
        new_config = {}
        for key, value in config.iteritems():
            new_config[key] = inject_environment(value)
    elif isinstance(config, list):
        new_config = []
        for value in config:
            new_config.append(inject_environment(value))
    elif isinstance(config, basestring) and \
            config.startswith(START_TOKEN) and \
            config.endswith(END_TOKEN):
        new_config = os.getenv(config[len(START_TOKEN):-len(END_TOKEN)])
    else:
        new_config = config

    return new_config

With YAML you can make this "just work" without magical values using a !EnvVar tag or something, which is a bit cleaner, but I think JSON is generally nicer and more common for configuration than YAML.
>      > <mailto:autobahnws+unsubscribe@googlegroups.com <javascript:>>.
>      > To post to this group, send email to autob...@googlegroups.com
>     <javascript:>
>      > <mailto:autob...@googlegroups.com <javascript:>>.
>      > To view this discussion on the web visit
>      >
>     https://groups.google.com/d/msgid/autobahnws/e5c1cc32-c880-49e6-8dfd-47ca0bb7e973%40googlegroups.com
>     <https://groups.google.com/d/msgid/autobahnws/e5c1cc32-c880-49e6-8dfd-47ca0bb7e973%40googlegroups.com>
>
>      >
>     <https://groups.google.com/d/msgid/autobahnws/e5c1cc32-c880-49e6-8dfd-47ca0bb7e973%40googlegroups.com?utm_medium=email&utm_source=footer
>     <https://groups.google.com/d/msgid/autobahnws/e5c1cc32-c880-49e6-8dfd-47ca0bb7e973%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
>      > For more options, visit https://groups.google.com/d/optout
>     <https://groups.google.com/d/optout>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Autobahn" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to autobahnws+...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages