I'm using node.js as part of a cloud-based, multi-tenanted server environment, with each node process specific to one of the tenants. (Their most common job is to communicate between some external system and the shared server.) For security reasons, it's a requirement to keep the node processes from interfering with each other, since, in principle, they could be running third-party-written code. After some analysis, we came up with the following specific requirements:
1. Each node processes will be given a file system directory, which will - Contain the JavaScript it runs - Be usable as a scratchpad, e.g. to buffer large datasets read from an external system - Other than that, the process will have no access to any other part of the file system 2. Each node process creates a socket that it uses to receive requests from the server. To prevent different node processes from communicating with each other directly, the ability of node processes to connect to sockets will be restricted. Access to Unix Domain Sockets will be turned off completely. 3. On general principles, Node processes will not be allowed to - kill other processes - change their default directory - change their effective user or group id 4. Node processes will not be allowed to create subprocesses (which might overcome the restrictions above). 5. Node processes will not be allowed to load native-code extensions (which might overcome the restrictions above.)
The only third-party code running in the system will be the node.js JavaScript. In particular, the server that starts up the node.js processes is trusted, so this can be implemented by starting node with command-line flags that force the behavior described above. I've defined three:
- *--restricted-outgoing-addresses* *address-list* takes a comma-separated list of IP addresses to which access will be restricted. This would most commonly be all the IP addresses for the current machine. - *--allowed-outgoing-ports **port-list* takes a comma-separated list of ports to which access is allowed even on restricted addresses. This would represent local services to which the node service is granted access. - *--safe-mode *implements the other restrictions shown above (file system, process restrictions, etc.)
I've forked 0.8.3 and implemented this at https://github.com/mikeatemotive/node.js-safe-mode . I'm quite interested in any comments, and in particular on whether there's interest in bringing this functionality into node.
On Wed, Aug 1, 2012 at 12:56 AM, MikeS <m...@emotive.com> wrote:
> I'm using node.js as part of a cloud-based, multi-tenanted server
> environment, with each node process specific to one of the tenants. (Their
> most common job is to communicate between some external system and the
> shared server.) For security reasons, it's a requirement to keep the node
> processes from interfering with each other, since, in principle, they could
> be running third-party-written code. After some analysis, we came up with
> the following specific requirements:
> Each node processes will be given a file system directory, which will
> Contain the JavaScript it runs
> Be usable as a scratchpad, e.g. to buffer large datasets read from an
> external system
> Other than that, the process will have no access to any other part of the
> file system
> Each node process creates a socket that it uses to receive requests from the
> server. To prevent different node processes from communicating with each
> other directly, the ability of node processes to connect to sockets will be
> restricted. Access to Unix Domain Sockets will be turned off completely.
> On general principles, Node processes will not be allowed to
> kill other processes
> change their default directory
> change their effective user or group id
> Node processes will not be allowed to create subprocesses (which might
> overcome the restrictions above).
> Node processes will not be allowed to load native-code extensions (which
> might overcome the restrictions above.)
> The only third-party code running in the system will be the node.js
> JavaScript. In particular, the server that starts up the node.js processes
> is trusted, so this can be implemented by starting node with command-line
> flags that force the behavior described above. I've defined three:
> --restricted-outgoing-addresses address-list takes a comma-separated list of
> IP addresses to which access will be restricted. This would most commonly
> be all the IP addresses for the current machine.
> --allowed-outgoing-ports port-list takes a comma-separated list of ports to
> which access is allowed even on restricted addresses. This would represent
> local services to which the node service is granted access.
> --safe-mode implements the other restrictions shown above (file system,
> process restrictions, etc.)
> I've forked 0.8.3 and implemented this at
> https://github.com/mikeatemotive/node.js-safe-mode . I'm quite interested
> in any comments, and in particular on whether there's interest in bringing
> this functionality into node.
No. I appreciate the effort but it sounds too much like PHP's safe
mode and everyone knows how broken that is. Containerization should be
done at the OS level, not the application level.
I take your point about php safe mode. Its failings, as I see them, are:
1. It gets in the way of normal functionally, so people turn it off. 2. There are ways around it, so It provides only the illusion of safety.
So far as I can tell, neither of these is true of the node changes. In the this case, which is a multi-tenanted server containing untrusted code, the functionality of the application needs to be restricted. My model was the Java Security Manager, which lets you define specifically the capabilities if the application and the capabilities of untrusted code (e.g. code loaded by specific class-loaders.) In fact, if I develop this further, it will be to make the notion of safety more granular, as Java does. This makes failing (1) less likely. And, given that the only way for node.js to access external resources like the file system and the network are via native-code modules, the combination of
- Checking for allowed capabilities inside the core node modules, and - Disallowing the loading of additional native code
If anyone uses native addons this is irrelevant. Also, if process spawning
of any tool that uses the restricted resources it is irrelevant. I have
spent a lot of time trying to restrict what ports a process can work with
and the only somewhat safe solutions come from passing between parent and
child and overriding cstdlib etc. Even after those chamges port jacking is
a problem if you have services on nonprivileged ports.
Right, that's why native add-ons and subprocess creation are disabled -- allowing them means you lose all control. It would be possible to allow a "blessed" set of addons in a number of ways:
1. Building them into the node.js executable. 2. Creating a directory that JavaScript doesn't have write access to, and allowing add-ons to be loaded only from there. 3. Specifying via a command line flag which add-ons are allowed (and somehow preventing spoofing)
The problem with continually restricting things like child processes and native addons, is that these are popular features, and you still face problems like port/fd hijacking. Indeed removing child processes would help as well as native addons, but this is a sieve. You expect the process to protect you instead of the OS. fork() cluster etc. would have to be turned off as well as running in a "nobody"-like user in order to help with file system access combined with chroot, and the spawning process would have to prevent file descriptor attacks. In order to prevent fd attacks and port hijacking you would need your master process to give the actual fds to the child after verifying them before opening them for the child in the first place. After that, you would want to completely restrict the C++ level access instead of Javascript as well, just to be safe that you did not miss any dangling references (which you did at the node level, but ideally at libsystem etc.). Other considerations are edge cases like INET_PORT_ANY.
The list goes on an on and on for potential exploits and we may eventually list most of them but it is improbable that we ever list all of them. I compare this approach to opt-in security in Windows or using the VM module to executed code, it is nice, but no guarantee. I recommend VM level separation if you are serious about security combined with OS level protection (file system attributes + RBAC etc.).
The problem with this kind of "hardened" approach is that Node has not
been built with that in mind.
True security containment is not trivial. It's not a feature you add
onto a platform later. It's something that you really have to bake
into the architecture from the very start, and evaluate all your
trade-offs in that light. And, if it's done halfway or badly, you end
up with something that performs terribly and has lots of sticky bugs.
Just look at the many trade-offs that have been made in the world of
client-side JavaScript sandboxing, and the awful mess that is php safe
mode.
If you want containerization, I'd recommend using SmartOS and putting
the contained programs in a zone. Trying to do this at the app level
will always lead to headaches.
Furthermore, doing multi-tenancy in this way is extremely dated, and
unnecessary. There's better technology now.
On Thu, Aug 2, 2012 at 4:52 PM, Bradley Meck <bradley.m...@gmail.com> wrote:
> The problem with continually restricting things like child processes and
> native addons, is that these are popular features, and you still face
> problems like port/fd hijacking. Indeed removing child processes would help
> as well as native addons, but this is a sieve. You expect the process to
> protect you instead of the OS. fork() cluster etc. would have to be turned
> off as well as running in a "nobody"-like user in order to help with file
> system access combined with chroot, and the spawning process would have to
> prevent file descriptor attacks. In order to prevent fd attacks and port
> hijacking you would need your master process to give the actual fds to the
> child after verifying them before opening them for the child in the first
> place. After that, you would want to completely restrict the C++ level
> access instead of Javascript as well, just to be safe that you did not miss
> any dangling references (which you did at the node level, but ideally at
> libsystem etc.). Other considerations are edge cases like INET_PORT_ANY.
> The list goes on an on and on for potential exploits and we may eventually
> list most of them but it is improbable that we ever list all of them. I
> compare this approach to opt-in security in Windows or using the VM module
> to executed code, it is nice, but no guarantee. I recommend VM level
> separation if you are serious about security combined with OS level
> protection (file system attributes + RBAC etc.).
On Fri, Aug 3, 2012 at 2:51 PM, Isaac Schlueter <i...@izs.me> wrote:
> The problem with this kind of "hardened" approach is that Node has not
> been built with that in mind.
> True security containment is not trivial. It's not a feature you add
> onto a platform later. It's something that you really have to bake
> into the architecture from the very start, and evaluate all your
> trade-offs in that light. And, if it's done halfway or badly, you end
> up with something that performs terribly and has lots of sticky bugs.
> Just look at the many trade-offs that have been made in the world of
> client-side JavaScript sandboxing, and the awful mess that is php safe
> mode.
> If you want containerization, I'd recommend using SmartOS and putting
> the contained programs in a zone. Trying to do this at the app level
> will always lead to headaches.
> Furthermore, doing multi-tenancy in this way is extremely dated, and
> unnecessary. There's better technology now.
> On Thu, Aug 2, 2012 at 4:52 PM, Bradley Meck <bradley.m...@gmail.com> wrote:
>> The problem with continually restricting things like child processes and
>> native addons, is that these are popular features, and you still face
>> problems like port/fd hijacking. Indeed removing child processes would help
>> as well as native addons, but this is a sieve. You expect the process to
>> protect you instead of the OS. fork() cluster etc. would have to be turned
>> off as well as running in a "nobody"-like user in order to help with file
>> system access combined with chroot, and the spawning process would have to
>> prevent file descriptor attacks. In order to prevent fd attacks and port
>> hijacking you would need your master process to give the actual fds to the
>> child after verifying them before opening them for the child in the first
>> place. After that, you would want to completely restrict the C++ level
>> access instead of Javascript as well, just to be safe that you did not miss
>> any dangling references (which you did at the node level, but ideally at
>> libsystem etc.). Other considerations are edge cases like INET_PORT_ANY.
>> The list goes on an on and on for potential exploits and we may eventually
>> list most of them but it is improbable that we ever list all of them. I
>> compare this approach to opt-in security in Windows or using the VM module
>> to executed code, it is nice, but no guarantee. I recommend VM level
>> separation if you are serious about security combined with OS level
>> protection (file system attributes + RBAC etc.).
I've used both Solaris Zones and Linux Containers. They work well and
cover most of these problems at the OS level. Also depending on how
much sandbox you need, just using a child-process optionally in a
chroot may be enough. But child-process + chroot is NOT a complete
solution and evil users can cause trouble from within those.
On Fri, Aug 3, 2012 at 1:04 PM, Gustavo Machado <machad...@gmail.com> wrote:
> Isaac, I am particularly interested in multi-tenancy, could list a few
> of the technologies that you mention in your last reply?
> Thanks,
> Gustavo
> On Fri, Aug 3, 2012 at 2:51 PM, Isaac Schlueter <i...@izs.me> wrote:
>> The problem with this kind of "hardened" approach is that Node has not
>> been built with that in mind.
>> True security containment is not trivial. It's not a feature you add
>> onto a platform later. It's something that you really have to bake
>> into the architecture from the very start, and evaluate all your
>> trade-offs in that light. And, if it's done halfway or badly, you end
>> up with something that performs terribly and has lots of sticky bugs.
>> Just look at the many trade-offs that have been made in the world of
>> client-side JavaScript sandboxing, and the awful mess that is php safe
>> mode.
>> If you want containerization, I'd recommend using SmartOS and putting
>> the contained programs in a zone. Trying to do this at the app level
>> will always lead to headaches.
>> Furthermore, doing multi-tenancy in this way is extremely dated, and
>> unnecessary. There's better technology now.
>> On Thu, Aug 2, 2012 at 4:52 PM, Bradley Meck <bradley.m...@gmail.com> wrote:
>>> The problem with continually restricting things like child processes and
>>> native addons, is that these are popular features, and you still face
>>> problems like port/fd hijacking. Indeed removing child processes would help
>>> as well as native addons, but this is a sieve. You expect the process to
>>> protect you instead of the OS. fork() cluster etc. would have to be turned
>>> off as well as running in a "nobody"-like user in order to help with file
>>> system access combined with chroot, and the spawning process would have to
>>> prevent file descriptor attacks. In order to prevent fd attacks and port
>>> hijacking you would need your master process to give the actual fds to the
>>> child after verifying them before opening them for the child in the first
>>> place. After that, you would want to completely restrict the C++ level
>>> access instead of Javascript as well, just to be safe that you did not miss
>>> any dangling references (which you did at the node level, but ideally at
>>> libsystem etc.). Other considerations are edge cases like INET_PORT_ANY.
>>> The list goes on an on and on for potential exploits and we may eventually
>>> list most of them but it is improbable that we ever list all of them. I
>>> compare this approach to opt-in security in Windows or using the VM module
>>> to executed code, it is nice, but no guarantee. I recommend VM level
>>> separation if you are serious about security combined with OS level
>>> protection (file system attributes + RBAC etc.).
If you do use chroots as jails, beware of chroot jail breaking using fd exploits and other techniques. Ensure your used is still non-privileged on the filesystem as appropriate.
On Fri, Aug 3, 2012 at 9:23 PM, Bradley Meck <bradley.m...@gmail.com> wrote:
> If you do use chroots as jails, beware of chroot jail breaking using fd
> exploits and other techniques. Ensure your used is still non-privileged on
> the filesystem as appropriate.
