there's no config parameter for this, afaik. The kernel gateway has a parameter to limit the number of kernels, but the notebook server does not. And even the kernel gateway will not automatically stop a running kernel. Rather, it would refuse to create new kernels until an old one has been stopped.
You could implement and configure a custom kernel manager that does what you want. See config property "NotebookApp.kernel_manager_class".
hope that helps,
Roland