This is a repost because it appears that I replyed incorrectly to sukar and not to this group. Hopefully this will not show up twice.
Not sure if any of my comment will be helpful, but I thought I would at least mention my recent experience in hopes it will assist you in your plans.
I just started using the Janus WebRTC code last week, so my experience with it is very limited and I am still trying to learn and understand it. But in my short time using it, I can say that I am very impressed with it's design and implementation. From what you describe of your requirements, it appears to me that Janus will be a natural and appropriate fit. I would suggest you clone it and start playing with it to determine if it is suitable for you.