Proposal for Route Services

[Dieu Cao]

Hi All,

We've put together a proposal for Route Services [1].

The goal of this feature is to enhance the router to be able to forward requests for a particular route to a route service, which then could choose to respond immediately or forward the request to the application. Developers can opt in to various types of filters and content transformations for their application.  For example, a developer could bind a rate limiting, metering, authorization, or caching service to his/her app.

We'd like to be able to iterate on this document quickly, so please keep your input and feedback in the comments of the document.

-Dieu

CF Runtime PM

[1] https://docs.google.com/document/d/1bGOQxiKkmaw6uaRWGd-sXpxL0Y28d3QihcluI15FiIA/edit?usp=sharing

[Alexander Lomov]

As far as I can see, all application traffic will be passed through route service and this means we will be able to monitor and manage it by putting all kind of traffic policies. This is great solution for bunch of problems. And I don't think overhead will be significant in this case, will think on what we can add to your proposal.

Thank you, Dieu.

------------------------

Alex Lomov

Altoros — Cloud Foundry deployment, training and integration

Twitter: @code1n GitHub: @allomov

------------------------

Alex Lomov

Altoros — Cloud Foundry deployment, training and integration

Twitter: @code1n GitHub: @allomov

--
You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group.
To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/a319b2aa-ee27-4445-a92c-501b3fdf0ea1%40cloudfoundry.org.

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

[John McTeague]

I very much like the proposal, however Im not sure I like the treatment of these services as separate from the other services (from the marketplace) and the syslog drains, which are a specialized form of service.

I would prefer to see a unification of service types and a single, consistent way of consuming these. I would want these route services to be part of the marketplace just like any other service (db, messaging etc). Syslog drains can already be presented and consumed via the marketplace.

John

[Mike Heath]

I agree with John. The inconsistencies (and bugs) that came from user provided services being a one-off service were bad enough. I would hate to see these kinds of specialized services propagate.

I get the impression that the user provided service was created as a specialized service because the services API is too rigid. Rather than created specialized services, I would rather take these use cases as an opportunity to make the services API more flexible.

-Mike

[Mike Youngstrom]

Interesting proposal.  Some goals of this work seems to conflict with the recent work to expose ip and port as environment variables. https://www.pivotaltracker.com/n/projects/966314/stories/82311924  This story seems to imply that CF is making strides towards accepting direct connections to application instances, bypassing the routers.

If the routers can be bypassed that would seem to defeat the value of many of these route proxy use cases.

Do you have any thoughts on this potential conflict in functionality?  Should all containers be forced to only accept connections from a router and provide some mechanism for the app to opt out of this restriction?  Or perhaps automatically apply router only ingress rules anytime an app is bound to a route service?

Thoughts?

Mike

--

[Dieu Cao]

Thanks for your feedback Mike and John.

I've updated the proposed cli interaction to use cf cups, cf bind-service, and cf services instead of wholly new commands.

I've also added a note under Non Goals that service brokers for route services are not part of MVP but could be added in future.

Sound good?

-Dieu

[Dieu Cao]

We don't plan on allowing direct connections to application instances through gorouter.

The environment variable work to expose the DEA's host ip within the container is to support certain service discovery use cases, where, for example, you have a private cloud foundry deployment and you trust your apps to talk to each other, you could relax security group rules to allow those apps to communicate directly.  

Also, once work on placement pools is completed, you could imagine deploying sets of apps to a particular placement pool and relaxing security groups rules just for apps deployed to that placement pool.

-Dieu

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+unsubscribe@cloudfoundry.org.

[Mike Youngstrom]

Thanks for the response Dieu.  I understand the direct to DEA host use cases.  And I like those use cases as an option for my customers.  My concern is that with route services I see a conflict in functionality.  For our deployment we want to suppose route services for apps that want to use the router as the front door AND direct connections for apps that want to support "certain service discovery use cases".  My hope is that we can come up a way to support both use cases without having to fragment my deployment.

If there were some way to manage ingress access to an perhaps we could do both?

Mike

To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/53d57e08-e0c7-41e3-8af8-9580c5d35d5e%40cloudfoundry.org.

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

[Mike Youngstrom]

If there were some way to manage ingress access to an perhaps we could do both?

Not for the same app obviously but perhaps at least within the same deployment or placement pool?

[zrob...@pivotal.io]

Hi Mike,

I'm not sure I understand what you mean by fragmenting your deployment.  Do you have a more concrete use-case to help explain your concern? 

Thanks,

Zach

[Mike Youngstrom]

Hi Zach,

Let me start over.  I, and I'm sure many others, were excited by the completion of story https://www.pivotaltracker.com/n/projects/966314/stories/82311924  This enables applications with their own service discovery mechanisms to bypass the routers and work in a cloud foundry deployment.  However, as Dieu called out to enable this type of app you'd have to open security groups to allow direct access to DEAs and warden port ranges for these apps.  Today opening up direct access to DEA ports is not a big deal because there is little functional difference between hitting an app instance directly and hitting it through the router.

However, with the concept of Route Services this changes.  For example, if I have an "auth" Route Service that injects a header to identify the authenticated user to an application using that auth service.  You cannot allow direct access to that application because that would enable a header injection attack on that application.  Another example, if the Route Service were a rate limit service then direct access to application instances would allow apps to circumvent rate limits.  Applications using a Route Service require that the app only be accessed through a router and not any other way.  If I have a security group that enables direct access to DEAs then apps with that security group enabled will be able will be able to circumvent the Route Service.

Their may be ways to get around this restriction using shared secrets between Route Services and the app but they'd require apps to code to the Route Service explicitly which I think would be sub optimal.

I want my deployment to support both apps that can do their own route discovery and apps that can use Route Services.

Dieu proposed that I could use something like placement pools (when they come) to fragment my deployment.  And require apps that want to do their own service discovery be deployed to their own placement pool.  I'm not a fan of this type of deployment fragmentation.

I think their are other options that could allow both service discovery apps and Route Services apps to exist side by side without the need to fragment my deployment.  Here are some options I see:

* Add support for ingress rules to security groups.  Spaces that wish to deploy Service Discovery apps would enable broader inbound access.  By default spaces could be restricted to only accept requests from routers.

* If the DEA notices that an application is using Route Services, only allow, that application to receive connections from routers.

* Have the router sign a header to prove the request is coming from it.  Then have the DEA filter requests into the container to validate the signed header to ensure the request came from a router before passing the request onto the app.

Those are a few thoughts.  Does that help clarify my concern Zach?

Thanks,

Mike

--

You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group.

To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/6003fa20-2348-40cf-8dc7-bfc0ebdae606%40cloudfoundry.org.

[Dieu Cao]

Thanks Mike for the clarification.  I think that use case makes sense.

We'll review and update.

-Dieu

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+unsubscribe@cloudfoundry.org.

[George Georgovassilis]

Hello Dieu,

Will route services enable path-based routing? So far the router maps HTTP requests to applications only based on the domain. I could imagine writing a single route service through which the main router pushes every HTTP request; this route service could inspect the HTTP request and route it based on the path. Will that be possible?

BR,
G.

[Dieu Cao]

Hi Mike,

We like the idea of support for ingress rules for application security groups.  

We don't plan to work on extending application security groups to support ingress rules for MVP of route services.  

We may take on that work in the future.  

We would be happy to work with you on a PR for ingress rules if you are interested.  

I've updated the proposal to include a note about this under Non Goals.

-Dieu

[Dieu Cao]

Hi George,

We think that should be possible.

We plan to do a spike on this proposal [1] in the next few weeks and we'll know more after the spike is completed.

-Dieu

[1] https://www.pivotaltracker.com/story/show/86843974

[Mike Youngstrom]

Sounds fair.  I doubt I'll have time to take on such a PR but perhaps one of those who pushed through 82311924 will be interested in working on such a PR.

Thanks for acknowledging the use case. :)

Mike 

To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/a3d73d2b-959f-4748-8fb3-d99e182957fc%40cloudfoundry.org.

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

[Nicolas Regez]

Hello Dieu

I only catched this topic today morning, I hope my input is not coming much too late. I yould like to share two points which came into my mind and would love to get your or anybody else's opinion on them.

[1] Many aspects you'd like to apply to your routes should be coupled to apps rather than routes. For example if you have multiple domains under which your app is accessible, you want aspects like authorization to be present and identical under all routes. So what about binding route-services to apps?
[2] Would it be conceivable to model the http access to cc itself as a route (api.*) which route-services can be bound to? This would open a veriety of nice options for service providers.

Many thanks for this initiative and looking forward to your feedback.

Best regards
Nicolas

[James Bayer]

dieu can fix any mis-steps, but here's a quick reply to your points.

[1] Many aspects you'd like to apply to your routes should be coupled to apps rather than routes. For example if you have multiple domains under which your app is accessible, you want aspects like authorization to be present and identical under all routes. So what about binding route-services to apps?

the route-service proposal was modified so that route services are modeled as "user-provided services" with a special attribute, like syslog drains. therefore binding an app to the user-provided service is the way it is proposed to be expressed, which means it would apply to all routes for the app.

[2] Would it be conceivable to model the http access to cc itself as a route (api.*) which route-services can be bound to? This would open a veriety of nice options for service providers.

the CC is treated like any other route by the router, so from the routers perspective this type of interaction should be possible. CF does not push the cc as an app, so there would need to be an alternative form of UI/config to register route services for the CC.

--
You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group.

To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/4970e984-9e52-4fe0-9541-373a1f21ca65%40cloudfoundry.org.

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

--

Thank you,

James Bayer

[Dieu Cao]

We haven't yet started work on this track of work. Thanks for your feedback.

I've added a note to the proposal that the MVP would not include support for route services for cc, uaa, other services but that it could be added in the future.

-Dieu

CF Runtime PM

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+unsubscribe@cloudfoundry.org.

[Guillaume Berche]

Dieu, it seems the proposal google document is protected and requires requesting autorization to access it. Any chance to make it publicly readeable as to ease its read access, and getting feedback ?

Thanks in advance,

Guillaume.

ps: I've also requested access in the mean time.

[James Bayer]

guillaume, the permissions were updated, please try now.

--

You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group.

To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/0e4358f1-e61b-43e7-bd6b-e2a210eaad09%40cloudfoundry.org.

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

[Guillaume Berche]

thanks, the document can now be read without being logged in.

Guillaume.

--
You received this message because you are subscribed to a topic in the Google Groups "Cloud Foundry Developers" group.
To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAB%3Dt-sXY26J3u-7_2LJAKbO3tJycj9r%2BuLib6MwjviD5mD-B3w%40mail.gmail.com.