Envoy IP range filters question: how to do allow, then deny, default allow?

[Ofer Inbar]

[ I tried the Envoy slack but it seems mostly inactive so trying here.
I haven't joined this list, please CC any responses to c...@aaaaa.org. ]

I'm not very familiar with Envoy and don't know how much I'll be
working with it, but I have a need to modify an existing (and large)
Envoy configuration, and having trouble figuring out if what I want
can be done.

Configuration is generated by a homegrown script, which is what I'll
actually modify, to autogenerate a modified config. Currently, this
configuration has:
- multiple listeners
- each listener has one or more filterChains
- each filterChain has envoy.filters.network.rbac as its first filter
- most filterChains then have a second filter that does other things
- the network.rbac filters are all identical (generated from the script);
each one has rules.action=DENY, rules.policies with a list of
principals which are all directRemoteIp - a set of IP ranges to deny

In other words, the current configuration applies a deny list of IP
ranges to every listener, and the default is to allow if the source
IP is not in one of those ranges.

What I need to add is an allow list that pre-empts the deny list:
a list of IP ranges to always allow, even if they match the deny
list. The rest of the behavior needs to stay the same. In other
words, the new behavior would be:
- If source IP is on the allow list, allow
- If it is not, then if it is on the deny list, deny
- If on neither list, then allow

I'm having a very hard time figuring out how to do this, even though
it seems like it should be easy and straightforward. Here's what I
think I've learned:

1. It is impossible to define a single rbac filter with a short
circuit chain of rules like this. The filter can either be entirely a
DENY filter (that is, default allow, but deny what matches the
policies) or entirely an ALLOW filter (that is, default deny, but
allow what matches the policies). Each filter can only have a single
"rules", which can only have one "action" (why "rules" not "rule"?)

2. Since each RBAC "rules" is part of a filter that is part of a
filterChains, and a filterChains can actually have multiple filters,
I considered whether I can use this chaining to get this behavior.
However, it looks like each chain will always go through all of the
filters in its list in turn, so having an allow-list filter before
the deny-list filter would have no effect - those IPs would still
get blocked.

3. Even if I'm wrong about #2 and filterChains can be made to have
with a short circuit, where first match wins, that would still not
work here. We need the subsequent rule that comes after the RBAC
filter to be processed whenever traffic is allowed; an allow list
that prevents that last filter from being processed would be bad.

4. "matcher" appears to be an alternative to "rules" that may be
able to do exactly what I want. But it has a big warning in the
docs saying that it's basically not supported yet, so we can't
use it for production configs.

Please correct me if any of my impressions above are wrong.

Creating a simple chain of allow and deny rules seems pretty basic, so
I'm surprised it's so hard to do with Envoy. Am I missing something?

-- Cos

[Dmitry Ilyevsky]

Using the matchers field rather than the rules should allow you to do this. Example: https://github.com/envoyproxy/examples/blob/6aa9c3730c91eddaf0c69cc7cbe5294d3da74310/rbac/envoy.yaml#L29

You would need two matchers - one for allow list, second for deny list, and allow on no_match.

Thanks,

-Dmitry

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/20240804221243.GO26064%40miplet.aaaaa.org.

[Ofer Inbar]

I wrote:
> > 4. "matcher" appears to be an alternative to "rules" that may be
> > able to do exactly what I want. But it has a big warning in the
> > docs saying that it's basically not supported yet, so we can't
> > use it for production configs.

Dmitry Ilyevsky <ds...@apoxy.dev> wrote:
> Using the matchers field rather than the rules should allow you to do this.
> Example:
> https://github.com/envoyproxy/examples/blob/6aa9c3730c91eddaf0c69cc7cbe5294d3da74310/rbac/envoy.yaml#L29
>
> You would need two matchers - one for allow list, second for deny list, and
> allow on no_match.

Thanks for the example, that's helpful.

I think you're confirming that "matcher" is an alternative to "rules"
- that is, I can use one in place of the other. And matcher can do
what I want.

Unfortunately it looks like it is not safe to use "matcher" yet:
https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/network/rbac/v3/rbac.proto

This API feature is currently work-in-progress. API features marked
as work-in-progress are not considered stable, are not covered by
the threat model, are not supported by the security team, and are
subject to breaking changes. Do not use this feature without
understanding each of the previous points.

Does anyone here know more about this status?

I found an old github issue open asking for this capability to be
added to "rules", but nobody from envoy has updated in a long time:
https://github.com/envoyproxy/envoy/issues/9376

Is "matcher" intended as a replacement for "rules" in the future?

-- Cos