Query logs-* with rights only on logs-user1-* ?

[foot...@gmail.com]

Hello,

This is more of a theoretical question, because i couldn't find a definitive answer in the docs (ES6.3.1 / SG 6.3.1-22.3)

I'm looking to provide a common read-only shared Kibana dashboard to several separate users who cannot see each others' data.

I split the user's data into several indexes:

- logs-user1-YYYY.MM.DD (1 by day)

- logs-user2-YYYY.MM.DD (1 by day)

...

- logs-userN-YYYY.MM.DD (1 by day)

But I run into at least one major issue, and i don't know if this is by design or it's something I don't do correctly.

I added privileges to user1 onto logs-user1-* (INDICES_ALL), and I'm able to query the state of these indexes by querying /logs-user1-*; but if I run a query on /logs-* with the same user, the permission is denied.

Is this behavior expected because Search Guard denies or allows based on the content, but is not able to filter the results themselves ?

Thanks,

Best regards,

Aurélien

[Jochen Kressin]

Search Guard has two ways of dealing with insufficient privileges when querying multiple indices (like in your case all the logs-*) indices.

1. Deny the request completely

2. Only return data from allowed indices

The default is 1), because filtering out results (silently) may lead to strange results, especially when using aggregations. But it depends on the use case which option is "better".

So to activate 2), add this to your sg_config.yaml:

searchguard:

  dynamic:

    kibana:

      do_not_fail_on_forbidden: true

 Setting do_not_fail_on_forbidden to false will give you behavior 1), setting it to true will give you behavior 2).

[foot...@gmail.com]

Hello,

Thanks for your quick response.

Le mercredi 11 juillet 2018 15:49:27 UTC+2, Jochen Kressin a écrit :

 Setting do_not_fail_on_forbidden to false will give you behavior 1), setting it to true will give you behavior 2).

I had already tried this one, but it doesn't seem to be effective ?

curl -u user1:pass 'es:9200/_cat/indices?index=access_log-*' -> permission denied
{"error":{"root_cause":[{"type":"security_exception","reason":"no permissions for [indices:monitor/stats] and User [name=user1, roles=[customer], requestedTenant=null]"}],"type":"security_exception","reason":"no permissions for [indices:monitor/stats] and User [name=user1, roles=[customer], requestedTenant=null]"},"status":403}

curl -u user1:pass 'es:9200/_cat/indices?index=access_log-user1-*' -> 200 with index statuses
green open access_log-user1-2018.07.03 yDSuaz92S9iTM0izkV7Wfg 5 1 1104624 0 1.1gb 564.7mb
green open access_log-user1-2018.07.08 g8j24cSYS9qkiHJHWv6y7g 5 1 1862963 0 1.8gb 952.6mb
green open access_log-user1-2018.07.10 bNuWnlbURXyK3Xc0R6VYCQ 5 1 2030975 0   2gb     1gb
green open access_log-user1-2018.07.11 m1pXTJuWTrGIfR51ZpOl8Q 5 1 1331561 0 1.2gb 659.8mb
green open access_log-user1-2018.07.06 iYFrm3tkSNCcVvun-4KQBQ 5 1 2316625 0 2.2gb   1.1gb
green open access_log-user1-2018.07.02 yf-Q1-RjQdeLdBrwfzQ8Sg 5 1 1567440 0 1.4gb 761.9mb
green open access_log-user1-2018.07.07 M1rtzMTgSaqs_WohIhzvjg 5 1 1822786 0 1.8gb 938.9mb
green open access_log-user1-2018.07.09 M8PjznBdSyGTWCnhR2JqXA 5 1 2137226 0 2.1gb     1gb
green open access_log-user1-2018.07.05 zIlMiTKGQLCzbhRIpINyQg 5 1 1517486 0 1.5gb 776.5mb

sg_config.yml:

searchguard:
  dynamic:
    kibana:
      do_not_fail_on_forbidden: true

    http:
      anonymous_auth_enabled: false
      xff:
        enabled: false
        internalProxies: '192\.168\.0\.10|192\.168\.0\.11' # regex pattern
        #internalProxies: '.*' # trust all internal proxies, regex pattern
        remoteIpHeader:  'x-forwarded-for'
        proxiesHeader:   'x-forwarded-by'
    authc:
      basic_internal_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 4
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: intern

[Aurélien GUILLAUME]

Hello,

On 11 Jul 2018, at 17:40, foot...@gmail.com wrote:

Hello,

Thanks for your quick response.

Le mercredi 11 juillet 2018 15:49:27 UTC+2, Jochen Kressin a écrit :

 Setting do_not_fail_on_forbidden to false will give you behavior 1), setting it to true will give you behavior 2).

I had already tried this one, but it doesn't seem to be effective ?

Searching further in the code : is this setting even used for something other than the _searchguard/kibana info page ?

BCS-NBK-AGU [footy] ~/Projects/search-guard % grep -r forbidden .

./sgconfig/sg_config.yml:      #do_not_fail_on_forbidden: false

./src/main/java/com/floragunn/searchguard/configuration/PrivilegesEvaluator.java:                && getConfigSettings().getAsBoolean("searchguard.dynamic.kibana.do_not_fail_on_forbidden", false);

./src/main/java/com/floragunn/searchguard/tools/SearchGuardAdmin.java:                System.out.println("     a configuration error and is therefore forbidden now.");

./src/main/java/com/floragunn/searchguard/rest/KibanaInfoAction.java:                    builder.field("not_fail_on_forbidden_enabled", evaluator.notFailOnForbiddenEnabled());

!130! BCS-NBK-AGU [footy] ~/Projects/search-guard % grep -r notFailOnForbidden .

./src/main/java/com/floragunn/searchguard/configuration/PrivilegesEvaluator.java:    public boolean notFailOnForbiddenEnabled() {

./src/main/java/com/floragunn/searchguard/rest/KibanaInfoAction.java:                    builder.field("not_fail_on_forbidden_enabled", evaluator.notFailOnForbiddenEnabled());

Thanks,

Best regards,

Aurélien

[Jochen Kressin]

I think we now have a guess regarding this issue: Am I right in the assumption that you are using the Community Edition? If this is the case, please try the 6.3.1-compliance-2 SG plugin.

Background: The feature I mentioned was once an Enterprise feature, but we made it a Community feature. This has been implemented in the SG Compliance codebase. We are in the process of merging the codebases, but for now it is only available in the compliance builds.

You can use these builds and disable all enterprise features as you probably did with the 22.3 version.

[Aurélien GUILLAUME]

Hello,

In fact I thought this was an enterprise feature, and I enabled the Trial licence to check that. I also did try the compliance edition:

[2018-07-20T15:36:44,978][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Search Guard License Info: SearchGuardLicense [uid=00000000-0000-0000-0000-000000000000, type=TRIAL, features=[COMPLIANCE], issueDate=2018-07-03, expiryDate=2018-09-02, issuedTo=The world, issuer=floragunn GmbH, startDate=2018-07-03, majorVersion=6, clusterName=*, allowedNodeCount=2147483647, msgs=[], expiresInDays=43, isExpired=false, valid=true, action=, prodUsage=Yes, one cluster with all commercial features and unlimited nodes per cluster., clusterService=org.elasticsearch.cluster.service.ClusterService@176bcbb5, getMsgs()=[], getExpiresInDays()=43, isExpired()=false, isValid()=true, getAction()=, getProdUsage()=Yes, one cluster with all commercial features and unlimited nodes per cluster.]

But there is no change in the outputs.

Using com.floragunn:search-guard-6:6.3.1-compliance-2 right now; with the same errors.

Privilege lookup log is the following:

[2018-07-20T15:42:12,298][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=user1, roles=[customer], requestedTenant=null] Resolved [aliases=[], indices=[access_log-2018.06.18, access_log-2018.06.19, access_log-global-2018.07.19, access_log-user2-2018.07.20, access_log-global-2018.07.17, access_log-global-2018.07.18, access_log-global-2018.07.15, access_log-global-2018.07.16, access_log-global-2018.07.13, access_log-global-2018.07.14, access_log-global-2018.07.11, access_log-global-2018.07.12, access_log-global-2018.07.10, access_log-user1-2018.07.15, access_log-user1-2018.07.16, access_log-user1-2018.07.17, access_log-user1-2018.07.18, access_log-user1-2018.07.11, access_log-user1-2018.07.12, access_log-user1-2018.07.13, access_log-user1-2018.07.14, access_log-user1-2018.07.19, access_log-global-2018.07.08, access_log-global-2018.07.09, access_log-global-2018.07.06, access_log-test-2018.07.02, access_log-user2-2018.07.10, access_log-global-2018.07.07, access_log-user2-2018.07.11, access_log-global-2018.07.05, access_log-user2-2018.07.12, access_log-user2-2018.07.13, access_log-global-2018.07.02, access_log-test-2018.07.06, access_log-user2-2018.07.14, access_log-global-2018.07.03, access_log-test-2018.07.05, access_log-user1-2018.07.20, access_log-user2-2018.07.15, access_log-user2-2018.07.16, access_log-test-2018.07.03, access_log-user2-2018.07.17, access_log-user2-2018.07.18, access_log-test-2018.07.09, access_log-user2-2018.07.19, access_log-test-2018.07.08, access_log-test-2018.07.07, access_log-user1-2018.07.05, access_log-user1-2018.07.06, access_log-user1-2018.07.07, access_log-user1-2018.07.02, access_log-user1-2018.07.03, access_log-2018.07.02, access_log-user1-2018.07.08, access_log-2018.06.30, access_log-user1-2018.07.09, access_log-2018.07.01, access_log-test-2018.07.13, access_log-test-2018.07.12, access_log-test-2018.07.11, access_log-test-2018.07.10, access_log-test-2018.07.17, access_log-user2-2018.07.02, access_log-test-2018.07.16, access_log-user2-2018.07.03, access_log-test-2018.07.15, access_log-user2-2018.07.05, access_log-test-2018.07.14, access_log-user1-2018.07.10, access_log-user2-2018.07.06, access_log-user2-2018.07.07, access_log-test-2018.07.19, access_log-user2-2018.07.08, access_log-test-2018.07.18, access_log-user2-2018.07.09, access_log-2018.06.27, access_log-2018.06.28, access_log-2018.06.29, access_log-2018.06.23, access_log-2018.06.24, access_log-2018.06.25, access_log-2018.06.26, access_log-2018.06.20, access_log-2018.06.21, access_log-2018.06.22, access_log-test-2018.07.20, access_log-global-2018.07.20], allIndices=[access_log-2018.06.18, access_log-2018.06.19, access_log-global-2018.07.19, access_log-user2-2018.07.20, access_log-global-2018.07.17, access_log-global-2018.07.18, access_log-global-2018.07.15, access_log-global-2018.07.16, access_log-global-2018.07.13, access_log-global-2018.07.14, access_log-global-2018.07.11, access_log-global-2018.07.12, access_log-global-2018.07.10, access_log-user1-2018.07.15, access_log-user1-2018.07.16, access_log-user1-2018.07.17, access_log-user1-2018.07.18, access_log-user1-2018.07.11, access_log-user1-2018.07.12, access_log-user1-2018.07.13, access_log-user1-2018.07.14, access_log-user1-2018.07.19, access_log-global-2018.07.08, access_log-global-2018.07.09, access_log-global-2018.07.06, access_log-test-2018.07.02, access_log-user2-2018.07.10, access_log-global-2018.07.07, access_log-user2-2018.07.11, access_log-global-2018.07.05, access_log-user2-2018.07.12, access_log-user2-2018.07.13, access_log-global-2018.07.02, access_log-test-2018.07.06, access_log-user2-2018.07.14, access_log-global-2018.07.03, access_log-test-2018.07.05, access_log-user1-2018.07.20, access_log-user2-2018.07.15, access_log-user2-2018.07.16, access_log-test-2018.07.03, access_log-user2-2018.07.17, access_log-user2-2018.07.18, access_log-test-2018.07.09, access_log-user2-2018.07.19, access_log-test-2018.07.08, access_log-test-2018.07.07, access_log-user1-2018.07.05, access_log-user1-2018.07.06, access_log-user1-2018.07.07, access_log-user1-2018.07.02, access_log-user1-2018.07.03, access_log-2018.07.02, access_log-user1-2018.07.08, access_log-2018.06.30, access_log-user1-2018.07.09, access_log-2018.07.01, access_log-test-2018.07.13, access_log-test-2018.07.12, access_log-test-2018.07.11, access_log-test-2018.07.10, access_log-test-2018.07.17, access_log-user2-2018.07.02, access_log-test-2018.07.16, access_log-user2-2018.07.03, access_log-test-2018.07.15, access_log-user2-2018.07.05, access_log-test-2018.07.14, access_log-user1-2018.07.10, access_log-user2-2018.07.06, access_log-user2-2018.07.07, access_log-test-2018.07.19, access_log-user2-2018.07.08, access_log-test-2018.07.18, access_log-user2-2018.07.09, access_log-2018.06.27, access_log-2018.06.28, access_log-2018.06.29, access_log-2018.06.23, access_log-2018.06.24, access_log-2018.06.25, access_log-2018.06.26, access_log-2018.06.20, access_log-2018.06.21, access_log-2018.06.22, access_log-test-2018.07.20, access_log-global-2018.07.20], types=[*], isAll()=false, isEmpty()=false] [Action [indices:monitor/stats]] [RolesChecked [sg_own_index, sg_customer]]

[2018-07-20T15:42:12,300][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for [indices:monitor/stats] 

--
You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
To post to this group, send email to search...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/8c6f70fe-15a8-4f2f-b4c3-8ba39fe20fcb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Aurélien GUILLAUME]

Hello,

So, is it a bug or did I misunderstood the feature ?

Thanks,

Best regards,

-- 

Aurélien GUILLAUME

foot...@gmail.com

[Jochen Kressin]

I don't think it is a bug. I have now tried to replicate the problem and I think somehow the "do not fail on forbidden" feature is not active. Here's what I did:

I set up three sample indices with one document each:

yellow open logs-user3-2018.1       QwzklrtbTUGrA5QLtS-fIw 5 1 1 0  4.5kb  4.5kb
yellow open logs-user2-2018.1       4qx9VaNgSyaXr76PKSA5DA 5 1 1 0  4.5kb  4.5kb
yellow open logs-user1-2018.1       -Vj1d4cMRzyU-KhJEcuFTw 5 1 1 0  4.5kb  4.5kb

I created a user "user1" with the following role:

sg_user1:

  cluster:

    - UNLIMITED

  indices:

    'logs-user1-*':

      '*':

        - INDICES_ALL

And enable dnfof with:

searchguard:

  dynamic:

      kibana:

        do_not_fail_on_forbidden: true

When I query 

https://<eshost>:9200/logs-*/_search

I get data from the index:

logs-user1-2018.1

without an error. Tested with the compliance edition.

Can you check whether the dnfof feature is really active by pulling your current configuration from the cluster and posting the sg_config.yml here? You can use sgadmin and the -r/--retrieve switch which will pull the active configuration from the cluster and place them in the current directory.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.

[Aurélien GUILLAUME]

Hello,

I'm able to replicate this.

But now, when you make a generic dashboard in Kibana for your customers (e.g. a public generic read-only .kibana for your customer - in my case I use ), you need to add logs-* as a log pattern.

And the inventory of the indexes then fails: please try with the user: /_cat/indices?v&index=logs-* (vs /_cat/indices?v&index=logs-user1-* which works).

But perhaps this is a known behavior.

Thanks,

Best regards,

-- 

Aurélien GUILLAUME

foot...@gmail.com

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.

To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/1434cc0c-2709-4ba4-9f14-a670a9ccd6c0%40googlegroups.com.

[Jochen Kressin]

Ok, thanks for the info, that gives the error a better frame. I will try to replicate it with Kibana as well.

[Jochen Kressin]

Ok, I see what you mean with /_cat/indices?v&index=logs-* vs /_cat/indices?v&index=logs-user1-*. We need to dig a bit deeper into this.

However, I could not reproduce this behavior in Kibana. I logged in with user1, created an index pattern "logs-*" and a visualization and a dashboard. Everything worked fine here. Could you please describe how to reproduce this exactly in Kibana?