DLS Searchguard2

[djtecha]

Is the DLS currently available in the 2.3.2 BETA 2 release? It claims that it should be part of the 2nd beta, but looks to be disabled according to my ES logs "FLS/DLS not enabled" Also, is this being considered part of the open source part of SG or will I need to purchase a license? It would really be a shame if one needed to use a license for this feature as it basically makes the project unusable for us and not much different form Shield. 

[SG]

See https://github.com/floragunncom/search-guard/wiki

DLS/FLS and LDAP ist not yet available, it will be released on Jun-17 under a dual license which means that it will be open source (in terms of: you can inspect the source code on github) and you can use it for non-commercial and non-production purposes free of charge. For commercial purposes (when deployed in a production cluster) you need to purchase either a license (which is licensed per cluster, not per node) or you purchase a support subscription (which then includes also the license). Its also free for academic use. Hope this fits you need.

> Am 19.05.2016 um 23:06 schrieb djtecha <djt...@gmail.com>:
>
> Is the DLS currently available in the 2.3.2 BETA 2 release? It claims that it should be part of the 2nd beta, but looks to be disabled according to my ES logs "FLS/DLS not enabled" Also, is this being considered part of the open source part of SG or will I need to purchase a license? It would really be a shame if one needed to use a license for this feature as it basically makes the project unusable for us and not much different form Shield.
>

> --
> You received this message because you are subscribed to the Google Groups "Search Guard" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> To post to this group, send email to search...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/497fb3ef-3e42-43bf-ab6c-c2cc2e9bd513%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[djtecha]

What will be the price of the license? Is it a one time purchase or annual license?

[SG]

Pls. see https://floragunn.com/searchguard/searchguard-license-support/ for details.
Its a annual license but if you're interested in a one time purchase i think we could also make this possible (pls. contact us through the contact form on the site or per mail via in...@floragunn.com).

> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/1cb82075-d523-4a8b-a121-7c978b85c92c%40googlegroups.com.

[djtecha]

So, should this be working in the first release? I'm trying to do something like:

sg_roles.yml

sg_public:

  indices:

    '*':

     '*':

        - READ

        - indices:admin/mappings/fields/get*

        - indices:admin/validate/query*

        - indices:admin/get*

    '?kibana':

      '*':

        - indices:admin/exists*

        - indices:admin/mapping/put*

        - indices:admin/mappings/fields/get*

        - indices:admin/refresh*

        - indices:admin/validate/query*

        - indices:data/read/get*

        - indices:data/read/mget*

        - indices:data/read/search*

        - indices:data/write/delete*

        - indices:data/write/index*

        - indices:data/write/update*

  _dls_: '{ "term" : {"type" : "elasticsearch" } }'

And I see the user get passed through and added to the sg_public role only, but that user can still search ALL types in kibana. I also downloaded the jar for all nodes.

[SG]

try this: https://gist.github.com/floragunncom/fc14ae597faccab7d0a9eb4aff8891b8 (note that _dls_ is on the same level as the type, so its bound on the index pattern)

See also https://github.com/floragunncom/search-guard/blob/master/sgconfig/sg_roles.yml as an example

> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/76582837-ddd5-4fef-ac18-adc8aaea7cc0%40googlegroups.com.

[Daniel Kasen]

Brings me back to this issue with the status page:

plugin:elasticsearch Document_already_exists_exception] [config][4.5.0]: document already exists, with: {"shard":"0","index":".kibana"}

Full sg_roles.yml

# Allows everything

# but not changes to searchguard config/index

sg_admin:

  cluster:

    - '*'

  indices:

    '*':

      '*':

        - '*'

        

sg_public:

  indices:

    '*':

      '*':

        - READ

        - indices:admin/mappings/fields/get*

        - indices:admin/validate/query*

        - indices:admin/get*

      _dls_: '{ "term" : {"type" : "elasticsearch" } }'

    '?kibana':

      '*':

        - indices:admin/exists*

        - indices:admin/mapping/put*

        - indices:admin/mappings/fields/get*

        - indices:admin/refresh*

        - indices:admin/validate/query*

        - indices:data/read/get*

        - indices:data/read/mget*

        - indices:data/read/search*

        - indices:data/write/delete*

        - indices:data/write/index*

        - indices:data/write/update*

sg_readonly_and_monitor:

  cluster:

    - CLUSTER_MONITOR

  indices:

    '*':

      '*':

        - ALL

sg_kibana4:

  cluster: 

      - cluster:monitor/nodes/info

      - cluster:monitor/health

  indices:

    '*':

      '*':

        - READ

        - indices:admin/mappings/fields/get*

        - indices:admin/validate/query*

        - indices:admin/get*

    '?kibana':

      '*':

        - indices:admin/exists*

        - indices:admin/mapping/put*

        - indices:admin/mappings/fields/get*

        - indices:admin/refresh*

        - indices:admin/validate/query*

        - indices:data/read/get*

        - indices:data/read/mget*

        - indices:data/read/search*

        - indices:data/write/delete*

        - indices:data/write/index*

        - indices:data/write/update*

sg_kibana4_server:

  cluster:

      - cluster:monitor/nodes/info

      - cluster:monitor/health

  indices:

    '?kibana':

      '*':

        - ALL

sg_logstash:

  cluster:

    - indices:admin/template/get

    - indices:admin/template/put

  indices:

    '*':

      '*':

        - indices:data/write/bulk

        - indices:data/write/bulk\[s\]

        - indices:data/write/delete

        - indices:data/write/update

        - indices:data/read/search

        - indices:data/read/scroll

        - CREATE_INDEX

sg_curator:

  cluster:

    - MONITOR

  indices:

    '*':

      '*':

        - indices:admin/delete

        - indices:admin/settings/update

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/7F69F4AB-F39F-4669-A10A-7DFD02DEC248%40search-guard.com.

[SG]

I guess that because you limit ALL indices to type:elasticsearch (including the .kibana index)

Try something like this

sg_public:
indices:
'the_indexpattern_i_want_dls_be_applied':

'*':
- READ
- indices:admin/mappings/fields/get*
- indices:admin/validate/query*
- indices:admin/get*
_dls_: '{ "term" : {"type" : "elasticsearch" } }'
'?kibana':
'*':
- indices:admin/exists*
- indices:admin/mapping/put*
- indices:admin/mappings/fields/get*
- indices:admin/refresh*
- indices:admin/validate/query*
- indices:data/read/get*
- indices:data/read/mget*
- indices:data/read/search*
- indices:data/write/delete*
- indices:data/write/index*
- indices:data/write/update*

Maybe it would be a good idea to exclude .kibana in general from dls/fls because that really makes no sense.
What do you think?

> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/CAArf371DXgFLtHfT-%2B5W0ZKTF_2dAyJHd3DS20tJ1yY2VutVNg%40mail.gmail.com.

[Daniel Kasen]

Ok, lets assume I only want users to be able to search on an index called logstash-2016.06.21 that have the type:elasticsearch in kibana. This does not work:

sg_public:

  indices:

    'logstash-2016.06.21':

      '*':

        - READ

        - indices:admin/mappings/fields/get*

        - indices:admin/validate/query*

        - indices:admin/get*

      _dls_: '{ "term" : {"type" : "elasticsearch" } }'

    '?kibana':

      '*':

        - indices:admin/exists*

        - indices:admin/mapping/put*

        - indices:admin/mappings/fields/get*

        - indices:admin/refresh*

        - indices:admin/validate/query*

        - indices:data/read/get*

        - indices:data/read/mget*

        - indices:data/read/search*

        - indices:data/write/delete*

        - indices:data/write/index*

        - indices:data/write/update*

Do you have a working example you would like to share? Because the previous one gives me errors like:

Error: [security_exception] no permissions for indices:data/read/field_stats at respond (http://corp-logs.redfintest.com/bundles/kibana.bundle.js?v=9889:64202:16) at checkRespForFailure (http://corp-logs.redfintest.com/bundles/kibana.bundle.js?v=9889:64165:8) at http://corp-logs.redfintest.com/bundles/kibana.bundle.js?v=9889:62783:8 at processQueue (http://corp-logs.redfintest.com/bundles/commons.bundle.js?v=9889:41836:29) at http://corp-logs.redfintest.com/bundles/commons.bundle.js?v=9889:41852:28 at Scope.$eval (http://corp-logs.redfintest.com/bundles/commons.bundle.js?v=9889:43080:29) at Scope.$digest (http://corp-logs.redfintest.com/bundles/commons.bundle.js?v=9889:42891:32) at Scope.$apply (http://corp-logs.redfintest.com/bundles/commons.bundle.js?v=9889:43188:25) at done (http://corp-logs.redfintest.com/bundles/commons.bundle.js?v=9889:37637:48) at completeRequest (http://corp-logs.redfintest.com/bundles/commons.bundle.js?v=9889:37835:8)

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/C87A2D51-AA57-4BCD-A9A5-DC1B7487D965%40search-guard.com.

[SG]

i ll provide a working solution in the next two days - we are currently working hard on the documentation for all this stuff.

> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/CAArf370Qn%2BOj%3Dd%3DofvPqdb8c0xKXZDdBCiCrLyTPAOO_%2BeePWQ%40mail.gmail.com.

[Daniel Kasen]

Well this seems to work, though I have concerns about this regex (applies the dls to all indices such as logstash-2016.06.21 and marvel-2016.06.21):

sg_public:

  indices:

    '*':

      '*':

        - READ

        - indices:admin/mappings/fields/get*

        - indices:admin/validate/query*

        - indices:admin/get*

    '*-*':

      '*':

        - READ

        - indices:admin/mappings/fields/get*

        - indices:admin/validate/query*

        - indices:admin/get*

      _dls_: '{ "term" : {"type" : "elasticsearch" } }'

    '?kibana':

      '*':

        - indices:admin/exists*

        - indices:admin/mapping/put*

        - indices:admin/mappings/fields/get*

        - indices:admin/refresh*

        - indices:admin/validate/query*

        - indices:data/read/get*

        - indices:data/read/mget*

        - indices:data/read/search*

        - indices:data/write/delete*

        - indices:data/write/index*

        - indices:data/write/update*

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/E496598A-0651-433E-8F0A-7B03F3EAF175%40search-guard.com.

[SG]

Does this work for you?

sg_public:
indices:
'logstash-2016*06*21':
'*':
- READ
- ....
- ...

or

sg_public:
indices:
'/logstash-2016?06?21/':
'*':
- READ
- ....
- ...


or

sg_public:
indices:
'/logstash-2016\\S*06\\S*21/':
'*':
- READ
- ....
- ...


Remember: You cannot use a . (dot) in a field name

> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/CAArf370jhy7aeE_3X-naZ_hsCocVdYD35Tmn613bseu-rf%3DV3w%40mail.gmail.com.