Warning logs

[Roman Kournjaev]

When asking questions, please provide the following information:

* Search Guard and Elasticsearch version

* Used enterprise modules, if any

* JVM version and operating system version

* Search Guard configuration files

* Elasticsearch log messages on debug level

I have finally set up search-guard, though it was quite tricky todo , since we provision our ES instances with CHEF scripts.

ES 5.4.2

S-G-5 : search-guard-5:5.4.2-12

java version "1.8.0_131"

1. What is the proper way to setup search guard on a production instance without the use of 'install_demo_configuration.sh' , since it explicitly says not to use it on production env. What I would do is to run it and then run some scripts to update the updated elasticsearch.yml and alter all the sg_config files , not sure thats the way the develop intended it to be, but it works.

2. I have some strange log lines that you might help me explain:

[2017-08-01T18:13:42,984][INFO ][o.e.n.Node               ] [elasticsearch-app-dev-fpjs] started

[2017-08-01T18:13:43,332][ERROR][c.f.s.a.BackendRegistry  ] Not yet initialized (you may need to run sgadmin)

[2017-08-01T18:13:43,915][INFO ][o.e.l.LicenseService     ] [elasticsearch-app-dev-fpjs] license [0c10aaae-9db8-40ed-9ccf-2cf7421e8a3b] mode [trial] - valid

[2017-08-01T18:13:43,917][INFO ][o.e.g.GatewayService     ] [elasticsearch-app-dev-fpjs] recovered [7] indices into cluster_state

[2017-08-01T18:13:44,159][ERROR][c.f.s.a.BackendRegistry  ] Not yet initialized (you may need to run sgadmin)

[2017-08-01T18:13:44,520][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Node 'elasticsearch-app-dev-fpjs' initialized

[2017-08-01T18:13:44,956][INFO ][o.e.c.r.a.AllocationService] [elasticsearch-app-dev-fpjs] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[products_latest][3], [.monitoring-es-2-2017.08.01][0]] ...]).

[2017-08-01T18:13:45,430][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:13:45,430][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:13:45,433][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:13:48,942][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:04,555][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:04,555][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

Thanks,

Roman

[Roman Kournjaev]

Also wrapping my head around with the monitoring exporter:

[2017-08-01T18:35:48,341][ERROR][o.e.x.m.e.h.BackwardsCompatibilityAliasesResource] org.elasticsearch.xpack.monitoring.exporter.http.BackwardsCompatibilityAliasesResource$$Lambda$1724/1875992075@318ffaf2

org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/.marvel-es-1-*?filter_path=*.aliases: HTTP/1.1 403 Forbidden

{"error":{"root_cause":[{"type":"security_exception","reason":"no permissions for indices:admin/get"}],"type":"security_exception","reason":"no permissions for indices:admin/get"},"status":403}

at org.elasticsearch.client.RestClient$1.completed(RestClient.java:354) ~[?:?]

at org.elasticsearch.client.RestClient$1.completed(RestClient.java:343) ~[?:?]

at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119) ~[?:?]

at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177) ~[?:?]

at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436) ~[?:?]

at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326) ~[?:?]

at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) ~[?:?]

at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81) ~[?:?]

at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39) ~[?:?]

at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) ~[?:?]

at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) ~[?:?]

at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) ~[?:?]

at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) ~[?:?]

at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) ~[?:?]

at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) ~[?:?]

at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588) ~[?:?]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]

The following syntax seems not work

sg_monitor:

  cluster:

    - "cluster:admin/xpack/monitoring/*"

    - "indices:admin/template/get"

    - "indices:admin/template/put"

    - "indices:admin/*get"

    - "cluster:admin/ingest/pipeline/get"

    - CLUSTER_MONITOR

    - CLUSTER_COMPOSITE_OPS

  indices:

    '*monitoring*':

      '*':

        - INDICES_ALL

    '*marvel-es*':

      '*':

        - INDICES_ALL

What works for me is : 

sg_monitor:

  cluster:

    - "cluster:admin/xpack/monitoring/*"

    - "indices:admin/template/get"

    - "indices:admin/template/put"

    - "indices:admin/*get"

    - "cluster:admin/ingest/pipeline/get"

    - CLUSTER_MONITOR

    - CLUSTER_COMPOSITE_OPS

  indices:

    '*':

      '*':

        - INDICES_ALL

[SG]

In you case you can ignore these two log messages:

- Not yet initialized (you may need to run sgadmin)
- _all does not exist in cluster metadata

The proper way to setup SSL for production is to use your own PKI. If you do not have one you maybe want to establish one (depends if you maybe need SSL certificates in the future for other services as well).
If you do not have a PKI and want not setup a company wide one then you can you our scripts also for production, but you do it on your own risk. It depends a bit wether your elasticsearch cluster is exposed to the public or if you can install root certifcates into the browsers of your users.. If its public or if you cannot install root certificates into the browsers you can of course also buy a commercial SSL certificate from verisign, thawte, ... or go with letsencrypt for free.

To make xpack monitoring work pls update to SG 14. This should work then out of the box. See also https://github.com/floragunncom/search-guard/blob/ves-5.4.3-14/sgconfig/sg_roles.yml

> --
> You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> To post to this group, send email to search...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/f3aaa0d2-bb30-416c-bd04-f49b513aac76%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Roman Kournjaev]

Thanks,

Any way I can get rid of the '_all does not exist in cluster metadata' log line  ? It practically writes this line every second , and i would like to keep my log lines clean.

I guess setting a specifc logger only to 'ERROR' level would do the job.

[Jochen Kressin]

Adding this to conf/log4j2.properties should to the trick:

logger.pe.name = com.floragunn.searchguard.configuration.PrivilegesEvaluator

logger.pe.level = error