Securing Es with elasticsearch-jetty

537 views
Skip to first unread message

Николай Колев

unread,
Aug 7, 2013, 4:58:42 AM8/7/13
to elasticse...@googlegroups.com
Hi all,

I am evaluating ES for our needs to provide internally for all departments. So far things are OK but we need security and some sort of access control that only certain amount of people can access specific index and can execute some commands. Second important thing is that there must be a way to display some information in form of dashboard using Kibana 3.
To do this I am trying to evaluate elasticsearch-jetty.  I need to provide an option that ES accept connection with authentication on public interface and without authentication on localhost

I have added to elasticsearch.yml this lines:
http.type: com.sonian.elasticsearch.http.jetty.JettyHttpServerTransportModule
sonian.elasticsearch.http.jetty:
    config: jetty.xml,jetty-hash-auth.xml,jetty-restrict-writes.xml,jetty-ssl.xml,jetty-local-server.xml
    ssl_port: 9443
    keystore_password: "OBF:1nc01vuz1w8f1w1c1rbu1rac1w261w9b1vub1ndq"
#    jetty.bind_host: "192.168.104.134", "127.0.0.1"

And this config files.

jetty.xml:
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure.dtd">

<Configure id="ESServer" class="org.eclipse.jetty.server.Server">

    <!-- ==================================================== -->
    <!-- ElasticSearch Handler.                               -->
    <!-- This handler redirects all requests to ElasticSearch -->
    <!-- ==================================================== -->
    <Set name="handler">
        <New class="com.sonian.elasticsearch.http.jetty.handler.JettyHttpServerTransportHandler"
             id="HttpServerAdapterHandler">
            <Set name="transport"><Ref id="ESServerTransport"/></Set>
        </New>
    </Set>

    <!-- ==================================================== -->
    <!--   Reduce verbosity of jetty default error handler    -->
    <!-- ==================================================== -->
    <Call name="addBean">
        <Arg>
            <New class="com.sonian.elasticsearch.http.jetty.error.JettyHttpServerErrorHandler"
                 id="HttpServerErrorHandler">
            </New>
        </Arg>
    </Call>

    <!-- ======================================== -->
    <!--         Add HTTP connector               -->
    <!-- ======================================== -->
    <Call name="addConnector">
        <Arg>
            <New class="org.eclipse.jetty.server.nio.SelectChannelConnector">
                <Set name="host">192.168.104.134<Property name="jetty.bind_host"/></Set>
                <Set name="port">9200<Property name="jetty.port"/></Set>
                <Set name="maxIdleTime">600000</Set>
                <Set name="Acceptors">2</Set>
            </New>
        </Arg>
    </Call>
</Configure>

jetty-local-server.xml:
<Configure id="ESLocalServer" class="org.eclipse.jetty.server.Server">

    <!-- ==================================================== -->
    <!-- ElasticSearch Handler.                               -->
    <!-- This handler redirects all requests to ElasticSearch -->
    <!-- ==================================================== -->
    <Set name="handler">
        <New class="com.sonian.elasticsearch.http.jetty.handler.JettyHttpServerTransportHandler"
             id="HttpServerAdapterHandler">
            <Set name="transport"><Ref id="ESServerTransport"/></Set>
        </New>
    </Set>

    <!-- ==================================================== -->
    <!--   Reduce verbosity of jetty default error handler    -->
    <!-- ==================================================== -->
    <Call name="addBean">
        <Arg>
            <New class="com.sonian.elasticsearch.http.jetty.error.JettyHttpServerErrorHandler"
                 id="HttpServerErrorHandler">
            </New>
        </Arg>
    </Call>

    <!-- ======================================== -->
    <!--         Add HTTP connector               -->
    <!-- ======================================== -->
    <Call name="addConnector">
        <Arg>
            <New class="org.eclipse.jetty.server.nio.SelectChannelConnector">
                <Set name="host">127.0.0.1</Set>
                <Set name="port">9201<Property name="jetty.port"/></Set>
                <Set name="maxIdleTime">600000</Set>
                <Set name="Acceptors">2</Set>
            </New>
        </Arg>
    </Call>

</Configure>

But I get this error:
[2013-08-06 18:33:09,292][INFO ][org.eclipse.jetty.server.Server] [ES Test Node] jetty-8.1.4.v20120524
[2013-08-06 18:33:09,503][WARN ][org.eclipse.jetty.util.component.AbstractLifeCycle] [ES Test Node] FAILED SelectChann...@192.168.104.134null:92009200: java.lang.IllegalArgumentException: port out of range:92009200
java.lang.IllegalArgumentException: port out of range:92009200
        at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
        at java.net.InetSocketAddress.<init>(InetSocketAddress.java:224)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:181)
        at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:311)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:260)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
        at org.eclipse.jetty.server.Server.doStart(Server.java:273)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
        at com.sonian.elasticsearch.http.jetty.JettyHttpServerTransport$1.onPortNumber(JettyHttpServerTransport.java:157)
        at org.elasticsearch.common.transport.PortsRange.iterate(PortsRange.java:68)
        at com.sonian.elasticsearch.http.jetty.JettyHttpServerTransport.doStart(JettyHttpServerTransport.java:100)
        at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)
        at org.elasticsearch.http.HttpServer.doStart(HttpServer.java:89)
        at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)
        at org.elasticsearch.node.internal.InternalNode.start(InternalNode.java:213)
        at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:120)
        at org.elasticsearch.bootstrap.Bootstrap.main(Bootstrap.java:204)
        at org.elasticsearch.bootstrap.ElasticSearch.main(ElasticSearch.java:32)
[2013-08-06 18:33:09,757][INFO ][org.eclipse.jetty.util.ssl.SslContextFactory] [ES Test Node] Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[2013-08-06 18:33:09,791][INFO ][org.eclipse.jetty.server.AbstractConnector] [ES Test Node] Started SslSelectCha...@0.0.0.0:9443
[2013-08-06 18:33:09,792][WARN ][org.eclipse.jetty.util.component.AbstractLifeCycle] [ES Test Node] FAILED org.eclipse.jetty.server.Server@28ee31cd: java.lang.IllegalArgumentException: port out of range:92009200
java.lang.IllegalArgumentException: port out of range:92009200
        at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
        at java.net.InetSocketAddress.<init>(InetSocketAddress.java:224)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:181)
        at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:311)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:260)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
        at org.eclipse.jetty.server.Server.doStart(Server.java:273)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
        at com.sonian.elasticsearch.http.jetty.JettyHttpServerTransport$1.onPortNumber(JettyHttpServerTransport.java:157)
        at org.elasticsearch.common.transport.PortsRange.iterate(PortsRange.java:68)
        at com.sonian.elasticsearch.http.jetty.JettyHttpServerTransport.doStart(JettyHttpServerTransport.java:100)
        at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)
        at org.elasticsearch.http.HttpServer.doStart(HttpServer.java:89)
        at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)
        at org.elasticsearch.node.internal.InternalNode.start(InternalNode.java:213)
        at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:120)
        at org.elasticsearch.bootstrap.Bootstrap.main(Bootstrap.java:204)
        at org.elasticsearch.bootstrap.ElasticSearch.main(ElasticSearch.java:32)
[2013-08-06 18:33:09,793][ERROR][com.sonian.elasticsearch.http.jetty.JettyHttpServerTransport] [ES Test Node] Jetty Startup Failed
java.lang.IllegalArgumentException: port out of range:92009200
        at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
        at java.net.InetSocketAddress.<init>(InetSocketAddress.java:224)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:181)
        at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:311)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:260)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
        at org.eclipse.jetty.server.Server.doStart(Server.java:273)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
        at com.sonian.elasticsearch.http.jetty.JettyHttpServerTransport$1.onPortNumber(JettyHttpServerTransport.java:157)
        at org.elasticsearch.common.transport.PortsRange.iterate(PortsRange.java:68)
        at com.sonian.elasticsearch.http.jetty.JettyHttpServerTransport.doStart(JettyHttpServerTransport.java:100)
        at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)
        at org.elasticsearch.http.HttpServer.doStart(HttpServer.java:89)
        at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)
        at org.elasticsearch.node.internal.InternalNode.start(InternalNode.java:213)
        at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:120)
        at org.elasticsearch.bootstrap.Bootstrap.main(Bootstrap.java:204)
        at org.elasticsearch.bootstrap.ElasticSearch.main(ElasticSearch.java:32)

In addition to this there are some problems with Kibana (v.3 is javascript based) and CORS.

My question is
1)    Is it possible with elasticsearch-jetty to achieve what I need to do or I have to look for other options?
2)    What is wrong with the config files that prevents ES to start?

Thanks in advance.

Best regards,
Nickolay Kolev



Pulkit Singhal

unread,
Aug 7, 2013, 8:09:54 AM8/7/13
to elasticse...@googlegroups.com, elasticse...@googlegroups.com
<Set name="port">9200<Property name="jetty.port"/></Set>

That configuration is resulting in a port 92009200 ... Which is way over 65k limit for port #s

Remove the hard coded 9200 from that line

Sent from my iPad

Николай Колев

unread,
Aug 8, 2013, 7:43:31 AM8/8/13
to elasticse...@googlegroups.com

Hi,

 

I manage to make it run. The documentation is misleading and unclear. Even the install procedure does not work with ES 0.9.x.

In my opinion it will be nice to make it work with Kibana 3 – I manage to do this with nginx in front of ES.

Unfortunately last time I have wrote in java was 15 years ago and  I am not able to make the changes and to debug the code properly.

Thanks all for the patience.

b.r.

Nickolay kolev

Sergio Ariel de la Campa Saiz

unread,
Sep 2, 2014, 7:06:23 AM9/2/14
to elasticse...@googlegroups.com
Hello... did you finally enable authentication in Elasticsearch-Jetty and at the same time use Kibana? I´m trying to use jetty over elasticsearch but when I enable authentication, Kibana can´t connect to elasticsearch.
 
Regards
[2013-08-06 18:33:09,503][WARN ][org.eclipse.jetty.util.component.AbstractLifeCycle] [ES Test Node] FAILED SelectChannelConnector@192.168.104.134null:92009200: java.lang.IllegalArgumentException: port out of range:92009200

java.lang.IllegalArgumentException: port out of range:92009200
        at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
        at java.net.InetSocketAddress.<init>(InetSocketAddress.java:224)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:181)
        at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:311)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:260)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
        at org.eclipse.jetty.server.Server.doStart(Server.java:273)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
        at com.sonian.elasticsearch.http.jetty.JettyHttpServerTransport$1.onPortNumber(JettyHttpServerTransport.java:157)
        at org.elasticsearch.common.transport.PortsRange.iterate(PortsRange.java:68)
        at com.sonian.elasticsearch.http.jetty.JettyHttpServerTransport.doStart(JettyHttpServerTransport.java:100)
        at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)
        at org.elasticsearch.http.HttpServer.doStart(HttpServer.java:89)
        at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)
        at org.elasticsearch.node.internal.InternalNode.start(InternalNode.java:213)
        at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:120)
        at org.elasticsearch.bootstrap.Bootstrap.main(Bootstrap.java:204)
        at org.elasticsearch.bootstrap.ElasticSearch.main(ElasticSearch.java:32)
[2013-08-06 18:33:09,757][INFO ][org.eclipse.jetty.util.ssl.SslContextFactory] [ES Test Node] Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[2013-08-06 18:33:09,791][INFO ][org.eclipse.jetty.server.AbstractConnector] [ES Test Node] Started SslSelectChannelConnector@0.0.0.0:9443
Reply all
Reply to author
Forward
0 new messages