Netty exception when used with HaProxy

[elr1c...@gmail.com]

I get this log continuously when I start a SockJS server on my local machine with HaProxy in front :

14:19:50.115 [vert.x-acceptor-thread-0] WARN  io.netty.bootstrap.ServerBootstrap - Failed to set channel option 'SO_KEEPALIVE' with value 'false' for channel '[id: 0x2e26a870, L:/0:0:0:0:0:0:7f00:1:8082 - R:/0.0.0.160:0]'

io.netty.channel.ChannelException: java.net.SocketException: Invalid argument

at io.netty.channel.socket.DefaultSocketChannelConfig.setKeepAlive(DefaultSocketChannelConfig.java:196)

at io.netty.channel.socket.DefaultSocketChannelConfig.setOption(DefaultSocketChannelConfig.java:112)

at io.netty.bootstrap.AbstractBootstrap.setChannelOption(AbstractBootstrap.java:457)

at io.netty.bootstrap.AbstractBootstrap.setChannelOptions(AbstractBootstrap.java:449)

at io.netty.bootstrap.ServerBootstrap$ServerBootstrapAcceptor.channelRead(ServerBootstrap.java:248)

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)

at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)

at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)

at io.netty.channel.nio.AbstractNioMessageChannel$NioMessageUnsafe.read(AbstractNioMessageChannel.java:93)

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)

at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:451)

at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)

at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)

at java.lang.Thread.run(Thread.java:748)

Caused by: java.net.SocketException: Invalid argument

at sun.nio.ch.Net.setIntOption0(Native Method)

at sun.nio.ch.Net.setSocketOption(Net.java:334)

at sun.nio.ch.SocketChannelImpl.setOption(SocketChannelImpl.java:190)

at sun.nio.ch.SocketAdaptor.setBooleanOption(SocketAdaptor.java:271)

at sun.nio.ch.SocketAdaptor.setKeepAlive(SocketAdaptor.java:370)

at io.netty.channel.socket.DefaultSocketChannelConfig.setKeepAlive(DefaultSocketChannelConfig.java:194)

... 19 common frames omitted

Is there something special to do to avoid this warning or should I simply put my logback netty logger level to 'error' ?

Thx.

[Julien Viet]

I don't think this is related to HA proxy.

can you double check ?

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/575f8a87-e1ad-4385-94b0-6ffc7f4902d1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[elr1c...@gmail.com]

Actually, when I start it standalone (without HaProxy), I don't see this log.

[elr1c...@gmail.com]

Other info : my sockJS server (vertx 3.5.2) is started in cluster mode, with Hazelcast.

[Thomas SEGISMONT]

There must be other differences in the deployment environment: is the application deployed on the same system? The same way? ...etc

This warning is logged before any connection has been made to the system. So HaProxy is very unlikely involved.

2018-07-19 14:44 GMT+02:00 <elr1c...@gmail.com>:

Actually, when I start it standalone (without HaProxy), I don't see this log.

--

You received this message because you are subscribed to the Google Groups "vert.x" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/vertx.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/c1c20818-289e-4112-bcd9-25caa9a8b924%40googlegroups.com.

[elr1c...@gmail.com]

Actually, I have tested also without cluster mode, and I get the same warnings (SO_KEEPALIVE or TCP_DELAY).

My test is to start an HaProxy with 2 instances of sockjs server on my local machine (Mac). As soon as I put a HaProxy in front of only one instance, I get warnings continuously. However, the applications seems to work fine : the load balancer detects my 2 instances and sends requests with success to both.

Just in case , here is my haproxy config :

# Ansible managed

##########################
#         GLOBAL         #
##########################
global
    daemon
    user       toto
    group      staff
    log        127.0.0.1    local0 debug

##########################
#        DEFAULT         #
##########################
defaults
  mode http
  log global
  option httplog
  option  http-server-close
  option  dontlognull
  option  redispatch
  option  contstats
  retries 3
  backlog 10000
  timeout client          25s
  timeout connect          5s
  timeout server          25s
  timeout tunnel        3600s
  timeout http-request    15s
  timeout queue           30s
  timeout tarpit          60s
  default-server inter 3s rise 2 fall 3
  option forwardfor

##########################
#         STATS          #
##########################
listen stats
    bind       *:9001
    mode       http
    maxconn    10
    stats      enable
    stats      refresh 20s
    stats      show-legends
    stats      show-node
    stats      hide-version
    stats      uri /stats
    stats      realm Haproxy\ Statistics
    stats      auth haproxy-stats:mypassword!!
    timeout    client 100s
    timeout    server 100s
    timeout    connect 100s
    timeout    queue 100s
    bind-process 1

frontend ft_web
  bind *:8080 name http
  maxconn 60000
  acl host_ws hdr_beg(Host) -i ws.
  use_backend bk_ws if host_ws
  acl hdr_connection_upgrade hdr(Connection)  -i upgrade
  acl hdr_upgrade_websocket  hdr(Upgrade)     -i websocket
  use_backend bk_ws if hdr_connection_upgrade hdr_upgrade_websocket
  default_backend bk_web

backend bk_web                                                   
  balance roundrobin                                             
  option httpchk HEAD /                                          
  server websrv1 localhost:8081 maxconn 100 weight 10 cookie websrv1 check
  server websrv2 localhost:8082 maxconn 100 weight 10 cookie websrv2 check

backend bk_ws                                                    
  balance roundrobin
  acl hdr_connection_upgrade hdr(Connection)                 -i upgrade
  acl hdr_upgrade_websocket  hdr(Upgrade)                    -i websocket
  acl hdr_websocket_version  hdr_cnt(Sec-WebSocket-Version)  eq 13
  http-request deny if ! hdr_connection_upgrade ! hdr_upgrade_websocket ! hdr_websocket_version
  server websrv1 localhost:8081 maxconn 30000 weight 10 cookie websrv1 check
  server websrv2 localhost:8082 maxconn 30000 weight 10 cookie websrv2 check

Thx

[elr1c...@gmail.com]

Actually, I have checked again, and when I remove the 'backend bk_ws' part, I don't get warnings anymore.

Do you have an idea why ?

Thx.

[Thomas SEGISMONT]

I'm not an haproxy expert but this page seems to indicate that bk_ws should be used when the web and websocket server are separated:

https://www.haproxy.com/fr/blog/websockets-load-balancing-with-haproxy/

Have you tried asking the haproxy community

--

You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/304c9fae-56e0-40f2-a279-3bd43fc627a3%40googlegroups.com.