tcp error: write: broken pipe

[Luanjuan Mao]

hello,

I have 2 tidb servers,  and i found many errors in tidb.log on tidb1, but no errors on tidb0, what is this mean?

terror.go:342: [error] write tcp ****:4000->****:45425: write: broken pipe,

 /home/jenkins/workspace/build_tidb_2.0/go/src/github.com/pingcap/tidb/server/conn.go:690: 
/home/jenkins/workspace/build_tidb_2.0/go/src/github.com/pingcap/tidb/server/conn.go:127:"

version: 2.0

centos7.4,ssd 800G,16 cpu,64G memory

[tl]

do you connect to tidb1 and then close the connection?

在 2018年6月1日星期五 UTC+8上午10:59:51，Luanjuan Mao写道：

[Luanjuan Mao]

i use nginx for tcp balance, can't specify tidb when client connect, so it indeed confused me why there is no error on tidb0

[Liu Tang]

what is your nginx configuration, which method for the load-balance, round-robin, or others?

Do you set a read/write timeout? can you show us the full TiDB 0 and 1 logs?

[Luanjuan Mao]

global

        user haproxy

        group haproxy

        daemon

        maxconn 10240

defaults

        log     global

        mode    tcp

        log 127.0.0.1 local2

        balance leastconn

        retries 3

        timeout connect         10s

        timeout client          1m

        timeout server          1m

listen  admin_stats

        bind 0.0.0.0:8888

        mode        http

        stats uri   /dbs

        stats realm     Global\ statistics

        stats auth  admin:admin

listen tidb_server

        bind 0.0.0.0:4001

        mode tcp

        option mysql-check user haproxy-check

        server tidb0 172.31.251.75:4000

        server tidb1 172.31.251.78:4000

now i change to haproxy, but the errors still exist , the above is configuration

please check the attachments

[Luanjuan Mao]

i remove the tidb which has errors from backend servers, still report errors

so maybe the errors not because of client connect

[shenli]

I guess the error is caused by the keep-alive message of haproxy. You can upgrade your cluster to the latest master. We think those logs are useless, so we removed those logs.

[Luanjuan Mao]

after i  removed tidb1 from haproxy backend, it still report this errors, and i found these errors when i use nginx first

the nginx configuration:

#user  nobody;

worker_processes  4;

error_log  /usr/local/system/logs/error.log;

#error_log  logs/error.log  notice;

#error_log  logs/error.log  info;

#pid        logs/nginx.pid;

events {

    worker_connections  10240;

}

stream {

    log_format basic '$remote_addr [$time_local] '

                     '$protocol $status $bytes_sent $bytes_received '

                     '$session_time';

    access_log  /usr/local/system/logs/access.log basic;

    upstream tidb_backend {

        hash $remote_addr consistent;

        server 172.31.251.78:4000;

        server 172.31.251.75:4000;

    }

    server {

        listen 4001 so_keepalive=on;

        proxy_pass tidb_backend;

    }

}

--
You received this message because you are subscribed to the Google Groups "TiDB user group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tidb-user+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/tidb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/tidb-user/9bfd91da-2dc1-443c-8645-40c84c1afa2b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Luanjuan Mao]

how can i fix the problem expect upgrade tidb? or  just ignore?

[Morgan Tocker (PingCAP)]

On Friday, June 8, 2018 at 10:59:08 AM UTC+8, Luanjuan Mao wrote:

how can i fix the problem expect upgrade tidb? or  just ignore?

I remember seeing this bug fix recently:

https://github.com/pingcap/tidb/pull/8812 

HAProxy uses an older server handshake, and I wonder if your error is related.  Do you mind testing 3.0-beta and verifying if it fixes the problem?

- Morgan