Why syslog-ng on the forward node lose a lot of packets?

[gary tan]

Dear，

 

  The processing speed of logstash on the master seems to be very slow, about 8k/s. 

  Already tried to adjust pipeline.batch.size and pipeline.workers, but the output on logstash still grows slowly and does not improve.

  I'm not sure if it's the logstash problem or the syslog-ng problem

  How to optimize performance with syslog-ng or logstash？

 

cpu：24core

mem：256g

 pipeline.batch.size = 3000

 pipeline.workers = 24

root@forward:~# syslog-ng-ctl stats

SourceName;SourceId;SourceInstance;State;Type;Number

destination;d_syslog;;a;processed;1580

source;s_bro_snmp;;a;processed;0

source;s_network;;a;processed;0

destination;d_console_all;;a;processed;0

dst.tcp;d_logstash#0;tcp,127.0.0.1:6050;a;dropped;1026215253

dst.tcp;d_logstash#0;tcp,127.0.0.1:6050;a;processed;1110610094

dst.tcp;d_logstash#0;tcp,127.0.0.1:6050;a;stored;10000

root@master:~$ curl -XGET 'localhost:9600/_node/stats/events?pretty'

{

  "host" : "28b68e64638b",

  "version" : "6.5.4",

  "http_address" : "0.0.0.0:9600",

  "id" : "8c30511e-3f88-4968-8a82-e408f797d450",

  "name" : "28b68e64638b",

  "events" : {

    "in" : 439097612,

    "filtered" : 342097336,

    "out" : 247097300,

    "duration_in_millis" : 29279274,

    "queue_push_duration_in_millis" : 872520

  }

[Wes]

You may want to start with the following (which will push more records to Redis at a faster rate):

https://securityonion.readthedocs.io/en/latest/redis.html#tuning

Thanks,

Wes