How to achieve fault tolerance with logstash forwarder?

Question!

I am using Logstash Forwarder to process tomcat logs.

My logstash forwarder config file contains :

 "files": [
{
  "paths": [ "/usr/share/tomcat/impression_logs/impressions.log" ]
}]

I am using log rotation so it will archive log file to different folder after some time or when file size exceeds 1MB.

If logstash-forwarder is down for few minutes logs are getting archived and it does not process those logs.

I am using monit to monitor logstash forwarder, but still there is chance of losing logs.

Is there a way to achieve fault tolerance with logstash forwarder?



Answers

I am using logstash-forwarder to send all logs from 4-5 tomcats per server and it's working fine and no logs are lost. May be you just need to set "dead time" not to 24 hours, for me it is 10 minutes. Cause if forwarder will fall and then will be started afer more than a couple of hours you can get high loading when it starts to send all logs for 24 hours.

By : Eddy


This video can help you solving your question :)
By: admin