I am using Logstash to read and parse logs from a file and send them to a Rest based API. My shipper is working fine, but I am experiencing a strange behavior.
When Logstash shipper parses the first log entry, it does not send it, It keeps it in the pipeline. When it parses the second log entry, it sends the first log entry to the API. Hence one message always remains in the pipeline and it is not being sent towards my API.
Whenever I stop my Logstash shipper process, then it sends the last remaining message as well. So, In a sense no message is lost, but shipper always is one message behind.
Question: Why is Logstash unable to flush out its pipeline and send message to the API as soon as it receives.