What is ideal directory for kafka messages

Divs asked:

The default value of logs.dir=/tmp/kafka-logsin server.properties. Usually /tmp is avoided from keeping any important files and we are storing messages and offsets!

Any particular reason why one may not choose /var/log/kafka-logs or /opt/kafka-logs

NOTE – Assuming /tmp, /var/log are all same file-system type.

My answer:


You’ll always find me placing files in standard directories or as close to them as possible.

The reason for this is so that future admins can find them later — because very often that future admin is me!

Consider logs, for instance, since that’s what you have brought up. I would create a subdirectory in /var/log to store these, such as /var/log/kafka. Not coincidentally, this is actually the default log directory if you haven’t overridden it. So it’s where most admins will go first to look for the logs.

If it turns out that you need to mount a disk partition to store logs, you don’t have to change the log directory; instead, you can mount the new disk space directly at /var/log/kafka.

And /opt is intended for large third party packages; it’s not where I expect to find most things.


View the full question and any other answers on Server Fault.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.