tail: inotify cannot be used, reverting to polling: Too many open files

user23577 asked:

tail: inotify cannot be used, reverting to polling: Too many open files

I’m running apache and tomcat servers on Ubuntu (AWS ec2).
Whenever I try to tail the catalina.out of tomcat I get too many open files.
However I am able to view it using vi.

After searching around on internet, I tried the following command:

lsof | awk '{ print $2; }' | sort -rn | uniq -c | sort -rn | head

With results below

 17 5650
 17 5178
 13 5972
 10 5976
 10 5974
  9 5977
  9 5975
  9 5973
  8 5978
  4 9

When I just ran lsof the process ids: 5650 were bash, 5178 was again bash and others were sshd,top and apache2.

Why is there bash, top, sshd opening files in huge number?
How can I close these files?
Will killing these processes do any good ?
Will the number decrease by itself or do I have to do anything?
Right now everything is working as expected except that tail -f gives me too many open files.

I use top and ssh to server a lot. But why don’t they release the files?
OR am I connecting the wrong dots.

My answer:

Probably you ran out of inotify watches. By default it’s an absurdly low 8192.

Check your current value by:

sysctl fs.inotify.max_user_watches

Then change it to something more reasonable by editing /etc/sysctl.conf or a file which it includes, and adding:

fs.inotify.max_user_watches = 524288

(or whatever value) and then running sysctl -p to have it take effect.

View the full question and any other answers on Server Fault.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.