Can't delete corrupt files on Linux

quickshiftin asked:

So I got hit by a script kitte… Fortunately the box is Ubuntu and was able to replace w/ binaries from a comparable system, however,

Some of the files I couldn’t delete, and am still stumped on this. The hijacked files are sitting in the /_bin directory which is writeable by root.

[email protected]:~$ ls -ld !$
ls -ld /_bin
drwxr-xr-x 2 root root 4096 Mar 12 18:00 /_bin

Ok, those are the perms on the directory, now for the files within:

[email protected]:~$ ls -l /_bin
total 268
-rwxr-xr-x 1 root root  39696 Nov 19 22:25 ls
-rwxr-xr-x 1 root root 119800 Mar 31  2012 netstat
-rwxr-xr-x 1 root root 101240 Dec 12  2011 ps

Now when I try to delete one of these files (as root):

[email protected]:/home/nathan# rm /_bin/ls
rm: cannot remove `/_bin/ls': Operation not permitted

Or if I try to delete the entire _bin directory (again as root):

[email protected]:/home/nathan# rm -rf /_bin
rm: cannot remove `/_bin/ls': Operation not permitted
rm: cannot remove `/_bin/netstat': Operation not permitted
rm: cannot remove `/_bin/ps': Operation not permitted

So how can I delete these files?

Edit:

Sure enough the immutable bit has been set, however, removing it does not let me delete the files.

[email protected]:/home/nathan# lsattr /_bin
s---ia--------- /_bin/ls
s---ia--------- /_bin/netstat
s---ia--------- /_bin/ps

[email protected]:/home/nathan# chattr -R -i /_bin
[email protected]:/home/nathan# lsattr /_bin
s----a--------- /_bin/ls
s----a--------- /_bin/netstat
s----a--------- /_bin/ps

[email protected]:/home/nathan# rm -rf /_bin
rm: cannot remove `/_bin/ls': Operation not permitted
rm: cannot remove `/_bin/netstat': Operation not permitted
rm: cannot remove `/_bin/ps': Operation not permitted

Also verified /_bin doesn’t have immutable bit:

[email protected]:/home/nathan# lsattr -d /_bin
--------------- /_bin

My answer:


Most likely the attacker has set the immutable attribute on the files and directory. This is commonly done by rootkits to make cleanup more difficult.

To confirm this, try:

lsattr /_bin

To remove the immutable attribute, use:

chattr -R -i /_bin

You’ll also want to clear the a and s attributes, since these may affect your ability to remove the files.

chattr -R -i -a -s /_bin

See the chattr man page for a full explanation of what all the attributes are and what they do.


View the full question and any other answers on Server Fault.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.