full /etc/ recovery from git

stvn asked:

I have a CentOS 6.9 server.

The full /etc directory was under version control.

I accidentally ran ‘git rm -R’ on it. The server stopped working right away, my ssh session got lost.

I managed to have an ip console attached to the server, and booted from usb with a live linux, I ran 'git reset --hard'.

Now index/working dir is clean. I rebooted the system, but it still doesn’t work. it responds to ping but i can’t ssh to it still.

On the IP console, I can’t even log in, after keying any user and it asks for user again.

From the recovery linux, I checked the errors in /var/log/boot.log — first error is regarding iptables-restore

error while loading shared libraries libip4tc.so.0 : cannot open shared object file: No such file or directory

Next is with rsyslog.d

CONFIG ERROR: could not interpret master config file '/etc/rsyslog.conf'

The file is clearly there if I check from the recovery live linux. I suspect some permissioning issue but no idea really.

Any help appreciated. Thanks.

My answer:

The ownership and permissions are probably wrong. Since you weren’t tracking these, you’ll have to fix them manually, for every single file.

By itself, git doesn’t track ownership and permissions of files. That’s not what it’s for. This metadata was not tracked at all and was therefore lost.

This is what systems like etckeeper are for. They wrap around git to keep /etc in version control, while also keeping the metadata that git doesn’t, and restoring it if necessary. To prevent this happening in future, consider using etckeeper or a similar tool.

You may also want to institute some sort of backups. You currently don’t appear to have any sort of backups, and this is a recipe for disaster. You’ve been lucky to escape with as little data loss as you have. You may not be so lucky next time.

View the full question and any other answers on Server Fault.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.