CentOS with duplicated packages all over system?

Garret Wilson asked:

I set up a CentOS 7.2 x64 server on DigitalOcean. Later I updated the kernel following the DigitalOcean directions to DigitalOcean GrubLoader v0.2 (20160714) CentOS.

This is one of the simplest server setups imaginable. This is about all I did to get started:

yum -y install epel-release
yum -y groupinstall "Development Tools"
yum -y install bind-utils curl-devel dos2unix expat-devel gettext-devel libtool lrzip openssl openssl-devel p7zip perl-CPAN perl-devel screen unzip whois zlib-devel
yum -y install httpd mod_ssl certbot-apache

I’m sure I did some other things, but that’s the gist of it. I used sudo yum update from time to time to keep things up-to-date.

I set up some static sites on Apache. I started and enabled certbot-renew.timer so that my Let’s Encrypt certificates would always be updated.

I knew something was up when I recently received an email from Let’s Encrypt saying that my certificates were about to expire. If I just issue the command certbot by itself, I get:

An unexpected error occurred:
VersionConflict: (certbot 0.19.0 (/usr/lib/python2.7/site-packages), Requirement.parse('certbot==0.14.1'))
Please see the logfile '/tmp/tmp3fGvUd' for more details.

Oh, but that’s just the start. I tried to do sudo yum update and got Multilib version problems found like here. So I followed the directions and removed and reinstalled grub2-tools and related packages. Then I tried again. Problems with NetworkManager. Then I got problems with glibc.

I kept getting all sorts of duplicate package problems, so I found other directions that said to do sudo package-cleanup --cleandupes. It found loads of duplicates, but said:

Error: Trying to remove "systemd", which is protected
Error: Trying to remove "yum", which is protected
 You could try using --skip-broken to work around the problem

Other sites are telling me that in no way shape or form should I use --skip-broken, so I didn’t.

Now when I go back to trying to do a simple sudo yum update it finds 2 packages to install, 98 to upgrade, and 3 to remove. After happily downloading 100 packages, it starts trying to update things. But a lot of these failed with messages like this:

error: Couldn't fork %post(cronie-anacron-1.4.11-17.el7.x86_64): Cannot allocate memory

But htop shows I’m only using 121M/489M!

I also see things like this:

httpd-2.4.6-45.el7.centos.4.x86_64 was supposed to be removed but is not!

At this point I figure maybe something updated, so I did a sudo systemctl reboot.

After the reboot, now I can run certbot! I did a sudo yum check-update and a sudo yum update, and the things that said “was supposed to be removed but is not” seem to have been installed. Although I did get some error

Re-declaration of type docker_t
Failed to create node
Bad type declaration at /etc/selinux/targeted/tmp/modules/100/docker/cil:1
/usr/sbin/semodule:  Failed!

I rebooted again and tried package-cleanup --dupes to see if I still had duplicate package. I have—and loads of them!

openssh-clients-7.4p1-13.el7_4.x86_64
openssh-clients-6.6.1p1-35.el7_3.x86_64
nss-tools-3.28.4-1.2.el7_3.x86_64
nss-tools-3.28.4-15.el7_4.x86_64
perl-5.16.3-291.el7.x86_64
perl-5.16.3-292.el7.x86_64
iputils-20160308-8.el7.x86_64
iputils-20160308-10.el7.x86_64
libX11-1.6.5-1.el7.x86_64
libX11-1.6.3-3.el7.x86_64
systemtap-devel-3.0-7.el7.x86_64
systemtap-devel-3.1-4.el7_4.x86_64
…

So I tried sudo package-cleanup --cleandupes again. It said it would remove 199 packages, and… this time it completed! Now I don’t have duplicates! Now no updates are available, so maybe they are all installed! My sites are available! (I will wait to see if certbot updates the certificates automatically.)

Now what I want to know is how did my system get in a state with all those duplicated packages? What did I do wrong on such a simple system configuration? And how do I prevent it from happening again!?

My answer:


You end up with duplicated packages when yum crashes during an installation.

Since you apparently have a 512MB droplet, you should be aware that yum frequently needs more memory than that to complete an upgrade of any significant number of packages.

It’s common on 512MB virtual machines for yum to run out of memory during installation, and the solution is to set up swap space.


View the full question and any other answers on Server Fault.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.