I recently got one
Undelivered Mail Returned to Sender while sending my newsletter to one of my 1500 customers. My website uses a double-opt-in procedure to make sure, the user explicitly want to receive my newsletter.
The error message:
smtp; 554 ... Swisscom AG IP: 126.96.36.199, You are not allowed to send us mail. Please refer to xyz.com if you feel this is in error.
I got an example spam mail (from the mail provider of the receiving mailserver):
Received: from mail.com ([188.8.131.52]) by smtp-27.iol.local with SMTP id itOWeYZ6O42IFitOWe35TR; Tue, 13 Feb 2018 03:54:09 +0100 From: "Servizi online - Poste Italiane" <[email protected]> Subject: Abbiamo ricevuto una segnalazione di accredito Date: Mon, 12 Feb 2018 11:32:03 -0500
The provider also stated, that my server seems to be hacked. He further stated, that “the recipient mail server has simply recorded the rDNS presented to it by the connecting IP, in this case
mail.com ([184.108.40.206])” – which is definitely NOT as I configured my rDNS entry (mail.lotsearch.de) for my IP address.
So if I understood rDNS correctly, the receving mail server queries the sender IP for a rDNS entry (220.127.116.11 => should resolve to => mail.lotsearch.de, which it definitely does, when I test it from my local machine via
$ host 18.104.22.168).
How is it possible to spoof rDNS? I can not imagine any way how this can technically work (only with a man-in-the-middle attack somewhere in the infrastructure between the receiving mailserver and my server).
The provider mentioned also, that “it is likely that a machine connecting from my IP has been compromised and sending these messages via direct connections to the recipient mail sever (also known as direct MX)”. What does
direct MX mean? Someone stole or found leaked mail credentials to one of my mail accounts and used them for mail sending?
What I have done so far to make sure my server is NOT / will not be hacked:
- searched the mail logs (
var/log/mail*): nothing special in there
- checked the ssh login logs (
lastb): nothing unusual
- checked if postfix is does relaying: no it does not (checked via telnet)
- checked for malware via clamav: no results
- installed and configured fail2ban for ssh, postfix and dovecot
- installed the latest patches / updates for Ubuntu 16.04 (I do so every week)
- checked if my IP address is on any blacklist: it is not
- verified rDNS entry in the management console of my hosting provider: it is correctly set to
- changed passwords of all mail accounts
- changed public keys for shell access
More important: There was no information about
[email protected] in the logs. So if my server would have been misused by a spammer (f.e. because of leaked smtp credentials of one of the mail accounts) I would see that in the log files.
The last possibility I can think of is that an intruder placed malware at my server I didn’t find yet.
How can I monitor outgoing mail traffic (per process and per port)?
Only monitoring outgoing port 25 would not help as this would only trap irregular mails sent via postfix, but not the mail traffic caused by a potential malware infection (if the malware uses another port than 25 for directly sending mails / communicating with recipient mail servers). If I monitor outgoing traffic on all ports I will get a way to huge log file which I cannot search through for suspicious activity efficiently.
EDIT – Added test for open relay:
$ telnet mail.lotsearch.de 25 $ HELO [email protected] 250 mail.lotsearch.de $ MAIL FROM: [email protected] 250 2.1.0 Ok $ RCPT TO:<[email protected]> 454 4.7.1 <[email protected]>: Relay access denied
EDIT – Running webapps
Before I get to my suggestion, I want to comment a bit on something your provider said to you:
Received: from mail.com ([22.214.171.124]) by smtp-27.iol.local with SMTP id itOWeYZ6O42IFitOWe35TR; Tue, 13 Feb 2018 03:54:09 +0100
This does not indicate that the reverse DNS for 126.96.36.199 is (or was) mail.com. Rather, it indicates that the SMTP client sent
mail.com in its
EHLO) line. (A well-configured mail server would have rejected this connection entirely, but that’s on Swisscom, not you…) This line does not indicate any reverse DNS entry. If it did, it would have appeared within the parentheses. For example:
Received: from mail-io0-f197.google.com (mail-io0-f197.google.com [188.8.131.52])
In this case, the first hostname is what the mail server identified itself as in its
EHLO. The second hostname is the reverse DNS recorded at the time the connection was made.
RFC 5321 section 4.4 explains the format of the Received: line, with a formal grammar.
In your case, no reverse DNS was recorded. Since your IP address has a PTR record, this may be because they didn’t look it up, or there was a temporary DNS failure.
Now, it appears you run a web hosting service and have numerous web apps. If one of these is compromised, it may begin sending spam. These often make direct connections to remote mail servers by looking up their MX records and connecting to port 25, as if they were actually a mail server themselves, rather than delivering mail to the local mail spool or an authenticated mail service on ports 587 or 465 as legitimate web apps do.
One way I stop this is by implementing a firewall rule that prevents outgoing connections on port 25 unless the user is the mail server user. For example:
iptables -I OUTPUT -m owner ! --uid-owner postfix -m tcp -p tcp --dport 25 -j REJECT
Web apps can no longer deliver mail directly to remote SMTP servers, but must use the local mail spool or an authenticated mail service.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.