Fixing Postfix After An Ubuntu Upgrade

As part of a set of security updates one of the servers in a cluster was upgraded from Ubuntu 16.04LTS to 20.04LTS. Everything seemed to be fine until we found that Postfix was no longer sending email via the Amazon AWS SES service. The past few hours have been spent fixing Postfix after an Ubuntu upgrade.

Ubuntu Changed How Postfix Works

Turns out this upgrade wasn’t completely innocuous when it comes to Postfix and the ability to send mail. It didn’t take long to uncover multiple errors in the server’s /var/log/mail.log file.

Name service error for name=email-smtp.us-east-1.amazonaws.com type=MX

Turns out this is a bit of red herring. Running through the postfix configuration did uncover some oddities that needed to be fixed, however.

A local lookup shows there are no MX records for that host available from the DNS servers.

host -t mx email-smtp.us-east-1.amazonaws.com

Re-running the initial AWS recommended Postfix configuration changes helped change this error, but not resolve it. It morphed to a new error.

Note: The AWS documentation is outdated and is not completely accurate for the Ubuntu 20.04 or newer releases. Instead you’ll need some tricks from the Ubuntu 20.04 Postfix configuration manual.

Instead of using the restart command listed on the AWS docs, use this command:

sudo systemctl restart postfix.service

Name service error for name=email-smtp.us-east-1.amazonaws.com type=AAAA

A lookup of the IPv6 host did not work from the server. Same command as above, from a standard privileged user.

host -t AAAA email-smtp.us-east-1.amazonaws.com

Testing the A record, a typical IPv4 record, worked from a standard user.

host -t A email-smtp.us-east-1.amazonaws.com

Telling Postfix To Lookup IPv4 Records

You’ll need to change the Postfix configuration file, /etc/postfix/main.cf, and set the internet protocol to IPv4. Change these lines:

#inet_protocols = all
inet_protocols = ipv4

Restart the service…

sudo systemctl restart postfix.service

Name service error for name=email-smtp.us-east-1.amazonaws.com type=A

This is a new error, but now a standard privileged user can find this host:

host -t A email-smtp.us-east-1.amazonaws.com

But becoming the Postfix user causes this to fail:

sudo su - postfix
host -t A email-smtp.us-east-1.amazonaws.com

Turns out the postfix user is running in a chroot jail. That means the standard system files are not what is in use when the Postfix app is running (it runs in the same environment as the postfix user — in a “jailed” or restricted set of OS files).

Fixing the chroot jail for Postfix

Turns out the Ubuntu update did NOT update the postfix chroot jail. No bueno.

The chroot jail is at /var/spool/postfix.

You’ll need to manually copy over the resolv.conf and services files from the main user as a privileged user:

sudo su -

cd /var/spool/postfix/etc
cp /etc/resolv.conf .
cp /etc/services .

sudo systemctl restart postfix.service

Closer, but not quite… we still have an authentication problem.

(host email-smtp.us-east-1.amazonaws.com[23.22.149.87] said: 530 Authentication required

After checking the /etc/postfix/sasl_passwd and the main.cf settings, everything seems intact. Yet any attempts to send messages via the server through AWS SES services fails.

Turns out that Ubuntu 22.04LTS is using Dovecot to manage the mail service authentication handshake.

Follow the Ubuntu 22.04 Postfix manual for configuring Dovecot. From the manual…

Next, edit /etc/dovecot/conf.d/10-master.conf and change the following:

service auth {
  # auth_socket_path points to this userdb socket by default. It's typically
  # used by dovecot-lda, doveadm, possibly imap process, etc. Its default
  # permissions make it readable only by root, but you may need to relax these
  # permissions. Users that have access to this socket are able to get a list
  # of all usernames and get results of everyone's userdb lookups.
  unix_listener auth-userdb {
    #mode = 0600
    #user = 
    #group = 
  }

  # Postfix smtp-auth
  unix_listener /var/spool/postfix/private/auth {
    mode = 0660
    user = postfix
    group = postfix
  }
 }

Restart the dovecot service:

sudo systemctl restart dovecot.service

Retry sending the email

During this process a couple of files helped make for some quick email testing.

The testemail.sh bash script:

#!/bin/bash
sendmail -f my@email.address another@email.address < testmail.txt

The testmail.txt

From: Sender Name <sender@example.com>
Subject: Amazon SES Test                
This message was sent using Amazon SES.                
.

Yay! Success. The server is now sending email again.

More testing is needed to ensure the setup is working after a reboot, but we are well on our way to sending email via Amazon SES services with Ubuntu 20.04LTS.

post image via Pixabay