How Do I Warm Up an IP Address?
After years on the waiting list, May First was just given a /24 block of IP addresses. Excellent.
Now we want to start using them for, among other things, sending email.
I haven’t added a new IP address to our mail relays in a while and things seems to change regularly in the world of email so I’m curious: what’s the best 2024 way to warm up IP addresses, particularly using postfix?
Sendergrid has a nice page on the topic. It establishes the number of messages to send per day. But I’m not entirely sure how to fit messages per day into our setup.
We use round robin DNS to direct email to one of several dozen email relay servers using postfix. And unfortunately our DNS software (knot) doesn’t have a way to add weights to ensure some IPs show up more often than others (much less limit the specific number of messages a given relay should get).
Postfix has some nice knobs for rate limiting, particularly:
default_destination_recipient_limit
and
default_destination_rate_delay
If default_destination_recipient_limit is over 1, then
default_destination_rate_delay is equal to the minimum delay between sending
email to the same domain.
So, I’m staring our IP addresses out at 30m - which prevents any single domain from receiving more than 2 messages per hour. Sadly, there are a lot of different domain names that deliver to the same set of popular corporate MX servers, so I am not sure I can accurately control how many messages a given provider sees coming from a given IP address. But it’s a start.
A bigger problem is that messages that exceed the limit hang out in the
active queue until they can be sent without violating the rate limit. Since I
can’t fully control the number of messages a given queue receives (due to my
inability to control the DNS round robin weights), a lot of messages are going
to be severely delayed, especially ones with an @gmail.com domain name.
I know I can temporarily set relayhost to a different queue and flush
deferred messages, however, as far as I can tell, it doesn’t work with
active messages.
To help mitigate the problem I’m only using our bulk mail queue to warm up IPs, but really, this is not ideal.
Suggestions welcome!
Update #1
If you are running postfix in a multi-instance setup and you have instances that are already warmed up, you can move active messages between queues with these steps:
# Put the message on hold in the warming up instance
postsuper -c /etc/postfix-warmingup -h $queueid
# Copy to a warmed up instance
cp --preserve=mode,ownership,timestamp /var/spool/postfix-warmingup/hold/$queueid /var/spool/postfix-warmedup/incoming/
# Queue the message
postqueue -c /etc/postfix-warmedup -i $queueid
# Delete from the original queue.
postsuper -c /etc/postfix-warmingup -d $queueid
After just 12 hours we had thousands of messages piling up. This warm up method was never going to work without the ability to move them to a faster queue.
[Additional update: be sure to reload the postfix instance after flushing the queue so messages are drained from the active queue on the correct schedule. See update #4.]
Update #2
After 24 hours, most email is being accepted as far as I can tell. I am still getting a small percentage of email deferred by Yahoo with:
421 4.7.0 [TSS04] Messages from 204.19.241.9 temporarily deferred due to unexpected volume or user complaints - 4.16.55.1; see https://postmaster.yahooinc.com/error-codes (in reply
So I will keep it as 30m for another 24 hours or so and then move to 15m. Now that I can flush the backlog of active messages I am in less of a hurry.
Update #3
Well, this doesn’t seem to be working the way I want it to.
When a message arrives faster than the designated rate limit, it remains in the active queue.
I’m entirely sure how the timing is supposed to work, but at this point I’m
down to a 5m rate delay, and the active messages are just hanging out for a lot
longer than 5m. I tried flushing the queue, but that only seems to affect the
deferred messages. I finally got them re-tried with systemctl reload. I
wonder if there is a setting to control this retry? Or better yet, why can’t
these messages that exceed the rate delayed be deferred instead?
Update #4
I think I see why I was confused in Update #3 about the timing. I suspect that when I move messages out of the active queue it screws up the timer. Reloading the instance resets the timer. Every time you muck with active messages, you should reload.