I'm getting mighty ticked off with Yahoo! Mail lately. I have industry contacts, family, and friends that use yahoo.com email accounts and I've been getting this "defer" messages lately. Yahoo is using an anti-spam tactic where they "defer" the message from your SMTP server. It's also known as "greylisting". Yahoo mail denies the first delivery attempt of an email (421 Message temporarily deferred). Yahoo assumes that spammers don't try sending the same email twice, so they put the IP address of that email server in a list and then if the delivery is retried within a short amount of time, they assume that it is a good email and "accept" the SMTP connection for mail delivery. It's too resource intensive for spammers to retry emails, which is why spammers mass blast programs give up after one try.
The SMTP protocol, configured on valid email servers, was designed such that it will retry to send an email several times before failing. Unfortunately, Yahoo's greylisting has gotten super aggressive, often deferring tmcnet.com emails to yahoo.com email addresses so many times that eventually our email server gives up.
Even worse, often times 24 hours goes by before I get an email undeliverable message! I had an important email that I assumed was delivered and I didn't know until a day later that it bounced!
Here's an example:
This message could not be delivered. The TMC I.C.E. Box will not make any further attempts to deliver the message.
A record of the delivery attempts made follows:
We were unable to connect to the destination server(s):
On Thu Nov 13 at 14:51 EST, Message exceeded queue lifetime. Bouncing.
On Thu Nov 13 at 14:51 EST, Connected to 216.39.53.2 but greeting failed. \ Remote host said: 421 Message from (206.252.203.30) temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Thu Nov 13 at 11:38 EST, 68.142.202.247 failed after I sent the message. \ Remote host said: 451 Message temporarily deferred - [70] On Thu Nov 13 at 08:38 EST, Connected to 66.196.82.7 but greeting failed. \ Remote host said: 421 Message from (206.252.203.30) temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Thu Nov 13 at 05:51 EST, Connected to 206.190.53.191 but greeting failed. \ Remote host said: 421 Message from (206.252.203.30) temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Thu Nov 13 at 03:18 EST, 209.191.88.247 failed after I sent the message. \ Remote host said: 451 Message temporarily deferred - [70] On Thu Nov 13 at 00:58 EST, Connected to 209.191.118.103 but greeting failed. \ Remote host said: 421 Message from (206.252.203.30) temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 22:51 EST, Connected to 216.39.53.1 but greeting failed. \ Remote host said: 421 Message from (206.252.203.30) temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 20:58 EST, Connected to 66.196.97.250 but greeting failed. \ Remote host said: 421 Message from (206.252.203.30) temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 19:18 EST, Connected to 216.39.53.3 but greeting failed. \ Remote host said: 421 Message from (206.252.203.30) temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 17:51 EST, Connected to 66.196.97.250 but greeting failed. \ Remote host said: 421 Message from (206.252.203.30) temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 16:38 EST, Connected to 67.195.168.31 but greeting failed. \ Remote host said: 421 Message from (206.252.203.30) temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 15:38 EST, Connected to 209.191.118.103 but greeting failed. \ Remote host said: 421 Message from (206.252.203.30) temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 14:51 EST, 66.196.82.7 failed after I sent the message. \ Remote host said: 421 Message temporarily deferred - 4.16.51. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 14:18 EST, Connected to 209.191.118.103 but greeting failed. \ Remote host said: 421 Message from (206.252.203.30) temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 13:58 EST, Connected to 209.191.88.247 but greeting failed. \ Remote host said: 421 Message from (206.252.203.30) temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 13:51 EST, Connected to 209.191.118.103 but greeting failed. \ Remote host said: 421 Message from (206.252.203.30) temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
You'll notice that the IP addresses are all different, since Yahoo has tons of email servers. TMC uses a similar greylisting feature from Sendio's anti-spam appliance, but I have it set to "permissive" and not "aggressive". Were I to set it to "aggresive", then if an email came in from a different IP address, our Sendio appliance would defer the message again even if the email was previously deferred from a different IP address. By setting it to "permissive" it allows the same domain email to come in via different IP addresses.
My guess is Yahoo has their email servers set to "aggresive" and they don't have a centralized "defer" database. So when I send an email to a Yahoo address, it's going to be pure luck if I hit the same server again after being deferred. Thus, this will result in bouncebacks after being deferred ad infinitum.
Check out Yahoo's own Q&A, which essentially puts the onus on the "sender" and places no blame on themselves for failed email delivery
:421 Message temporarily deferred - [numeric code]
If you are seeing the error "421 Message temporarily deferred - [numeric code]", where "[numeric code]" shows a specific diagnostic code (e.g., "4.16.51", "4.16.52") in your SMTP logs, this indicates that:
- the message you attempted to send exhibited characteristics indicative of spam,and/or
- emails from your network have been generating complaints from Yahoo! Mail users.
This is a temporary error and your mail server may automatically re-try sending emails at a later time. However, we do encourage you to examine your outbound queues to ensure that spammers are not abusing your mail server.
Are you seeing the same error consistently over an extended period of time? If so, we encourage you to provide us with detailed information, including the error and diagnostic code(s) you see in your logs, so that we can help diagnose your problem.
For bulk mailers, please visit this page to review our best practice recommendations and request assistance.
If your mail server does not primarily send bulk mailings (e.g., you run a personal, corporate, educational, or ISP mail server), please fill out this form instead.
If you're not the administrator of the mail server in question, please contact the administrator directly with the error message you're receiving.
This question/answer wasn't very helpful either:Q: As a sender, how can I ensure uninterrupted SMTP access and prioritized delivery?
A: Yahoo! Mail has become more aggressive in its acceptance of SMTP connections and denies connections by IP address when these connections do not conform to Internet standard practices. To continue uninterrupted access and prioritized delivery, you should ensure that your email address lists are well maintained. If you are conforming to these standards, then these policies should only affect you positively. We anticipate improvements in delivery times and in available connections.
If you consistently see 451 SMTP error codes you should review your email practices and policies, especially the following:
- Remove email addresses that bounce. Bounces are an indication that the mail could not be delivered because the user does not exist, no longer exists, or is unable to accept your email. List managers should remove addresses that generate bounces. A particularly popular technique for managing bounces is to use VERP to identify the recipient address that has failed.
- Examine your retry policies. Messages that receive permanent errors, such as emails sent to accounts that do not exist or are over quota, should not be retried. Permanent errors that are retried increase the likelihood that delivery will not receive the priority it deserves.
- Pay attention to the responses from our SMTP servers.The SMTP protocol defines response codes that tell your server what to do next. In particular, our server will send "500" SMTP response codes to indicate problems that you need to investigate. For example, if an email is sent to an invalid recipient, our servers will respond with a "500" range SMTP code, indicating a permanent error. Large numbers of emails sent to non-existant accounts may be indicative of a systemic problem. Many companies following best email list practices treat email addresses that have received a "500" range SMTP codes specially. For instance, many companies place these addresses into a special queue that tests for continued bounces after longer and longer periods of time. This helps them maximize delivery to email addresses that exist and minimize wasted bandwidth costs.
- Don't send unsolicited email. Make sure that all email addresses are confirmed with an opt-in process that ensures the recipient wants to receive your mail. Obtaining permission from a third party to send an email does not ensure the email is solicited. Probably the best way to confirm an email addresses before adding them to a mailing list is by using closed-loop confirmation (sometimes referred to as "full confirmation," "full verification," "confirmed voluntary subscription," or "double opt-in"). In this process, after you receive a subscription request, you send a confirmation email to that address which requires some affirmative action before that email address is added to the mailing list. Since only the true owner of that email address can respond, you will know that the true owner has truly intended to subscribe and that the address is valid.
- Provide a method of unsubscribing from your list in each mail you send.
- Ensure that your mail servers are not open relays, and that your servers attempt to detect and deny connections to open proxies. At a minimum, your SMTP servers should identify the originating IP addresses that of the email and indicate this in the email headers to help you diagnose spam problems. There are a variety of open relay testers available.
Technorati
Del.icio.us
Slashdot
Digg
That's completely unacceptable. I had a similar situation with Hotmail a year or two ago (which is probably still going on, good thing most of my friends switched to gmail) where Hotmail would try sending the email to the server in the A Record instead of the MX record.
So randomly I'd get an email from a friend that had an error forwarded to me that they got when they tried sending me an email. Resending would usually go through. Now WHY would they even bother trying the A Record?
Don't even get me started on Hotmail's spam blocking policies (VERY dumb).
So I completely understand how angry you must be right now. Big email companies take an aggressive stand, which is fine, but if they're not going to do it right or at least be able to take some feedback on bugs, then they might as well just stop using email since they're breaking the standards.
OK, rant off. Let us know if this ever gets fixed on Yahoo's end.
I know I am getting sick of hearing that a reply to an email I initiated was sent, but I never got it. I like my yahoo account and Id really prefer that they get their act together and solve the problem (without blocking my husband's emails!) so I dont have to open a whole new account.
If you consistently see 451 SMTP error codes you should review your email practices and policies, especially the following:
bounce list checking? yes
domain keys? yes
opt in users? yes
unsubscribe link in all emails? yes
whois lookup for ip address? yes
same ip always sends email? yes
emails still not getting through to yahoo? yes
About 26% of my userbase uses yahoo.com email. I get about 5-10 support requests/day from users who can't receive emails from my server. ALL 100% of the reports come from yahoo.com users. I get ZERO reports from any users with other email providers. Why? Because the emails send successfully to everyone except for yahoo. What makes it even odder is about 50% of the emails go through to yahoo. Why block some and not others? Why, after I've taken every possible measure to ensure delivery are you still blocking me.
My newest form of attack is telling users to email yahoo complaining they can't receive emails. But I doubt anyone is really attempting that, nor would that work anyways.
Hi, Tom:
Yahoo! does not attempt to "greylist" incoming traffic -- we do not attempt to defer legitimate traffic. We do defer mail that our users have complained about through the "spam" button, and sometimes legitimate mailers with good intentions get blocked because their mail server handles traffic for multiple clients, or spammers are somehow exploiting the mail server, or a misleading subject line led some of our users to mark your mail as spam.
If you or any of your readers are encountering a problem like the one you describe, we want to know about it. Please have someone complete the form at http://help.yahoo.com/fast/help/us/mail/cgi_defer (referenced in the help page)
I hope that helps.
I clicked on the link "Mark" included- here's what I got
Yahoo! Postmaster WL Application
If emails sent by your servers are being delivered to the Yahoo! Bulk Mail folder and you would like to prevent this from occurring, please complete this application in detail to help us determine where your email is most appropriately delivered.
Yahoo! Mail Bulk Sender Form
Use this form if you send legitimate bulk mail and your mailings are not being delivered to the Inbox.
Yahoo! Mail Delivery Issues Form
Use this form to report problems your servers are experiencing when sending to Yahoo! Mail.
Yahoo! Mail ISP Issues Form
Use this form if you administer a mail server for an ISP (or a hosted email service) and your server is encountering problems sending to Yahoo! Mail.
None of this applies to me. I send emails with subjects like "Planning for the Weekend" "dog adoption"- nothing erronious about those. and the person who replies to me (mainly my husband- the auther of this blog) responds back- except it takes the reply as long as 24 hours for me to recieve. I get these same problems from the group I work with "labs4rescue".
This is not a "bulk mail folder" issue (heck Id be happy to get it in my bulk mail- at least I'd get it)
I am not sending legitimate bulk mail- these are individuals I am emailing.
MY servers are not the problem.
I am not an administrator.
I dont have to fill out any complaint forms to Hotmail, because it's never an issue.
Perhaps instead of defending Yahoo, Yahoo aught to actually solve the legitimate problem Tom raises.
I eagerly await the update notice from Yahoo!
Mark,
I did indeed submit a report. I have no idea if the problem is solved though, since Yahoo email delivery is so sporadic.I can send 10 daily emails to Yahoo and 10 go through. But a few days later I send 5 emails and 4 go through and 1 doesn't. Makes it very hard to troubleshoot the problem. Our corporate domain tmcnet.com does send thousands of emails each day to our daily newsletter subscribers. I'm sure a lot of them are on Yahoo accounts, so perhaps Yahoo is seeing the quantity of emails and assuming we are spammers, when we are not. I block the SMTP port on our firewall, so only legitimate outbound email should be going out via our Exchange Server. However, anything is possible - a client PC could be infected. Would be nice if Yahoo's online report would point to any zombie/virus-based spam being sent out via our servers. I doubt it's the case, but no IT administrator is perfect, so an online tool that tells us we're "clean" or "dirty" would be helpful. Include a sample spam email online that came from our IP address for example. Would help an IT person trace any internal virus infection.
Anyway, here's my case report I filed. I took out my email address obviously:
Hello Thomas,
Thank you for writing to Yahoo! Mail.
Please note that if you are not the mail server administrator for the
mail server(s) affected, we encourage you to contact the administrator
so they can address the possible issues regarding mailings from the mail
server. We also appreciate your report as your input helps us to
identify ways to constantly maintain and improve our service. We
apologize for any inconvenience this has caused you.
If you notice any further difficulties when delivering to Yahoo! Mail
accounts, after waiting a reasonable amount of time, please let us know
by replying directly to this email. However please note the following:
If you are receiving the error code "421 Message from x.x.x.x
temporarily deferred" or "451 Message temporarily deferred - 4.16.50" in
your SMTP logs, this indicates that the message you attempted to send
was not accepted by Yahoo! Mail servers. It is not a permanent error;
your system will automatically re-try later. If, after attempting to
deliver to Yahoo! and receive the 4.16.50 error repeatedly and conform
to the best sending practices outlined in the following page, we
encourage you to apply for prioritized sending at the link below.
Recommended Best Practices:
http://help.yahoo.com/l/us/yahoo/mail/postmaster/postmaster-15.html
Prioritized Delivery (Please note that this form can also be used by
senders whose mailings are being directed to the Bulk Folder):
http://help.yahoo.com/l/us/yahoo/mail/postmaster/bulk.html
If you are receiving a separate error than that above, please provide
the text of any error messages you may have received. Also, providing
the specific IP address of the mail server that experienced the delivery
issue will help us to troubleshoot the issue more efficiently.
For assistance with delivery issues to Yahoo! Mail, please visit:
http://postmaster.yahoo.com/
We appreciate your patience while we work to improve and maintain our
service.
Thank you again for contacting Yahoo! Mail. Your case number for this
issue is 55103003. Please reference it in all future communication about
this particular issue.
Regards,
Ashlar
Yahoo! Customer Care
55103003
For assistance with all Yahoo! services please visit:
http://help.yahoo.com/
Original Message Follows:
-------------------------
Mail-Id:
w1.help.re1.yahoo.com-/l/us/yahoo/mail/postmaster/defer.html-1226611331-
581
1. What is your name?
----------------------
Name: Thomas Keating
2. What is your email address?
-------------------------------
Email Address: tom's email here
3. What domain(s) are you contacting us about?
-----------------------------------------------
Domains: tmcnet.com
4. About the problem:
----------------------
Subject: Receiving abnormal amount of bounces
5. Indicate the specific IP address of the mail server experiencing the
issue.
------------------------------------------------------------------------
-------
206.252.203.30
6. Enter the full text of the error code(s) your mail server received
from Yahoo!:
------------------------------------------------------------------------
-----------
We were unable to connect to the destination server(s):
On Thu Nov 13 at 14:51 EST, Message exceeded queue lifetime. Bouncing.
On Thu Nov 13 at 14:51 EST, Connected to 216.39.53.2 but greeting
failed. \ Remote host said: 421 Message from (206.252.203.30)
temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Thu Nov 13 at 11:38 EST, 68.142.202.247 failed after I sent the
message. \ Remote host said: 451 Message temporarily deferred - [70]
On Thu Nov 13 at 08:38 EST, Connected to 66.196.82.7 but greeting
failed. \ Remote host said: 421 Message from (206.252.203.30)
temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Thu Nov 13 at 05:51 EST, Connected to 206.190.53.191 but greeting
failed. \ Remote host said: 421 Message from (206.252.203.30)
temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Thu Nov 13 at 03:18 EST, 209.191.88.247 failed after I sent the
message. \ Remote host said: 451 Message temporarily deferred - [70]
On Thu Nov 13 at 00:58 EST, Connected to 209.191.118.103 but greeting
failed. \ Remote host said: 421 Message from (206.252.203.30)
temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 22:51 EST, Connected to 216.39.53.1 but greeting
failed. \ Remote host said: 421 Message from (206.252.203.30)
temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 20:58 EST, Connected to 66.196.97.250 but greeting
failed. \ Remote host said: 421 Message from (206.252.203.30)
temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 19:18 EST, Connected to 216.39.53.3 but greeting
failed. \ Remote host said: 421 Message from (206.252.203.30)
temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 17:51 EST, Connected to 66.196.97.250 but greeting
failed. \ Remote host said: 421 Message from (206.252.203.30)
temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 16:38 EST, Connected to 67.195.168.31 but greeting
failed. \ Remote host said: 421 Message from (206.252.203.30)
temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 15:38 EST, Connected to 209.191.118.103 but greeting
failed. \ Remote host said: 421 Message from (206.252.203.30)
temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 14:51 EST, 66.196.82.7 failed after I sent the message.
\ Remote host said: 421 Message temporarily deferred - 4.16.51.
Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 14:18 EST, Connected to 209.191.118.103 but greeting
failed. \ Remote host said: 421 Message from (206.252.203.30)
temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 13:58 EST, Connected to 209.191.88.247 but greeting
failed. \ Remote host said: 421 Message from (206.252.203.30)
temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
On Wed Nov 12 at 13:51 EST, Connected to 209.191.118.103 but greeting
failed. \ Remote host said: 421 Message from (206.252.203.30)
temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
7. Copy and paste the contents of the email here:
--------------------------------------------------
Friendly Service Station.
Online on AAA website it says transaction was Monday the 27th not the
29th like our paper statement says.
http://maps.google.com/maps?f=q&hl=en&geocode=&q=friendly+service+station+danbury,ct&sll=37.0625,-95.677068&sspn=40.545434,79.101563&ie=UTF8&ll=41.407169,-73.449869&spn=0.018798,0.038624&t=h&z=15
Either of these two gas stations on the map look familiar?
8. Enter additional information here:
--------------------------------------
Some emails go through, but getting a few 421 defers. tmcnet.com
is a major news outlet and we have 100,000+ newsletter subscribers (many
yahoo.com subscribers) so perhaps you are being too aggressive in
deferring our email server?
"Information Passed In":
------------------------
"from_url" : http://help.yahoo.com/l/us/yahoo/mail/postmaster/basics-55.html
------------------------
While Viewing: http://help.yahoo.com/l/us/yahoo/mail/postmaster/basics-55.html
Form Name: http://help.yahoo.com/l/us/yahoo/mail/postmaster/defer.html
Yahoo ID:
Other ID:
Machine: PC
OS: WinVista
Browser: Firefox 3.0
REMOTE_ADDR: 206.252.203.146
REMOTE_HOST: 206.252.203.146
Date Originated: Thursday November 13, 2008 - 13:22:11
-------
Mark,
Let me also add, that many users are "lazy" and click the 'spam' button in their Inbox for newsletters they no longer want because they are too lazy to actually unsubscribe. So it's possible out of the thousands of newsletter subscribers we have, that a dozen or so just click the 'spam' button, thus reaching Yahoo's pre-set threshold (whatever that may be) and classifying tmcnet.com emails as spam.
Just a theory...
We have double opt-in practices, and very good opt-out options, so we're certainly not spamming.
I welcome your thoughts.
Hi, all:
Nicole: The form we intended you to complete was the "Yahoo! Mail Delivery Issues Form," which should address problems where messages are not being delivered. It sounds like the phrase "your servers" was misleading because you don't consider yourself the owner of the server? In either case, please complete that form and someone should address your issue.
Tom: The issue referenced by that case number has now been resolved. Hopefully you are seeing an improvement, and have also received a note from the Postmaster team. I'm tracking to see why it wasn't resolved more quickly for you -- certainly you shouldn't have to post to a blog to get a response from us.
On your comment about users who click "spam" because they're lazy, you're right, and it's definitely something we take into account. We consider myriad factors before blocking mail -- our goal is not just to block spam, but also to ensure that users receive all the mail they want -- and apply a weighting to spam complaints to avoid these false positives. We also plan to relaunch our complaint feedback loop very shortly, which will allow you as a sender to receive a list of the users who are complaining, to help better tailor your messages and avoid these "lazy" complaints.
i read your entire blog, and i have had the same issues. Did they give you a number or was all of the contact thru the blog. I would love to get this yahoo issue resolved.
I have like you added
SPF records,
domain keys,
RDNS records,
and filled out the forms so many times i cant think straight. I would love to know if they actually fixed the issue for you.
thanks