When setting up email authentication, one question I frequently encounter is whether to use SPF hardfail (-all) or softfail (~all) in your SPF record, especially when DMARC is also in play. My initial thought, like many, was that hardfail offers superior security by strictly rejecting unauthorized emails. However, the reality, particularly with modern email authentication protocols, is more nuanced and often leads to the recommendation of softfail alongside a robust DMARC policy.
It's a common misconception that an SPF hardfail provides the highest level of protection. While it might seem intuitive, the interplay between SPF, DKIM, and DMARC means that a hardfail can sometimes cause legitimate emails to be rejected, even if they would otherwise pass DMARC through DKIM authentication. This is a critical point that impacts email deliverability and overall sender reputation. We'll explore why softfail is often the smarter choice when you've implemented DMARC.
Understanding SPF qualifiers: hardfail (-all) vs softfail (~all)
Sender Policy Framework (SPF) is a DNS text record that specifies which mail servers are authorized to send email on behalf of your domain. It helps prevent email spoofing by allowing recipient mail servers to verify that an incoming email from a domain comes from an IP address authorized by that domain's administrators. The all mechanism at the end of an SPF record defines the policy for emails sent from unauthorized servers. This is where -all (hardfail) and ~all (softfail) come into play.
Hardfail (-all)
A hardfail, indicated by -all, tells recipient mail servers that any email originating from a server not explicitly listed in your SPF record should be immediately rejected. This is the strictest policy and is intended to prevent unauthorized sending. In a world without DMARC, this would be the most secure option. Here is an example:
Example SPF Record with Hardfaildns
v=spf1 include:_spf.google.com -all
Softfail (~all)
A softfail, denoted by ~all, indicates that emails from unauthorized servers should be accepted but marked as suspicious. This usually means they might be routed to the spam folder or flagged, rather than outright rejected. It's a more lenient approach, often used during the initial stages of email authentication setup or for domains that have complex sending environments. Take a look at this example:
Example SPF Record with Softfaildns
v=spf1 include:mail.yourdomain.com ~all
Understanding the immediate action taken by receiving servers based on these qualifiers is crucial for managing your email deliverability. For a deeper dive into SPF basics, refer to this DMARC RFC document.
The role of DMARC in email authentication
DMARC, or Domain-based Message Authentication, Reporting, and Conformance, builds upon SPF and DKIM to provide a comprehensive framework for email authentication. Its primary function is to give domain owners the ability to tell receiving mail servers what to do with emails that fail authentication (SPF or DKIM) and to receive reports about those failures. This is where DMARC truly shines, as it unifies the results of SPF and DKIM checks and applies a policy based on your domain's instructions.
With DMARC in place, the decision of whether an email is accepted, quarantined, or rejected shifts from the individual SPF or DKIM results to the DMARC policy itself. DMARC requires at least one of SPF or DKIM to pass and, crucially, to be aligned with the From: domain. If an email fails both SPF alignment and DKIM alignment, the DMARC policy comes into effect. This means that whether your SPF record uses -all or ~all, DMARC will treat it as an SPF failure if it doesn't align correctly. Learn more about a simple guide to DMARC, SPF, and DKIM.
The key policies for DMARC are p=none (monitoring), p=quarantine (send to spam), and p=reject (block entirely). These policies dictate the actual handling of non-compliant emails. This distinction is vital because DMARC's robust policy enforcement means that the strictness of your SPF all mechanism becomes less critical, as DMARC effectively takes over the decision-making for authentication failures. For example, if p=reject is set in DMARC, an SPF ~all will still result in the email being rejected by DMARC. I have found this to be extremely powerful in protecting domains. Check out more information about when you should use DMARC policies.
Why SPF hardfail can be problematic with DMARC
Despite the intention behind SPF hardfail, using -all can lead to significant deliverability problems when DMARC is active. The primary issue is that some receiving mail servers may perform SPF checks first and immediately reject emails that hardfail, without even considering the DMARC record or a successful DKIM authentication. This means a legitimate email, properly signed with DKIM and intended to pass DMARC, could still bounce if its SPF check results in a hardfail due to common email handling quirks.
A classic example of this is email forwarding. When an email is forwarded, its SPF authentication often breaks because the forwarding server's IP address isn't listed in the original sender's SPF record. If your SPF policy is -all, the forwarded email might be rejected outright, even if the DKIM signature remains valid and DMARC alignment would have passed. This problem with forwarded emails is a strong argument against using hardfail when DMARC is active.
It is because of these potential issues that I advise against SPF hardfail when DMARC is in use. DMARC provides the mechanism to enforce your email policy, allowing DKIM to serve as a backup authentication method when SPF fails (for example, due to forwarding). Using ~all in your SPF record, combined with a DMARC policy of p=quarantine or p=reject, effectively achieves the same security outcome as hardfail, but with significantly less risk of legitimate email rejection. You can find more details on this in Al's comprehensive SPF explanation.
The risk of SPF hardfail with DMARC
When your SPF record uses -all and DMARC is implemented, legitimate emails can still be rejected. This happens because some email providers perform the SPF check first. If it's a hardfail, the message is dropped before the DMARC policy is even evaluated, even if DKIM would have passed. This leads to deliverability issues for your valid emails, impacting sender reputation and campaign effectiveness. It's a key reason why SPF -all is often considered obsolete in modern DMARC deployments.
Recommendations and best practices for SPF with DMARC
Based on my experience and industry best practices, I strongly recommend using SPF ~all (softfail) in your SPF record when you have DMARC deployed. This combination provides robust security without the risk of over-rejection that -all can introduce. Remember, DMARC's policy (e.g., p=reject) is what ultimately determines the action on failed emails, not the SPF qualifier.
The transition to a stricter DMARC policy, like p=reject, should be a gradual process. I typically advise starting with p=none to monitor your email traffic and ensure all legitimate sending sources are authorized by SPF and DKIM. Once you're confident, you can move to p=quarantine, and eventually p=reject. This phased approach helps avoid disruption to your email flow. For guidance on this, see how to roll out DMARC enforcement.
Always ensure your DKIM records are correctly configured and aligned, as DKIM is often more resilient to forwarding issues than SPF. If DMARC passes but SPF fails, it means DKIM likely saved your email from being rejected. This underscores the importance of a multi-layered approach to email authentication, rather than relying solely on SPF's -all mechanism. You can check out this article on if DMARC passes but SPF fails.
Traditional SPF view (without DMARC)
Hardfail (-all): Considered the most secure option, as unauthorized emails are immediately rejected. This prevents spoofing at the SPF level.
Softfail (~all): Less secure, as unauthorized emails are merely marked as suspicious or sent to spam, but not outright blocked. This leaves a window for potential malicious delivery.
Primary Goal: To block unauthorized senders at the earliest possible stage through SPF alone.
SPF with DMARC deployed (recommended view)
Hardfail (-all): Can cause legitimate emails to bounce prematurely if SPF is checked before DMARC or if DKIM is the only passing authentication. Increases risk of false positives.
Softfail (~all): Recommended for domains with DMARC. SPF failures (including softfail) are passed to DMARC, allowing DKIM to also contribute to an authentication pass. DMARC policy then dictates disposition.
Primary Goal: To ensure all legitimate mail is authenticated (via SPF or DKIM) and DMARC policy enforces actions for unauthenticated mail.
Views from the trenches
Best practices
Always include SPF and DKIM records for all domains that send email. Both are crucial for robust email authentication and DMARC alignment.
Start DMARC implementation with a p=none policy and SPF ~all to gather data without impacting deliverability, then gradually increase enforcement.
Regularly monitor your DMARC reports to identify legitimate sending sources that might be failing authentication and adjust your SPF and DKIM records accordingly.
Ensure your DKIM signatures are correctly configured and aligned, as DKIM is often more resilient to email forwarding than SPF.
If using SPF ~all, pair it with a DMARC policy of p=quarantine or p=reject to enforce actions on unauthorized emails.
Keep your SPF record concise and avoid unnecessary lookups to prevent hitting the 10 DNS lookup limit, which can cause SPF to fail.
Common pitfalls
Using SPF -all with DMARC can lead to legitimate emails being rejected prematurely by some mail servers, even if DKIM would have passed.
Neglecting DMARC reports means you miss critical insights into your email authentication failures and potential spoofing attempts.
Not having DKIM set up or having misconfigured DKIM records, which leaves your email authentication vulnerable, especially when SPF fails.
Attempting to go straight to DMARC p=reject without monitoring, leading to a high rate of legitimate email rejections.
Ignoring the SPF 10-lookup limit, causing SPF validation to return a permError, potentially impacting deliverability.
Failing to account for email forwarding scenarios, where SPF can break, leading to legitimate forwarded emails being rejected.
Expert tips
Consider that the SPF 'all' mechanism primarily informs DMARC about whether SPF passed or failed, with DMARC dictating the final action.
Remember that modern email authentication is a layered approach where SPF and DKIM feed into DMARC's decision-making process.
A well-configured DMARC policy at p=reject offers stronger protection against spoofing than SPF hardfail alone, as it considers both SPF and DKIM.
DMARC's reporting functionality provides the necessary visibility to safely transition from a relaxed SPF policy to a more stringent DMARC enforcement.
Prioritize a comprehensive DMARC implementation strategy, as it effectively deprecates the strictness of SPF's hardfail qualifier.
Email deliverability is an ongoing challenge; continuous monitoring and adaptation of authentication protocols are essential for maintaining inbox placement.
Expert view
An expert from Email Geeks says they recommend softfail because some mail transfer agents (MTAs) will evaluate SPF hardfail and bounce messages even if they are fully DMARC compliant with DKIM.
2024-05-22 - Email Geeks
Expert view
An expert from Email Geeks says that -all is obsolete in the world of DMARC and recommends using ~all outside of rare cases.
2024-05-22 - Email Geeks
Making an informed choice
Ultimately, the decision between SPF hardfail (-all) and softfail (~all) in your SPF record, when DMARC is present, leans heavily towards softfail. While hardfail might seem more secure on its own, it introduces a risk of false positives that DMARC is designed to mitigate. By using ~all with a strong DMARC policy, you achieve the same level of security against spoofing and phishing without unnecessarily blocking legitimate emails, especially those that have been forwarded.
The key takeaway is to let DMARC handle the enforcement. It provides a more intelligent and adaptable approach to email authentication by considering both SPF and DKIM. This layered security ensures that your legitimate emails reach the inbox while unauthorized messages are properly handled according to your specified DMARC policy. Always prioritize DMARC as your primary email authentication and enforcement mechanism.
