How to fix SPF record exceeding DNS lookup limit?

Summary

The core problem is exceeding the SPF record's DNS lookup limit of 10, which leads to authentication failures and impacts email deliverability. The solutions converge on reducing the number of lookups. Removing unnecessary includes, implementing SPF flattening (replacing includes with IPs), using a dedicated sending domain, setting up subdomains with separate SPF records, and consolidating includes are frequently recommended. The potential for ESP includes to cause issues due to nested lookups is a recurring theme. Diagnostic tools are available for identifying these issues, and adhering to RFC 7208 is essential. Having one SPF record and utilizing dedicated IPs are also advised.

Key findings

  • RFC 7208 Limit: SPF records are limited to a maximum of 10 DNS lookups as per RFC 7208.
  • Nested Lookups from ESPs: Includes from ESPs can lead to excessive nested DNS lookups.
  • SPF Flattening: SPF flattening converts includes to IP addresses, reducing DNS lookups.
  • Single SPF Record: Only one SPF record should exist for a domain to avoid authentication issues.
  • Dedicated IPs: Dedicated IPs from ESPs can prevent issues related to shared domains and nested lookups.
  • Diagnostic Tools: Tools are available to diagnose SPF issues and identify nested lookups.

Key considerations

  • Remove Unnecessary Includes: Regularly audit and remove any unused includes from the SPF record.
  • Utilize CNAME for SendGrid: Use a CNAME record for SendGrid to decrease the number of lookups.
  • Consider SPF Flattening: Evaluate the feasibility of SPF flattening to reduce DNS lookups.
  • Testing Before Deployment: Always test SPF records before deploying them to prevent deliverability issues.
  • Review and Update: Periodically review and update the SPF record to maintain accuracy and effectiveness.
  • Separate Subdomains: Create separate subdomains if your primary domain can't meet the lookup requirements

What email marketers say
11Marketer opinions

The primary issue addressed is exceeding the SPF record's DNS lookup limit of 10, which causes authentication failures and deliverability problems. Common solutions involve reducing the number of DNS lookups by removing unnecessary includes, implementing SPF flattening (replacing includes with IP addresses), using dedicated sending domains, or setting up subdomains with separate SPF records. Tools are available to test SPF records and identify nested lookups. It's crucial to have only one SPF record per domain and consolidate mechanisms where possible. Some sources suggest that ESPs can provide dedicated IPs to bypass shared domains and nested lookups.

Key opinions

  • SPF Lookup Limit: SPF records are limited to 10 DNS lookups to prevent denial-of-service attacks and long processing times.
  • Nested Lookups: Includes from ESPs can cause nested lookups, exceeding the limit due to their own extensive lists.
  • SPF Flattening: SPF flattening involves resolving 'include' statements to IP addresses to reduce DNS lookups.
  • Single SPF Record: A domain should have only one SPF record to avoid authentication issues.
  • Dedicated IPs: Using dedicated IPs from ESPs can prevent shared domains with nested lookups.

Key considerations

  • Remove Unnecessary Includes: Carefully review and remove any includes that are not actively used for sending email.
  • Testing: Test SPF records before deployment to identify and fix issues.
  • Subdomains: Consider using subdomains with separate SPF records if the primary domain cannot meet lookup limits.
  • CNAME for SendGrid: Use a CNAME record for SendGrid to reduce the number of required lookups
  • SPF Record Updates: Regularly check and update the SPF record to ensure its accuracy
Marketer view

Marketer from Email Geeks shares a cautionary tale that includes from ESPs can lead to excessive SPF lookups due to their own extensive listings.

October 2022 - Email Geeks
Marketer view

Email marketer from SuperOffice explains that a properly configured SPF record can improve deliverability and prevent spammers from forging your domain. It's important to keep the record updated and accurate.

July 2023 - SuperOffice
Marketer view

Email marketer from dmarcian clarifies that you cannot have multiple SPF records for a single domain. Having multiple SPF records will cause authentication issues, and the best practice is to consolidate all mechanisms into a single record.

December 2024 - dmarcian
Marketer view

Email marketer from MXToolbox suggests testing your SPF record before pushing it live. Their tool will give you warnings and help identify nested includes.

November 2024 - MXToolbox
Marketer view

Email marketer from EasyDMARC details that SPF flattening involves resolving all the 'include' statements in your SPF record to their corresponding IP addresses to stay within the DNS lookup limit.

October 2021 - EasyDMARC
Marketer view

Email marketer from EmailQuestions advises that a possible route is to ask your ESP for dedicated IPs. This will prevent any shared domains that might be in their includes that might have nested lookups.

August 2024 - EmailQuestions Forum
Marketer view

Marketer from Email Geeks suggests using a CNAME for SendGrid to reduce the number of SPF lookups, along with removing unnecessary includes as Matt V mentioned. Provides a link to Sendgrid documentation.

May 2024 - Email Geeks
Marketer view

Email marketer from AuthSMTP explains that the limit exists to prevent denial-of-service attacks and to ensure that SPF checks don't take too long. Suggests removing unnecessary includes and using SPF flattening.

May 2024 - AuthSMTP
Marketer view

Email marketer from Stack Overflow suggests examining all include statements for nested lookups, and that third parties often have many, and combining is one method to get below limits.

April 2021 - Stack Overflow
Marketer view

Email marketer from Reddit suggests that if you can't get under the lookup limits, consider setting up a subdomain to send those emails from, and set up a different SPF record that can bypass the limits.

October 2024 - Reddit
Marketer view

Email marketer from Mailhardener Blog suggests using techniques like SPF flattening, removing unnecessary includes, and using a dedicated sending domain to reduce SPF lookups and stay within the limit.

July 2022 - Mailhardener Blog

What the experts say
4Expert opinions

Experts agree that exceeding the SPF DNS lookup limit is a common deliverability issue. Identifying unnecessary 'include' mechanisms and nested lookups within the SPF record is crucial. Tools, such as the one offered by Word to the Wise, can help diagnose SPF issues. Optimization strategies include removing unused includes, consolidating includes, and using IP addresses instead of domain names to minimize DNS queries.

Key opinions

  • Exceeding Lookup Limit: The withwayfinder.com domain exceeds the SPF DNS lookup limit, requiring 11 lookups when the limit is 10.
  • Unnecessary Includes: Many services listed in an SPF record (e.g., Mailchimp, HubSpot) might not actively send emails, making their inclusion unnecessary.
  • Diagnostic Tools: Tools exist to diagnose SPF issues and identify nested lookups contributing to the limit.
  • Optimization Techniques: Optimizing SPF records involves removing unused includes, consolidating includes, and using IP addresses instead of domain names.

Key considerations

  • Regular SPF Audit: Regularly audit the SPF record to ensure only necessary services are included.
  • Utilize Diagnostic Tools: Use diagnostic tools to identify and address SPF issues, including nested lookups.
  • Prioritize IP Addresses: When appropriate, use IP addresses instead of domain names to reduce DNS lookups.
  • Consolidate Includes: Consolidate multiple includes into a single include mechanism where possible to reduce the number of lookups.
Expert view

Expert from Email Geeks advises that many includes in the SPF record may not be necessary, suggesting that services like Mailchimp and HubSpot might not be actively used for sending emails from the domain, and therefore can be removed.

June 2022 - Email Geeks
Expert view

Expert from Spam Resource shares tips on optimizing SPF records, including removing unused includes, consolidating multiple includes into a single include where possible, and using IP addresses instead of domain names when appropriate to avoid DNS lookups. They also mention the tool from Word to the Wise for checking.

August 2021 - Spam Resource
Expert view

Expert from Email Geeks identifies the multiple includes in the SPF record for withwayfinder.com that are causing it to exceed the DNS lookup limit of 10, highlighting that the record requires 11 lookups.

February 2023 - Email Geeks
Expert view

Expert from Word to the Wise explains that their authentication tool helps diagnose SPF issues, including identifying nested lookups that contribute to exceeding the limit. The tool will give you a count of lookups.

April 2022 - Word to the Wise

What the documentation says
5Technical articles

Multiple documentation sources highlight the importance of adhering to the SPF DNS lookup limit of 10, as specified in RFC 7208. Exceeding this limit can lead to SPF check failures and negatively impact email deliverability. Streamlining SPF records is advised, and best practices for creating and maintaining SPF records are emphasized across different platforms. The documentation underscores the role of SPF in preventing spoofing and the necessity of understanding the syntax and parameters of SPF records.

Key findings

  • RFC 7208 Limit: RFC 7208 mandates a maximum of 10 DNS lookups per SPF check.
  • Google's Stance: Google emphasizes streamlining SPF records to prevent failures due to exceeding the lookup limit.
  • Microsoft's Guidance: Microsoft highlights SPF's role in preventing spoofing and provides best practices, referring to RFC 7208.
  • Cloudflare's Observation: Cloudflare notes that exceeding the 10 DNS lookup limit is a common issue.
  • OpenSPF's Syntax: OpenSPF details the syntax of SPF records, emphasizing the 'v=spf1' TXT record structure.

Key considerations

  • Adherence to RFC 7208: Ensure SPF implementations adhere to the RFC 7208 specification regarding lookup limits.
  • Record Streamlining: Simplify SPF records to minimize DNS lookups and improve email deliverability.
  • SPF Syntax: Understand and correctly implement SPF record syntax, as detailed by OpenSPF.
  • Preventing Spoofing: Leverage SPF's capabilities to prevent email spoofing.
  • Best Practices: Follow best practices for creating and maintaining SPF records.
Technical article

Documentation from OpenSPF specifies that the syntax of an SPF record is a TXT record that begins with v=spf1. It also details what all the parameters are and how they work in relation to each other.

December 2021 - OpenSPF
Technical article

Documentation from Cloudflare outlines the standard syntax of SPF records and how they work, but also what the most common issues are. The most common problem is exceeding the DNS lookup limit of 10.

June 2023 - Cloudflare
Technical article

Documentation from Microsoft states that SPF prevents spoofing and provides the best practices for the creation and maintenance of SPF records and what the limits are. It refers back to RFC 7208

January 2024 - Microsoft
Technical article

Documentation from RFC Editor specifies that SPF implementations MUST limit the number of mechanisms and modifiers that cause DNS lookups to at most 10 per SPF check, including any lookups caused directly or indirectly by these mechanisms and modifiers.

August 2022 - RFC Editor
Technical article

Documentation from Google explains that exceeding the 10 DNS lookup limit can cause SPF checks to fail. Suggests streamlining SPF records and using techniques to minimize lookups for improved email deliverability.

February 2023 - Google