What Is Dynamic SPF Flattening? (Visually Explained)

Updated August 28th, 2024

The Sender Policy Framework (SPF) is an important addition to email security. However, senders with large email volumes quickly run into its limitation of prohibiting more than 10 DNS lookups. This means that their SPF record cannot host all sending services that the sender uses to send their emails.

Dynamic SPF Flattening solves SPF’s limitation of max. 10 DNS lookups: The SPF record is reformatted to host all sending services with fewer DNS lookups by removing include terms and adding their IP addresses directly to the main SPF record. An automatic system keeps this reformatted record up-to-date.

In this article you will learn everything you need to know – including how SPF flattening works in detail, how you can do it yourself, why an automated solution is important, what the alternatives are, and how to double-check that you even need it.

Do You Need SPF Flattening?

If you observe any of the following then you need to use SPF flattening:

1. Your emails bounce with a 5.7.26 SMTP error code (or any other 5.7.x error code).
2. Your DMARC report contains SPF PERMERROR entries for your sending sources.
3. This SPF record checker tells you that you exceed the SPF lookup limit.
4. You are planning to add a new sending service that will add too many extra lookups to your SPF record.

If you observe either 1 or 2 then please use this SPF record checker to verify that the issue is caused by exceeding the SPF lookup limit. These issues may have other causes as well.

We are offering a Dynamic SPF service that is arguably the easiest solution to overcome the lookup limit. But this article is intended to give you the necessary knowledge so that you can solve the issue yourself.

How SPF Flattening Works

Let us use this SPF record as an example which we will flatten:

v=spf1 ip4:93.184.216.34 include:spf.protection.outlook.com include:_spf.google.com ~all

By analyzing its includes, we can visualize this SPF record and its 5 lookups:

Before I show you the flattening steps, let me show you the actual SPF records that are involved here.

At the time of writing, the SPF record for spf.protection.outlook.com contains a list of IP addresses and no further includes:

v=spf1 ip4:40.92.0.0/15 ip4:40.107.0.0/16 ip4:52.100.0.0/14 ip4:104.47.0.0/17 ip6:2a01:111:f400::/48 ip6:2a01:111:f403::/49 ip6:2a01:111:f403:8000::/51 ip6:2a01:111:f403:c000::/51 ip6:2a01:111:f403:f000::/52 -all 

Google, however, has 3 includes in their _spf.google.com record:

v=spf1 include:_netblocks.google.com include:_netblocks2.google.com include:_netblocks3.google.com ~all 

Each of the three included SPF records contain a list of IP addresses:

• _netblocks.google.com:

v=spf1 ip4:35.190.247.0/24 ip4:64.233.160.0/19 ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:74.125.0.0/16 ip4:108.177.8.0/21 ip4:173.194.0.0/16 ip4:209.85.128.0/17 ip4:216.58.192.0/19 ip4:216.239.32.0/19 ~all 

• _netblocks2.google.com:

v=spf1 ip6:2001:4860:4000::/36 ip6:2404:6800:4000::/36 ip6:2607:f8b0:4000::/36 ip6:2800:3f0:4000::/36 ip6:2a00:1450:4000::/36 ip6:2c0f:fb50:4000::/36 ~all 

• _netblocks3.google.com:

v=spf1 ip4:172.217.0.0/19 ip4:172.217.32.0/20 ip4:172.217.128.0/19 ip4:172.217.160.0/20 ip4:172.217.192.0/19 ip4:172.253.56.0/21 ip4:172.253.112.0/20 ip4:108.177.96.0/19 ip4:35.191.0.0/16 ip4:130.211.0.0/22 ~all 

To flatten this SPF record...

...we simply remove all include terms and add all IP addresses to the main SPF record:

v=spf1 ip4:93.184.216.34 ip4:40.92.0.0/15 ip4:40.107.0.0/16 ip4:52.100.0.0/14 ip4:104.47.0.0/17 ip6:2a01:111:f400::/48 ip6:2a01:111:f403::/49 ip6:2a01:111:f403:8000::/51 ip6:2a01:111:f403:c000::/51 ip6:2a01:111:f403:f000::/52 ip4:35.190.247.0/24 ip4:64.233.160.0/19 ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:74.125.0.0/16 ip4:108.177.8.0/21 ip4:173.194.0.0/16 ip4:209.85.128.0/17 ip4:216.58.192.0/19 ip4:216.239.32.0/19 ip6:2001:4860:4000::/36 ip6:2404:6800:4000::/36 ip6:2607:f8b0:4000::/36 ip6:2800:3f0:4000::/36 ip6:2a00:1450:4000::/36 ip6:2c0f:fb50:4000::/36 ip4:172.217.0.0/19 ip4:172.217.32.0/20 ip4:172.217.128.0/19 ip4:172.217.160.0/20 ip4:172.217.192.0/19 ip4:172.253.56.0/21 ip4:172.253.112.0/20 ip4:108.177.96.0/19 ip4:35.191.0.0/16 ip4:130.211.0.0/22 ~all 

Although the SPF record is correct, it is quite long. And keep in mind that this is only a small example. Real-world SPF records will be much longer. So long that they exceed the maximum DNS record length.

According to the SPF specification, an SPF record must not be longer than 512 characters. Our example has 785 characters.

To fix this, we need to split the record up into multiple records and then string them together with includes:

The image shows an SPF record that is split up into 3 parts. Our example, however, only needs two parts because its 785 characters fit into 2 times 512 characters.

The final two records, that are the flattened version of the original SPF record at the top, are these:

• Main SPF record

v=spf1 ip4:93.184.216.34 ip4:40.92.0.0/15 ip4:40.107.0.0/16 ip4:52.100.0.0/14 ip4:104.47.0.0/17 ip6:2a01:111:f400::/48 ip6:2a01:111:f403::/49 ip6:2a01:111:f403:8000::/51 ip6:2a01:111:f403:c000::/51 ip6:2a01:111:f403:f000::/52 ip4:35.190.247.0/24 ip4:64.233.160.0/19 ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:74.125.0.0/16 ip4:108.177.8.0/21 ip4:173.194.0.0/16 ip4:209.85.128.0/17 ip4:216.58.192.0/19 ip4:216.239.32.0/19 ip6:2001:4860:4000::/36 include:_spf2.example.com ~all

• _spf2.example.com

v=spf1 ip6:2404:6800:4000::/36 ip6:2607:f8b0:4000::/36 ip6:2800:3f0:4000::/36 ip6:2a00:1450:4000::/36 ip6:2c0f:fb50:4000::/36 ip4:172.217.0.0/19 ip4:172.217.32.0/20 ip4:172.217.128.0/19 ip4:172.217.160.0/20 ip4:172.217.192.0/19 ip4:172.253.56.0/21 ip4:172.253.112.0/20 ip4:108.177.96.0/19 ip4:35.191.0.0/16 ip4:130.211.0.0/22 ~all

We reduced 5 lookups required by the original SPF record to just one!

Word of caution: Stringing together multiple records, as we just did above, produces DNS lookups. Hence we need to pay attention to the 10 DNS lookup limit again. The main SPF record can be strung together with a maximum of 10 additional records. If you exceed this limit when flattening your own SPF record, you will need a dynamic SPF service that is truly limitless.

How To Manually Flatten Your SPF Record With Free Tools

To manually flatten your SPF record you need to follow these steps:

1. Create flattened SPF records based on your original SPF record. (Many free tools help with that.)
2. Host the flattened SPF records on your DNS server.
3. Regularly repeat steps 1 and 2 to ensure that the records remain up-to-date. (They get outdated quickly! More about this below.)

Step 1 – Create flattened SPF records based on your original SPF record

You don’t need to do the flattening steps manually. Many free tools take your original SPF record and generate the flattened SPF records for you.

Here are some tools that you can try:

• Free SPF Flattening Tool – Online tool where you simply enter your domain name
• sender policy flattener – A Python-based command line tool

Step 2 – Host the flattened SPF records on your DNS server

Replace your original SPF record with the first record of the flattened records. The additional flattened records need to be hosted as TXT records for the subdomains that are used in the includes that chain all flattened records together.

Since your original SPF record gets replaced, it is a good idea to host your original SPF record on an unused subdomain, e.g. _spf-original.yourdomain.com. This allows your team to update the original record as needed and then you can ask the tools in step 1 to query this subdomain to use it for their flattening process.

Step 3 – Regularly update your flattened records

Your flattened records get out of date for two reasons:

1. Your email infrastructure grows and you need to add a new sending service to your SPF records.
2. An existing sending service updates its email infrastructure by adding or removing IP addresses.

The second reason happens more often than you think!

Although you and your team can make sure that you update the flattened records every time your email infrastructure grows – which does not happen very often – you never know when the email infrastructure of your existing sending services change. It can happen at any time.

You should check your SPF records at least daily to ensure that your flattened SPF records are never outdated for too long. This means that you should automate this process.

Why You Need Dynamic SPF Flattening

Dynamic SPF Flattening services automate all steps from creating flattened SPF records to keeping them up-to-date on your DNS server. The full automation of all manual steps is necessary because changes occur frequently – most notably because 3rd party sending services add new IP addresses.

Large email volume senders usually use a long list of 3rd party sending services to deliver their emails. Thus their SPF record contains includes to these sending services, e.g. include:spf.protection.outlook.com include:_spf.google.com.

These include terms have the benefit that they automatically adopt if e.g. Outlook adds new IP addresses to their IP address pool. The team at Outlook simply adds the new IP addresses to their spf.protection.outlook.com SPF record and then all senders who use Outlook’s email infrastructure automatically allow the new IPs to send emails on their behalf.

However, if you flatten your SPF record then you lose the flexibility of these include terms. Simply because flattening removes the include terms. If a sending service adds a new IP address, your flattened SPF record gets out of date and emails sent from the new IP address will fail the SPF verification.

Hence you need automation that keeps your flattened SPF records up-to-date. It regularly checks the sending service’s SPF records and if the list of IP addresses changes, a new set of flattened SPF records gets created and published to your DNS server.

We offer a Dynamic SPF service that does everything I described for you, is easy to set up, and has none of the limitations that traditional SPF flattening still has.

What Are The Alternatives To SPF Flattening?

SPF flattening has its own limitations and also adds complexity to your SPF setup. For these reasons, let us look at alternatives that either add less complexity or overcome the remaining limitations.

• Macro-based Dynamic SPF

  SPF flattening is itself limited by the 10 DNS lookup limit because it chains multiple SPF records together which produce their own lookups. This means that it can host a maximum of ~230 IP addresses / ranges.

  If you have a large email infrastructure for which this is too limiting, or if you want to ensure that this limit won’t be an issue for you in the foreseeable future, then you need a solution based on SPF macros that is truly unlimited.

  In case you are curious, I once shared the technical details of how this approach works on Reddit. Like SPF flattening, this macro-based approach reformats the SPF records but does it in a hierarchical way instead of stringing the records together. This way the lookup count stays the same no matter how many sending services are added.

  Our own Dynamic SPF solution works this way.

• Cleaning up your SPF record

  Chances are that your SPF record contains terms that are not needed and that waste valuable lookups. By removing them you may free up some lookups that you can use for your sending service include terms.

  Do you have +a +mx in your SPF record? You may have added them because many SPF record generators add them by default. However, they are not needed oftentimes. If you are able to remove both, you free up two lookups.

  The +a term points to the webserver. It is usually not needed because the webserver does not act as an email server itself. Even if the webserver sends emails it is usually done by calling the API of a sending service such as sendgrid.

  The +mx term points to the email servers that receive incoming emails for the domain. It is not needed if the receiving email servers are hosted by one of the included sending services. Listing their include terms is enough and the +mx term only duplicates some of the IP addresses.

• Splitting your SPF record up based on your inboxes

  It may be the case for you that your contact@example.com inbox only uses your customer support sending service, your sales@example.com inbox only uses your billing system sending service, and do on. If this is the case then you can split your SPF records up.

  You simply do this by replacing your main SPF record with:

  v=spf1 include:%{l}._spf.example.com ~all

  The %{l} is an SPF macro that stands for the inbox name, i.e. contact and sales. Hence you would create an SPF record for each inbox:

  • Under the subdomain contact._spf.example.com for contact@example.com:

    v=spf1 include:your.customer-support.service ~all

  • Under the subdomain sales._spf.example.com for sales@example.com:

    v=spf1 include:your.sales-system.service ~all

  • And finally under the catch-all subdomain *._spf.example.com for all other inboxes:

    v=spf1 include:remaining.service.one include:remaining.service.two ~all

  By splitting up the SPF records you are generating one extra lookup – from the main SPF record to one of the split records – but each of the split records can list fewer sending services. This may be enough to not exceed the 10 lookup limit anymore.

  Furthermore, the advantage of this approach is that you don’t need an automated system that keeps your records up-to-date.