Why are automated scripts and crawlers opening my emails, and how can I identify and exclude them from tracking?
Summary
What email marketers say12Marketer opinions
Email marketer from ZeroBounce.net explains that implementing a double opt-in to confirm each email address can reduce invalid signups. This is one of the first lines of defense in preventing bots from skewing open rates.
Email marketer from EmailonAcid.com shares that security programs are scanning emails as a means of providing security to their users. Recommends using a combination of methods to filter bots, including excluding known bot IPs, identifying common bot user agents (like python-requests), and analyzing open patterns (like very fast opens after sending).
Email marketer from StackOverflow notes that automated systems often open emails for security checks. The user suggests implementing a unique tracking pixel per recipient and monitoring unusual patterns like rapid opens or opens from uncommon user agents.
Marketer from Email Geeks identifies AHC/2.1 as associated with SchoolMessenger (<https://www.schoolmessenger.com/student-email-safety/>) and notes the IPs are usually AWS, indicating non-human activity.
Email marketer from EmailVendorSelection.com responds that bots and email security software can trigger opens without genuine engagement. Recommends analyzing IP addresses, user agents, and open times to identify and exclude bot traffic from email metrics.
Email marketer from Mailjet.com answers that bots inflate email open rates and suggests using a double opt-in process to confirm genuine subscriptions, and implementing a reCAPTCHA on the signup form. They also suggest regularly cleaning your email list to remove unengaged subscribers.
Marketer from Email Geeks shares a link to AWS's JSON file for IPv4 and IPv6 ranges: <https://ip-ranges.amazonaws.com/ip-ranges.json>.
Email marketer from Litmus.com suggests using a combination of methods to filter bots, including excluding known bot IPs, identifying common bot user agents (like python-requests), and analyzing open patterns (like very fast opens after sending) to improve email marketing reports.
Email marketer from SparkPost.com explains that bots and automated systems can inflate open rates, and provides the following advice: Examine IP addresses, user agents and frequency of opens to help exclude them. Also, create a double opt-in system to help avoid fake sign-ups.
Email marketer from Reddit explains that many security appliances open emails to check for malicious content, inflating open rates. Suggested solutions include identifying these 'false' opens by their user agent or IP and filtering them out.
Email marketer from SenderPulse.com notes that Apple's Mail Privacy Protection (MPP) loads images on behalf of the user thus inflating open rates, much like a bot would. Users can use their software to identify email opens that are potentially protected by MPP.
Email marketer from NeilPatel.com shares that bot traffic inflates open rates and suggests implementing CAPTCHAs on signup forms, using double opt-in, and regularly cleaning email lists to minimize the impact of bots.
What the experts say4Expert opinions
Expert from Word to the Wise shares that bot traffic from security scans is often misattributed and suggests monitoring user agent strings, and identifying patterns in opens to identify these non-human opens. They also recommend segmenting and testing your mailings.
Expert from Spam Resource explains that one reason for automated opens is security software scanning emails for threats. They share to identify these opens, monitor user-agent strings like 'python-requests' or look for rapid opens after the email is sent.
Expert from Email Geeks suggests that any AWS IPs can be excluded because end-users do not typically use AWS as their internet connection.
Expert from Email Geeks advises treating open traffic from cloud services like AWS, GCP, DO, and Azure suspiciously, considering them as non-human interactions and not 'true' opens for reporting.
What the documentation says5Technical articles
Documentation from IETF provides detailed technical standards for SMTP, including user agent conventions. These documents are used to understand the expected behavior and format of legitimate email clients and identify anomalies associated with bot traffic.
Documentation from Amazon Web Services shares that they publish a JSON file containing all their public IPv4 and IPv6 address ranges. This list can be used to identify and filter out bot traffic originating from AWS infrastructure. The ip-ranges.json file is updated frequently and should be checked regularly.
Documentation from developers.google.com explains that Googlebot crawls the web to index content. These crawls may trigger opens, but should ideally be identified via user-agent strings and IP ranges to avoid skewing email analytics.
Documentation from Spamhaus.org shares that they maintain blacklists of IPs and domains used by spammers and bots. Checking email traffic against these lists can help identify and block malicious bot activity.
Documentation from learn.microsoft.com provides information that Microsoft uses crawlers to index websites for its search engine, Bing. These crawlers can be identified by user agent strings and IP addresses provided in Microsoft's documentation.