Email deliverability monitoring: a system operators can actually use

Most email deliverability monitoring fails for a simple reason: it collects scores without defining decisions.

One dashboard says the domain is healthy. Another says an IP is listed. Open rates moved. Gmail data is missing. Microsoft deferred a batch. The team has plenty to look at and no agreement on what any of it should trigger.

A useful monitoring system has five parts:

1. The identity being monitored.
2. The signal and its source.
3. A normal baseline for that identity.
4. The person who owns the response.
5. The next action when the signal changes.

Build those five parts before buying another tool.

Monitor the layers separately

“Deliverability” is an outcome across several systems. Keep the layers separate long enough to find the problem.

The layers affect each other, but they do not share one universal score.

A domain can authenticate correctly while the audience is wrong. An IP can look normal while one mailbox is restricted. A campaign can generate fewer replies without a delivery failure. Monitoring should help you distinguish those cases.

Start with an infrastructure inventory

You cannot monitor what you cannot map.

For each active sending unit, record:

• Business or client owner.
• Sending domain.
• Mailbox addresses or cohort.
• Google Workspace, Microsoft 365, or SMTP transport.
• DKIM signing domain and return-path domain.
• Tracking domain, if used.
• Sending platform or sequencer.
• Public IP or pool, when visible and relevant.
• DNS owner and registrar access owner.
• Daily pattern and campaign start date.
• Suppression source.
• Person allowed to pause the send.

Keep the record small enough that someone will maintain it. The goal is to answer, “What else shares this dependency?” when a signal moves.

One domain per workspace can make that ownership boundary clearer. Cheap Inboxes uses that structure for the workspaces it manages. It does not guarantee delivery, but it reduces ambiguity when an operator needs to trace a domain, its inboxes, and the people responsible for them.

Establish a baseline before setting alerts

An alert without a baseline is usually noise.

Pick a stable period for each active cohort and save the distribution—not only the average. A ten-domain program can have the same overall bounce rate while one domain fails completely and nine remain normal.

Useful baseline cuts include:

• By receiving provider: Gmail, Microsoft, Yahoo, and other.
• By sending domain.
• By mailbox cohort.
• By campaign and audience source.
• By transport or IP when you control it.
• By day of week and sending stage.

For each cut, record volume as well as rate. One complaint out of 50 messages and 20 complaints out of 10,000 may display the same percentage after rounding, but they do not carry the same confidence.

Do not invent a global “good” threshold because a vendor needs a red line on a chart. Use official provider requirements where they exist, then set internal warnings based on your own normal ranges and tolerance for risk.

Google, for example, tells bulk senders to keep reported spam rates below 0.1% and avoid reaching 0.3% or higher. That is a provider guideline. It is not permission to run every program at 0.09%, and it does not describe Microsoft recipients.

The monitoring matrix

This is the core of the system. Each signal needs a scope, cadence, owner, and action.

Change the frequencies to match your volume. Do not change the requirement that every row has an owner and a first action.

Use provider tools for the identity they cover

Google Postmaster Tools

Google Postmaster Tools is useful for eligible traffic to personal Gmail accounts. It can show dashboards for spam rate, IP or domain reputation, authentication, encryption, and delivery errors when Google has enough data.

Missing data does not automatically mean the domain is healthy or broken. The verified domain may not have enough eligible traffic, the wrong domain may be selected, or the reporting window may not be complete.

Compare Google’s data with Gmail-specific SMTP results and campaign outcomes. Do not mix every destination into one average and then try to explain a Gmail dashboard.

Microsoft SNDS and JMRP

SNDS is mainly useful for sending IP space you can prove you control. JMRP provides eligible junk reports connected to enrolled IPs.

Standard Google Workspace and Microsoft 365 customers generally do not control the providers’ shared transport IPs. A mailbox reseller does not gain control simply by selling those mailboxes. If you operate dedicated SMTP infrastructure, SNDS can be much more actionable.

Read the Microsoft SNDS guide before adding it to the routine. The access model tells you whether it belongs in your stack.

DMARC aggregate reports

DMARC reports help show which systems are sending mail using a domain and whether SPF or DKIM alignment passed. They are valuable for detecting unauthorized or forgotten senders and verifying a migration.

They do not show inbox placement and usually do not tell you whether recipients wanted the messages. Keep DMARC monitoring in the authentication layer.

Blocklists and third-party tests

Blocklist monitors, seed tests, content scanners, and scoring products can be useful supporting evidence. They are also easy to overread.

Before acting on one:

1. Confirm the tested domain, IP, mailbox, and message.
2. Confirm the tool’s observation window.
3. Check whether the same identity is failing with real recipients.
4. Look for a corresponding provider response or campaign change.
5. Record what the tool can and cannot measure.

A seed test shows what happened to a controlled group of test accounts. It does not reproduce every real recipient’s history, filtering, or engagement. Use it for comparisons and change detection, not as a universal placement percentage.

Opens are not a reliable center of the system

Open tracking is affected by image blocking, privacy proxies, corporate security tools, and automated scanners. A reported open may not be a person. A missing open may not mean the message was filtered.

That does not make open data useless. It makes it directional and context-dependent.

Give more weight to:

• Full SMTP responses.
• Authentication results from received messages.
• Provider complaint data.
• Hard bounces and mailbox restrictions.
• Human replies and positive replies.
• Unsubscribes and explicit negative responses.
• Qualified downstream outcomes.

If an open-rate change is the only evidence of a delivery problem, keep investigating before replacing infrastructure.

Preserve the error, not just the category

Sending platforms often normalize dozens of provider responses into a few friendly labels. Keep the original response wherever possible.

An operations record should preserve:

• Timestamp.
• Sending mailbox and domain.
• Receiving domain or provider.
• SMTP status code and enhanced status code.
• Full response text.
• Message or campaign identifier.
• Retry behavior.
• Final outcome.

Then map responses into an action taxonomy:

Do not route every 4xx response to “warm-up” and every 5xx response to “replace domain.” The actual text matters.

Run controlled investigations

When a signal moves, write the problem as a comparison:

Microsoft deferrals increased for mailboxes on Domain A after Tuesday, while Domain B using the same sequencer and audience source remained within baseline.

That is investigable. “Deliverability is down” is not.

Use this sequence:

1. Define the affected identity and destination.
2. Define the first abnormal date.
3. Pull raw responses and received headers.
4. Check provider data covering that identity.
5. List every change in the preceding period.
6. Find an unaffected control that shares some of the same setup.
7. Change one supported variable.
8. Observe the same signals through a defined window.

Possible changes include a new audience source, a higher daily pattern, new copy, a new tracking domain, DNS work, mailbox credential changes, a sequencer migration, or routing to a different SMTP pool.

If five things changed, the change log should say so. The investigation may need a reset to a known configuration rather than another experiment.

Avoid false positives

Monitoring systems create their own failures when they react to noise.

Low sample size

A rate based on a handful of messages can swing wildly. Always display the numerator and denominator.

Reporting delay

Provider dashboards can lag. Do not compare a partial day with a complete historical day without labeling it.

Audience mix

A campaign targeting a new segment may reply differently even when the infrastructure is unchanged. Separate business response from technical acceptance.

Automated activity

Security scanners can click links or load pixels. Do not treat every click as buyer intent.

Shared dependencies

Several domains can appear to fail independently when they share one tracking domain, SMTP pool, template, or list source. The inventory should expose that dependency.

Tool methodology changes

A vendor may change how it calculates a score. Preserve raw evidence and annotate tool changes in the same log as infrastructure changes.

Require corroboration before a destructive response. One vague score should not retire a domain.

Daily, weekly, and monthly routines

Daily: catch active damage

• Review mailbox restrictions and authentication errors.
• Review new hard bounces and suppress permanent failures.
• Review provider policy failures by destination.
• Process complaint reports and unsubscribes.
• Pause a clearly failing identity when continued sending would add harm.

The daily review should be short. It is an exception queue, not a meeting.

Weekly: find patterns

• Compare key signals against the baseline by provider, domain, and campaign.
• Review Postmaster, SNDS, JMRP, and relevant blocklist evidence where applicable.
• Compare replies and complaints against changes in audience and copy.
• Review the infrastructure change log.
• Assign one owner and next action for every material exception.

Keep “watch” separate from “act.” Not every movement needs a change.

Monthly: protect ownership

• Audit domain registration, renewal, DNS, and recovery access.
• Confirm which mailboxes and domains are active.
• Remove or archive assets nobody owns.
• Test a sample of SPF, DKIM, and DMARC results externally.
• Review whether every dashboard still covers the identities the team thinks it covers.
• Update baselines when the sending program has intentionally changed.

This is also the time to review the monitoring system itself. Delete metrics that never lead to a decision.

The incident record

When something meaningful changes, open one record with:

• Clear problem statement.
• Affected and unaffected identities.
• First abnormal time.
• Raw evidence links or attachments.
• Recent changes.
• Current containment.
• Working hypothesis, labeled as a hypothesis.
• Owner and next check time.
• Decision and result.

Do not paste secrets, mailbox credentials, customer records, or sensitive recipient data into a general ticket. Store those in the system designed for them and link only to an access-controlled record.

The incident is not closed when a dashboard turns green. It is closed when the team understands the likely cause, has stopped the failure, and has changed the process that allowed it.

What a good monitoring system produces

A good system produces fewer dramatic reactions.

It catches authentication failures before a full campaign runs. It suppresses bad recipients quickly. It separates Gmail problems from Microsoft problems. It tells you when a shared dependency affects several domains. It gives provider support a specific evidence packet. And it prevents the team from rotating infrastructure because one generic score moved.

You do not need a perfect view of every mailbox provider. You need enough evidence to make the next decision without guessing.

That is the standard: identity, baseline, owner, action.

Design the dashboard around exceptions

A monitoring dashboard should make the abnormal unit obvious without forcing an operator to open twenty charts.

The first view should show missing data, authentication changes, restricted mailboxes, bounce classes outside baseline, provider-specific failures, complaint evidence, renewal risk, and changes made during the reporting window.

Every exception should link to raw evidence and the infrastructure inventory. A red indicator with no SMTP response, header, or provider record creates more work than it saves.

Use three states:

1. Watch: movement is real but sample size or corroboration is incomplete.
2. Investigate: evidence crossed the cohort’s threshold and needs an owner now.
3. Contain: continued sending is likely to add harm or the technical path is clearly broken.

The system should never pause activity solely because a generic third-party score moved. Containment needs a rule grounded in an identity the team owns and evidence it can inspect.

Define missing-data behavior

Missing data is a state, not zero.

If Google Postmaster Tools has no eligible data, display “unavailable” and the last complete date. If SMTP-log ingestion stops, alert on the ingestion failure. If a cohort sent no messages, show zero volume separately from zero failures.

This prevents a broken monitor from producing a perfect-looking report.

Reporting for agencies and business units

Separate the operator view from the client or executive report.

Operators need raw responses, mailbox identities, dependencies, and change history. A client needs the material signal, what it affects, the response, the owner, and the result.

A weekly summary should state the domains, cohort, campaigns, reporting window, message volume, material exceptions, response, outcomes, recent changes, and next action. Do not average unrelated clients into one deliverability rate. Do not expose credentials, recipient-level complaint data, or another client’s infrastructure.

Each client should have its own inventory, suppression boundary, evidence store, and authority to pause. The agency can reuse the method without pooling sensitive records.

Monitor migrations more closely

Increase monitoring around a DNS-host migration, mailbox-provider migration, new sequencer, tracking-domain change, new SMTP pool, major audience-source change, or significant volume change.

Create a pre-change packet with current public DNS, known-good external headers, recent SMTP-response distribution, provider baseline, active identities, and rollback owner. After the change, test the same identities and destinations.

Do not declare success because an admin interface saved the setting. Success means the expected message path works, authentication aligns, errors remain visible, and the operator can reverse or troubleshoot the change.

A 30-day implementation plan

Week 1: inventory and raw evidence

Map active domains, mailboxes, transports, sequencers, tracking domains, owners, and suppression sources. Preserve full SMTP responses and an external header sample for each provider path.

Week 2: baseline and ownership

Build the provider, domain, and campaign cuts. Label missing data. Assign owners for DNS, inboxes, recipient data, campaigns, and incident containment.

Week 3: exceptions and runbooks

Implement the three alert states. Write first actions for authentication failure, mailbox restriction, hard bounce, provider policy response, complaint, and relevant blocklist evidence.

Week 4: controlled test

Simulate a non-destructive exception, such as a test cohort with a known disabled connection. Confirm the monitor detects it, routes it, preserves evidence, and closes with a documented result.

Only then decide whether another paid monitoring product fills a real gap.

Six exception playbooks

Alerts become useful when the first response is already defined.

SPF, DKIM, or DMARC changes from pass to fail

Scope the problem to a domain and sending path. Query authoritative DNS, then inspect the headers of a new external message sent through the same application.

Compare the current result with the known-good sample. Look for a DNS migration, provider change, new gateway, altered From domain, different return path, or Microsoft/Google tenant change.

Contain the affected path when the intended identity is no longer authenticated. Do not change the DMARC policy first. Fix the source or alignment problem and prove the result externally.

Hard bounces rise

Split the response classes. If mailbox-not-found failures increased, pause the affected list source, suppress permanent failures, and sample the records against the original data.

If the errors are provider policy responses, keep them out of the invalid-recipient bucket. Route them to the infrastructure and campaign owners with the exact text.

Ask what changed in the audience, enrichment provider, verification date, catch-all rule, or import. Rechecking the same poor list with a different verifier may produce different labels without repairing the source.

Gmail performance changes

Separate Gmail recipients and the affected domains. Inspect Google Postmaster data where eligible, the full SMTP responses, external authentication, complaint movement, and the campaign change log.

Compare against a stable domain or campaign using enough of the same setup to be informative. Do not assume a Gmail-only change is a universal sender failure. Do not assume missing Postmaster data means nothing happened.

Microsoft deferrals increase

Preserve the enhanced status codes and full response text. Split consumer Microsoft domains from Exchange Online organizations where the evidence allows it.

If you control the sending IP, review current SNDS/JMRP evidence and map the IP to traffic. If Microsoft or another provider controls the transport, use the account, tenant, message trace, provider notice, and support paths available to you.

Check whether a new audience, tracking dependency, sending pattern, or mailbox cohort is concentrated in the failure.

One mailbox stops while its cohort continues

Inspect the connection, credential or token state, mailbox license, provider restriction, and last successful event. Test the provider-supported connection path without changing the other mailboxes.

Pause that mailbox. Do not replace the domain because one account is restricted. Record whether the failure belongs to the sequencer connection, the mailbox provider, or the user account.

Several domains move at the same time

Look for a shared dependency: sequencer deployment, SMTP pool, tracking domain, DNS host, list source, template, or operational change.

The domains may appear independent in the campaign UI while sharing the same failure underneath. Use the inventory to identify the common layer and keep an unaffected control outside it.

Decide what software must do

Once the manual system works, software should reduce collection and routing work.

A deliverability-monitoring product or internal dashboard should be able to:

• Preserve full SMTP responses.
• Group results by recipient provider and sending identity.
• Distinguish unavailable data from zero.
• Store provider, source, and observation time.
• Link a signal to the infrastructure inventory and change log.
• Show volume beside every rate.
• Route an exception to a named owner.
• Maintain client and business-unit access boundaries.
• Export the evidence required for a provider-support case.

It should not flatten all signals into one unexplained score or claim universal inbox placement from a small seed list.

Build versus buy

Use existing sequencer, provider, DNS, analytics, and CRM data before building a custom platform. A small operation may only need a daily exception export, weekly review, and change log.

A custom pipeline becomes more reasonable when several clients or business units share the method, the same manual joins repeat every week, and the team can own ingestion failures and schema changes.

Buying software is reasonable when it covers a specific missing source or operational job. Test it with known events. Confirm which providers, domains, IPs, mailboxes, and destinations it can observe.

The purchase decision should name the manual action being removed. “Better visibility” is too vague.

Measure the monitoring system

The monitoring process needs its own quality checks:

Do not optimize for the number of alerts closed. An operator can close noise quickly while a real failure remains hidden in a blended average.

Review one incident each month from signal to outcome. Ask whether the system identified the right layer, whether the first action helped, and what evidence was missing. Update the runbook—not only the dashboard.

Monitoring is successful when it changes the right decision early. It does not need to predict every mailbox provider’s private filtering model.

Sources

• Email sender guidelines — Google Workspace Admin Help
• Postmaster Tools dashboards — Gmail Help
• Set up Postmaster Tools — Gmail Help
• Smart Network Data Services — Microsoft
• Junk Mail Reporting Program — Microsoft
• Email authentication in Microsoft 365 — Microsoft Learn