What Happened to Our Metrics After We Stopped Sending So Much Email

Last fall, my teammate, Pamela Vaughan, and I made a terrifying and exciting decision. We were on a big mission to reduce our graymail, and we decided to do something drastic. 

We created a workflow that initially unsubscribed 250,000 people from our emails -- which was roughly half of our list -- and would continue to unsubscribe unengaged people over time. Plus, we completely eliminated the option to receive instant notification emails from us. 

Though we had done some preliminary number crunching to estimate the impact of these changes, we weren't 100% sure how it would go. Soon after we announced the changes, we got some great pieces of qualitative feedback, such as:

You’re fabulous. Thanks for giving me some breathing room. And yes, I can use it."

And this:

I think this is a good move. I like HubSpot, and think that you provide a great product/service, but I'm feeling very bombarded by you guys to be honest."

But only time would tell how this whole plan would affect our numbers. 

So we waited. And waited. 

Today, the waiting is over. Today, we have our answer.

Quick Refresher: A Visual Recap of What Changed

Before we enacted these changes, here's roughly how our blog email subscriptions worked. 

You signed up to get email notifications from us on an instant, daily, or weekly cadence. We'd send you those emails until you decided to manually unsubscribe. The end. 

We did this for years. So our list was big ... but had a large portion of old email addresses that didn't work anymore and overwhelmed subscribers who tuned us out. Not only is that a really bad experience for our subscribers, but it's also not good news for the health of our email deliverability. 

Now, here's how it works. 

You sign up to get email notifications from us on a daily or weekly cadence. We send you emails for six months. If you click on an email in those six months, you stay subscribed. If you don't, we send you an email letting you know we're going to unsubscribe you. No harm, no foul. (And any time in this process, you can also manually unsubscribe from our emails.)

The goal with the changes was to reduce our graymail and stop bombarding our subscribers with emails. But it turns out, this new strategy had a big impact on our metrics, too. 

What Happened to Our Metrics After We Stopped Sending So Much Email

1) Traffic from our email notifications increased.

Despite sending fewer emails, traffic from our email notifications went through the roof -- views grew by 21% between October 2015 (a record traffic month for us and the month before we implemented the changes) and Jan 2016 (when traffic is usually back to normal after the holidays).

This means our emails are working better, right?

Well ... kind of. Traffic to the blog from our email notifications gives us an indication of how these emails are doing, but it's not a full picture of how well our emails themselves are doing.

For that full picture, we had to turn to opens and clicks (and their corresponding rates) for our blog notification emails.

2) Our blog email notification opens, clicks, open rates, and clickthrough rates (mostly) increased.

To gauge the health of our blog notification emails, I pulled the following numbers for all daily and weekly emails sent between October and February:

• Opens per email send
• Clicks per email send
• Open rate (opens/delivered) per email send
• CTR (clicks/delivered) per email send
• The number of emails delivered

To reduce the variability in the day of the week that the daily emails are sent, I averaged all the stats for the daily emails in a given week. (The weekly email subscriptions didn't need the same treatment -- they're sent the same day every week.)

Then, I compared the stats of daily and weekly emails sent in early October to those sent in early February. Here's what we found. 

Note: In the graphs below, I've added the dotted line at the date we implemented those changes so you can better gauge the effects.

For the daily email notifications, our opens, open rates, clicks, CTRs, and emails delivered all went up. 

• Number of Emails Delivered: +6.9%
• Number of Opens: +24.1%
• Number of Clicks: +43.4%
• Open Rate: +16.1%
• CTR: +34%

Our weekly emails' metrics, on the other hand, experienced some mixed results. The number of emails delivered and opened went down, but the rest were much higher than before (especially our CTR).

• Number of Emails Delivered: -48.2%
• Opens: -18.8%
• Clicks: +45.7%
• Open Rate: +57.4%
• CTR: +177.3%

So what do these metrics tell us? Did this strategy actually work? 

Is This Strategy Working?

In short: yes. 

Most of our metrics are up and to the right, but I'd like to draw your attention to two of the most important: Clicks and CTR.

Because both of these metrics are higher than before for both email types, we're doing something right here. If we only increased CTR, it could simply be because we're delivering to fewer people. Seeing that the volume of clicks also grows between October and February despite having way smaller list sizes than before means that we're sending the right people the right content at the right cadence. 

No longer are we padding our metrics with vain, inflated subscribers who ultimately harm our deliverability and email performance. Instead, we're keeping our email list smaller, but way healthier and more effective. 

But I know what you're thinking: What about the huge drops in opens and delivers on the weekly email subscription? Aren't you worried about that? 

Truthfully ... I'm not. 

Delivers is down because we unsubscribed way more people from this list when the workflow first kicked off. And the reason why we unsubscribed more people from this list is because the way most people become subscribers is through a checkbox on our landing pages. It's effective at acquiring new subscribers, but may not be great at acquiring engaged subscribers. 

As for opens, I also attribute that to the unsubscribe campaign, too. We used clicks as a way to measure engagement (it's more reliably tracked than opens), so it's very possible we unsubscribed people who opened our emails but never clicked on them.

All of those things considered, I am optimistic that this new approach is better -- but as always, we're going to keep an eye on this and make changes if necessary. We'll keep you posted. ;)

What's Next?

The unsubscribe workflows are here to stay, and our instant list is going to stay firmly in the trash.

That said, we aren't going to stop trying to find better ways to keep our subscribers in the loop on what's been published. These changes are only the beginning to building a smarter, more lovable blog subscription strategy.