How Takeout Improved Our Candidate Experience: A HubSpot Experiment

Here at HubSpot, we take culture pretty seriously. After all, we have an entire code dedicated to it, and it doesn’t just apply to our internal environment -- it also shows up when we’re recruiting new people to join our team. We have an inbound recruiting mission of attracting top talent through a world-class candidate experience.

That’s why, one January night that started like any other -- watching Netflix in my pajamas and eating chicken tikka masala from my favorite Indian takeout joint -- I decided to respond to a mobile customer satisfaction survey from the food delivery platform that I use, called Grubhub.

I had been thinking a lot about mobile, and how it could play a role in our inbound recruiting efforts at HubSpot. So, when I got an automated text that night saying "Grubhub here! Tell us about your order from India Quality Restaurant," I wanted to know: How could we recreate that kind of seamless experience for HubSpot’s candidates?

So read on -- and find out how takeout inspired our approach to recruiting and interviewing.

How Takeout Improved Our Candidate Experience

The Hypothesis

We all know that texting for business is nothing new. You might get a text message when you pay your wireless, make a hair appointment, or confirm a time slot at the dentist. But texting hasn't played a role in HubSpot's recruiting and hiring process since its earliest days, when developer recruiting was essentially managed on one VP's smart phone.

Eleven years later, whether or not a candidate receives or accepts an offer to work at HubSpot at the end of her interview process, we want her to enjoy their time with us. That experience has a big impact on whether or not candidates advocate for HubSpot in the future, the Glassdoor reviews they leave, and the likelihood that they refer friends or pursue future opportunities with us. That’s why we ask for feedback -- so we can learn how we can improve. And until recently, we used what we called a Net Promoter Score survey (NPS), that was distributed via email.

But when you consider that, today, people spend more time browsing on mobile than they do on desktop, we couldn’t help but wonder if following up with candidates via text, instead of emailing them for feedback, would make it that much easier and engaging for them to actually respond.

So, we came up with a bit of a unique hypothesis: If our candidate NPS survey was more like Grubhub's, and the survey was sent via mobile instead of email, response rates would increase. The objective, then, was to get a higher volume and quality of feedback that could enable us to even better improve the recruiting experience.

The Experiment

The Candidate NPS Survey Today: Email

When candidates globally have a face-to-face interview with HubSpot, they receive an automated email at 7:00 PM that evening with a link to take a short survey with three questions:

1. On a scale of 1-10, how likely are you to recommend HubSpot to a friend based on your experience?
2. Which department did you interview with?
3. Anything else you'd like to share with us?

From there, my colleague Danielle McLellan analyzes the results and synthesizes the feedback, so the recruiting team has some tangible insights into what is (not) working well.

For context, the response rate to date on the email survey hovers around 55%.

The Introduction of Mobile

There was just one hiccup with implementing my mobile survey distribution idea: I'm not mobile development savvy, so executing an automated text survey was pretty foreign. But, I could still perform research with the best of 'em, so I gathered some information on mobile survey vendors, and chatted with HubSpot's developer team about the feasibility of two options:

1. Buying the tech to create this feedback system.
2. Building that tech internally for the same purpose.

We ended up going with a scrappy, but reliable, solution that combined two platforms: Textit.in + Twilio.

Textit.in is a mobile messaging platform with pretty intuitive usability. You build a visual workflow of the desired text series, upload contacts' phone numbers, and schedule the workflow to start at a certain time and date. While it's not highly sophisticated, for the purposes of this experiment, it provided the solution we needed.

Twilio, a cloud communications platform, provided us with a virtual phone number that could easily be connected to Textit.in. All in all, this technology cost us about $60, showing how frugally something like this experiment could be replicated by other marketers.

Only candidates interviewing in our Cambridge office received the mobile survey, as establishing an international virtual phone number would have required additional steps that, considering our deadline, time simply didn't permit. That said, we had a population of roughly 220 Cambridge candidates to work with.

The Process

We ran the experiment from February 13 - March 31. Once the technology was up and running, here's how it worked:

1. Collect Phone Numbers. Our recruiting coordinators added candidates' mobile phone numbers to a shared Google spreadsheet by 4:00 PM daily, with information like the candidate's full name, date of her face-to-face interview, and the department she interviewed with.
2. Schedule Text Workflow. I then imported that information into Textit.in, and set up a workflow to have that group of contacts receive the mobile feedback survey via text at 7:00 PM on the evening of their respective interviews. To make sure we didn't change any variables aside from the method of communication, we kept the following variables consistent across all messages:
   • Time of send.
   • The series of questions asked of candidates, with the exception of one trigger question: "Hi, HubSpot CareerBot here! Thanks for interviewing today, we'd love your feedback. Will you answer a few quick questions about your experience? Y/N". (By the way -- shout out to my colleague Noah Gilman for coming up with the "CareerBot" name.)
3. Collect results. I then created another spreadsheet to track the following results:
   • Who did (not) respond to the survey.
   • Each person's NPS score.
   • Any open-ended comments.
4. Analyze results. McLellan analyzed those results to look for patterns and other actionable outcomes.

For the sake of privacy, the above example isn't one from an actual candidate, but one that I made up to illustrate how the system works -- though, the part about Dave's beard is true.

Results

As predicted, the volume of responses to the mobile surveys outnumbered those from email considerably. Even better: We saw an improvement to our percentage of promoters and the overall candidate NPS score via this medium.

And thanks to the higher volume of responses, we also were able to gain better insights from more open-ended comments. We observed that, no matter what the medium, our recruiters were receiving high praise from candidates, and -- although, yes, I'm biased -- the best part of this experiment was getting to read that feedback. Here are a few of our favorites:

Julia Blatt and Kelsey Freedman were awesome coordinators and made my day great! Everything was timely and went smoothly. Thanks to Becky, Gus and Amy!"

The entire interview experience was amazing. The recruiter was flexible to accommodate my scheduling requests at every stage of the interview. I was well informed on each stage of the interview which certainly helps the preparation. The follow ups have been prompt and timely."

My recruiter, Noah, has done an absolutely amazing job throughout the process. Very transparent and informative. Send him my thanks."

(P.S. We're hiring.)

Next Steps and Takeaways

Across the board, the mobile results indicate an improved follow-up experience for candidates -- and that candidates are more likely to give us feedback via mobile. And based on these positive results from the experiment, we know there's a place for mobile in our recruiting process ... and that there could be in yours, too. Remember the statistic from above about how many people are browsing on mobile than desktop? Keep that in mind next time you're looking to improve a user experience.

We'll continue to carry out additional tests of this kind to collect more directional benchmarks -- things like language modification, timing, and scalable technology. Also, we'd like to expand the experiment to our global offices and will be formulating a timeline and corresponding plan to do so.

If you're curious about what else is going on in our world of inbound recruiting, check out the Move On Up blog, which gives readers a peek inside culture and careers at HubSpot.

And in the meantime -- I would highly recommend India Quality Restaurant to a friend.

How has your team enhanced its recruiting experience? Let us know about your best experiments in the comments --and hey, we might even feature it on our blog.