How to Design Customer Surveys That Lead to Actionable Insights

Last month’s metrics came in, and you’re not exactly happy.

Maybe the metrics are improving too slowly for you to reach your goals. Perhaps they’re stagnant. Or, *gasp* they’re getting worse.

You ask around your team or experts or consultants, but you want more.

You want the truth.

You know that optimizing starts with the people who actually use your store. You also know that quantitative analytics will only get you so far. More than bounce rates, page views, CTRs, and sales data, you need real, human feedback.

And so -- with the best of intentions -- you think through a few key areas of your business: maybe with your team, maybe all alone.

You settle on 15 questions -- some about your website, some about fulfillment, and some about the products themselves.

You input them into your email service provider’s survey creator and hit send.

Then … crickets.

It’s a tale as old as ecommerce itself.

You need the kind of deep insights qualitative research provides, but some surveys can yield notoriously low completion rates and even more notoriously unclear (23.4% customers agree that our shipping is below average) responses.

To help you overcome both, let’s take a look at the three sets to designing a customer survey that people will actually fill out: (1) who, (2) what, and (3) how.

1. Who Should You Survey?

The question of who to survey largely depends on what you want to learn.

If you want to make improvements to shipping, survey customers who bought in the last 30 days. If you want to launch a loyalty program, survey frequent buyers.

The foundation for a rock-solid survey strategy should always be built on a segmented customer database.

Having this segmentation in place enables you to ask targeted questions and get more out of your surveys. What’s especially important is the ability to segment based on last order date. Think about it — a person who shopped a week or month ago has much more useful and relevant feedback than someone who hasn’t visited your store in over a year.

Depending on what you’re trying to learn, your survey segments could include:

• First time customers
• Frequent buyers
• Infrequent buyers
• Longtime customers
• Big spenders
• Based on demographic data (age, sex, location, occupation etc)
• Interests
• On-site browsing history
• Registered non-buyers

Whichever segment(s) you choose to run a survey on, every question in there has to have a well-defined purpose and be relevant to the customers giving answers.

2. What Questions Should You Ask Participants?

Open vs. Closed Questions

In surveying, there are two main types of questions:

Closed-ended questions solicit yes or no, right or wrong, true or false responses. They also include multiple choice answers, rating scales, and rankings. “Did you receive your product on time?”

Open-ended questions, on the other hand, cannot be answered with a simple yes or no. They require the responder to think and formulate an answer in their own words. “What did you think of the the unboxing experience?” is open.

Because of this, open-ended surveys yield more substantive results. At the same time, they also tend to have lower completion rates and take more time than their closed counterparts.

Thankfully -- as we’ll see in just a moment -- ecommerce surveys lend themselves to a healthy combination of open- and closed-ended questions.

Good vs. Bad Questions

Whoever said, “There’s no such thing as a stupid question,” wasn’t talking about qualitative research. Good questions generally follow two rules:

Be Concise

Keep questions short and ask one question at a time. The language used should be simple in both words and phrases - everyone has to be able to understand the question asked without hesitation

To achieve that, avoid technical jargon, sentences that take longer than a single breath to read out loud, and negative -- as well as double negative -- expressions.

As for the length of your survey, SurveyMonkey discovered, “On average … respondents take just over a minute to answer the first question in a survey (including the time spent reading any survey introductions) and spend about 5 minutes in total, answering a 10 question survey.”

In other words, the fewer questions you include, the more thorough and accurate the responses will be.

Be Objective

You should be on the lookout for questions that lead towards a particular answer and get rid of them as soon as you see them.

As SurveyMonkey’s classic illustration goes: instead of saying, “We have recently upgraded SurveyMonkey’s features to become a first class tool. What are your thoughts on the new site?” instead, ask: “What are your thoughts on the upgrades to SurveyMonkey?”

Also, eliminate emotionally charged words and value judgments. Help Scout has a great example of this kind of question being used in deciding disciplinary action with kids: “Should a smack as part of good parental correction be a criminal offence in New Zealand?" Ironically, the question assumes that smacking is an inherent part of “good” parental correction. Results to loaded questions will inevitably be skewed.

Lastly avoid universal words -- like “always”, “never”, “only”, and “just” -- as well as imprecise words -- like “often,” “usually,” “generally.” For one person, the word “often” could mean once or twice a week, while for someone else it could be once or twice a month.

Importance of Sequencing

The sequencing of questions can seem like such a minor detail, but it's one that could significantly alter the number of replies you get.

In a properly designed survey, the questions asked increase progressively in difficulty. In a way, the easier questions create a "priming" effect for the more difficult ones.

Questions regarding demographics and personal information are a good example of the kind of more "difficult" ones that are best to be asked towards the end of survey when respondents have had time to develop confidence in the survey (and your company) to give those kind of more personal answers.

Additionally, all surveys should always try to stimulate recall and flow in an orderly fashion. Want to ask questions about shipping? Those questions should be among the first ones as shipping is the last step in the ordering process and thus is easiest to recall and give relevant answers to.

Sample Survey Questions

While the goals for launching an online survey vary greatly, there’s no need to start from scratch. Here’s a slightly adapted list of questions ConversionXL -- one of the most trusted names in optimization -- uses with their clients:

• How did you find out about [blank]?

• Why did you buy [blank]?

• What are you using [blank] for?

• Have you recommended [blank]?

• Why or why not?

• Did you consider any alternatives to [blank]?

• If so, who or what?

• What doubts did you have before completing the purchase?

• What questions did you have prior to purchase?

• Were you able to find answers?

• Anything else you would like to tell us?

3. How Should You Send Your Survey Out? Also, What To Write.

For any online merchant, or really any merchant in general, the most logical way of distributing surveys is using email. If your customers are already used to getting emails from you, they are more likely to respond. 

The first thing to consider is the email subject line. Keep it concise and either add an incentive upfront or a personal touch of curiosity, goodwill, and humor:

• Have 30 seconds? We need your feedback!
• “You can’t handle the TRUTH!” (but we can) …
• Way more than a penny for your thoughts [$20 gift card]
• 5 questions, 5 minutes, 5 ways to win
• Can you do me a favor?
• I’d ask my mom … but I think she’d sugarcoat it

As for the main body of the email, think of it like a birthday invitation: friendly, with all the relevant details. For a survey, those details include (1) what it’s about, (2) how long it takes to complete, and (3) the incentive.

Once you feel like you’re ready to send out your survey … don’t.

Instead, test both the email and the survey with a few participants. Those can be a small group of customers or anyone outside your organization willing to be ruthlessly honest with you. This preliminary round will provide valuable feedback and bring out any ambiguities with either wording or design as well as ensure it actually works.

After you’ve refined your survey based on their feedback, send it to the people you’d planned to.

For inspiration, here’s how Amazon’s survey request looks:

And here’s one from Airbnb:

Image via Promoter.io

Final Thoughts

Done right, customer surveys can deepen relationships with your customers instead of ending them. They often reveal existing problems and opportunities as well as unlock your company's growth.

We all need the kinds of deep insights that qualitative research provides.

And the best way to overcome the challenges of low-response rates and bad data is to start by understanding (1) who you’re reaching out to, (2) what you’re asking, and (3) how to start a conversation with them.