Survey Research and Questionnaires
Survey research is a commonly used method of collecting information about a population of interest. There are many different types of surveys, several ways to administer them, and many methods of sampling. There are two key features of survey research:
- Questionnaires -- a predefined series of questions used to collect information from individuals
- Sampling -- a technique in which a subgroup of the population is selected to answer the survey questions; the information collected can be generalized to the entire population of interest
The two most common types of survey questions are closed-ended questions and open-ended questions.
- The respondents are given a list of predetermined responses from which to choose their answer
- The list of responses should include every possible response and the meaning of the responses should not overlap
- An example of a close-ended survey question would be, "Please rate how strongly you agree or disagree with the following statement: 'I feel good about my work on the job.' Do you strongly agree, somewhat agree, neither agree nor disagree, somewhat disagree, or strongly disagree?"
- A Likert scale, which is used in the example above, is a commonly used set of responses for closed-ended questions
- Closed-ended questions are usually preferred in survey research because of the ease of counting the frequency of each response
- Survey respondents are asked to answer each question in their own words
- Responses are usually categorized into a smaller list of responses that can be counted by the study team for statistical analysis
Considerations for Designing a Questionnaire
- It is important to consider the order in which questions are presented. Sensitive questions, such as questions about income, drug use, or sexual activity, should be put at the end of the survey. This allows the researcher to establish trust before asking questions that might embarrass respondents. Researchers also recommend putting routine questions, such as age, gender, and marital status, at the end of the questionnaire
- Double-barreled questions, which ask two questions in one, should never be used in a survey. An example of a double barreled question is, "Please rate how strongly you agree or disagree with the following statement: 'I feel good about my work on the job, and I get along well with others at work.'" This question is problematic because survey respondents are asked to give one response for two questions
- Researchers should avoid using emotionally loaded or biased words and phrases
Visit the following websites for more information about questionnaire design:
Glossary terms related to questionnaire design:
Surveys can be admininistered in three ways:
- Through the mail
- Advantage: Low cost
- Disadvantage: Low response rate
- By telephone
- Advantages: Higher response rates; responses can be gathered more quickly
- Disadvantage: More expensive than mail surveys
- Advantages: Highest response rates; better suited to collecting complex information
- Disadvantage: Very expensive
Visit the following website for more information about survey administration:
Glossary terms related to survey administration:
One of the primary strengths of sampling is that accurate estimates of a population's characteristics can be obtained by surveying a small proportion of the population. Four sampling techniques are described here:
Simple Random Sampling
- Simple random sampling is the most basic form of sampling
- Every member of the population has an equal chance of being selected
- This sampling process is similar to a lottery: the entire population of interest could be selected for the survey, but only a few are chosen at random
- Researchers often use random-digit dialing to perform simple random sampling. In this procedure, telephone numbers are generated by a computer at random and called to identify individuals to participate in the survey
- Cluster sampling is generally used when it is geographically impossible to undertake a simple random sample
- Cluster sampling requires that adjustments be made in statistical analyses
For example, in a face-to-face interview, it is difficult and expensive to survey households across the nation. Instead, researchers will randomly select geographic areas (for example, counties), then randomly select households within these areas. This creates a cluster sample, in which respondents are clustered together geographically.
- Stratified samples are used when a researcher wants to ensure that there are enough respondents with certain characteristics in the sample
- The researcher first identifies the people in the population who have the desired characteristics, then randomly selects a sample of them
- Stratified sampling requires that adjustments be made in statistical analyses
For example, a researcher may want to compare survey responses of African-Americans and Caucasians. To ensure that there are enough Afrian-Americans in the survey, the researcher will first identify the African-Americans in the population and then randomly select a sample of African-Americans.
- Common nonrandom sampling techniques include convenience sampling and snowball sampling
- Nonrandom samples cannot be generalized to the population of interest. Consequently, it is problematic to make inferences about the population
- In survey research, random, cluster, or stratified samples are preferable
Visit the following websites for more information about sampling procedures:
Glossary terms related to sampling procedures:
Measurement error is the difference between the target population's characteristics and the measurement of these characteristics in a survey. There are two types of measurement error: systematic error and random error.
- Systematic error is more serious than random error
- Occurs when the survey responses are systematically different from the target population responses
- For example, if a researcher only surveyed individuals who answered their phone between 9 and 5, Monday through Friday, the survey results would be biased toward individuals who are unemployed
- Sources of bias include
- Nonobservational error -- Individuals in the target population are systematically excluded from the sample, such as in the example above
- Observational error -- When respondents systematically answer surveys question incorrectly. For example, surveys that ask respondents how much they weigh will probably underestimate the population's weight because respondents are likely to underreport their weight
- Random error is an expected part of survey research, and statistical techniques are designed to account for this sort of measurement error
- Occurs because of natural and uncontrollable variations in the survey process, i.e., the mood of the respondent
For example, a researcher may administer a survey about marital happiness. However, some respondents may have had a fight with their spouse the evening prior to the survey, while other respondents' spouses may have cooked the respondent's favorite meal. The survey responses will be affected by the random day on which the respondents were chosen to participate in the study. With random error, the positive and negative influences on the survey measure balance out.
Visit the following website for more information about measurement error:
Glossary terms related to measurement error:
Nonresponse Rate Bias
Ethics of Survey Research
Respondents should give informed consent before participating in a survey. In order for respondents to give informed consent,
- The researcher must inform the respondents of the study's purpose, content, duration, and potential risks and benefits
- The researcher must inform the respondents that they do not have to answer all the survey questions
- The researcher must inform the resondents that they can stop participating in the study at any point
Confidentiality and Anonymity
It is absolutely imperative that researchers keep respondents' identities confidential. To ensure confidentiality, researchers should not link respondents' identifiers to their survey responses when using data. Common identifiers include names, social security numbers, addresses, and telephone numbers.
Anonymity is an even stronger safeguard of respondent privacy. If a researcher assures anonymity, it means that the researcher is unable to link respondents' names to their surveys.
Visit the following websites for more information about anonymity:
Glossary terms related to ethics:
Advantages and Disadvantages of Survey Research
- Sample surveys are a cost-effective and efficient means of gathering information about a population
- Survey sampling makes it possible to accurately estimate the characteristics of a target population without interviewing all members of the population
Survey sampling is particularly useful when the population of interest is very large or dispersed across a large geographic area.
- Surveys do not allow researchers to develop an intimate understanding of individual circumstances or the local culture that may be the root cause of respondent behavior
- Respondents often will not share sensitive information in the survey format
- A growing problem in survey research is the widespread decline in response rates
2. A simple thank you
Power up your survey introduction with a thank you note.
Hey, it’s not only a sign of appreciation, it’s the least you can do!
Your participants are giving up their time for you to benefit from.
They are not gaining anything from doing this (except maybe your super relevant incentive). Try your best to make this experience as human and “spontaneous” as possible by adding a personal touch, especially by thanking your respondents.
(Thank you, by the way. You’re doing an awesome job reading all the way to the end… )
Use this template: We personally want to thank YOU for every second invested in our research. You rock!
If your company and research allows it, make this even more personal.
Give your brand a face.
Adding a real person’s name works just as well in surveys as it does in a newsletter, blog post or podcast.
Use this template: Thanks for helping us out. From all of us at [company name], [your name] [your title (optional)]
3. Less is more
Take everything you’ve learned in the previous steps…
And now scratch that!
Because a little less will do.
We recommend to always write your survey introduction text as short as possible.
… Sorry, I know!
People – especially those in a hurry – don’t want to waste their time reading page-long introductions before finally being able to start your survey. Use only the essentials from the above tips. Then go ahead and just make your point.
By writing up your survey introduction as short as possible, you force yourself to only focus on the most important message. And you don’t waste respondent’s time even before they’ve taken the survey. Getting them in is what’s most important.
4. Inviting atmosphere is key
The first step for your introduction is making sure people will enter your survey and answer questions. Your second priority, is making sure they’re honest.
Here’s how to get that done:
Honesty:“You get what you give. What you put into things is what you get out of them.” – Jennifer Lopez.
Don’t expect your participants to blindly answer in a truthful way if they don’t know the full picture surrounding your study. Offer all corresponding information from the very beginning to avoid sloppy data which could lead to “brand dilution”.
Neutrality: Try to remain neutral throughout your entire survey, not just your introduction. You often see companies using one liners like “leading company in our niche” or “Bringing you the best service”. Don’t do this! As it will only create confusion and prejudice instead of confidence and reliable data.
Now let’s summarize, and get to the good stuff:
The perfect survey introduction example
Followed all of the steps above? Nice!
You will have a survey introduction that is perfect and by the book!
Does it look like it got a bit too long?
Too much info?
Simply not as appealing as you thought it would be?
We wrote up a short, generic and to-the-point version for you to use. The perfect survey introduction example:
Use the ultimate template:
Hey, glad to see you around here!
First of all, let me thank you for taking our [survey duration] survey. You are a great help! 🙂
We at [company name] are on a daring quest to collect the right data about [survey subject]. Our target audience involves everyone who [target audience]. This is why we chose you!
And don’t worry, your data is just for [where you will use it] ]so [be clear about their privacy]. We promise!
– Get started and take your chance to WIN [a grand prize]
It’s most important to make this introduction represent you as a brand, organization or person.
It’s the first step of starting up a conversation.
And don’t be afraid to entertain: Don’t bore, get more. 😉
Extra tips & inspiring introduction examples
1. Increase brand recognition
Hey it’s you!
By adding your logo at the top of your intro screen, you’ll increase brand recognition without having to push it forward during the entire survey.
Make people feel like they’re talking to an old friend.
2. Use a conversational tone
Most people still associate surveys with these boring tasks that are basically – let’s face it – a waste of their time.
Spice up your language and bring some humanity into your questions.
Making your surveys more conversational will benefit your participation and completion rate tremendously!
EXTRA: If your brand and tone of voice allow it, throw in an interjection here and there. A “yee-hah” to show joy or an “ooh-la-la” to let respondents know they can win a prize?
Works like a charm.
Take a look at this list of interjections, for exclamations in every kind of situation.
3. Turn a frown upside down with emoticons
Go back 15 years in time and nobody would even THINK of using a smiley face.
Well, that period is over and now it’s totally fine.
In fact, did you know that the use of emoticons in your communication increases the empathy towards your brand?
Perfect to express the mood of your survey, if your message is a playful one.
4. What’s in a name? Don’t use the word “survey”
Long, boring, difficult, too much work, … the word “survey” brings out some awful associations. We cannot blame our respondents. Instead, ask people to answer “a few questions” or to “spare a minute of their time”.
By avoiding the actual word you’ll see an increase in clicks and actual responses. This works in the introduction screen of your questionnaire, but in your email invites as well for example.
Keep in mind though that you should stick to just 4 questions if that’s what you said!
5. Show off the (incentive) goodies
A reward for answering a couple of questions lowers the threshold tremendously! Even more so if there’s a game or contest connected to the reward. The thrill of playing and possibly winning something is a perfect addition to your survey introduction.
6. In all seriousness
There’s a time and place for everything, so if your questionnaire is much too serious for smileys or “whoopees”, you can still write up a longer introduction that eases the respondent into the setup.
A competency assessment, like the above example, is something that requires more information. They managed to put quite a lot of it in the introduction, but decided to refer to an informative page via a hyperlink in case an employee would like to read up before getting started.