Student Booklet
CREDITS
3
LEVEL
2
Questionnaire Design
SORTED THEMES
Savings, KiwiSaver, Investing
AS91263 (version 3)
Mathematics and Statistics
Design a questionnaire
Te waihanga i te uiui
pāngarau
Contents
1
Topic One:
The purpose and audience of questionnaires..................... 04
2
Topic Two:
Defining the target and sample populations....................... 12
3
Topic Three:
Ethical considerations...................................................................
19
4
Topic Four:
Question types.................................................................................
26
5
Topic Five:
The importance of good question design............................
36
6
Topic Six:
Structuring your questionnaire.................................................
44
7
Topic Seven
Doing a desk review.......................................................................
50
8
Topic Eight:
Doing a pilot survey.......................................................................
54
9
Topic Nine:
Reflecting on the process............................................................
60
02
Sorted in Schools AS91263 | Student Booklet
Nau mai haere mai!
Welcome to the Questionnaire Design module.
Please read through the Student Guide for an overview of the
module and assessment before starting this Student Booklet
and your questionnaire design.
03
Sorted in Schools AS91263 | Student Booklet
Topic One:
The purpose and audience
of questionnaires
Learning outcome for Topic One
Understand data collection through questionnaire design
Success criteria
• I can give at least two examples of the types of information
people can gather using a questionnaire
• I can explain at least two strengths of questionnaires
as a research tool.
1
04
Sorted in Schools AS91263 | Student Booklet
This topic focuses on understanding why questionnaires
are a useful tool for gathering data.
Imagine that your school is thinking about a change that will impact on
the whole school community, for example, making changes to the school
timetable, introducing a new uniform, or implementing a new approach to
teaching and learning. Who should the school involve in that decision?
Why? And how can the school find out what dierent groups of people
think about the proposed change?
To find out what people think about the proposed change, the school needs
to gather data. There are two main types of data: quantitative and qualitative.
Quantitative data is data that involves
numbers, for example, things that you
can count or measure. Questions that
gather quantitative data often start
with sentence starters such as how
many, how much, or how often. One
advantage of quantitative data is that
it is easy to analyse.
Qualitative data describes qualities
or characteristics; for example, gender,
cultural identity, or opinions. Qualitative
data can be hard to analyse in a precise
way, but it’s an important and rich part
of research.
s o r t e d
05
Sorted in Schools AS91263 | Student Booklet
Ways to gather data
There are several ways to gather data
about people’s thoughts, feelings, and
opinions.
For example:
The school might set up focus groups,
inviting small groups of people to
discuss an idea.
Another option is one-to-one interviews,
which involves asking people a series of
questions face-to-face. These methods
are useful for gathering in-depth
information about what people think or
feel about an idea. However, they can
also be very time consuming and, as a
result, the number of people invited to
contribute ideas tends to be small.
Questionnaires (a set of written
questions) are a useful way to gather
data from large groups of people.
They can provide statistical information
about people’s behaviours and
circumstances, and provide ways for
people to share their beliefs, attitudes,
and opinions; although usually with
less depth than focus groups and
one-to-one interviews.
Complete Topic 1 Activity 1
in the Student Practice Booklet
06
Sorted in Schools AS91263 | Student Booklet
Plan
Identify what you
will measure and how
you will measure it,
plan how to gather
data.
Problem
Define a problem
and write an
investigative
question.
Data
Collect data that will
help you to answer
your investigative
question.
Analysis
Explore the
data using graphs
and statistics;
make connections
between your data
and the context.
Conclusion
Answer your
investigative question;
identify further
questions;
communicate
your findings.
Statistical Enquiry
Cycle
Questionnaires – getting started
Coming up with a set of questions sounds simple right? But designing a
questionnaire takes a lot of thought, time, and testing. If you don’t ask the
right questions in the right way, you may not get the information you need.
More than that, poorly designed questions can make people feel oended,
annoyed, or distrustful.
Survey or questionnaire – what’s the dierence?
In everyday use, there’s not much dierence between a questionnaire and
a survey. People use both terms to describe a set of written questions. But
when it comes to research, there is a dierence: a questionnaire is a set of
questions, while a survey is a wider process of sending out a questionnaire,
collecting responses, and analysing the responses to reach a conclusion.
In other words, a survey is an example of the PPDAC statistical enquiry
process in action:
07
Sorted in Schools AS91263 | Student Booklet
Assessment tip:
In your assessment, you’ll mainly be focusing on the first two parts of the
process (problem and plan), although you will be collecting some data to
test your questionnaire and doing a short analysis to check that the data
your questionnaire gathers can be analysed.
Complete Topic 1 Activities 2–6
in the Student Practice Booklet
Establishing the purpose and audience of a questionnaire
The purpose of a questionnaire is to gather information about a particular
context so that you can answer an investigative question or questions. The
audience is the person or group(s) who want to use the information your
questionnaire gathers.
Questionnaires are used by individuals, organisations,
iwi, businesses, and community groups to gather
statistical information that help to inform their decision-
making. Councils and the government use questionnaires
to help plan changes to education, health, housing and
transport services.
Assessment tip:
In your assessment, you need to be very clear about why you are gathering
information, who will use it, and why it is useful. It’s possible that at the
moment your main motivation for creating a questionnaire is just to get
credits. But you’ll find the process of designing a questionnaire much more
interesting and meaningful if you can link it to something you care about or
are genuinely interested in.
08
Sorted in Schools AS91263 | Student Booklet
You need to be able to answer each of these questions:
• What do you want to find out?
• Why is the data you are gathering useful? (For example, how might
it inform a decision or provide feedback on a situation.)
• How can the data be analysed?
• Who do you need to give the questionnaire to to make sure you get
the information that your audience needs? (You’ll be learning more
about this last question in Topic Two.)
To get Excellence, you need to make connections to the purpose of your
questionnaire throughout your report, including how changes you have
made to your questionnaire help it to achieve its purpose. You also need
to show that you have carried out research that relates to the purpose of
the questionnaire.
You might decide to refine the purpose of your questionnaire as you work
your way through the questionnaire design process, for example, if you
discover that there are specific information needs of the survey and the
groups who will use the data.
It may take you a few days to identify
an audience and develop a purpose
for your questionnaire. Checkpoint
One includes a brainstorming activity
that you can use to start coming up
with ideas.
Once you have chosen a context for
your questionnaire, the next step is to
define your research goals (the specific
information you want to gather).
Assessment tip:
When you’re creating a
questionnaire on behalf of an
individual or a group, it’s
important that you talk to them
first to find out what types of
information would be useful for
them. This will help you to clarify
the specific information needs of
the survey and of the groups who
will use the data you gather.
09
Sorted in Schools AS91263 | Student Booklet
Who is CFFC?
CFFC is a government organisation
that works to improve the financial
capability of New Zealanders at all
stages of life. They believe that most
New Zealanders can make smart
decisions about money if they have
the right tools and encouragement.
Their overarching vision is that all New
Zealanders can retire with confidence.
Metiria asks her contact at the CFFC
what types of information they are
looking for. The CFFC is interested in
understanding the financial choices
young people make and the factors
that shape these decisions.
How will the data be used?
The CFFC knows that factors such
as an ageing population and an ever
changing world can present challenges
that will aect young people both once
they start working and into their
retirement. They want to develop tools
that will equip young people to make
smart decisions about money. This is
because improved financial capability
gives young people more choices and
the opportunity to achieve better social
and health outcomes for themselves
and their family and whānau, making
it easier for them to actively participate
in economic life. To make these tools
useful and relevant for young people,
the CFFC wants to gather data on what
factors influence their attitudes towards
money and what their current money
behaviours are.
Here is an example:
Metiria is a year 12 student who
wants to research young people’s
attitudes and behaviours related to
planning for the future. Her audience
is the Commission for Financial
Capability (CFFC).
$
$
10
Sorted in Schools AS91263 | Student Booklet
Before moving on to Topic Two, check that you understand:
What a questionnaire is and why people use them
The strengths and weaknesses of questionnaires as a research tool.
Based on this information, Metiria decides on three research goals:
Complete Topic 1 Activity 7
in the Student Practice Booklet
11
Sorted in Schools AS91263 | Student Booklet
2
Topic Two:
Defining the target and
sample populations
Learning outcomes for Topic Two
Understand target and sample populations through questionnaire design
Understand sampling and non-sampling errors through questionnaire design.
Success criteria
• I can explain the dierence between a census and a sample
• I can identify the target population and sample population of a
questionnaire
• I can explain the dierence between sampling and non-sampling errors.
12
Sorted in Schools AS91263 | Student Booklet
Remember that a key purpose of a questionnaire is to
gather information that is useful for making decisions.
This means that it’s important to think about which individuals or groups your
audience need to hear from. This group of people is called the target population.
Assessment tip:
Throughout your questionnaire design process, it’s important that everything
you do has a clear link to the purpose and audience of your questionnaire.
For example, you need to be able to explain how your target population
matches the purpose of your questionnaire and the needs of your audience.
Defining the target population
The target population is the entire
group that you would like to get
information from and it’s important
to be very specific about who this is.
For example: The target population
for Metiria’s study is:
“Year 12 and 13 students in Aotearoa
New Zealand in 2021”.
This relates to the purpose and
audience of Metiria’s questionnaire
because many of these students will
be thinking about what they want to
do when they finish school. Finding
out about the attitudes and behaviours
of this target audience will help the
CFFC design tools to support young
people as they transition from school
into work or tertiary study.
13
Sorted in Schools AS91263 | Student Booklet
This target population is focused on
school year level not age; the study
is only exploring schools in Aotearoa
New Zealand; the focus is a snapshot
of a particular year.
In an ideal world, Metiria would gather
information from everyone in the
target population. This is called a
census.
Gathering information from everyone in
your target population would make
the data you gather very reliable
(trustworthy). However, in real life this
is seldom possible because:
• It may not be possible to contact
everyone in the target population,
for example, if you are sharing the
questionnaire electronically, people
who don’t have access to the
internet won’t be able to participate
• Inviting everyone in the target
population is often time-consuming
and expensive.
Using a sample population
Because of the challenges of doing a
census most people doing a survey use
a group from the target population.
This group is called the sample
population.
Listing your sampling frame
A sampling frame is a list of the people
that you will take your sample from.
Ideally, the sampling frame will be a list
of everyone in your target population.
For example, if your target population
is students at your school, the sampling
frame would be a list of all the students
at the school.
Assessment tip:
Sometimes there will be people
on a sampling frame who are not
in your target population. When
you create your questionnaire,
it’s useful to have a screening
question at the start to check that
the person filling it meets your
criteria. If not, you can save them
and yourself time by letting them
know that they don’t need to
complete the questionnaire.
MAIL
MAIL
14
Sorted in Schools AS91263 | Student Booklet
Ideally, the people in the sample will
be similar to the population that they
come from. For example, if you
surveyed a sample of the students at
your school about a timetable change,
you would need to make sure that your
sample includes students from all year
levels, students who live far away from
school, and students who have outside-
of-school commitments that might
aect their ability to start school early
in the morning.
Statistical errors
In everyday use, the word “error”
means that a mistake has been made.
In statistics, the word error has a more
specific meaning.
A statistical error is the dierence
between the value you get from a data
collection process and the true value
for a population.
If the error is small, the data you have
gathered accurately represents the
population. If the error is large, the data
you have collected is not very accurate.
No one likes errors, but they are an
accepted part of the survey process.
Next are some examples of statistical
errors that relate to surveys.
Please tick just one.
Yes
No
Proposed
Timetable Change
Please tick just one.
Yes
No
15
Sorted in Schools AS91263 | Student Booklet
Sampling error A sampling error is the error that happens when you
use a sample rather than doing a census. Because a
sample is only a part of the population, we can never
guarantee that the information that a questionnaire
gathers is a perfect match for the population; if we
take a dierent sample, we will probably get a
dierent result. Using a large sample and making sure
that everyone has an equal chance of being in the
survey can help to reduce sampling error.
Non-sampling errors Non-sampling error is the error that comes from
factors in the data collection process other than doing
a sample. Even if you do a census, this doesn’t mean
that your gathered data will be 100 percent reliable.
For example, not everyone will want to fill in or
complete a questionnaire – they might choose not to
take part or might only complete some of it. If the
people who choose not to complete the questionnaire
have a characteristic in common, this can lead to bias.
There are lots of dierent types of non-sampling
errors.
Sampling bias is a statistical error that relates to the
way that a sample is selected from the target
population. For a sample to be unbiased, everyone in
the target population needs to have an equal chance
of being selected for the sample.
Response bias is when respondents don’t answer
questions accurately or truthfully. Here are some
examples of factors that can lead to response bias.
Some people may have barriers that make it dicult
for them to participate or to understand a question,
for example, if they don’t speak English
People aren’t always honest when they complete a
questionnaire because they don’t want to look bad
People might interpret a question or answer
dierently to the way you intended.
16
Sorted in Schools AS91263 | Student Booklet
Assessment tip:
One of the requirements for achievement with Excellence is to “integrate
your statistical and contextual knowledge”. This means making connections
between statistical concepts and your questionnaire. One way to do this is to
discuss sampling and non-sampling errors and how these might relate to the
purpose or design of your questionnaire.
The diagram below shows how
challenging it can be to gather accurate
data. The ideal situation would be to
have a sampling frame that includes
everyone in the target population. The
sample is the group of people that you
select to be part of your survey. The
respondents are the people who
complete and return the survey.
Target population
Sampling frame
Sample
Respondents
Target population
Sample=
respondents
In real life, it can be hard or impossible
to find a sampling frame that is a perfect
match for the target population. The
sampling frame might miss some people
out or include people who are not in the
target population. Once you’ve selected
a sample, you might find that not
everyone in your sample chooses to
complete and return your questionnaire.
Ideal Situation Reality
17
Sorted in Schools AS91263 | Student Booklet
Complete Topic 2 Activities 1–5
in the Student Practice Booklet
Now it’s time to work towards completing
Checkpoint one: Purpose, audience, and
target population.
To meet the requirements of this checkpoint, you
need to:
• identify the context of your investigation and the
research objectives of your questionnaire
• explain what information the questionnaire will
provide
• explain the audience of your questionnaire
• define your target population.
WHAI
WHAKAARO
CHECKPOINT
Complete Topic 2 Activity 6
in the Student Practice Booklet
Before moving on to Topic Three, check that you understand:
What a target population is and how to define one
The dierence between a census and a sample
What a sampling frame is
What a sampling error is
18
Sorted in Schools AS91263 | Student Booklet
19
Sorted in Schools AS91263 | Student Booklet
3
Topic Three:
Ethical considerations
Learning outcome for Topic Three
Understand the ethics of surveys through questionnaire design
Success criteria
• I can explain why privacy, confidentiality, and anonymity are important
• I can identify techniques I can use to meet ethical standards.
The focus for this topic is understanding the importance
of meeting ethical questionnaire standards.
Ethics involves working in ways that protect people’s rights and dignity.
Ethical standards are guidelines that help you to act in ways that are fair,
kind, and respectful.
We’re going to focus on four key aspects of ethical surveys: transparency,
informed consent, privacy, and inclusivity.
T
r
a
n
s
p
a
r
e
n
c
y
I
n
f
o
r
m
e
d
C
o
n
s
e
n
t
I
n
c
l
u
s
i
v
i
t
y
P
r
i
v
a
c
y
20
Sorted in Schools AS91263 | Student Booklet
Transparency
The people who complete your
questionnaire have a legal right to
control who has access to their
information and what is done with it.
When you ask someone to complete
a questionnaire it’s important that you
tell them:
• Who you are, for example, your
name, year level, and contact details.
One of the reasons for providing
your contact details is so that
participants can contact you to
request their data if they want
• The purpose and audience of your
questionnaire (why you’re doing the
survey and who you are collecting
data from)
• Whether their responses are
anonymous or identifiable and
whether the data is confidential
• That they have a right to request
that their data is deleted or
removed from your survey.
“By completing this questionnaire,
you are indicating your consent to
participate in this research.”
Informed consent
Once people know about the purpose
of the questionnaire, they need to show
that they agree to be part of your
research.
You can meet this requirement by using
a simple phrase such as:
21
Sorted in Schools AS91263 | Student Booklet
Asking personal questions
For this assessment, it’s important not
to ask personal questions that might
invade people’s privacy. For example, in
your questionnaire, it’s not appropriate
to ask questions about people’s mental
health, their well-being, sexuality, or
drug use. This is to protect both you
and your participants. If you ask
personal questions you might:
• make the person filling in the
questionnaire feel uncomfortable,
upset, or unsafe
• find out sensitive information
without being able to provide
professional support.
Another issue is that these types of
questions may mean someone is no
longer willing to be part of your survey.
It can be hard to get a good response
rate to any survey, but writing a
questionnaire that is easy and safe to
complete can help.
If you’re not sure about a question you
want to ask, check with your teacher.
Privacy
Privacy is the right to keep your
personal information secret. Protecting
people’s privacy involves:
• not requiring them to answer
personal questions
• securely storing and protecting their
data
• keeping their identity secret.
Dr Carla Houkamau:
Protecting our member’s privacy
is a priority for us. We take great
precautions to secure all the
information people provide for
us. All personal details are
encrypted and stored separately
from questionnaire data.
22
Sorted in Schools AS91263 | Student Booklet
Confidentiality
Confidentiality includes not sharing
information about someone with
another person. In the context of a
questionnaire, this means that when
you share information gathered through
your questionnaire, you should avoid
using names or anything that can be
used to identify the people who
completed it, or anything that links
people with their responses. One way
to do this is to make their responses
anonymous (unnamed). If their data is
anonymous, even you as the researcher
won’t know whose responses you are
reading when you look at a completed
questionnaire.
Ways to respect people’s privacy
• Only gather information that relates
to your purpose; if you can’t explain
why some of the information in your
questionnaire is important or useful,
take it out
• Let people skip questions if they
want to and make it clear that they
can do this
• Let people know who will see the
information that they give you, and
whether their responses are
anonymous
• Let people know that filling in the
questionnaire is optional.
Dr Carla Houkamau:
We know that responding to
questions that ask about personal
matters might cause discomfort
for some people. However, for our
particular study, it is important to
ask questions that are personal.
Some people may think personal
questions are irrelevant or
perhaps an invasion of people’s
privacy. A person’s point of view
will be influenced by many things
such as their life experiences,
background or challenges they
may face from day-to-day. All the
factors we ask about (age,
gender, health, personality and
relationships) help to create a
picture of a person’s lifestyle and
personal preferences. These
things are a major factor in
determining a person’s values and
priorities – as well as their
economic choices. If, at any time,
people don’t want to answer any
of the questions they do not have
to. What they do and do not
respond to is totally up to them.
23
Sorted in Schools AS91263 | Student Booklet
Inclusivity
Inclusivity involves making sure that
no-one feels excluded (left out or
denied access) by your questionnaire.
A key area of consideration is how you
ask about people’s gender.
Remember that any question you ask
needs to match the purpose of your
questionnaire – if you ask questions
about gender there needs to be a
clear reason for doing so.
If you do ask about gender, you need
to include more options than male and
female so that you don’t exclude
people who are gender diverse. It’s
also important to make questions
about gender (and any other personal
information) optional so that people
can choose not to answer them.
Here is an example:
Dr Carla Houkamau:
Only I, Professor Chris Sibley and
trusted research assistants
working on the MIFAS in secure
conditions have access to
participants’ contact details.
Participants’ contact details are
used solely for the purposes of
contacting them to continue their
participation each year and to
provide them with information
and feedback about research
findings from the MIFAS.
Complete Topic 3 Activity 1
in the Student Practice Booklet
Are you:
Female
Male
Gender diverse
Prefer not to say
24
Sorted in Schools AS91263 | Student Booklet
Before moving on to Topic Four, check that you understand:
Why questionnaires need to meet ethical standards
What information you need to give people about your questionnaire
before they fill it in
How to meet the ethical standards of transparency, informed consent,
privacy and inclusivity
Assessment tip:
In your assessment, you can explain why you didn’t include certain questions.
Linking this to an explanation of ethical standards is one way to “integrate
contextual and statistical knowledge” into your report, which is a
requirement for excellence.
Complete Topic 3 Activities 2–3
in the Student Practice Booklet
25
Sorted in Schools AS91263 | Student Booklet
26
Sorted in Schools AS91263 | Student Booklet
4
Topic Four:
Types of questions
Learning outcome for Topic Four
Understand the advantages and disadvantages of dierent
types of questions through questionnaire design
Success criteria
• I can identify whether a question is closed or open
• I can give two advantages and two disadvantages of closed
and open questions.
In this topic we explore some dierent types of
questions that you can use in your questionnaire.
Using a variety of questions can make filling in your questionnaire more
interesting and enjoyable. This can increase the likelihood that your
respondent will complete it, which could reduce the non-sampling error.
There are two main categories of questions: closed and open. Let’s take
a look at closed questions first.
Assessment tip:
In your questionnaire, you should aim to have at least three types of questions.
To gain a Merit grade or higher, you need to explain decisions you have
made in the design process. This includes explaining the type of questions
you have used. Whichever question type you use, remember that all of the
questions you ask need to have a clear link to the purpose and audience of
your questionnaire.
Closed questions
Closed questions are questions that
can be answered with a single word
or a short phrase. In questionnaires,
closed questions are often followed
by a limited number of response
options, for example, yes/no or
multiple choice answers.
27
Sorted in Schools AS91263 | Student Booklet
Advantages of closed questions Disadvantages of closed questions
Closed questions are often quick and
easy to answer. This can make it more
likely that respondents complete the
questionnaire, reducing non-sampling
errors.
Responses to closed questions are
easy to analyse and display.
It can be frustrating for respondents
if the answer they want to give is not
oered.
Respondents who don’t have an
opinion might just select a response
at random.
Closed questions can reduce complex
ideas down to overly simple responses.
Closed questions are usually easy
and quick to answer. The data they
gather is also easy to analyse because
there are a limited number of
responses.
The types of closed questions
explored in this topic are:
• yes/no questions
• multiple choice questions
• rating scales
• Likert scales.
28
Sorted in Schools AS91263 | Student Booklet
Yes/no
questions
A Yes/no question is
useful when these are
the only two options
for a question.
Example
Do you have any
savings?
Yes No
Troubleshooting
Limit your use of Yes/no
questions to screening
questions.
Consider including a third
option such as “I’m not sure”
or “prefer not to say” so
that you don’t force
people to answer
the question.
Disadvantages
Yes/no questions can
come across as a bit
blunt.
Yes/no questions don’t
allow people to explain
their response or
provide alternative
answers.
Advantages
Yes/no questions can
be a useful screening
question. In the example,
if you have further questions
about more specific types of
savings you can instruct people
who tick “No” to skip them
and go to a subsequent
numbered question.
29
Sorted in Schools AS91263 | Student Booklet
Multiple
choice questions
Multiple choice questions
allow participants to select
one or more responses
from a list that you
provide.
Disadvantages:
As the designer, you’re
in control of the response
options and you might not
provide all the necessary
options. This can lead to bias or
participants might choose
randomly if the answer they
want isn’t there, which can
reduce the accuracy of
responses.
Troubleshooting:
Include an “other” option
and a comment field that
lets people add their own
option.
Use a pilot survey to check
whether you need to add more
options (you’ll be learning
about pilot surveys in
Topic Eight).
Advantages:
Multiple choice
questions are easy
to answer and
to analyse.
Example
Do you use any of the
following financial services
or have any of these types of
investment?
Iwi-managed
investments
KiwiSaver
Other retirement
savings
Shares
Personal savings
A home loan/
mortgage
Home/contents
insurance
Other
(please state):
30
Sorted in Schools AS91263 | Student Booklet
Rating
scales
Rating scale questions
provide a scale of options,
for example, rating
something from 1 to 10. The
respondent can choose a
number that matches
their response.
Disadvantages:
Without clear
instructions, people might
interpret the scale
incorrectly.
People may not have an
opinion or the question
might not apply to
them.
Troubleshooting:
Make sure you explain
what the scale means.
Allow participants to
indicate if they don’t
have an opinion or
skip the question.
Advantages:
Easy to use
and analyse.
Easy to answer.
Example
How well has school
prepared you to make good
money choices?
Not at all
1 2 3 4 5 6 7 8 9 10
Extremely well
31
Sorted in Schools AS91263 | Student Booklet
Likert scales
(Pronounced: Lick-urt)
Likert scales involve giving
respondents statements
and asking them to rate
how much they agree
or disagree with
them.
Advantages:
Most people will be
familiar with Likert scales and
will understand how to respond
to them.
They’re quick to fill in.
Likert scales provide quantitative
data that is easy to analyse.
Using a scale means that people can
express a range of responses
(instead of just “agree” or
“disagree”).
Example
How much do you agree
with the following statement:
KiwiSaver should be compulsory
once you have turned 18.
Strongly
disagree
Disagree Neither
agree nor
disagree
Agree Strongly
agree
Troubleshooting:
Make sure that the scale
is balanced (for example, from
very unlikely to very likely).
Use the same scale throughout
your survey to avoid confusion. For
example, if you decide to use an
even-pointed scale (the forced-
choice method), use this for all
the Likert scale questions in
your questionnaire.
Disadvantages:
Likert scales don’t provide
ways for people to explain their
ideas or opinions in depth.
People will interpret the interval
between points on the scale dierently.
People sometimes avoid choosing
extreme answers, so there may be non-
sampling errors.
Forced-choice questions might force
people to indicate agreement or
disagreement that they
don’t genuinely feel.
Likert Scales continued on next page
32
Sorted in Schools AS91263 | Student Booklet
Open questions
Open questions are questions that
respondents can answer in an unlimited
number of ways. They are a useful way
to gather information on opinions,
beliefs, and feelings. Open questions
often begin with words such as: why,
how, or describe.
Open questions often require an
answer longer than one or two words,
especially if they require respondents
to think or reflect.
Likert scales are a useful way to measure people’s opinions or attitudes
towards something in a way that can be quantified (analysed using
numbers). A Likert scale is usually “symmetrical” which means it has equal
numbers of positive and negative items, with a neutral value in the middle.
A format of a typical five-level Likert scale could be:
Strongly
disagree
Disagree Neither agree
nor disagree
Agree Strongly
agree
You can also use an even-pointed scale. This is sometimes called a forced-
choice method because it removes the option of a neutral response.
A format of this type of Likert scale could be:
Strongly
disagree
Disagree Somewhat
disagree
Somewhat
agree
Agree Strongly
agree
You can read more about Likert scales on Wikipedia.
33
Sorted in Schools AS91263 | Student Booklet
What is your
opinion about...
Why do you
think…
What are some
examples of…
For this reason, they are more commonly used as part of a focus group interview
or a one-to-one interview with oral rather than written responses. In these settings
the researcher can ask further questions to make sure that they understand what
the person is saying.
Examples of open questions include:
Here are two ways to write open-questions that can be quite quick to answer:
Word associations
Give the respondents some key words and ask them to write down the first
word that comes to mind when they read them, for example:
Wealth Debt Retirement
Sentence completions
Ask the respondents to complete a sentence, for example:
When I think about leaving school, I feel...
34
Sorted in Schools AS91263 | Student Booklet
Advantages of open questions Disadvantages of open questions
Allow for any response.
Can provide qualitative data.
An open question might introduce
ideas that you haven’t considered.
Open questions provide more
opportunities for in-depth opinions,
explanations, and feelings than
closed questions.
Open questions can be more time
consuming for the respondents and
require more thought.
Responses to open-ended questions
can be harder to analyse objectively.
For more information on types of
questions see: Survey Monkey Types
of survey questions
Before moving on to Topic Five,
check that you understand:
The dierence between
closed and open questions
The advantages and
disadvantages of dierent
question types.
Complete Topic 4 Activities 1–3
in the Student Practice Booklet
35
Sorted in Schools AS91263 | Student Booklet
36
Sorted in Schools AS91263 | Student Booklet
5
Topic Five:
The importance of
good question design
Learning outcome for Topic Five
Understand the value of good questionnaire question design
Success criteria
• I can identify flaws in questions
• I can explain ways that questions can be improved.
In this topic we explore what makes a good question.
The quality of the questions you ask will determine the quality and usefulness
of the data you gather.
Deciding what to ask
All of the questions you ask need to be based on the purpose of your
questionnaire and the information needs of your audience. To check this, you need
a clear and precise purpose statement.
In Checkpoint one, you developed a set of research goals that relate to your
context. Your next step is to write draft questions related to each goal.
For every question, you should be able to explain:
• how the question relates to the purpose of the questionnaire
• what data the questionnaire will gather
• how you can analyse or display the data you gather.
Other considerations include:
• Whether respondents will be willing and able to answer the question?
• How can I make it as easy as possible for the respondents to give accurate data?
Two key considerations are:
• What will you ask?
• How will you ask?
37
Sorted in Schools AS91263 | Student Booklet
Dr Carla Houkamau:
Survey items need to be written a certain way in order to
make sure we get valid responses.
Some people get frustrated because they cannot see their
answers in the list of options provided. Developing a compact
list of responses is the hardest part of writing a survey.
Deciding how to ask questions
Your questions need to be clearly understood by the people filling in your
questionnaire. This will reduce misunderstandings and make your questionnaire
easier to complete.
Watch this short Statistics Learning Centre video in which Dr Nic
summarises how to write good survey questions.
Complete Topic 5 Activity 1
in the Student Practice Booklet
38
Sorted in Schools AS91263 | Student Booklet
Using clear language
It’s important that the people filling in your questionnaire can understand
the questions they are being asked and that it’s easy for them to answer
the questions.
Use everyday language
Write your questions in a simple and
straightforward way. Avoid complex
language or technical terms that people
might not understand.
Flawed question:
How does your financial identity influence your economic decisions?
Issue:
The concept of “financial identity” is complex and may not be widely understood.
For young people “economic decisions” could be replaced by questions
about how they “spend, save, share, and grow” their money.
Revised question:
What is one piece of advice a family member or friend
has given you about saving?
Avoid double negatives
Double negatives occur when two
negative words are used in the same
sentence. Double negatives have to be
read carefully to be understood
correctly and can be confusing. If your
respondent misinterprets the question,
you will have a non-sampling error.
2
D
1
I
1
S
1
A
2
G
1
R
1
E
1
E
1
S
4
H
1
O
1
U
1
L
2
D
1
N
T
1
39
Sorted in Schools AS91263 | Student Booklet
Flawed question:
Do you disagree that people shouldn’t have to save for retirement?
Issue:
The double negative in this question makes it hard to understand.
Revised question:
Do you think saving for retirement should be compulsory?
Focus on one idea at a time
Avoid double or triple questions. If you
put more than one idea in a question,
it can be hard to know how to answer it.
Flawed question:
Do you save and invest your money?
Yes No
Issue:
If someone saves their money but doesn’t invest it, they may not know how to
answer this question.
Revised questions:
Do you have any savings?
Yes No Prefer not to say
Do you have any investments?
Yes No Prefer not to say
40
Sorted in Schools AS91263 | Student Booklet
Be realistic
Make sure that the questions you
ask can be accurately answered.
For example, avoid asking people
to remember too far back.
Flawed question:
How much money did you spend on snack food last month?
Issue:
It’s hard for people to accurately remember how much money they spent
on snack food over a whole month.
Revised questions:
Over the past week, approximately how much money did
you spend on snack food?
Keep questions neutral and bias free
When it comes to designing questions,
it’s important that your questions are
not biased – that they don’t show your
preferences or opinions. People need
to be free to give their own ideas. Avoid
questions that make assumptions.
Flawed question:
Approximately how much of your weekly income do you save each week?
none a bit about half most all
$
$
41
Sorted in Schools AS91263 | Student Booklet
Issue:
This question assumes that the respondent has a weekly income. They might
tick “none” to signal that they don’t have an income (and therefore don’t save
any of it) but this is not what the question is asking. This is an example of a
non-sampling error.
Revised questions:
Do you have a weekly income?
Yes No
If you answered yes, continue to question 2. If you answered no,
skip question 2 and go to question 3.
Provide complete response options
Make sure that you have provided a
complete set of options for each
question.
Flawed question:
Where do you get financial advice from?
Family Friends School The internet
Issue:
The list of answer options doesn’t cover all possible options, for example, students
might get financial advice from financial experts, apps, podcasts, banks, or tv.
Revised questions:
Provide an “other” category that gives people an option
if their situation isn’t listed.
1
2
3
4
5
Other
42
Sorted in Schools AS91263 | Student Booklet
In Topic Eight you’ll learn about the importance of doing
a pilot survey, which involves testing your questionnaire
on a small group of people and getting their feedback on
it. This is a useful way to check whether the response
options you have provided are suitable. If lots of people
choose “other”, you may need to add some more options.
For more information on how to write
good questions, see the StatsNZ
resource A Guide to Good Survey
design, pages 38 to 47.
Before moving on to Topic Six, check that you understand:
The importance of aligning your questions to your purpose
Ways to write clear, straightforward questions
Complete Topic 5 Activity 2
in the Student Practice Booklet
43
Sorted in Schools AS91263 | Student Booklet
44
Sorted in Schools AS91263 | Student Booklet
6
Topic Six:
Structuring your
questionnaire
Learning Outcome for Topic Six
Understand the importance of structure in questionnaire design
Success criteria
• I can create a draft of my questionnaire
• I can explain choices I have made about the structure of my
questionnaire.
It’s nearly time for you to start putting together the first
draft of your questionnaire.
Before you do, here are some guidelines on how to structure your questionnaire.
Give your questionnaire a title
Give your questionnaire a short and
meaningful title that helps your
participants know what the purpose
of your questionnaire is.
Questionnaires with titles are
usually seen as more credible than
questionnaires that don’t have a title.
Include an introduction
Explain who you are and why you are
doing the study. Let the participants
know how their data will be stored,
used, and whether or not their
responses will be anonymous.
Questionnaire Title
Kia ora,
1.
Please tick just one.
Yes
No
Other, (please specify)
If yes please answer question 2,
if no then skip to question 3
2.
Please tick just one.
Other, (please specify)
3.
Please tick just one.
4.
Select your stance on the scale below.
How much do you agree with the below statement
Strongly
disagree
Disagree Neither
agree nor
disagree
Agree Strongly
agree
5.
Select your stance on the scale below.
How much do you agree with the below statement
Strongly
disagree
Disagree Neither
agree nor
disagree
Agree Strongly
agree
45
Sorted in Schools AS91263 | Student Booklet
Assessment tip:
If your questionnaire seems to be
too long, read each question and
ask, “How does this information
relate to my purpose?” If there’s a
strong link keep it. If there’s not a
clear reason to include the
question, get rid of it. Make a note
in your log book of your decision
and explain why.
Keep the questionnaire short
If your questionnaire is short people
are more likely to complete it. This
reduces the non-sampling error of
non-completion and allows you to
have more confidence in your results.
Provide instructions
Write clear and concise instructions
on how to complete the questionnaire.
Use short sentences and basic
vocabulary.
Order the questions carefully
To increase the response rate, you
want to make the process of filling in
the questionnaire as easy and as
enjoyable as possible.
6.
Select your stance on the scale below.
How much do you agree with the below statement
Strongly
disagree
Disagree Neither
agree nor
disagree
Agree Strongly
agree
7.
8.
Select your stance on the scale below.
How much do you agree with the below statement
Strongly
disagree
Disagree Neither
agree nor
disagree
Agree Strongly
agree
Other, (please specify)
9.
Please tick just one.
Other, (please specify)
10.
Please tick just one.
Other, (please specify)
11.
Do you have any comments or suggestions
abouthow we could improve this study?
Thank you
for giving up your time to share
your information and opinions.
I would like to receive a copy of my responses
Not at all Extremely well
1 2 3 4 5 6 7 8 9 10
46
Sorted in Schools AS91263 | Student Booklet
Here are some tips to help you choose the order of your
questions:
• Begin with some questions that are easy to answer
and interesting. If the first questions you ask are
complicated or boring, people may choose not to
fill in the rest of the questionnaire
• Use screening questions towards the start of your
questionnaire to check that the person filling it in is
part of your target population. If they’re not, let them
know that they don’t need to complete the rest of
the questionnaire
• Put the most important questions in the first half of
the questionnaire so that if someone doesn’t complete
it, you will still have some useful information
• Provide variety in the type of questions used
• Group the questions into categories – this will
improve the flow from one question to the next
• Unless you are using them to screen for your target
population, put the demographic questions at the end
(demographic questions are questions that help you to
understand the identity of the respondent, for example,
what year level they are in, their age, gender, or family
size).
Check your formatting
Take care when formatting your questionnaire. For example, make sure you use
spacing well. In the example that follows, a respondent might tick the wrong box:
monthly weekly daily
47
Sorted in Schools AS91263 | Student Booklet
Provide an opportunity for feedback
Asking respondents for feedback on the questionnaire provides an opportunity
for them to let you know if they had any diculties filling it in or whether there is
anything else they would like to let you know. This will make you aware of potential
gaps or shortcomings in your questionnaire and might open up some future paths
of investigation.
Example from the MIFAS:
We are always looking for feedback to help improve the Māori Identity and
Financial Attitudes Study. Have we missed anything important? Or do you
have any other general comments or suggestions about how we can improve
the study next year?
Say thanks!
Include a short thank you to your respondents for giving up their time to share
their information and opinions with you.
Thank you
for giving up your time to share
your information and opinions.
I would like to receive a copy of my responses
48
Sorted in Schools AS91263 | Student Booklet
Before moving on to Topic Seven, check that you understand:
Key components to include in your questionnaire
How to structure your questionnaire.
Complete Topic 6 Activity 1
in the Student Practice Booklet
Now it’s time to work towards completing
Checkpoint two: First draft.
To meet the requirements of this checkpoint,
you need to:
• provide a draft set of questions
• use dierent question types
• explain how the questions you have asked
to relate to your purpose
• explain why you have chosen particular
types of questions.
WHAI
WHAKAARO
CHECKPOINT
49
Sorted in Schools AS91263 | Student Booklet
50
Sorted in Schools AS91263 | Student Booklet
7
Topic Seven:
Doing a desk review
Learning outcome for Topic Seven
Understand the purpose of a desk review through questionnaire design
Success criteria
• I can conduct a desk review of my questionnaire and/or ask someone
else to do a desk review
• I can make changes to my questionnaire based on a desk review
• I can explain changes I have made based on the desk review.
Now that you’ve created the first draft of your
questionnaire, it’s time to give it a “desk review”.
A desk review involves carefully reading through the draft questionnaire
checking for things such as:
Target population – have you screened for your target population?
Wording – are the questions clear and easy to follow?
Spelling and grammar – are there any spelling or grammar errors?
(If this isn’t one of your strengths, ask someone to check this for you.)
Format – is the font easy to read? Is the numbering correct?
Clarity – have you used everyday language?
Tone – is the language you have used friendly and polite?
Instructions – are there instructions for each set of questions?
Categories – are similar questions grouped together?
Flow – does the order of questions make sense? Do the questions
link together?
Length – are there too many questions?
Purpose – will the questions help to answer your research objectives?
You can do the desk review yourself or you can ask someone else to do it for
you. Make sure your reviewers know what the purpose and audience of your
questionnaire are.
51
Sorted in Schools AS91263 | Student Booklet
If you ask another student to review
it for you, it’s helpful if they know
something about questionnaire design
so that they can give you in-depth
feedback. If you ask a friend to review
it, encourage them to be critical and
to mention even small things that could
be improved.
If you are designing your questionnaire
for a specific client, it’s useful to ask
them for feedback on the draft
questionnaire. That way, they can let
you know whether it will give them
the information they need. Their
suggestions can both improve your
questionnaire and give them more
confidence in the results.
Any time you make changes to your
questionnaire, you should do a desk
review – it’s easy for mistakes to slip
in or for instructions to be left out.
Assessment tip:
Remember that to get a merit
grade you need to explain
decisions that you make in the
design process. This includes
explaining changes to questions
following the desk review.
52
Sorted in Schools AS91263 | Student Booklet
Before moving on to Topic Eight, check that you understand:
What a desk check review involves and why they are important
Complete Topic 7 Activity 1
in the Student Practice Booklet
Now it’s time to work towards completing
Checkpoint three: Desk review and Checkpoint
four: Second draft.
To meet the requirements of this checkpoint,
you need to:
• provide evidence of your own desk review
• provide evidence of someone else’s review of
your questionnaire
• identify what you need to change in your
questionnaire based on the desk review
• revise your questionnaire
• explain the changes you have made.
WHAI
WHAKAARO
CHECKPOINT
53
Sorted in Schools AS91263 | Student Booklet
54
Sorted in Schools AS91263 | Student Booklet
8
Topic Eight:
Doing a pilot survey
Learning outcome for Topic Eight
Understand the purpose of a pilot survey through questionnaire design
Success criteria
• I can conduct a pilot survey of my questionnaire
• I can gather feedback on my questionnaire from pilot survey
participants
• I can make changes to my questionnaire based on the pilot survey
• I can explain changes I have made based on the pilot survey.
The next step in the
questionnaire design process
is to do a pilot survey
– this means giving the questionnaire
to a small sample from the target
population to see whether it will work
in real life. It also involves collecting
and processing the data, for example,
checking that the results can be
analysed.
Doing a trial run will help you to identify
any issues with the questionnaire.
Why is it important to use people
from the target population in the
pilot survey?
Although it’s only done on a small
scale, a pilot survey should be as
similar as possible to the actual survey.
This will show you:
• whether people in your target
population can accurately complete
the questionnaire
• whether the questionnaire will help
you to gather information that you
can analyse and answers your
questions.
Example from the MIFAS:
We spent about six months
researching the content of the
survey and drafting it. We also
piloted the survey and had it peer
reviewed. We also submitted the
survey for ethical review and
removed several questions that
we deemed less important and
perhaps somewhat controversial.
55
Sorted in Schools AS91263 | Student Booklet
Your first step is to organise a small
group of people from your target
population who are willing to be part
of your pilot study. Explain that you
are asking them to complete the
questionnaire as part of a pilot survey
to see whether the questionnaire will
help you to achieve your research goals.
Encourage them to pay attention to
how easy or hard it is to complete the
questionnaire and to mark any areas
that they think could be improved.
Let them know that they can ask you
if any of the questions are unclear.
Once the questionnaire has been sent
out, this won’t be an option, so it’s
important to find out whether there are
any problems through the pilot survey.
Approaches to pilot surveys
1
56
If it looks like they are finding a question
hard to answer, stop and ask them why
– it could be that the wording of the
question is unclear or perhaps the
answer options are not complete. You
should also ask them questions about
the process once they have finished
filling the questionnaire in.
If the participants have answered, “does
not apply” or “don’t know”, ask them
why. You may like to probe a little deeper
by asking them what they understand
certain words or concepts to mean.
If you can’t be present while the pilot
survey respondents are filling in the
questionnaire, you could give them a
checklist to complete afterwards that
asks them if they had any problems
completing the questionnaire and any
suggestions for changes (for example, an
alternative answer option for a question).
Once you’ve completed a pilot survey,
you can make final changes to your
questionnaire. You may need to
complete another desk review to check
that you haven’t introduced any new
errors (for example, spelling or
numbering mistakes).
2
3
57
Sorted in Schools AS91263 | Student Booklet
Here are some questions that you can use to help you reflect on the findings of
your pilot survey. The list of questions is also on page ** of your workbook.
Reflections:
Did people understand
what the questions were
asking?
Did you get
any unexpected
answers?
Is the questionnaire
measuring what you
want it to measure?
Will the information
gathered give you what
you need for the
purpose of the
questionnaire?
Were there many “I
don’t know” or “Does
not apply” responses? If
so, does the question
need to be changed? Do
you need to add some
more response options?
What was the response
rate? If it was low, is
there a way to improve
this?
58
Sorted in Schools AS91263 | Student Booklet
Before moving on to Topic Nine, check that you understand:
What a pilot survey involves and why they are important
Approaches to doing a pilot survey
Complete Topic 8 Activity 1
in the Student Practice Booklet
Now it’s time to work towards completing
Checkpoint five: Pilot survey and Checkpoint six:
Final questionnaire.
To meet the requirements of this checkpoint,
you need to:
• conduct a pilot survey using your draft
questionnaire
• provide the responses to your pilot survey
• analyse your questionnaire based on the pilot
survey responses
• explain what you need to do to refine your
questionnaire
• make and explain changes to your questionnaire
• do a desk to make sure the changes haven’t
introduced any new errors
• provide a final copy of your questionnaire.
WHAI
WHAKAARO
CHECKPOINT
59
Sorted in Schools AS91263 | Student Booklet
60
Sorted in Schools AS91263 | Student Booklet
9
Topic Nine:
Reflecting on the process
Learning outcome for Topic Nine:
Understand how to evaluate your questionnaire and the process
you used to develop it
Success criteria
• I can evaluate the eectiveness of my final questionnaire in terms
of its purpose
• I can evaluate the process I used to develop my questionnaire.
Assessment tip:
In real life, the next step would be to select a random sample from your
target population, ask them to complete the questionnaire, and then analyse
the results to reach a conclusion. However, these steps aren’t required for
this internal assessment, so your final step is to reflect on your final
questionnaire and on the questionnaire design process.
Now it’s time to work towards completing
Checkpoint seven: Reflecting on your
questionnaire design process.
WHAI
WHAKAARO
CHECKPOINT
Below are some questions you can use to help you reflect on your questionnaire
design process. Read all of the questions then choose at least three to answer in
the evaluation section of your report.
Ka pai!
You’ve nearly completed the questionnaire design process.
Complete Topic 9 Activity 1
in the Student Practice Booklet
61
Sorted in Schools AS91263 | Student Booklet
How important was
the desk review and/or
pilot survey in terms of
ensuring that your final
questionnaire fulfils its
purpose?
How well does your
questionnaire match
its purpose? Were
there any gaps?
What other avenues
of enquiry related to
your research goals
could you explore?
Which aspects of your
questionnaire design are
likely to reduce non-
sampling errors? How
would this help the
questionnaire to fulfil
its purpose?
How easy would it be
to analyse the data your
questionnaire would
gather? Did you have to
make any compromises
with the questions you
asked in order to make
analysis easier?
What advice would you
give to another student
doing this assessment
standard?
What connections can
you make between your
research, the purpose of
your questionnaire, and
its final design?
What are three things
you have learned about
questionnaire design?
What would you do
dierently next time?
Possible reflective questions:
62
Sorted in Schools AS91263 | Student Booklet