A questionnaire is a set of written questions used to collect data from a sample. A well-designed questionnaire produces reliable data; a poorly designed one introduces bias — results that systematically favour one outcome over another. GCSE statistics asks you to identify flaws in surveys and suggest improvements.
What is bias in statistics?
Bias occurs when data collection is unfair in a way that makes certain results more or less likely, regardless of reality. Bias does not come from random variation — it is a systematic distortion. The key sources of bias at GCSE are:
- Leading (biased) questions — wording that nudges the respondent toward a particular answer.
- Non-representative sampling — asking only a subset of people who share a characteristic not typical of the whole population.
- Inadequate response options — options that don't cover all possibilities, or overlap.
- Sensitive questions — questions where people may not answer honestly.
- Timing and context — asking people in circumstances that influence their answer.
What makes a question biased, and how do you improve it?
Example 1 — Leading question
Flawed: "Don't you agree that the school canteen serves unhealthy food?"
Problems: The phrase "Don't you agree" assumes the respondent will agree. It nudges people toward a "yes" response.
Improved: "How would you rate the healthiness of the school canteen's food?"
Options: Very healthy / Fairly healthy / Neither / Fairly unhealthy / Very unhealthy.
Example 2 — Response options that overlap or have gaps
Flawed: "How many hours of TV do you watch per day?"
Options: 0–1 hours / 1–2 hours / 2–3 hours / 3–4 hours
Problems: The options overlap (1 and 2 appear in two boxes each) and there is no option for more than 4 hours.
Improved: Options: Less than 1 hour / 1 to less than 2 hours / 2 to less than 3 hours / 3 hours or more.
Using "to less than" removes overlap and "or more" removes the upper-end gap.
Example 3 — Biased sampling
A researcher asks shoppers in a garden centre about their preferred leisure activity and concludes that 60% of adults prefer gardening.
Problem: The sample is drawn from a location where people who enjoy gardening are over-represented. This is called selection bias or sampling bias.
Improvement: Survey a random sample of the general population — for example, using systematic or stratified random sampling.
What are the features of a good questionnaire?
| Feature | Why it matters |
|---|---|
| Clear and unambiguous wording | Respondents interpret questions consistently |
| Response options that are exhaustive | Every possible answer has an option |
| Response options that are mutually exclusive | No answer belongs to two categories |
| Appropriate scale (e.g. 1–5) | Captures range without being overwhelming |
| No leading or emotive language | Does not influence the respondent's answer |
| Includes a pilot study | Identifies confusing questions before full data collection |
What is a pilot study and why is it useful?
A pilot study (or pilot survey) is a small-scale trial run of a questionnaire before full data collection. It reveals:
- Questions respondents find confusing or offensive.
- Response options that are missing or overlap.
- How long the survey takes to complete.
- Whether the data collected will actually answer the research question.
GCSE exam questions often ask: "Give one reason why a pilot study would be useful." The standard answer is: to identify and fix problems with the questions or response options before the full study begins.
How does sampling affect bias?
Even a perfectly worded questionnaire produces biased results if the sample is unrepresentative. Common sampling problems include:
- Convenience sampling — asking only people who are easy to reach (e.g. friends, family, or people in a specific location).
- Volunteer sampling — people self-select to respond; those with strong opinions are over-represented.
- Size — a very small sample is more likely to differ from the population by chance.
The solution is to use a sampling method that gives every member of the population an equal or known chance of being selected — such as simple random sampling, systematic sampling, or stratified sampling.
Frequently asked questions
What is the difference between a biased question and a biased sample?
A biased question influences how the respondent answers (wording problem). A biased sample means the wrong group of people was asked (selection problem). Both introduce bias, but from different sources and with different fixes.
How many response options should a question have?
For categorical answers, include every possible category plus an "Other" option if needed. For a scale, five options (e.g. 1 to 5 or Strongly agree to Strongly disagree) is standard at GCSE — enough to capture variation without being confusing. Always make the options exhaustive (cover all possibilities) and mutually exclusive (no overlap).
What is a "closed" versus an "open" question?
A closed question gives a fixed set of response options (e.g. Yes/No, or a tick-box scale). It is easy to analyse statistically. An open question allows any written response, giving richer information but harder to process. GCSE questionnaire questions almost always use closed questions with tick boxes.
Why might people not answer sensitive questions truthfully?
Social desirability bias causes people to answer in a way they think they are expected to answer — for example, over-reporting exercise or under-reporting unhealthy habits. One mitigation is to make questions anonymous. Another is to use indirect questions (asking about "most people" rather than the respondent themselves), though this goes beyond standard GCSE expectations.
For guided GCSE statistics revision with Socratic questions and worked examples — visit aitutors.me.