Experimental Design, Control and Sampling

Before a psychologist collects a single result, they have already made two decisions that shape everything: how participants are arranged across conditions (the experimental design) and who takes part (the sample). Get these wrong and the data is confounded or unrepresentative before the study even begins.

This is one of the most heavily assessed areas of the AQA A-level. Research methods accounts for at least 25–30% of the whole assessment, and design and sampling questions appear on every paper. Examiners reward precise use of terms: independent groups is not the same as repeated measures, and random allocation is not the same as random sampling.

Three ideas run through this whole topic:

• Experimental design — how participants are distributed across the conditions of an experiment.
• Control — techniques that stop unwanted variables from distorting the results.
• Sampling — how a smaller group is selected to represent a larger population.

An experiment compares performance across conditions. The design determines whether the comparison is fair, and the sample determines whether the findings apply to anyone beyond the room.

Get comfortable defining each term in one sentence. Most marks here are lost to vague wording, not to a lack of ideas.

An experimental design is the way participants are allocated to the different conditions of an experiment. There are exactly three you must know, and each trades one advantage against one cost.

Design	What it is	Strength	Limitation
Independent groups	Different participants take part in each condition	No order effects, as no one repeats a task	Participant variables differ between groups; more participants needed
Repeated measures	The same participants take part in every condition	Participant variables controlled; fewer participants needed	Order effects (practice, fatigue); participants may guess the aim
Matched pairs	Different participants matched on key characteristics, one of each pair per condition	Reduces participant variables without order effects	Matching is time-consuming and never perfect

A participant variable is an individual difference (age, IQ, memory ability) that could affect the results. In independent groups these differences are spread unevenly between conditions, which is a weakness. Repeated measures removes them entirely because the same people appear in every condition, but at the cost of order effects.

Learn the designs as a set of trade-offs. Independent groups avoids order effects but not participant variables; repeated measures avoids participant variables but not order effects; matched pairs is the compromise that tackles both, at a practical cost.

Matched pairs works by pairing participants who are similar on a relevant characteristic, then splitting each pair across the two conditions. It approximates the benefits of repeated measures using different people, which is why it avoids order effects.

An order effect happens in a repeated measures design because participants complete more than one condition. Their performance in the second condition may be affected simply by having already done the first:

• Practice effect — participants improve because they have had a go already.
• Fatigue or boredom effect — participants get tired or lose motivation, so performance drops.

The problem is that an order effect can be mistaken for a real effect of the independent variable. If everyone does condition A then condition B, and scores rise, you cannot tell whether B was genuinely better or whether people simply warmed up.

The standard fix is counterbalancing: vary the order in which participants meet the conditions so any order effect is balanced out across both. The simplest version splits the sample into two groups (AB/BA counterbalancing).

Group	First condition	Second condition
Group 1	A	B
Group 2	B	A

Half the participants do A then B; the other half do B then A. Any practice or fatigue effect now appears equally in both conditions, so it no longer favours one over the other.

Counterbalancing does not remove order effects — it spreads them evenly across the conditions so they cancel out. Saying it "eliminates" order effects is a common overstatement.

Beyond the design itself, researchers use control techniques to stop extraneous variables from creeping into a study. Three named techniques appear on the spec, and students routinely confuse them.

Random allocation assigns participants to the conditions of an experiment by chance (for example, by drawing names or using a random number generator). It is used in independent groups designs to spread participant variables evenly between conditions, so neither group is systematically stronger.

Randomisation uses chance to decide the order of materials or trials within a study. For example, presenting a memory word list in a randomly generated order for each participant stops the position of items from biasing the results.

Standardisation means keeping the procedure and instructions identical for every participant. Everyone gets the same briefing, the same timings and the same conditions, so the researcher's own behaviour is not an uncontrolled variable.

Technique	What chance decides / what is kept constant	Main purpose
Random allocation	Which condition each participant is placed in	Spread participant variables evenly across conditions
Randomisation	The order of materials or trials	Reduce bias from the order of items
Standardisation	Keeps procedures and instructions identical for all	Remove differences in how participants are treated

Random allocation deals with people going into conditions. Randomisation deals with the order of materials. Standardisation deals with keeping the procedure the same. Do not use them interchangeably.

A target population is the entire group a researcher wants to study and draw conclusions about. It is rarely possible to test everyone, so the researcher selects a sample — a smaller subset — and generalises the findings back to the population. How that sample is chosen determines how representative it is.

Method	How it works	Strength	Limitation
Random	Every member of the population has an equal chance (e.g. a lottery)	Free from researcher bias; likely representative	Needs a full list of the population; can still be unrepresentative by chance
Systematic	Select every nth person from a list (e.g. every 5th name)	Objective once the interval is set; avoids bias	Not truly random; a pattern in the list could distort it
Stratified	Sample reflects the proportions of subgroups (strata) in the population	Highly representative of the population structure	Time-consuming; identifying the strata can be complex
Opportunity	Use whoever is available and willing at the time	Quick, convenient and cheap	Unrepresentative; prone to researcher bias in who is chosen
Volunteer	Participants respond to an advert (self-selected)	Easy to gather; participants tend to be committed	Volunteer bias — respondents may share particular traits

The key evaluation point is bias and generalisation. Random and stratified samples tend to be more representative, so findings generalise more confidently. Opportunity samples are often unrepresentative because they draw on whoever happens to be nearby, and volunteer samples suffer volunteer bias because people who respond to adverts may differ from the wider population (for example, being more motivated or having more free time).

A "random sample" has a precise meaning: every member of the target population has an equal chance of selection. Grabbing convenient participants is an opportunity sample, not a random one, even if it feels arbitrary.

A pilot study is a small-scale trial run of a study, carried out before the main study, to identify and fix problems before committing time and resources. It lets a researcher check that the design works, the materials make sense, the instructions are clear and the procedure runs smoothly. Any weaknesses can then be corrected, saving a flawed full-scale study.

The aim of piloting is to find and fix problems with the design, materials or procedure in advance — for example, spotting a confusing questionnaire item before hundreds of people answer it.

Two threats to validity come from participants and researchers being human.

Demand characteristics are cues in a study that lead participants to guess its aim and then change their behaviour in response. A participant might try to help the researcher by giving the "expected" answer, or deliberately do the opposite. Either way the behaviour measured is no longer natural.

Investigator effects occur when a researcher unintentionally influences the outcome, through their expectations, tone, body language or the way they record data. For example, unconsciously smiling more during one condition could nudge participants' responses.

Standardisation, single-blind procedures and careful, neutral instructions all help reduce these effects, which is one reason control techniques matter so much.

When behaviour is observed rather than manipulated, researchers still need a clear design so that observations are systematic and can be repeated. Three tools appear on the spec.

Behavioural categories break a target behaviour down into observable, measurable components that are recorded whenever they occur. Instead of the vague label "aggression", an observer might record specific actions such as hitting, pushing and shouting. Clear categories make the observation objective and improve inter-observer reliability.

Two sampling methods decide when behaviour is recorded during an observation:

Method	How it works	Best for
Event sampling	Count each time a target behaviour occurs throughout the period	Behaviours that happen occasionally and can be counted
Time sampling	Record behaviour only at fixed intervals (e.g. every 30 seconds)	Continuous or frequent behaviour that is hard to count in full

Event sampling risks missing detail if several behaviours happen at once; time sampling is manageable but can miss behaviours that occur between the sampled intervals. The choice depends on how often the behaviour happens and how much detail is needed.

Behavioural categories define what is recorded; event and time sampling define when it is recorded. A strong observational design needs both.

1. Confusing counterbalancing with random allocation

Counterbalancing controls order effects in a repeated measures design by varying the order of conditions. Random allocation assigns participants to conditions in an independent groups design. They solve different problems in different designs — do not swap them.

2. Saying repeated measures needs matching

Matching is the defining feature of the matched pairs design, not repeated measures. In repeated measures the same people do every condition, so there is nothing to match. Only matched pairs pairs up different participants on key characteristics.

3. Muddling the sampling methods

Systematic sampling (every nth person) is not the same as random sampling (equal chance for all). Stratified sampling reflects the proportions of subgroups, which opportunity and volunteer samples do not. Define each precisely rather than treating them as loose synonyms for "picking people".

4. Forgetting that order effects apply to repeated measures

Order effects (practice, fatigue, boredom) only arise when participants do more than one condition, which is repeated measures. Independent groups and matched pairs use different people per condition, so order effects are not a limitation for them.

5. Confusing random allocation with random sampling

Random sampling selects participants from the target population (who takes part). Random allocation assigns those participants to conditions once they are in the study (which condition they go into). Naming the wrong one is one of the most common errors on this topic.