Coaching Centre
Analysing and evaluating results
Describing trends and correlations with data, why correlation is not causation (hidden variables, reverse causation and chance), writing a discussion, evaluating validity, reliability and accuracy with specific improvements, and writing a conclusion that answers the aim.
In 2000 a British science writer, Robert Matthews, published a paper showing that across 17 European countries, the number of breeding pairs of storks was correlated with the number of babies born. The link was statistically significant. Of course storks don't deliver babies: bigger countries have more storks and more people. Matthews wrote the paper as a joke to make a serious point. A correlation in data, however strong, does not tell you what causes what.
Describing a trend
When you describe results, say what happens to the dependent variable as the independent variable changes, and back it up with numbers from your data:
- Direction: positive (both increase), negative (one increases as the other decreases), or no correlation.
- Shape: linear (a straight line) or non-linear (a curve that levels off, gets steeper, or peaks).
- Strength: strong (points close to the line) or weak (widely scattered).
A weak description says: "More mulch means less water loss." A strong one says: "Water lost decreased as the depth of sugarcane mulch increased, from 214 g with no mulch to 96 g with 5 cm, but adding more than 5 cm made little further difference (91 g at 10 cm), so the relationship is non-linear and levels off."
Scientists often summarise the strength of a straight-line relationship with the correlation coefficient, r, which runs from −1 (perfect negative) through 0 (no relationship) to +1 (perfect positive). Spreadsheets calculate it with =CORREL().
Correlation is not causation
Two variables can rise and fall together for reasons that have nothing to do with one causing the other:
- A hidden (confounding) variable causes both. Hot weather increases ice-cream sales and swimming, and so drownings.
- Reverse causation: the effect actually causes the "cause". Do people who exercise sleep better, or do people who sleep well have the energy to exercise?
- Chance: search enough data sets and some will line up by coincidence.
This is why controlled experiments are so powerful. If you change only one variable on purpose, keep everything else the same and assign samples at random, the independent variable is the only explanation left for the change. Observational studies (surveys, field data) can find correlations and suggest causes, but need extra evidence before they can show causation. That extra evidence is how scientists showed that smoking causes lung cancer: many studies, a dose–response relationship (more cigarettes, more cancer), a mechanism, and experiments on cells and animals.
- For "Ice cream and drownings", guess the hidden variable, then colour the points by it. What do you notice about where the hot months sit?
- Keep the hidden variable fixed. What happens to r? What does that tell you?
- Try "Shoe size and reading" and "Firefighters and damage". Would giving children bigger shoes improve their reading?
- Now try the plant experiment. Why does the link survive when the other variable is fixed?
Writing the discussion
The discussion is where you show you understand your results, and it usually carries the most marks. A good discussion:
- Describes the trend with data (as above).
- Explains it using science: why did sugarcane mulch lose the least water?
- Compares it with your background research and the hypothesis. Do your results agree with other studies? If not, why might they differ?
- Explains anomalies: outliers, unexpected results and what might have caused them.
- Evaluates the method (see below).
- Suggests further research: what new question do your results raise?
Evaluating your investigation
Evaluation means judging how far your results can be trusted, using the terms from 8F. Be specific about each problem, which way it affected the results, and how to fix it.
| Ask | Weak evaluation | Strong evaluation |
|---|---|---|
| Validity: was it a fair test of the question? | "The experiment was fair." | "Pots near the edge of the carport got more wind and lost about 10 g more water. Randomising pot positions daily would remove this confounding variable." |
| Reliability: were results consistent? | "We should repeat it more." | "The four pots for each mulch were within 9 g of each other, so the differences between mulches (20 g or more) are reliable." |
| Accuracy: how close to the true values? | "There was human error." | "The balance read to 0.1 g, under 0.1% of the masses measured, so measurement uncertainty was negligible. The bigger issue was water draining out of the bottom, which was not captured." |
Writing the conclusion
A conclusion is a short paragraph that answers the aim directly, using the key data, and states whether the hypothesis was supported. It doesn't introduce new ideas.
Test yourself
Towns with more libraries have more crime. The most likely explanation is
Population is a hidden variable driving both.
A correlation coefficient of r = −0.92 means
Close to −1: as one increases, the other decreases consistently.
Which is the strongest evaluation statement?
It names a specific problem, its effect on the results, and a fix.
Why are randomised controlled experiments better than surveys at showing cause and effect?
Random assignment and control remove hidden variables.
Name each pattern.
Put the parts of a discussion in a sensible order.
A variable that causes two others to change together is a (confounding) variable. When the effect actually causes the supposed cause, it is reverse . A relationship where both variables increase together is a correlation. A graph that curves and levels off shows a non- relationship. A conclusion must answer the .
A survey of 500 Year 10 students found that those who ate breakfast every day had higher average exam marks than those who didn't. A news headline says "Breakfast boosts exam results". Evaluate this claim and describe an investigation that could test whether eating breakfast causes better exam performance.
The survey shows a correlation between eating breakfast and exam marks, not that breakfast causes higher marks. Students weren't assigned to eat breakfast or not; they chose, so the groups probably differ in other ways. Students from households with regular routines, more support at home or more money may be more likely to eat breakfast and also more likely to do well, and students who sleep enough may both have time for breakfast and concentrate better. These are hidden variables. The data was also probably self-reported, which may not be accurate. The headline overstates the finding. To test causation, a researcher could run a crossover experiment with consenting volunteers (and parent or carer consent): on some test mornings each student eats a standard breakfast and on others eats nothing until after the test, with the order randomised. The tests would be of equal difficulty, at the same time and place, and marked by someone who doesn't know which condition each paper came from. Because each student is compared with themselves and the order is random, breakfast becomes the only systematic difference, so a consistent difference in marks would be good evidence that breakfast affects performance. The trial would need ethics approval and short time periods so no one is harmed by skipping a meal.