8B

Researching and judging sources

Primary and secondary sources, how peer review works, how to judge whether a source is reliable (including AI chatbots), writing a short literature review, respecting First Nations knowledge as a source, and referencing in APA style.

In 1998 a paper in The Lancet, one of the world's most respected medical journals, suggested a link between the MMR vaccine and autism. It studied just 12 children. Other scientists tried and failed to find any link in studies of millions of children, an investigation found the data had been manipulated, and in 2010 the journal retracted the paper and the lead author was struck off the UK medical register. By then, vaccination rates had dropped and measles had returned. Even a famous journal can publish something wrong, so researchers learn to check every source.

Primary and secondary sources

The words "primary" and "secondary" are used in two slightly different ways, so be careful:

  • Primary data is data you collect in your own investigation. Secondary data was collected by someone else, such as Bureau of Meteorology temperature records.
  • A primary source in the scientific literature is the original report of research, written by the people who did it, usually a paper in a journal. A secondary source summarises or explains other people's research: textbooks, encyclopedias, news articles and most websites.

Background research for your project will mostly use secondary sources to understand the science, plus a few primary sources (actual studies) close to your question.

How science checks itself: peer review

Peer review. Experts check a paper before it is published; after publication, other scientists test it again.

When scientists send a paper to a journal, the editor asks two or three independent experts in the field to review it. They check whether the method actually tests the question, whether the conclusions follow from the data and whether anything is missing. Most papers are sent back for changes; many are rejected. Peer review isn't perfect, as the 1998 paper shows, which is why the real test comes afterwards, when other scientists try to repeat the work. A result becomes trusted when independent teams get the same answer.

Some kinds of source give stronger evidence than others. Reviews that combine many studies sit at the top; opinion and advertising at the bottom.

Judging a source

A website ending in .gov.au or .edu.au is often reliable, but not always, and many excellent sources end in .com or .org. Instead of judging by looks, ask four questions:

QuestionLook forWarning signs
Who wrote it?Named authors with expertise in the field; a known organisationNo author; an influencer or company with no science background
What's the evidence?Data, methods and references you can follow"Studies show…" with no studies named; one person's experience
When?A recent date, or a classic that is still acceptedNo date; old information in a fast-changing field
Why does it exist?To inform or report researchTo sell something, persuade, or get clicks

Fact-checkers use a trick called lateral reading: rather than reading a page from top to bottom, open new tabs and search for what other sources say about the author or organisation. A site can look very professional and still be run by a lobby group.

AI chatbots

Chatbots can be handy for explaining an idea or suggesting search terms, but they generate text that sounds right, and they sometimes invent facts and even references that look completely real, with plausible authors, journals and page numbers. Never cite a chatbot as a source, and never cite a paper you haven't found and read yourself. Your school will have rules about whether and how you may use AI, and you must say if you have.

InteractiveCheck the sources
Try this
  • For each source, use at least two of the four checks before you decide. Which sources change your mind once you check them?
  • Which source would you use for background, but cite something else instead?
  • Why is the AI chatbot's reference a problem even though it looks like a real journal article?
  • The journal review is strong evidence. What is its limitation for this particular question?

Writing a short literature review

The background section of your report is a mini literature review. It should do more than list facts: it should show what is already known, what isn't, and how your question fills a small gap. Group sources by idea rather than writing a paragraph about each one, and say where sources agree or disagree.

Example: weak and strong background

Compare two background paragraphs for the mulch project.

"Mulch is material put on top of soil. There are many types of mulch. Mulch is sold at garden shops. Mulch is good for gardens." (Facts with no sources, no science and no link to the question.)
weak
"Most water lost from bare soil in summer is lost by evaporation from the surface, which increases with temperature and wind (author, year). Organic mulches shade the soil and slow air movement over it, while gravel absorbs heat (author, year). Western Sydney now has more days over 35 °C than in the 1970s (author, year). Few studies compare common supermarket mulches in pots under these conditions, which this investigation tests."
strong

First Nations knowledge as a source

Aboriginal and Torres Strait Islander peoples hold detailed knowledge built up over tens of thousands of years, for example about seasons, fire, plants and animal behaviour. Much of it is now shared in published, community-approved forms, such as the Indigenous seasonal calendars developed by communities with CSIRO, or books and websites written by Aboriginal authors and organisations. Use and cite these like any other source, naming the people and community the knowledge comes from. Some knowledge is not public, and some is only for certain people. If you want to include knowledge that hasn't been published, you need permission first (see 8E).

Referencing

Referencing gives credit to other people's work, lets your reader check your sources, and protects you from plagiarism. Most Australian schools use APA style (7th edition) or Harvard, which are very similar. Each source appears once in an alphabetical reference list, and you cite it in the text with the author and year.

The parts of an APA reference for a journal article and a web page, and how they are cited in the text.

Write each reference into your logbook as soon as you use the source, including the date you accessed web pages. Hunting for a half-remembered website the night before the deadline wastes hours.

Test yourself

  1. Which is a primary source in the scientific literature?

  2. A chatbot gives you a reference to a paper, but you can't find the paper anywhere. You should

  3. Lateral reading means

  4. In APA style, the in-text citation for a 2024 report by the Bureau of Meteorology is

Label the levels of evidence.

Systematic reviews and meta-analysesSingle peer-reviewed studyTextbooks, expert and government sitesNews articles reporting on researchOpinion, anecdotes, ads, social mediastronger evidence

Put the peer review process in order.

A list of every source used, in alphabetical order, is the reference . Data you collect yourself is data. Experts checking a paper before publication is called peer . A paper withdrawn by a journal because it is seriously wrong has been . Using someone else's words or ideas without credit is .

5 marks

A student finds a website claiming that a magnetic bracelet improves sports performance. The site has a professional design, customer reviews and the line "clinically proven". Explain how the student should evaluate this source and decide whether to use it in a research project.