iDevice icon Reflection

Consider the following statements. Would you trust the conclusions from the following research findings? If not, why not?

  • In the UK there are estimated to be 4.8m households with one or more resident dog.. From a survey of 10 dog owners, picked at random in Winchester city centre, it was concluded that the most popular breed of dog is the labrador.
  • Elderly male subjects were asked whether they had ever smoked and were divided into two groups according to the answer. The group who had smoked was not found to suffer from significantly greater ill health than the group who had not.
  • In a study of exam results for 16 year old students, the examinees were divided into two groups. Those with birthdays between September and January were allocated to the group who received extra tuition and those with birthdays between February and August were allocated to the control group who received no extra tuition. It was found that the group receiving extra tuition did significantly better in their exams at the end of the year.
  • Two interviewers were used in a study about drug use. One was an ex-drug user. They obtained slightly different views from their interviewees about aspects of life as an addict

Bias can take many forms - some intentional and some not. Some of the more common types are:

  • Insufficient sample: A generalisation or conclusion is made from too small a sample size.
  • Observer bias: One observer consistently over or under records data values.
  • Information bias: Data is systematically incorrectly recorded.
  • Cherry picking: The deliberate choice of the data or scientific studies that support your view, while ignoring the data or studies that oppose your view.
  • Confirmation bias: People tend to remember things that confirm their beliefs and forget things that do not. For example they will more frequently remember times they made a correct prediction than times they were wrong and they will tend to look for information that confirms their established views rather than information that contradicts them.
  • Recall bias: Subjects forget details of their past history. For example: patients with needing replacement knees may remember earlier injuries to that joint. People without problems in their knees may not remember such incidents, particularly if they were minor inconveniences or a long time ago.
  • Timing bias: The timing of an experiment can affect the results you get.
  • Assessment bias: This can occur when people are aware of the thing being studied. This is a particular problem where you are looking for a subjective response for instance, double blinding (where neither the experimenter nor the subject knows whether they are in the control group or not) can help to alleviate this.
  • Omitted evidence: Leaving out evidence that would weaken or even cause the average reader to dismiss the claim.

Note that we are discussing systematic errors here that tend to skew the conclusions of your research in one particular direction.

In quantitative research, such bias can be avoided by careful experimental design and researcher integrity. In qualitative research we shall see below that the concept of eliminating bias is contentious and is not necessarily an aim of this kind of study at all.

In addition to systematic errors, all numerical data sets also contain what are called random variations or random errors that are unavoidable. We will discuss these further in the section on quantitative analysis.

iDevice icon Activity 4:Identifying bias

Complete the following activity online to receive feedback and in your log book as a record.

Task 1

Read the paragraph below and fill in the missing words that describe the type of bias that may have occurred:

1. A group of people in their late thirties were asked to give a list of their GCSE grades. Researchers found that the grades reported were generally slightly higher than the actual grades achieved. This is an example of JXUwMDJhJXUwMDE3JXUwMDA2JXUwMDAyJXUwMDBkJXUwMDAw bias.

2. A random sample of people was asked how they would rate the dentistry degree at Oxford University. The subjects said they would rate it very highly. In fact there is no dentistry degree at Oxford and this is an example of JXUwMDNiJXUwMDBjJXUwMDAxJXUwMDA4JXUwMDBmJXUwMDFiJXUwMDFmJXUwMDBjJXUwMDE1JXUw MDFkJXUwMDA2JXUwMDAx bias.

3. A poll during the World Cup showed that 90% of people regarded football as their favourite sport. It is most likely that these findings are affected by JXUwMDJjJXUwMDFkJXUwMDA0JXUwMDA0JXUwMDA3JXUwMDA5 bias.

4. In a survey on attitudes to environmental issues the data collectors were issued with a list of standard questions. Each data collector was allowed to design their own form to record the answers. As a result the findings were suspect to be distorted due to JXUwMDM3JXUwMDBkJXUwMDExJXUwMDE2JXUwMDE3JXUwMDA0JXUwMDEzJXUwMDE3 bias.

5. A sample of 5 people in a pub suggested that beer drinkers prefer real ales to lagers. It was, however, acknowledged that the sample was JXUwMDMxJXUwMDA3JXUwMDFkJXUwMDA2JXUwMDEzJXUwMDAwJXUwMDBmJXUwMDBhJXUwMDBhJXUw MDBjJXUwMDBiJXUwMDFh for such a generalisation.

6. An experiment was proposed to test whether boys were better at tennis than girls. The supervisor suggested that careful design would be needed and recommended a change of title to try to minimise any suggestion of JXUwMDNiJXUwMDBjJXUwMDAxJXUwMDA4JXUwMDBmJXUwMDFiJXUwMDFmJXUwMDBjJXUwMDE1JXUw MDFkJXUwMDA2JXUwMDAx bias.

7. In a study of back pain treatment subjects were divided into two groups and one group was treated with massage while the other group received no treatment. The results suggested that the subjects, who received treatment, rated the pain lower on a score of 1 to 5 than those without treatment. This may have more to do with JXUwMDM5JXUwMDEyJXUwMDAwJXUwMDE2JXUwMDE2JXUwMDAwJXUwMDFlJXUwMDA4JXUwMDBiJXUw MDFh bias than with successful treatment.

8. Of the 36 rabbits examined in the study, we present the data for 3 and conclude from it that most rabbits are white. A clear case of JXUwMDM3JXUwMDAyJXUwMDA0JXUwMDFkJXUwMDAwJXUwMDExJXUwMDAx JXUwMDNkJXUwMDEzJXUwMDFmJXUwMDBkJXUwMDAxJXUwMDBiJXUwMDBkJXUwMDA2 !

In a study where the data analysis is purely quantitative, bias can usually be eliminated by careful research design with the use of appropriate sampling strategies, data protocol, blinding and so forth. You should be aware, however that there is always an aspect of bias that comes from the researcher's interests, the choice of research question and the choice of the parameters that will be measured. This is inevitable, arising from the fact that research is undertaken by human beings with experience, ideas, prejudices and personal philosophies. It would be possible to worry about this so much that no research was ever started. A more realistic approach is to recognise that these personal aspects are often the drivers for a research study in the first place. In many cases, especially where human or animal subjects are involved in the study, the ethics committee plays a vital role in considering whether the research question is unacceptably biased. In a quantitative study it is the subsequent aspects of the design and the analysis that must be undertaken according to an established protocol to ensure bias elimination.

In qualitative approaches bias is harder, if not impossible to eliminate from the data collection and analysis stages of the study and it becomes necessary to acknowledge and account for it, rather than trying to remove it. For a discussion of the development of understanding of bias amongst a group of students on a Masters course in conflict resolution you may like to look at an article in The Qualitative Report; an online journal which discusses qualitative research: Bias in Qualitative Research: Voices from an Online Classroom by Beloo Mehra

In the next part, you will consider when it is reasonable to claim that a piece of research shows that one thing causes another.