Sampling

Sampling

Sampling

In order to acquire data it is normally necessary to identify the group you wish to study (people, animals, items etc) and then to investigate the attributes of a selection of them. The method of choosing which members of the group you will study is known as sampling and the sub-group selected in this way is called a sample. Watch the slide show about sampling and then carry out the activity below.

IDevice IconActivity 2: Sampling

Task 1:

  • Consider at least one of the four research projects described in the case studies, A to D.
  • Answer the questions that follow them in your log book.
Once you have had a go, look at the feedback to see if you agree.
iDevice icon Case Study A: 21st Century Science - Evaluation of teaching for scientific enquiry

This project is currently being carried out in the School of Education at Southampton with funding from the Nuffield Foundation:

Research Team: Professor Mary Ratcliffe, Ms Pam Hanley, Professor Jonathan Osborne

Description:

In 2003 21st Century Science GCSE was piloted in 80 schools in England. This evaluation study is one of three projects which the funders commissioned to inform further development of the course.

Research Aims: The research had the following objectives. To explore:

  • how effectively the two components of the 21st Century Science course, science explanations and ideas-about-science - are interrelated and recognised in the teaching of the course;
  • the extent to which teachers are successful in handling ideas-about-science and science-related issues that may involve a range of social, economic, political and ethical ideas;
  • the views of students about 21st Century Science, particularly with respect to their experience of ideas-about-science and open-ended issues in terms of their learning and motivation and the nature of teaching they experienced.

Design & Scope: The following methods were used to collect appropriate data. For the first and second objective:

  • a questionnaire to teachers
  • observation of teachers' practice
  • interviews with teachers

For the third objective:

  • a questionnaire to students
  • focus group discussions with students

Questions:

  • What are the populations of interest?
  • What sampling frame might you use to find your sample?
  • What sampling strategy would you use?
iDevice icon Case Study B: THE ECONOMICS OF SOIL EROSION: A model of farm decision-making

Derek Eaton - December 1996

ABSTRACT

Soil erosion is widely considered to be a serious threat to the long-term viability of agriculture in many parts of the world. The problem is particularly serious in certain developing countries. This paper examines key factors affecting smallholder farmers, decisions about soil depletion and conservation. The analysis focuses exclusively on the on-site productivity losses due to soil erosion in an attempt to understand farmer behaviour, thus ignoring any externality effects, or off-site costs.

The physical processes of soil erosion are described and its economic effects are reviewed, drawing on theoretical and empirical studies to date. Contrary to arguments that farmers are not aware of the extent and effects of erosion, an economic rationale for them to deplete their soil may be found in relatively simple conceptual models. While much of the research focuses on the North American context, this paper emphasises the relevance of economic models for analysing the situation in developing countries.

The two paragraphs above, form part of the abstract of a report which you can download from the web site for the International Institute for Environment and Development at: http://www.iied.org/pubs/display.php?o=8083IIED.

Questions

  • What would be the target population for this study?
  • What sampling frame might you use?
  • What sampling strategy would you select?
  • Might you need any exclusion criteria?
iDevice icon Case Study C: Older people's views of advice about falls prevention: a qualitative study.

Several of the authors of the study described below are members of the School of Psychology at the University of Southampton

Yardley, L., Donovan-Hall, M., Francis, K. and Todd, C.(2006). Health Education Research, 21, 508-517

Abstract

The aim of this study was to gain an understanding of older people's perceptions of falls prevention advice, and how best to design communications that will encourage older people to take action to prevent fall. Focus groups and interviews were carried out... using falls prevention messages to stimulate discussion. Thematic analysis revealed that participants interpreted 'falls prevention' principally as meaning hazard reduction, use of aids and restriction of activity. Only one participant was aware that falls risk could be reduced by carrying out exercises to improve strength and balance. Falls prevention advice was typically regarded as useful in principle, but not personally relevant or appropriate. Advice about falling was often depicted as common sense, only necessary for older, or more disabled individuals, and potentially patronising and distressing. Our findings suggest that older people do not reject falls prevention advice because of ignorance of their risk of falling, but because they see it as a potential threat to their identity and autonomy. Messages that focus on the positive benefits of improving balance may be more acceptable and effective than advice on falls prevention.

Questions:

  • What is the target population for this study?
  • What sampling frame might you use?
  • What sampling strategy would you select?
  • Might you need any exclusion criteria?

iDevice icon Case Study D: A survey of behavioural characteristics of pure-bred dogs in Italy

One of the authors of the study below is a member of staff in the psychology department at the University of Southampton. The paper is published in

Applied Animal Behaviour Science 103 (2007) 118-130

Lorella Notari Deborah Goodwin

Abstract

The selection of dog breeds for functionality has progressively lost its importance and behavioural attributes originally selected are often now considered problems in household contexts. This study investigated behavioural characteristics of the 49 most popular breeds of dogs in Italy. Questionnaires were sent to 112 Italian veterinarians and 56 non-veterinarians (trainers, behaviour counsellors and animal charity officers) who rated breed behavioural characteristics and compared the behaviour of males and females. Females were considered more trainable for obedience, more demanding of affection and more housetrainable. Males were rated higher than females for all other traits except playfulness and general activity. Principal factor analysis with varimax rotation, generated two principal factors (labelled aggressivity and reactivity/immaturity) that accounted for 56% of the total variance. Nine breed groups with different behavioural characteristics we regenerated by K-means cluster analysis. These groupings had similarities with the groupings presented in the USA and UK, e.g. of the seven breeds rated as high in aggressivity in this study, five were rated high in all three countries, the Miniature Schnauzer was rated high for aggressivity in Italy and the US, but the Yorkshire Terrier was rated high only in Italy. These results provide further evidence of the need for care when transposing breed behavioural advice or treatments between countries.

Questions:

  • What is the target population?
  • What sampling frames might be suitable?
  • What sampling strategy would you use?

IDevice IconActivity 3: Sampling in your own work

The following activity requires you to find a paper in your own field that uses a sample of a wider population and then answer the same questions as you considered for the case studies. You are also asked to reflect on how you would develop a sampling scheme, if you need to use one in your research. Now open your log book and complete this activity. Do not forget to save your work to your own file space after you have made your logbook entry.

If you want to know more about sampling you can watch a longer and more detailed slide show here.

IDevice IconHow big should your sample be?

How do you decide how many members are needed in a sample? This depends very much on your aims and how much you know about your target population.

For a pilot study you may start with a convenience sample to get a feel for the amount of variation you are likely to find in the population. A more refined sampling scheme can then be developed as you begin to understand the group(s) you are dealing with.

If you are only interested in giving summary statistics (e.g. mean and standard deviation) then the sample size is not critical - just sample enough to be broadly representative.

If you have reached a stage in your research where you have a hypothesis to test you can make a calculation of the number of members required in your sample based on assumptions about the statistics of the population. This is known as a power calculation. Note that the power calculation should be based on the number of actual responses so you may need to survey more than you need to allow for non-responses. The power calculation considers:

  • the level of precision you require - are you working to a precision of +/- 5%? +/-1%? If you want more precision you need more samples.
  • the confidence level - the more confidence you want that your sample is representative of the population the more samples you need to take
  • the degree of variability - if you have a population where the attribute you are studying does not vary very much (is homogeneous) you can get away with a smaller sample than for an attribute which varies a lot across the population (is heterogeneous).

Of course you can only use a power calculation for a hypothesis where you are planning random sampling. Also, if you are using anything other than simple random sampling you will need to make a power calculation for each sub-group.

Many power calculation methods are based on the assumption that the attribute you are studying is normally distributed across the population. You will need to think about whether you expect this to be true before you make your power calculation.

You can find out more via the following links:

This one (C D Florey (1993). Sample size for beginners, BMJ, 306(6886): 1181-1184) explains the concept of determining sample size clearly, while this one is a more mathematical look at power and power calculations to determine sample size.

Although there are now many web pages that offer to help you to make a power calculation, you should be wary of any software where you cannot see how the calculation is done. It may not be correct!

One good strategy if you are not sure what sample size to use is to look at the sample sizes used in similar published studies. This can be a helpful guide to get you started. We will look at sample size a little more in the section on quantitative approaches to research

Next we will look at bias in research and consider when it is appropriate to try to eliminate it and when simply to recognise its existence.