«1. Introduction The design of a sampling strategy for a qualitative study is as important as that for quantitative inquiry. A well-defined sampling ...»
Designing sampling strategies for qualitative social research: with
particular reference to the Office for National Statistics’ Qualitative
The design of a sampling strategy for a qualitative study is as important as that for
quantitative inquiry. A well-defined sampling strategy that utilises an unbiased and
robust frame can provide unbiased and robust results.
There is a tendency, particularly within a quantitative environment, to consider that the sampling strategy for qualitative research is of lesser importance to that where statistical inference is required. Indeed, it is not unknown for those unfamiliar with qualitative research methods to suppose that no more than a convenience sampling strategy is applied. That is to say the researcher makes no attempt, or only a limited attempt, to ensure that the sample is an accurate reflection of the population.
This paper attempts to show why it is as important to develop a robust sampling strategy, from a well-constructed sampling frame, for qualitative research practice.
The paper also discusses how the Office for National Statistics (ONS) puts theory into practice using a qualitative ‘Respondent Register’, developed for use for sample frame construction for qualitative social research.
1.1 Understanding qualitative research It is difficult to discuss the design of qualitative sampling strategies without first discussing the nature and purpose of qualitative research, and how its approach differs from quantitative research.
The terms ‘quantitative’ and ‘qualitative’ are used as overarching categories covering a wide range of approaches and methods within each. However, the very bases of those approaches differ.
Quantitative research, by definition, implies a measurement or numerical approach.
The methodology employed is based on the testing of hypotheses deduced from theory. Using statistical inference the results may be generalised to the population.
Qualitative research aims to provide an in-depth understanding of the world as seen through the eyes of the people being studied. It aims not to impose preordained concepts; hypotheses and theory are generated during the course of conducting the research as the meaning emerges from the data. Statistical inference is not the objective, although within government, results are used to inform policy and therefore some form of generalisation or transferability is implicit.
Qualitative research may stand alone or in conjunction with quantitative research, used before, along side or after. Where the studies are associated then the sampling strategy for both should ideally be considered at the same time.
2. Informing the design of a qualitative sampling strategy Different qualitative sampling strategies may be used at different stages of the research, or for different research purposes.
Questions which the researcher should ask themselves at the outset, and which will inform the design of the sampling strategy, are the similar for both quantitative and
qualitative research. They are:
• What are the research objectives?
• What is the target population?
• Who should be excluded from the sample?
• Who should be included in the sample?
• What is the budget?
• What is the reporting time period?
• How many qualified researchers are available to work on the project?
• What sampling technique(s) should be employed?
• How are the data to be analysed?
• What data collection methods should be employed?
• What are the sample criteria?
• How long will the interview be?
• What size should the sample be?
• What should be used as the sampling frame?
• How should potential respondents/participants be recruited?
All of the above are interdependent, however some of the questions require a more detailed discussion with regard to their application in a qualitative research environment.
2.1 Research objectives Clear definition of the research objectives is crucial. Time spent in clarification with the client is time well spent, as the objectives, of course, determine the route of the research. In qualitative research the objectives may be refined as the research progresses and new incites are realised.
Time, budgetary and other resource constraints may impact on the qualitative sample design but should not be allowed to undermine it. The nature of the data collection method (e.g. cognitive, in-depth, or group interview), the human resources available to the project and their skills base, are also important considerations.
2.3 Sampling technique The sampling technique employed is a crucial element of the overall sampling strategy.
At this point it is important to understand why probability sampling is inappropriate for qualitative research. In probability sampling members of the research population are chosen at random and have a known probability of selection. Groups are represented in the sample in their true proportions; or, where unequal probabilities are used the data are reweighted back to the true proportions. The aim is to produce a statistically representative sample, suitable for hypothesis testing.
Qualitative research uses non-probability sampling as it does not aim to produce a statistically representative sample or draw statistical inference. Indeed, a phenomenon need only appear once in the sample.
Purposive sampling is one technique often employed in qualitative investigation. With a purposive non-random sample the number of people interviewed is less important than the criteria used to select them. The characteristics of individuals are used as the basis of selection, most often chosen to reflect the diversity and breadth of the sample population.
However, there are different approaches to purposive sampling some of which focus on different aspects of the sample members, cases are chosen because they are considered more extreme, for example. One form of purposive sampling is ‘theoretical sampling’, developed from the ‘grounded theory’ approach (Glasser and Strauss, 1967). The term ‘grounded theory’ expresses the idea that theory is generated, through an iterative process, involving the continual sampling, collection and analysis of data to inform the next stage of the sample design, until ‘theoretical saturation’ is achieved; that is, no new ideas or theories emerge. The iterative nature of the theoretical sample design is important. It gives the researcher the opportunity to analyse the data as the sampling progresses and means that the researcher can add to or change the emphasis of the sample design, and in doing so ensure robustness of the theories generated. It is therefore valuable to have considered the analysis technique early on in relation to the qualitative sampling strategy.
Sometimes theoretical hypothesis generation is not the primary aim of the research.
Where the sample population is clearly defined, such as when testing already operational survey questions, and where resource and time constraints are in place, then a more constrained purposive sampling strategy can be devised that avoids iteration and does not necessarily achieve saturation, on the grounds of diminishing returns.
Whatever approach is used, some advance knowledge of the population under investigation is necessary when carrying out purposive sampling.
2.4 Sample criteria A decision will be required as to the sample selection criteria. That is, what characteristics will need to be reflected in the sample population to address the research question. The decision on which criterion to use will be informed by the policy advisor and other subject specialists, as well as a review of the current literature. The researcher will need to know whether particular sub-groups need to be included to ensure breadth. The criteria used may be based on demographic characteristics or behaviours or attitudes, and will need to be prioritised if purposive sampling is to be employed. This is partly influenced by the fact that qualitative research is often, but not always, based on a relatively small number of cases so it may not be possible to include all of the sample criteria in the sample design. Some criteria may be considered more important than others in relation to the research objectives.
2.5 Interview length
The intensity and therefore the length of the qualitative interview will also impact on the design of the qualitative sampling strategy and the decision of sample size. Longer interviews may provide more data than shorter interviews. A decision may be taken, depending on the nature of the study, to conduct a larger number of shorter interviews or a smaller number of longer interviews.
2.6 Sample size
A feature of qualitative sampling is this fact that the number of cases sampled is often small. This is because, as mentioned earlier, a phenomenon only need appear once to be of value. There is no need for scale as there is no need for estimates of statistical significance. Furthermore, because qualitative investigation aims for depth as well as breadth, the analysis of large numbers of in-depth interviews would simply be unmanageable because of a researcher’s ability to effectively analyse large quantities of qualitative data. However, the small-scale approach only works if the researcher has a strong sampling strategy (Ritchie and Lewis 2003).
The issues that should be considered when determining the sample size for qualitative investigation are dependent on the heterogeneous or homogeneous nature of the sample population, or requirements of the data collection methods employed; for example, focus groups tend to be more productive and manageable if participants have some commonality.
The number of selection criterion required and the degree to which criteria are nested (dependent on whether certain characteristics are to be controlled for e.g. age), are important considerations. The intensive nature of the study; whether multiple samples are required, the inclusion of a control sample for instance; and the resources available to conduct the study, are also important for determining sample size (Ritchie and Lewis 2003).
To provide some idea of the scale of qualitative investigation one might expect to achieve between 20 and 50 interviews for a one-to-one investigation and around 60 to 100 participants at group interview, depending on the research question.
The size of the sample required will of course also feed into the decision about the type of sampling frame to use.
2.7 Sampling frames A sampling frame is a list or map that identifies most units within the target population. (Missing units are referred to as undercoverage.) 2.7.1 Frame evaluation When evaluating the effectiveness and efficiency of any sampling frame for qualitative research, it is important, as with quantitative research, to consider whether the frame is comprehensive. That is, all of the target population are included. The full range of dimensions, and information needed to inform the sample selection, should be covered. This is because sections of society missing from the frame may have different characteristics and indeed different behaviours, opinions and attitudes from those covered by it. This undercoverage may affect the results if associated with the subject of enquiry for example, phenomena may not be raised or survey questions not tested thoroughly.
It is however, also important to consider overcoverage. Sample members may be listed more than once, or the list may contain members considered out of scope for the purposes of the study. Their inclusion in the study could impact on the findings and indeed on resources and the ultimate cost of the project.
Furthermore, the frame should also contain sufficient numbers in each sub-group to provide the sample size required, as not everyone who is eligible will be willing to take part. Three or four people may be contacted who fulfil the sample criteria before one agrees to take part.
A practical consideration is whether the frame can be easily manipulated in order to identify those with the relevant characteristics.
As with frames for quantitative research, geographical clustering is important because if the population are highly dispersed then fieldwork will be more resource intensive.
It is also important to know whether the potential respondent contact details are complete and up-to-date.
Lastly, whether the time and cost involved in using the frame is justified.
2.7.2 Types of frame There are basically two types of frame (or list) available for social research practice.
These are existing lists that can be used as frames, perhaps after some manipulation, or frames that need to be constructed.
Existing frames Existing frames usually comprise of records which were constructed for administrative purposes, for example, a published list of General Practitioners. As such, they tend not to have been designed with research purposes in mind and may not be very well maintained from a research perspective. Furthermore, some administrative records, for example, benefit records, will be covered by data protection and confidentiality issues, which can make them difficult to access.
Existing survey samples can provide a frame, but they may also not have been designed with the current research interest in mind. (Although if the qualitative research is a follow-up to a quantitative survey and has been planned as such, then the qualitative sample criteria can be built into the quantitative survey in advance.) Existing survey samples may be affected by undercovereage and response bias, not least because they are dependent on the survey response rate.