Steps in conducting survey research- Pre fieldwork phase

Daniel Armah-Attoh, indicated that the first activities to undertake, as discussed earlier, are to:

1) Activity 1: Identify and define the research issue/problem

2) Activity 2: Establish a clear research objective, by answering the questions, “what are the reasons for undertaking the planned survey research?”, “what is the survey intended to achieve?” He noted that the lack of a clear objective may severely affect the design of questions; selection of respondents and other downstream survey activities; and may have the effect that the survey results to a large extent would be unusable.

3) Activity 3: Identify the survey population for sampling. He noted that inaccurate identification of the survey population could result in a biased sampling frame and sample, and in biased survey results. A sample is used where it is impossible to survey the total population. A sample is “a selected number of representative elements from the target population. These form the core respondents to the survey”.

3.1 The sampling process and sampling techniques

The steps in the sampling process are as follows:

a) Identify the target population as accurately as possible and in a manner that is logical within the context of the survey’s objective.

b) Develop a good sample frame by listing all elements in the target population if possible. It was stressed, that there is no perfect sampling frame because omissions, double listing and wrong elements inclusion may occur. However, one should endeavour to minimize these problems.

c) Use appropriate sampling technique to select the sample. It was noted, in particular, that one should avoid arbitrariness in the selection. The choice of sampling technique should be informed by the nature of the survey and the characteristics of the population.

3.2 Types of Sampling Techniques

Basically, there are two main types of sampling techniques.

§ Probability sampling technique.

§ Non-probability sampling technique.

In PROBABILITY SAMPLING, each element of the population has a known non-zero chance (or probability) of being selected as part of the sample. However, this known chance may or may not be equal for all elements in the population.

Examples of probability sampling are:

§ Simple Random Sampling: which is most appropriate when the population is homogenous.

§ Stratified Sampling: which is most appropriate where the target population is heterogeneous

§ Cluster Sampling:

§ Multi-stage Sampling

§ Systematic Sampling: this technique demands prior information on the size of the target population.

In NON-PROBABILITY SAMPLING, there is no known chance (or probability) of being selected as part of the sample. There are two main types of non-probability sampling:

§ purposive sampling, where the researcher selects elements of the sample with specific purpose in mind. There are three types of purposive sampling techniques:

o Quota sampling: in which elements are selected using fixed quotas.

o Expert sampling

o Snowball sampling

§ accidental sampling, where the researcher picks members of the population to form survey sample without any purpose or characteristic of interest in mind. Examples of accidental sampling techniques are:

o Convenience sampling, and.

o Judgement sampling

3.3 Sample size

Armah-Attoh indicated that It is inappropriate, and more so unscientific, to choose the size of a sample arbitrarily. The decision regarding the sample size is informed by the following criteria:

§ The desired level of precision or sampling error tolerable (e.g. 5%);

§ The desired confidence level (e.g. 95%); and

§ The degree of variability in the population (e.g. 50-50);

§ The size of the sub-group for which estimates are needed.

There are various formulas which can be used to identify the most appropriate sample size. Sometimes the sample size is inflated, in order to deal with possible non-responses. There are also formulas to assist in estimating how many additional individuals need to be added to the sample. It may be useful to employ or consult a statistician to provide information on the most appropriate sample size, and to provide input on the sampling technique.

Questions and Discussion

How do we identify the most appropriate sampling technique as some techniques are not useful in some contexts. For example, mail and email in Burundi are not possible.

  • Armah-Attoh indicated that the characteristics of the population should inform sampling techniques. For example, if you are wanting to do a survey amongst street children you wouldn’t use a phone interview.
  • Hofmeyr added that one needs to look at the context in deciding technique. What are characteristics of country, identified group (e.g. street children). The method can also bias the information because it is related to whether you can get to the target or not (e.g. why use telephone when 60% of your target population have no telephones).

How would you sample specific groups, e.g. “victims of torture”?

The moment you begin to talk about victims of torture you have a purpose in mind which means that you would need to use purposive sampling. As discussed, above, there are different types of purposive sampling. For example, the survey research on the NRC in Ghana started with the information of the victims and their contact details from the NRC. Field workers then also used the ‘snowballing’ technique, where these initial victims identified to the fieldworker other victims. Often one combines more than one sampling technique

Relationship between qualitative and quantitative, and separating fact and opinion.

By doing survey is one separating fact and opinion? In qualitative research, an opinion is regarded as fact, and sometimes a lie is still good data, as there are reasons for the lie which one would explore analytically.

Armah-Attoh noted that separating fact from opinion is important in the initial stages when designing your survey objective. You need to consult with people to inform the objective but you need to be informed by fact at this point. For example, if you go into a community and are wanting to do a survey on teenage pregnancy and are told by the community that it’s not an issue and yet you can see young girls with children all over and no education or opportunities as a result of this, following opinion you would drop the survey, following fact you would proceed and design your objectives as such.

Explaining sampling to communities, and feeding back to results.

Armah-Attoh indicated that explaining sampling is obviously part of survey ethics and community entry, and needs to be managed as such. Daniel notes that in the African context it is difficult to conduct a survey in a community or talk to individuals – particularly in rural areas – without going through the ‘opinion leaders’ or gatekeepers first. He notes, from past experience, that by going to the leader first and interviewing them you bring them on board and make them feel respected.

Presentation continued: Pre-Fieldwork Phase

After deciding on the population and sample, the researcher then needs to

4) Activity 4: Decide on what Information to collect. For a survey to capture the relevant information, the researchers must bear in mind the problem and objective of the survey. When still faced with difficulties, the researcher can conduct brainstorming; focus group sessions or informal, less structured interviews with individuals (or groups) to stimulate critical thinking through. This can give an indication of the “nice to know” and “need to know” questions to ask in the main survey.

5) Activity 5: Choose appropriate survey method to use. Though many survey methods are available, no one method is superior. Each method has to be assessed and adopted based on its relevance to the survey topic and the target survey population. (See above, under Session 7)

6) Activity 6: Develop the survey instrument: Developing the survey research instrument is just a matter translating the survey issue/problem/idea into good questions. Three forms of survey questions are easily identifiable:

a. open-ended;

b. closed-ended; and

c. partially close-ended.

7) Activity 7: Training fieldworkers and conducting pilot survey. Be sure of the calibre of persons being recruited as either Field Supervisors (FSs) or Field Assistants (FAs). One should consider their level of education; honesty; past involvement in fieldwork etc.

a. Train FSs and FAs for at least a week depending on the size and nature of the research and the expected level of involvement of field assistants.

b. Possible content of training for FSs

i. Sampling processes including selection of random sampling start points in each Primary Sampling Unit (PSU).

ii. Identification of landmarks in the area maps. FSs should be persons with good knowledge of the survey locality.

iii. Effective supervision of fieldwork: Carrying out back-checks to find out from households whether FAs did conduct interviews; sampling procedures for household and respondents selection were properly applied; questions were asked correctly; and responses recorded accurately.

iv. Conducting daily debriefing sessions with FAs (i.e. scrutinizing each and every completed instrument)


c. Possible content of training for both FSs & FAs:

i. Introduction to the survey (background etc);

ii. Objectives and scope of survey (i.e. coverage: nation-wide or restricted);

iii. Survey ethics (community entry; interviewer identification; confidentiality; consent etc);

iv. Codes of conduct (e.g. time consciousness, no drinking on the job, etc.);

v. Familiarization with the questionnaire (both national language and translated versions). This will include going through all the instructions and coding processes as well as doing mock interviews.

vi. Responsibilities of FSs and FAs in the sampling processes in the fieldwork. Main issues for discussion may include: Sample design-overview; Understanding the principle of randomness; Selecting starting points; Household selection; and Respondent Selection.

d. Deploy FSs and FAs to conduct pilot survey to pre-test the instrument using both the translated and national language versions of instrument (i.e. first level quality control measure). After the pilot, conduct thorough debriefing session by

i. going through completed questionnaires;

ii. sharing field experiences (e.g. ease of asking questions; attitude of respondents and other environmental challenges).

iii. FSs and FAs review open-ended responses, and where necessary practice assigning codes; discuss the codes assigned; in order for them to develop thorough understanding of interviewer coding responsibilities;

iv. Fine-tune both national language and translated versions of the instrument.

v. Discuss all administrative issues (i.e. contract; insurance cover; accommodation and feeding; communication etc).

Question and Discussion

What age should a survey cover?

Armah-Attoh noted that it depends what you’re wanting to achieve. The Afrobarometer uses 18 years and above because that’s the voting population. However if what you’re trying to look at how the subject of your survey affects a younger population you would want to include them.

How can one ensure that an internet poll is statistically valid?

Armah-Attoh noted that it is very difficult to control validity on internet. There are some software packages that prevent a user from going back to the site over and over but you still have to consider that you are only reaching a certain segment of the population – those with internet access. He notes that at best internet polls can give you an indication, a signpost of general trends but they are not accurate in terms of the statistics itself.

Copyright © African Transitional Justice Research Network 2006-2010. Site by Interiority.