Self Report in Research and Online Response Biases (IX)

Tips for Structured Self Report

Self Report in Research and Online Response Biases, Developing Structured Self-Report Instruments, Tips for Developing Structured Self-Report Instruments, Tips for Developing Structured Self-Report Instruments.

    Although self-reports represent a powerful mechanism for obtaining data, researchers who use this approach should always be aware of the risk of response biases that is, the tendency of respondents to distort their responses. Perhaps the most pervasive problem is people's tendency to present a favorable image of themselves. 

    Social desirability response bias refers to the tendency of some individuals to misrepresent their responses consistently by giving answers that are consistent with prevailing social values. This problem is often difficult to combat. Subtle, indirect, and delicately worded questioning can sometimes help to alleviate this response bias. The creation of a permissive atmosphere and provisions for respondent anonymity also encourage frankness.

    Some response biases are most commonly observed in composite scales. These biases are sometimes referred to as response sets. Scale scores are seldom entirely accurate and pure measures of the critical variable. A number of irrelevant factors are also being measured at the same time. 

    Because response set factors can influence or bias responses to a considerable degree, investigators who construct scales must attempt to eliminate or minimize them. Extreme responses are an example of a response set that introduces biases when some individuals consistently select extreme alternatives (eg, “strongly agree”). 

    These extreme responses distort the findings because they do not necessarily signify the most intense feelings about the phenomenon under study. There is little a researcher can do to counteract this bias, but there are procedures for detecting it. Some people have been found to agree with statements regardless of content. Such people are called yea-sayers, and the bias is known as the acquiescence response set. 

    A less common problem is the opposite tendency for other individuals, called nay-sayers, to disagree with statements independently of question content. The effects of response biases should not be exaggerated, but it is important that researchers who are using self-reports give these issues some thought. 

    If an instrument or scale is being developed for general use by others, evidence should be gathered to demonstrate that the scale is sufficiently free from response biases to measure the critical variable.

Developing Structured Self-Report Instruments

    A well-developed interview schedule or questionnaire cannot be prepared in minutes or even in hours. To design useful, accurate instruments, researchers must carefully analyze the research requirements and attend to minute details. The steps for developing structured self-report instruments follow closely those outlined. However, a few additional considerations should be mentioned. 

    Once data needs have been identified, related constructs should be clustered into separate modules or areas of questioning. For example, an interview schedule may consist of a module on demographic information, another on health symptoms, a third on stressful life events, and a fourth on health-promoting activities. 

    Some thought needs to be given to sequencing modules, and questions within modules, to arrive at an order that is psychologically meaningful and encourages candor and cooperation. The schedule should begin with questions that are interesting, motivating, and not too sensitive. The instrument also needs to be arranged to minimize bias. 

    The possibility that earlier questions might influence responses to subsequent questions should be kept in mind. Whenever both general and specific questions about a topic are included, general questions should be placed first to avoid “coaching.” Every instrument should be prefaced by introductory comments about the nature and purpose of the study. In interviews, the introductory comments would be read to respondents by the interviewer, and often incorporated into an informed consent form. 

    In SAQs, the introduction usually takes the form of a cover letter that accompanies the questionnaire. The introduction should be carefully constructed because it represents the first point of contact with potential respondents. An example of a cover letter for a mailed questionnaire is presented. 

    When a first draft of the instrument is in reasonably good order, it should be discussed critically with people who are knowledgeable about questionnaire construction and with experts on the instrument's substantive content. The instrument should also be reviewed by someone capable of detecting technical problems, such as spelling mistakes, grammatical errors, and so forth. 

    When these various people have provided feedback, a revised version of the instrument can be pretested. The pretest should be administered to individuals who are similar to current participants. Ordinarily, 10 to 20 pretests are sufficient.

Tips for Developing Structured Self-Report Instruments

    Although we all are accustomed to asking questions, the proper phrasing of questions for a study is an arduous task. In this section, we provide some tips on wording questions and response options for self-report instruments. Although most advice is specific to structured self-reports, some suggestions are equally appropriate for qualitative interviews.

Tips for Wording

    Questions In wording questions for self-reports, researchers should keep four important considerations in mind. 

1.Clarity. Questions should be worded clearly and unambiguously. This is usually easier said than done. Respondents do not necessarily understand what information is needed and do not always have the same mind-set as the researchers. 

2. Ability of respondents to give information. Researchers need to consider whether respondents can be expected to understand the question or are qualified to provide meaningful information. 

3. Bias. Questions should be worded in a manner that will minimize the risk of response biases. 

4. Sensitive information. Researchers should strive to be courteous, considerate, and sensitive to the needs and rights of respondents, especially when asking questions of a private nature. Here are some specific suggestions with regard to these four considerations:

    Tips for Formatting an Instrument The appearance and layout of an instrument may seem a matter of minor administrative importance. However, a poorly designed format can have substantive consequences if respondents (or interviewers) become confused, miss questions, or answer questions they should have omitted. 

    The format is especially important in questionnaires because respondents typically do not have a chance to seek assistance. The following suggestions may be helpful in laying out an instrument.