What Is Evaluation
Evaluation
is a method for measuring the effect of some purposeful action on a particular
situation. It is often described as an assessment of worth. In evaluation, both
anticipated and unanticipated outcomes are important and are included in the
discussion of findings and the publication of results. The purpose of
evaluation is to provide information for decision makers who usually have some
stake in the outcome of the intervention.
Evaluation Methods or Types
Evaluation
methods have been categorized along a continuum ranging from simple assessment,
in which informal practices are used to look for indication of outcome, to
evaluation research, in which research methods are used to allow for
generalization to other comparable situations. In actuality, the use of
informal practices for determining intervention outcome is never appropriate.
Consequently, the term evaluation should suffice for all efforts in which a
systematic process is used to determine the effect of some intervention on some
anticipated outcome. The research component of the term is assumed. No matter
what the purpose of the evaluation, the issue of rigor is always foremost, and
the methods and measurement approaches used should involve the same level of
attention given to any research method.
First Type Evaluation
According
to Rossi and Freeman (1985), evaluations serve one of three purposes: (1) to
conceptualize and design interventions, (2) to monitor implementation of some
intervention, or (3) to assess the utility of some action. In the first type of
evaluation, studies focus on (a) the extent of the problem needing
intervention, (b) who should be involved in or targeted for the intervention,
(c) whether the proposed intervention will address the problem or the needs of
individuals, and (d) whether the chance for successful outcome has been
maximized.
Second Type Evaluation
In
the second type of evaluation, studies focus on what is done; they generally
are referred to as process evaluation studies. These studies also determine
whether the intervention is reaching the targeted population and whether what
is done is consistent with what was intended. Process evaluations are essential
for determining cause and effect, although they are not sufficient by
themselves for measuring impact. That is where evaluation researchers often get
into trouble. They stop collecting data once they describe what was done;
therefore, process evaluation methods have tended to be viewed with disfavor,
which is unfortunate. Although they are insufficient by themselves, they are
absolutely necessary for determining whether the intervention caused the
outcome and if so, how and if not, why not.
Third Type Evaluation
In
the third type, studies determine both the degree to which an intervention has
an impact and the benefit of the intervention in relation to the cost. The
degree of impact is referred to as the intervention's effectiveness, and the
degree of cost is referred to as its efficiency (Rossi & Freeman, 1985).
Recent
writings on evaluation focus on the need for theory to guide the investigation
and frame the results. Authors have identified theories that range from those
targeted solely for the purposes of designing evaluations to those directed at
the expected relationships between intervention and outcome. For example,
behavioral theories are often used to develop interventions targeted at
changing health behaviors; they also are used to select measures for
determining impact. Evaluation theories, on the other hand, focus on the
purpose of the study-whether it is for determining what goals or outcomes
should be examined, how the treatment should be developed and delivered, or
under what conditions certain events occur and what their consequences will be.
HT Chen (1990) has defined these two types of evaluation theory as normative
(the first type) and causative (the second), normative theory is derived from prior
knowledge, usual practice, or theory. Causative theory is empirically based and
specifies causal relationships between intervention and outcome.
Measuring the Effectiveness of Interventions
Measuring
the true effect of the intervention is often difficult. Evaluation studies are
subject to the same measurement and analysis problems associated with other
designs. In addition, Ingersoll (1996) has summarized several others that are
important to evaluation research. Among these is the need to measure the extent
of the intervention introduced, which is frequently absent from reports of
evaluation studies. This information assists in demonstrating cause-and-effect
relationships and clarifies what magnitude of the intervention is required
before an effect is seen. It also helps to prevent the potential for Type III,
IV, and V evaluation errors, which affect statistical conclusion validity and
generalizability validity.
Type
III evaluation error is an error in probability and results in solving the
wrong problem instead of the right problem. It usually occurs when the program
is not implemented as planned and when insensitive measures are used to
determine effect. Type IV error occurs when the evaluator provides information
that is useless to stakeholders. Type V error involves confusing statistical
significance with practical significance, which ultimately leads to Type IV
error (Ingersoll, 1996).
Benefits or Purposes
Evaluation
is key to measuring intervention magnitude and effect. To assure that
evaluations are useful, however, steps must be taken to design them according
to some meaningful conceptual framework; and close attention must be paid to
maximize the rigor of the methods, analysis, and rejection of alternative
hypotheses. Approaches to quality control recommended for other
non-experimental, quasi-experimental, and experimental designs are appropriate.
With attention to these aspects of the evaluation process, evaluations become
an effective means for extending nursing science.
Give your opinion if have any.