Evaluation Theories in Education

Nursing Educational Program Evaluation

Academic program evaluation is part of  nursing education in which different theories are given.

Program Evaluation Theories

    Program evaluation theories are either method-oriented or theory-driven, depending on underlying assumptions, preferred methodology, and general approach. Method-oriented theories emphasize methods for conducting assessments, while theory-driven approaches emphasize the theoretical framework for developing and implementing assessments. The most popular approaches have been method-oriented (Chen, 1990; Sha Dish et al., 1991).

    Method-oriented approaches generally focus on the relationship between program inputs and outputs and include an emphasis on a preferred method for conducting program evaluation. Many of the method-oriented approaches emphasize quantitative research methods. Some method-oriented approaches recommend naturalistic or qualitative methods for conducting program evaluation.

    An example of a method-oriented quantitative theory of program evaluation is the social science model of Rossi and Freeman (1993). These authors believe that the use of experimental research methods will produce the most effective program evaluation. The advantage of this approach is that the measurement techniques must be reliable and valid even if an experimental design is not used to perform the evaluation. One of the main limitations of this approach is that the focus on methodology can distract evaluators from other issues, such as B. Recognize the importance of the stakeholder perspective. Furthermore, experimental designs are often difficult to apply to some aspects of educational evaluation.

    An example of a qualitatively method-oriented theory of program evaluation is Guba and Lincoln's (1989) fourth-generation evaluation. Guba and Lincoln advocate naturalistic methods for program evaluation. One particular focus of their approach is the emphasis they place on integrating stakeholder views into program evaluation. A key benefit of their approach is that the use of qualitative methods allows evaluators 10 to gain a deeper understanding of the strengths and limitations of the program in a given context. The approach is limited in that it tends to overlook outcome evaluation, which generally requires a more quantitative methodology.

     Theory-based approaches to program evaluation begin with the development of program theory. Program theory is the framework that describes the elements of the program and explains the relationships between the elements. With this approach, the evaluation of the program should check whether the theory of the program is correct and has been implemented correctly. If the program is unsuccessful in achieving outcomes, a theory-driven approach allows the evaluator to determine whether the program's failure is due to flaws in program theory or faulty program implementation. The theory-based approach often requires a variety of research methods, as evaluators select the most appropriate methodology to answer evaluation questions (Chen, 1990).

    The model based on Chen's theory (1990) is one of the most complete models for program evaluation. Although the model was designed to evaluate social service programs, it is adaptable to educational programs. Here is a brief description of the Chen Model, along with suggestions on how it might be adapted to an educational program. A more detailed application of Chen's model in developing a program evaluation plan for nursing education is provided later in this chapter.

    In 2005, Chen provided practical guidelines for professional evaluators to help them choose evaluation strategies in his publication, Practical Program Evaluation. Chen modified some of the terms in his earlier work, such as B. Replace the word tag "intervening variable" with the tag "determinant." The general meaning of these terms has not changed. Chen continues to call for theory-driven program evaluation, and the supporters of his original model of program evaluation remain essentially the same. In this chapter, the authors have chosen to retain the original language of Chen's model published in 1990.

Theory-Based Program Evaluation

 Chen (1990) defines program theory as a framework that identifies program elements, prides itself on the rationale for interventions, and describes the causal links between element interventions and outcomes. According to Chen, program theory is necessary to determine desired goals, what should be done to achieve desired goals, how actions should be organized, and what outcome criteria should be examined. 

    The theory of the program is normative or causal. The normative theory is prescriptive and value-laden and defines what should happen. The normative theory can be broken down into three main areas: treatment, implementation environment, and outcome. Treatment theory defines the nature of treatment and its measurement. Implementation environment theory defines the environment in which processing is performed. Outcome theory defines the desired goals and outcomes of the program (Chen, 1990).

    The causal theory is descriptive, explaining how and why program elements are related. Causal theory can be broken down into three main areas: effect, mechanism of intervention, and generalization. The theory of effects explains how the treatment affects the desired results. The intervention mechanism describes the relationships between the causal processes that link treatment to outcomes. Generalization theory describes how evaluation results can be generalized and applied to other issues of interest to stakeholders (Chen, 1990).

    Chen defines program evaluation as the systematic collection of empirical evidence to assess the consistency between normative and actual program structures. Empirical evidence is needed to verify the program's effect, its underlying causal mechanisms, and its level of generalization. Through this systematic collection of evidence, the program can develop and refine program structure and operations, understand and strengthen program effectiveness and similarity, and facilitate policy-making (Chen, 1990).

    Six types are derived from the six domains of program theory. These six types of assessment are normative outcome (goals), normative treatment, implementation environment (seven dimensions), impact (expected and unintended outcomes), intervention mechanism, and generalization.

Evaluation Of The Normative Result (Goal)

The normative result evaluation tries to answer the question: "What do we want to achieve?" There are three main activities involved in this type of assessment. The goal disclosure assessment determines the goals and desired outcomes. Consensus on goal priority determines which goals and outcomes are considered most important by key stakeholders. The assessment of the feasibility of the goals assesses whether there is a match between the goals and the activities of the program. Program objectives are evaluated to determine if they cause difficulties in operating the program. 

    The evaluator works with stakeholders to develop a theoretical framework or program theory, to guide the choice of program goals, to define the appropriate activities through which to achieve the goals, and to establish the link between activities and program goals to explain. Methods for achieving a normative outcome assessment include stakeholder surveys and the use of focus groups. 

    Qualitative analysis can be used to compare and contrast program goals and conceptual frameworks with the organization's mission. Although normative outcomes assessment can take place after a program has been implemented, it also provides a framework for initial program development (Chen, 1990). For an educational program, the assessment of normative outcomes may involve soliciting input from health care leaders on the program goals through focus groups or advisory panels

Evaluation Of The Normative Treatment

    Since the evaluation of the goal is completed, the congruence of the programs between the expected and the delivered treatment needs to be evaluated. The treatment provided should be compared to the original program design to determine whether the treatment and surgeries understand and reinforce the effectiveness and usefulness of the program and facilitate policy making (Chen, 1990).

    Six evaluation types are derived from the six domains of program theory. These six types of assessment are normative outcome (goals), normative treatment, implementation environment (seven dimensions), impact (expected and unexpected results), intervention mechanism and general.