Conducting Evaluation In Nursing Education Its Purpose Scope and Resources
General Purpose of Evaluation
The purpose answers the question, “Why is the evaluation being conducted?” For example, the purpose might be to decide whether to continue a specific education program or to determine the effectiveness of the teaching process. If an individual or group has a primary interest in the results of an evaluation, input from that group can clarify the purpose.
To Whom, Why and Which Evaluation Is Necessary
An important note of caution.Why an evaluation is being conducted is not synonymous with who or what is being evaluated. For example, nursing literature on patient education commonly distinguishes among three types of evaluations:
(1) learner
(2) teacher
(3) educational activity
This distinction answers the question of who or what will be evaluated and is extremely useful in designing and conducting an evaluation. The question of why under take an evaluation of a learner, for example, is answered by the need to know how well the learner performed.
If the purpose for evaluating learner performance is to determine whether the learner gained sufficient skill to perform a self care activity (eg, foot care), the educator might design a content evaluation that includes one or more return demonstrations by an individual learner prior to hospital discharge.
If the purpose for evaluating learner performance is to determine whether the learner can conduct the same self-care activity at home on a regular basis, the educator might design an evaluation determining outcomes that includes both observing the learner perform the activity in his or her home environment and measuring other related clinical parameters (eg., skin integrity), which would improve and be maintained with regular and ongoing selfcare.
In stating the purpose of an evaluation, an excellent rule of thumb is to keep it singular. In other words, the evaluator should state, “The purpose is...,” not “The purposes are.... Keeping the purpose singular and focused on the audience helps avoid the frequent tendency to attempt to do too much in one evaluation . An exception to this rule is when undertaking a total program evaluation. As discussed later in this chapter, program evaluations are naturally broad in scope, focusing on learners, teachers, and simultaneously educational offerings.
Quarries Related to Evaluation
Questions to be asked must be directly related to the purpose for conducting the evaluation. must be specific, and must be measurable. Examples of such questions include “To what extent are patients satisfied with the cardiac discharge teaching program?” and “How frequently do staff nurses use the diabetes teaching reference materials?” Asking the right questions is crucial if the evaluation is to fulfill the intended purpose.
As discussed later in this chapter, formulated clear, concise, and appropriate questions is both the first step in selecting the evaluation design and the basis for analyzing the data that are collected from evaluation.
Scope Conducting Evaluation In Nursing Education
Scope considers the extent of what is being examined, such as, “How many aspects of education will be evaluated?,” “How many individuals or representative groups will be evaluated?,” and “What time frame is to be evaluated ?” For example, will the evaluation focus on one class or on an entire program? Will it focus on the learning experience for one patient or for all patients being taught a certain skill?
Evaluation could be limited to the teaching process during a patient education class, or it could be expanded to encompass both the teaching process and related patient outcomes of learning.The scope of an evaluation is determined in part by the purpose for conducting the evaluation and in part by available resources.
For example, an evaluation addressing learner satisfaction with educators for all programs conducted by a staff development department within a given year is necessarily broad and long term in scope; such a vast undertaking requires expertise in data collection and analysis.
By comparison, an evaluation to determine whether a patient understands each step in a learning session on how to self-administer insulin injections is narrow in scope, is focused on a specific point in time, and requires expertise in clinical practice and observation.
Resources Conducting Evaluation In Nursing Education
Resources include time, expertise, personnel, materials, equipment, and facilities. A realistic appraisal of which resources are accessible and available relative to the resources that are required is crucial in focusing any evaluation. Anyone who is conducting an evaluation should remember that time and expertise are required to collate, analyze, and interpret data as well as to prepare a report of the evaluation results.
Content Rating
The purpose of content evaluation is to determine whether learners have acquired the knowledge or skills taught during the learning experience. Abruzzese (1992) described content evaluation as taking place immediately after the learning experience to answer the guiding question, “To what degree did the learners learn what they were taught?” or “To what degree did learners achieve preset behavioral objectives?”
Asking a patient to give a return demonstration of a teaching session on psycho-motor skill development or asking participants to complete a cognitive test at the completion of a 1-day seminar are common examples of content evaluation.The RSA model shows content evaluation as the level in between process and outcome evaluation. In other words, the purpose of content evaluation is to focus on how the teaching-learning process affects immediate, short-term outcomes.
A question to be asked is: “Were specified objectives met based on the teaching that was done?” The evaluation design to answer this question is different from an evaluation that answers the question “Did learners achieve specified objectives?” An important point to be made here is that questions must be carefully considered and clearly stated because they dictate the basic framework for design and conduct of the evaluation. Evaluation designs are discussed in some detail later in this chapter.
The scope of content evaluation is limited to a specific learning experience and to specifically stated objectives for that experience. Content evaluation occurs immediately after completion of teaching but accounts for all teaching-learning activities included in that specific learning experience.
Data are obtained from all learners involved in a specific class or group teaching session. For example, if both parents and the adolescent patient with diabetes are taught insulin administration, all three are asked to complete a return demonstration. Similarly, all nurses attending a workshop are asked to complete the cognitive posttest at the end of the workshop.
Also, resources used to teach content can be evaluated as to how well that content was learned. For example, the exact equipment included in teaching a patient how to change a dressing also can be used by the patient to perform a return demonstration or teach-back. In the same manner, a pretest used at the beginning of a continuing education seminar can be re-administered as a post test at seminar completion to measure change resulting from program delivery.
Content evaluation, like process evaluation, focuses collecting on internal evidence to determine whether objectives for a specific group of learners were met. Gathering data about the learner prior to a teaching session and then collecting data again immediately after the teaching session can be used to compare if any change in learner behavior occurred.
For instance, Walker (2012) describes development and implementation of Skin Protection for Kids, a primary prevention education project aimed at school aged children, their parents, and their teachers to decrease unnecessary sun exposure. Content evaluation included pretests conducted prior to providing informational materials and post tests conducted 24 to 48 hours later.
Teachers' scores, for example, improved from an average of 56.25% on the pretest to an average of 87.5% on the post test, indicating that teachers improved their short term knowledge of sun safety. Also, this study supported the use of informational guidelines as an effective intervention for parents of school-aged children in kindergarten through fifth grade.
As another example, patients who undergo kidney transplant must become knowledgeable in self-management of a complex medication regimen. To provide nurses with the necessary knowledge and skill to teach post transplant patients, Mangold (2016) reports on use of standardized patients to provide transplant nurses an opportunity to practice the teach back they would eventually be using to evaluate patient learning.
Content evaluation of nurses' learning included test retest analysis of knowledge retention and confidence in the use of teach back method. Transplant nurses also participated in debriefing sessions immediately after training to reflect on what they had learned that could be taken back to their practice setting.
Cibulka (2011) also provides an example of content evaluation in her description of a continuing education program on research ethics in which nurses completed quizzes after each module to determine short-term knowledge retention. Cibulka's use of quiz scores already developed and integrated into a well known and widely used research ethics educational activity exemplifies an important rule of thumb.
Use existing data if those data are relevant and already available. Asking any individual to spend his or her time to complete a quiz, survey, or any data collection activity should be viewed by the person collecting those data as a promise to appropriately use the data collected.
Give your opinion if have any.