Models Types Levels and Scope of Evaluation In Nursing Education
Types of Evaluation and Their Components
Evaluation can be classified into different types, or categories, based on one or more of the five components just described. The most common types of evaluation identified include some combination of process (also known as formative) evaluation, and outcome (also known as summative) evaluation. Other types of evaluation include context, content, outcome, impact, and total program evaluations.
Models of Evaluation on the Bases Types
Evaluation models defining these types and their relationship to one another have been developed (Abruzzese, 1992; Frye & Hemmer, 2012; Losby, Vaughan, Davis, & Tucker Brown, 2015; Milne, 2007; Noar, 2012; Ogrinc & Batalden, 2009; Rankin & Stallings, 2005; Rouse, 2011; Stavropoulou & Stroubouki, 2014). Because not all models define fine types of evaluation in the same manner, the nurse educator should choose the model that is most appropriate for the given purpose and most realistic given the available resources.
Abruzzese (1992) constructed the Roberta Straessle Abruzzese (RSA) evaluation model for conceptualizing, or classifying, educational evaluation into different categories or levels. Although developed in 1978 and originally designed to evaluate staff development education, the RSA model remains useful for conceptualizing types of evaluation from both staff development and patient education perspectives.
Examples of use of the RSA model are given by Dilorio, Price, and Becker (2001) in their discussion of the evaluation of the Neuroscience Nurse Internship Program at the National Institutes of Health Clinical Center and by Underwood, Dahlen Hartfield, and Mogle (2004) in their study of nurses' perceived expertise following three continuing education programs.
More recently, Sumner, Burke, Chang, McAdams, and Jones (2012) used the RSA model to support development of a study to evaluate process, content, and outcome evaluation of a basic arrhythmia course taught to registered nurses.
The RSA model provides a visual of five basic types of evaluation in relation to one another based on focus, purpose, related questions, scope, and resources available. The five types of evaluation include process, content, outcome, impact, and total program. Abruzzese describes the first four types as levels of evaluation leading from the simple (process evaluation) to the complex (impact evaluation). Total program evaluation encompasses and summarizes all four levels.
Process (Formative) Evaluation In Nursing Education
The purpose of process evaluation (formative evaluation) is to make necessary adjustments to an educational activity as soon as they are identified, such as changes in personnel, materials, facilities, teaching methods, learning objectives, or even the educators' own attitude. One or more adjustments may need to be made after a class or teaching session and before the next is taught or even in the middle of a single learning experience.
Consider, for example, evaluating the process of teaching an adolescent with newly diagnosed type 1 diabetes and her family how to administer insulin. The nurse might facilitate learning by first injecting himself or herself with normal saline so that the learners can see someone maintain a calm expression during an injection. If the nurse educator had planned to have the parent give the first injection but the child seems less fearful, the nurse might consider revising the teaching plan to let the child first perform self injection.
Process (formative) evaluation is a component of the ongoing education cycle of assessment, planning, and implementation. Process evaluation helps the nurse anticipate and prevent problems before they occur or identify problems as they arise. Milne's (2007) evaluation framework consists of the elements of structure-content-outcomes-procedures-processes-efficiencies (SCOPPE).
Noar's (2012) framework includes the elements of audience-channel-message-evaluation (ACME). Both of these frameworks focus on the characteristics of the teacher and the learner as well as on examining teaching methods and instructional materials as aspects of process evaluation. Noar (2012) speaks to the importance of using a consistent theoretical framework for designing, conducting, and evaluating education.
Consistent with the purpose of process evaluation, the primary question here is, “How can teaching be improved to facilitate learning?” The nurse's teaching effectiveness, the elements of the education process, and the learner's responses are monitored on an ongoing basis.
Abruzzese (1992) describes process evaluation as a “happiness index.” While teaching and learning are ongoing, learners are asked their opinions about the educator(s), learning objectives, content, teaching methods, instructional materials, physical facilities, and overall learning experience.
Scope of Process of Evaluation In Nursing Education
The scope of process evaluation generally is limited in breadth of content and time frame allotted to a specific learning experience, such as a class or workshop, yet is sufficiently detailed to include as many aspects of the specific learning experience as possible while they ocheart. Thus, learner behavior, teacher behavior, learner teacher interaction, learner response to teaching methods and materials, and characteristics of the environment are all aspects of the learning experience within the scope of process evaluation.
Whether they engage in the educational program online or on-site, all learners and all teachers participating in a learning experience should be included in process evaluation. If resources are limited and participants include different groups, a representative sample of individuals from each group rather than everyone in the groups-may be included in the evaluation. Resources usually are less costly and more readily available for process evaluation than for the other types of evaluation, such as impact or total program evaluation.
Although process evaluation occurs more frequently-during and throughout every learning experience than any other type of evaluation, it occurs concurrently with teaching. Therefore, the need for additional time, facilities, and dollars to conduct process evaluation is minimal and limited.
From the perspective of EBP, process evaluation is important in providing patient-centered care based on clinical practice guidelines (CPGs). CPGs also are sometimes referred to as clinical pathways or critical pathways (Ferguson, 2010; Hoesing, 2016), although they are at different levels of precision and accuracy. A well-constructed CPG includes not only the specific intervention, such as teaching a patient how to take his discharge medication, but also how to evaluate the effectiveness of that intervention.
A CPG is intended as a guide in caring for all patients who have similar characteristics and learning needs. Its development requires a high level of rigor. As discussed later in this chapter, CPG development is conducted as evaluation research. Evaluation of the process of using a CPG, however, focuses on the fit of a CPG for a specific learner or learners. To the extent that a patient is the same as those patients who were studied in development of the CPG, the patient's nurse will follow the guideline as written.
To the extent that the patient has unique learning needs or responses to teaching, the nurse should use informal process evaluation to vary from the guideline to ensure that the patient is able to learn and is receiving individualized patient centered care. CPGs are also useful for orientation of new healthcare employees, for quality improvement, and for evaluating clinical learning in student precept situations.
These are pathways used by members of the healthcare team as cost-effective and time efficient measures to note learner progress and the achievement of goal-based outcomes. Implementing this education-focused evaluation method allows educators to choose teaching and learning activities specific to the behavioral outcomes to be achieved (Bradshaw, 2007).
EBP is not a “cookbook” approach to providing health care. Similarly, CPGs are not intended to disregard the individual learner's needs. Variance from a CPG, though, should not occur in a haphazard manner. As attention to practice-based evidence evolves, the importance of using internal evidence gathered from process evaluation is becoming an increasingly critical component of every teaching-learning experience.
For instance, Chan, Richardson, and Richardson (2012) describe a process evaluation conducted to examine which factors might support successful delivery of an intervention to improve symptom management for patients receiving palliative radiation therapy for their lung cancer . Although two nurses were using the same procedure to provide the intervention. patients were more satisfied with the nurse who had prior experience in oncology.
Patients also were less likely to practice their muscle relaxation protocol if they had not yet experienced the level of pain that the muscle relaxation was intended to alleviate. Based on these findings. Chan and colleagues modified the intervention protocol to have patient education be provided by nurses with prior oncology experience and to allow patients flexibility to decide on the frequency with which they practiced muscle relaxation techniques.
As other examples, Jones et al. (2016) used the context, input, process, product (CIPP) model to include stakeholders in determining what problems might be anticipated in teaching self-management skills to people with Parkinson's disease. During each education session, patients, caregivers, and healthcare professionals were surveyed for input as to what might be revised to improve the next session.
Vaartio-Rajalin et al. (2015), aware that patients undergoing chemotherapy may experience cognitive changes, in-volved patients with cancer and their caregivers on how, when, and by whom their education about treatment should be conducted.The importance of practice-based evidence to process evaluation encompasses staff education as well as patient education.
Rani and Byrne (2012) conducted a multi method evaluation of a training course geared toward teaching providers how to care for patients presenting with dual diagnosis of mental illness. A process evaluation conducted with an initial group of participants included daily collection of quantitative data using Likert scales to determine the extent to which participants found teaching methods and content covered that day to be helpful for their learning.
Milne's SCOPPE framework and Noar's ACME framework focus heavily on initial planning of the educational activity as an essential component of process evaluation that includes design of the educational activity and examination. The CIPP model used by Jones and colleagues also focuses on initial planning, including the context in which the education process should take place.
Grant, Dreischulte, and Guthrie (2017) also focus on initial planning as well as feedback during a multi component intervention to learn from clinicians what aspects of a quality improvement program were most likely to influence clinicians to reduce prescribing high-risk medications to patients seeking primary care.
Give your opinion if have any.