Chapter 6: Assessment Strategies that Support Learners’ Management of Uncertainty

Assessment is a word associated with some ambiguity, in that it can mean a variety of things to different people depending on context and experience, but in general terms, assessment refers to actions intended to obtain information about learner competency or performance (Schuwirth & van der Vleuten, 2019). Broadly speaking, assessments may be categorised by their purpose (e.g., summative, leading to decisions such as course progression or certification; or formative, intending to build learners’ understanding of their performance), setting (e.g., exam hall, computer based, or workplace), and type (e.g., written or performance based). Assessment grading varies across health professions education and includes graded and pass-grade-only approaches.

Assessment of health professions learners traditionally privileges evaluation of certainty through tests of knowledge, skills, and attributes that are intended to be objective and reliable (Ryan & Wilkinson, 2021; Schuwirth & van der Vleuten, 2019), using summative assessment strategies, assigning grades or values to learners’ performance on set tasks. Dominant assessment strategies include examinations using multiple-choice questions with single correct answers, and ‘objective’ assessments of skills through objective structured clinical examinations (OSCEs) with standardised simulated patients; these strategies have increased in popularity over recent decades due to their greater reliability (i.e., precision or consistency of scoring) when compared to short-answer questions and clinical assessments involving actual patients, such as long cases (Schuwirth & van der Vleuten, 2019). Reliability of assessments is important, because educators and regulatory bodies want to be as certain as possible that learners are performing at a standard at which they can safely and effectively commence practice, and learners want to be able to trust that they are being fairly assessed. Other factors contributing to the popularity of such approaches may be the lower costs of assessments (e.g., less educator marking time for multiple-choice questions compared to short-answer questions) and higher acceptability (e.g., perceptions of fairness by students when standardised patients are used). Educational impact is another consideration in determining the utility of an assessment (Schuwirth & van der Vleuten, 2019), as is validity  – that is, the extent to which an assessment evaluates the construct it purports to measure (American Educational Research Association, 2014; Schuwirth & van der Vleuten, 2019).

The discourse on assessment in health professions education has shifted to highlight the importance of balancing the learning impacts of assessments with reliability and that assessments align with the knowledge, skills, and attributes required in learners’ future careers (Ryan & Wilkinson, 2021; Schuwirth & van der Vleuten, 2019). Uncertainty is the reality of healthcare practice (see Chapter 1), and it is widely recognised that assessments can impact or drive’ learning (Scott, 2020). Therefore, assessment tasks and strategies that consider and include uncertainty may help motivate students to learn about uncertainty relevant to healthcare practice and to develop skillsets for managing the uncertainty.

In response to the greater focus on the learning impacts of assessments, formative assessments are increasingly utilised across health professions education. These tend to be lower-stakes assessments concerned with monitoring learner performance through ongoing feedback dialogues between the educator and learner that result in the learner understanding how their performance aligns with expected course outcomes (Carnegie Mellon University, n.d.). In part due to this emphasis on feedback, formative assessments may have great potential in supporting learner uncertainty tolerance. For instance, clinical prioritisation questions are a formative assessment tool designed to mimic complex healthcare environments and differential diagnoses (in which more than one answer is correct). Clinical prioritisation questions appear to enhance critical thinking and may improve uncertainty tolerance (Sam et al., 2020). Health professions education programs typically engage a combination of formative and summative assessments, due to their different but complementary purposes. The goal of assessment throughout a health professions education degree is to balance assessment tasks that support both learners and the needs of the program. As such, both forms of assessment are relevant to managing uncertainty.

Of the topics included in this handbook, assessment is the most under construction in relation to existing evidence. However, other evidence-based concepts in health professions learner assessment do provide some insights into how the field of uncertainty tolerance assessment may progress. For instance, authentic and programmatic approaches to assessment, which focus on enhanced preparation of students for their future workplaces and for lifelong learning, could be ideal frameworks for considering how managing uncertainty may be assessed. Drawing on available literature related to both the broader topic of assessment and the field of uncertainty tolerance, this chapter critiques historical approaches to uncertainty tolerance assessment (i.e., scales) and provides initial recommendations for future directions in assessment that may support learners’ development of uncertainty tolerance. The chapter includes an overview of relevant theory alongside practical approaches that educators may implement for supporting uncertainty tolerance assessment strategies.

Historical Assessment of Uncertainty Tolerance Using Scales

An example of how uncertainty tolerance is currently assessed in health professions learners is found in the Association of American Medical Colleges’ (2021, 2024) use of the Tolerance for Ambiguity scale for routine national surveys of medical students at both matriculation and graduation. The results of these surveys are provided to medical schools to indicate how well their programs are preparing students to manage the uncertainties they will encounter during medical practise. Many other scales have been developed with the intent of measuring individuals’ uncertainty tolerance, and calls been made to implement uncertainty tolerance scales as part of the selection criteria for medical students (Geller et al., 1990; Hancock & Mattick, 2020; Stephens, Karim et al., 2022). Consideration of the historical underpinnings of such scales is important to gain understanding of both the preponderance of uncertainty tolerance scales use in health professions education and their substantive limitations in relation to assessment of uncertainty tolerance.

Early research on uncertainty tolerance centred on the development of scales, using classical test theory, intended to measure the construct as a personality trait (Budner, 1962; Furnham & Marks, 2013). These scales typically include multiple statements or ‘items’ designed to assess an individual’s perceptions of or responses to uncertainty; the items are engaged with via a Likert scale (e.g., ‘strongly disagree’ to ‘strongly agree’) (Stephens, Karim, et al., 2022). Performance on a scale across all items is purported to indicate an individual’s uncertainty tolerance or intolerance, depending on the scale used (Stephens et al., 2023). Items may be generic (i.e., ask about uncertainty or ambiguity in general) or contextualised to healthcare (e.g., ask about uncertainty related to diagnosis, treatment, patient communication, etc.) (Stephens, Karim, et al., 2022). Commonly used uncertainty tolerance scales developed for health professions and learner populations include the aforementioned Tolerance for Ambiguity scale (Geller et al., 1993), versions of the Physicians’ Reactions to Uncertainty scale (Gerrity et al., 1990, 1995), and the Tolerance of Ambiguity in Medical Students and Doctors scale (Hancock et al., 2015).

Although the use of these scales may be appealing to educators interested in assessing learners’ development of uncertainty tolerance, there are considerable limitations in their validity evidence, particularly within the context of health professions learners (Stephens, Karim, et al., 2022; Stephens et al., 2023). Ways of evaluating validity have developed over time, from considering a couple of types of validity (i.e., content and criterion validity) to five sources of validity (i.e., test content, response processes, internal structure, relations with other variables, and consequences of testing); to build a validity argument, some approaches now use inferences, which consider factors contributing to validity in a stepwise fashion (Cook et al., 2015). In essence, the more contemporary approaches to building validity evidence consider a wider array of factors which together determine the extent to which a scale measures the intended construct as it relates to a specific context and purpose, alongside potential implications of the testing.

Stephens et al. (2023) considered uncertainty tolerance scale research in relation to contemporary approaches to developing and evaluating validity evidence and found that the research leans heavily on evidence evaluating a scale’s internal structure (i.e., the relations between items in the scale evaluated by approaches such as Cronbach’s alpha and factor analyses) and on relations between a scale and other variables (e.g., correlations with other scales and association with demographic factors). This research also identified key gaps in the validity evidence for commonly used uncertainty tolerance scales within health professions education, including evidence for response processes and consequences of testing.

Response processes can be evaluated through approaches such as cognitive interviewing, which explores participants’ thought processes when engaging with a scale, including whether they understand items in the way intended by the scale’s developers (Stephens et al., 2023). Understanding learners’ response processes is particularly important for comprehending validity evidence of uncertainty tolerance scales, because in their conceptualisation of uncertainty novices may differ from those with more established knowledge and careers (Stephens, Karim, et al., 2022; Stephens, Sarkar, et al., 2022b; Stephens et al., in press).

Consequences of testing considers how scales may impact participants (e.g., a low uncertainty tolerance score influencing a participant to choose a career perceived as less uncertain), including whether the scales impact participants differentially (e.g., bias related to gender, culture, etc.). It is unclear whether the most commonly used uncertainty tolerance scales assess uncertainty tolerance in a manner understood by participants and what impacts they have on educational and healthcare stakeholders.

Critically, validity evidence for a scale when using classical test theory is not established through a one-and-done process but must be determined specifically in relation to the context and intended purpose of the scale (Cook et al., 2015; St-Onge et al., 2017; Stephens et al., 2023). The uncertainty tolerance scales with the most comprehensive validity arguments are based on evidence from test populations consisting of physicians. Therefore, validity evidence to support the use of these scales in health professions other than medicine, and among learner populations, is critically lacking (Stephens et al., 2023).

The importance of considering the test population has been illustrated by a meta-analysis of the internal consistency of uncertainty tolerance scales that identified significantly lower internal consistency when scales were used with medical students rather than physicians, indicating that differences in these populations’ experiences of uncertainty may be substantive enough to impact the accuracy of scales to measure the construct (Stephens, Karim, et al., 2022). A qualitative research study with medical students completing clinical placements identified that participants’ conceptions of uncertainty centred on individual knowledge gaps rather than on the uncertainty inherent in patient care (Stephens, Sarkar, et al., 2022b). Therefore, an increase in uncertainty tolerance as measured by a scale could represent an increase in students’ knowledge and not necessarily improvement in students’ uncertainty tolerance (Stephens et al., in press).

Commonly used uncertainty tolerance scales also tend to focus on stress responses (e.g., worry, anxiety, and stress) as the major determinants of whether an individual tolerates uncertainty. However, more contemporary understandings of uncertainty tolerance highlight the need to additionally consider cognitions and behaviours as indications of individuals’ uncertainty management, as well as how effectively managing uncertainty will depend on the context (Hillen et al., 2017; Stephens et al., 2024). Accordingly, uncertainty tolerance scales may not be effective assessments of learners’ uncertainty tolerance, and the use of existing uncertainty tolerance scales in health professions learner populations is currently cautioned. This is especially the case when scale results are intended to inform higher-stakes decision-making, such as selection and progression (Stephens et al., 2023; Stephens, Karim, et al., 2022).

However, there may yet be a role for existing uncertainty tolerance scales in particular contexts and where they include validity evidence specific to health professions learners (Stephens et al., 2023). For instance, scales could be used as part of self-assessment (Lee et al., 2023), ideally in combination with approaches like reflective learning and feedback dialogues (Stephens & Lazarus, 2024). Advances in test theory that address the limitations of classical test theory applications (e.g., those that led to the existing uncertainty tolerance scale development) may also aid in developing scales with enhanced psychometric properties. For instance, Rasch (1966) modelling using item response theory may allow an assessor to account for variables influencing an individual’s responses to singular items on a scale. Scales developed using item response theory and Rasch modelling could account for how individual learners may respond to items which have more or less uncertainty, resulting in a score that is graded at the individual item level as opposed the whole scale level. Furthermore, scales developed with Rasch modelling could account for variables that might be influencing individual responses to items. In the case of uncertainty tolerance, this could include perceptions of the level of uncertainty for each item or the context within which the uncertain stimulus is occurring (e.g., healthcare encounter, learning environment, etc.) alongside the learner’s own uncertainty tolerance. Such yet-to-be-developed scales would allow assessors to provide learners with item-level feedback, enabling learners to understand their performance on each item and to receive feedback about their competency in managing uncertainty, instead of providing learners with an overall score, as is typical of existing healthcare-related uncertainty tolerance scales (Farlie et al., 2021). This could give insights into how learners perform on items with more or less uncertainty or in different learning contexts (e.g., classrooms or workplace-based learning), helping learners to identify where competency could be improved. While such scales provide an opportunity for future research in the field of health professions education, other, related areas are realising their value. Rasch modelling approaches are being applied to other complex health professions attributes, such as feedback quality instruments (Johnson et al., 2021) and scales for assessing uncertainty tolerance related to environmental factors (García-Pérez & Yanes-Estévez, 2022).

Until well-constructed validity arguments have been established for uncertainty tolerance scales, educators should instead consider how uncertainty can be acknowledged and incorporated into assessment strategies and how approaches to assessment can shape students’ experiences of uncertainty (Stephens et al., 2023; Stephens & Lazarus, 2024).

Authentic and Programmatic Assessment Strategies Can Incorporate Uncertainty

Uncertainty is an inherent feature of healthcare practice (Han et al., 2011; Simpkin & Armstrong, 2019), and effective uncertainty management is increasingly seen as a desired competency in health professions training frameworks (Accreditation Council for Graduate Medical Education, 2015; Royal Australian College of General Practitioners, 2016). The challenge for many institutions is in identifying and implementing assessment strategies which impact uncertainty tolerance development and evaluation in health professions learners (Stephens & Lazarus, 2024). Without assessment tasks intended to evaluate learners’ progress with managing uncertainty, the risk is that managing uncertainty isn’t valued or prioritised by learners, educators, or institutions (Scott, 2020; Stephens, Sarkar, et al., 2022a).

In a qualitative study conducted by Stephens, Sarkar, et al. (2022a) with medical students, participants described how assessment strategies centred on summative written examinations and OSCEs influenced their learning about uncertainty. They described how the perceived objective nature of assessments impeded their engagement with clinical uncertainties, leading them to prioritise library-based personal study over placement-based learning. This highlights the importance of considering the learning impacts of assessments, in which high-stakes assessments of knowledge may impact engagement with other aspects of learning.

Like best-practice assessments in health professions education more broadly, assessment of learners’ capacity for managing uncertainty should not be appraised using a single assessment at a single time point (Schuwirth & van der Vleuten, 2019; Stephens & Lazarus, 2024). And, as with learning activities intended to stimulate uncertainty (Chapter 4), assessment of uncertainty management as a competency may not need to come at the expense of assessment of other core competencies (Stephens & Lazarus, 2024). Ideally, learners’ capacity for managing uncertainty should be evaluated through a variety of methods over time, in manners which reflect the workplace applications they will experience when entering their careers. Such assessments are also valuable in communicating to learners their own progress in uncertainty management through their health professions degree (Stephens & Lazarus, 2024). Two key approaches to assessment in relation to uncertainty management are authentic assessments and programmatic assessments; these are discussed below.

Authentic Assessment

The concept of authentic assessment was introduced in the late 1980s and offers a counterpoint to more traditional forms of assessments (Ashford-Rowe et al., 2013; Villarroel et al., 2018). Although definitions vary, traditional assessments are typically characterised by written examinations conducted in exam halls with invigilators and are often focussed on lower-order cognitive tasks, such as remembering and understanding (Villarroel et al., 2018). By contrast, authentic assessments aim to engage students in tasks that are challenging and relevant to their potential future workplaces (Diug et al., 2021; Raymond et al., 2013; Villarroel et al., 2018). In a review of authentic assessments in higher education, Villarroel et al. (2018) described benefits that included improved quality and depth of learning, and enhanced capacity for self-regulation, autonomy, and motivation. They identified three core elements in the approach of authentic assessments: realism, cognitive challenge, and evaluation judgement; all three have important implications for uncertainty tolerance and are discussed below.

Educators may consider how learners will experience uncertainty in their future workplaces and how this can be reflected within assessment tasks using realism. Realism in this context refers to the presence of elements within an assessment that align with learners’ future workplaces (e.g., a clinical context or patient case). Learners apply their knowledge and skills to solve a problem that has similarities to problems encountered by those working in a healthcare context. The concept of realism may be helpful when considering how to integrate uncertainty into assessments, as a lack of realism in traditional assessments may underplay the presence of uncertainty within learners’ future workplaces.

Realism in assessments can be achieved to different extents. For instance, a degree of realism may be attained by adding context to examinations and written tasks (e.g., asking learners to apply anatomical knowledge to answer a question based on a patient case, instead of simply recalling an isolated anatomical fact) and by using performance-based evaluation for which students produce work or demonstrate knowledge that aligns with that expected of healthcare professionals (Villarroel et al., 2018). In this context, an OSCE may be considered an authentic assessment. As artificial intelligence is further integrated into healthcare practice, it may relevant to include such technologies as a further aspect that creates realism in health professions assessments (Ajjawi et al., 2024). For instance, learners could be asked to critique a management plan supported by artificial intelligence, including the learner communicating remaining uncertainties about the plan.

The cognitive challenge component of authentic assessment involves requiring learners to engage higher-order cognitive skills, which are often drawn on to manage uncertainty. These may be appraised through learners applying, analysing, evaluating, and creating, rather than simply remembering and understanding, the relevant content. This feature of authentic assessment recognises that simply reproducing knowledge in a decontextualised examination does not guarantee that the same knowledge can be applied in a future, uncertain work environment. For instance, recalling the questions to ask a simulated patient in a respiratory history during an OSCE station is less cognitively challenging than developing a differential diagnosis based on an incomplete history and examination of a patient presenting to a general practitioner with subtle respiratory symptoms. By considering the cognitive challenge of assessment tasks, educators can ensure the tasks not only reflect future workplaces in relation to context but also represent ways in which learners will need to apply their knowledge to manage uncertainty and ambiguity typical of healthcare practice.

Once a realistic and cognitively challenging assessment task relating to uncertainty is implemented, educators should consider how their feedback will ensure learners understand their progress in adaptive uncertainty management. Evaluative judgement is used in the educational literature to describe learners’ ability to judge their own performance and regulate their own learning (Villarroel et al., 2018). To achieve this, they need help with calibrating their judgements about their learning journey through regular educator feedback, which can occur through different processes, including formative assessments. Clear assessment criteria and exemplars can help learners compare their efforts to desired and expected standards, guiding evaluative judgement through benchmarking. To continue with the example of OSCEs, implementing this approach at the end of a unit of study with the provision of a numerical grade would provide learners with only limited opportunities to regulate their own learning. By contrast, implementing multiple OSCEs throughout a unit with numerical grades, qualitative feedback, and ongoing feedback dialogue might be a more effective way to facilitate learners’ self-regulation. Flexible assessments (a moderator introduced in Chapter 5) can provide learners with multiple options for completing an assessment task. (For a practical application of this approach in which learners can choose the subject of their assignment, see Chapter 13.) However, clear assessment criteria to support evaluative judgement should still be provided.

Local and regional assessment policies are important to consider and will likely impact the degree to which authentic assessments can be implemented (Ajjawi et al., 2024; Villarroel et al., 2018). For instance, some institutions may specify required forms of assessment, as is the case for the United States Medical Licensing Examination, for which the test and its delivery are predetermined and structured in only one format. Finding ways to work within the constraints of institutional assessment policies for supporting learners’ uncertainty tolerance development is, however, necessary. This may require finding a balance between traditional and authentic assessment approaches. Considering ways in which realistic and challenging uncertainties can be integrated within assessment tasks, alongside ongoing and regular feedback dialogues between educator and learner, may help to address the challenges posed by traditional assessments. Where traditional assessments may dissuade students from engaging with clinical uncertainties, authentic assessments may help learners develop adaptive strategies for managing uncertainty relevant to their future careers. This chapter’s Integrating Uncertainty within Assessment Tasks section gives examples of how common assessment tasks can incorporate aspects of authentic assessments relating to uncertainty.

Programmatic Assessment

While authentic assessment provides a framework for developing assessment tasks that integrate uncertainty, programmatic assessment provides a framework for understanding how different assessment tasks can be used in combination over time to build a rich understanding of learners’ uncertainty management competency (Ryan & Wilkinson, 2021; Stephens & Lazarus, 2024). The concept of programmatic assessment recognises that there is no single perfect assessment and that, rather, each assessment task will have its strengths and weaknesses in terms of reliability, validity, educational impacts, acceptability, and cost (Schuwirth & van der Vleuten, 2012). For example, case-based assessments of clinical skills (e.g., OSCEs) have strengths in terms of reliability (achieved through multiple stations and trained assessors), but the standardised approach with simulated patients can lessen their realism or authenticity, potentially reducing exposure and practice opportunities for learners to respond to uncertainty. Assessments involving actual patients in healthcare settings (e.g., long cases) can enhance authenticity and therefore exposure to uncertainty stimuli (Lazarus et al., 2024; Richmond, 2022). However, as single assessments, long cases are likely to be less reliable (ironically, due to the related uncertainties) and may pose challenges in acceptability, due to case differences experienced by different learners.

Programmatic assessment helps resolve these issues by allowing educators to work backwards from a competency and to determine, based on strengths and limitations, the different assessment tasks that may together provide a fuller picture of learner competence in specific knowledge, skills, or attributes related to uncertainty. In other words, the whole assessment picture is worth more than the sum of its individual components (Schuwirth & van der Vleuten, 2012; van der Vleuten et al., 2014). Detailed assessment maps or blueprints are useful tools for understanding where and how different competencies are assessed across an entire health professions program (Ryan & Wilkinson, 2021).

A further core aspect of programmatic assessment is its acknowledgement that decision-making should be disconnected from a single assessment task (Schuwirth & van der Vleuten, 2012). Instead of a traditional end-of-year examination determining whether a student progresses to the next year of a course of study, programmatic assessment involves collating portfolios or dossiers of information on learners’ performance on multiple assessment tasks: assessors evaluate and then make decisions about learners’ progress based on these multiple sources of evidence (Ryan & Wilkinson, 2021). The concept in programmatic assessment is similar to that in making a diagnosis in healthcare, in that, rather than using a single investigation result to determine whether a person has a particular health condition, a more holistic approach, involving a combination of history, examination, and relevant investigations, is more likely to lead to appropriate healthcare decision-making.

In programmatic assessment approaches, multiple forms of evidence are required to gauge whether a learner has developed a competency for communicating uncertainty in healthcare. Evidence may be gathered through assessment tasks such as OSCEs, which require communication of differential diagnoses to the examiner; simulations of medical emergencies, in which learners recognise the limitations of their skills and call for help; and direct observations of a patient encounter, in which learners must communicate uncertainties about management options. The assessment tasks may also test for other competencies (e.g., clinical examination in OSCEs and basic life support in simulations). While learners could theoretically achieve a pass mark for individual assessment tasks, a programmatic assessment approach would ensure that assessments of specific competencies (e.g., communicating uncertainty) are not lost within the overall performance of a task. For example, an 82 per cent overall score on an OSCE may qualify for a high distinction, but if the 18 per cent lost was all related to communicating uncertainty, and this was also identified as a weakness in a subsequent assessment task, it could suggest that this competency is yet to be achieved.

Wholesale implementation of programmatic assessment is resource intensive and often limited by institutional assessment policies, such as the need to couple assessment time points with decision-making due dates (van der Vleuten et al., 2014). However, the concept shows how assessment tasks which incorporate uncertainty can work together across a program of study and ensure learners have access to sufficient feedback and mentorship to improve their skills. Rather than reinventing the wheel, the first step to implementing programmatic assessment that incorporates uncertainty is mapping existing curricula and assessment tasks to identify where small adjustments can be made to include multiple opportunities in different tasks over time to assess uncertainty management. This is known as developing an assessment master plan (van der Vleuten et al., 2014): the plan should address the competencies related to managing uncertainty required for a graduate of a specific health profession.

Integrating Uncertainty Within Assessment Tasks

This section describes different forms of assessment and how they may be adjusted to integrate authentic assessment of learners’ management of uncertainty.

Written Assessments

Written assessments come in many formats, including multiple-choice questions, extended matching questions, short-answer questions, and essays. A common misconception is that open-ended questions assess higher-order cognitive skills relevant to authentic assessment principles and that multiple-choice questions and extended matching questions do not. This is not necessarily the case (Jolly & Dalton, 2019), as it is the content of the questions, and particularly whether the questions include realism through the provision of context, that determines the extent to which a learner is challenged to engage higher-order cognitive skills.

Providing rich context through case vignettes and posing questions with multiple possible correct answers are relatively simple ways in which authenticity and uncertainty can be incorporated into written assessments. Context-rich questions include a scenario or case with a question that requires learners to apply, analyse, or evaluate its information. Extended matching questions take a similar approach to multiple-choice questions but typically include a longer list of potential answers that apply to several clinical vignettes or scenarios. The clinical vignettes in extended matching questions are generally longer than in multiple-choice questions, with the list of options including multiple theoretically relevant answers (e.g., a case about a patient with chest pain including options for many causes of chest pain). Although multiple-choice questions and extended matching questions can have points awarded for a single correct answer only (e.g., awarding for certainty in a single answer), extended matching questions in particular lend themselves to having more than one correct answer and multiple points awarded accordingly, or a single best answer with other relevant but less correct options attracting partial points. Thus, extended matching questions can introduce uncertainty to a greater extent than typical multiple-choice questions. This approach can help reflect the complexity of healthcare, in which sometimes a single answer isn’t achievable or appropriate based on the available information.

Clinical Skills Assessment

Because of the inherent uncertainties in clinical practice, it is likely that existing approaches to clinical skills assessments may include opportunities to integrate uncertainty. In such cases, educators are encouraged to acknowledge these opportunities within assessment criteria (Stephens & Lazarus, 2024). Assessment of clinical skills or competence in health professions learners can occur in a variety of formats and settings amenable to incorporating uncertainty. These include controlled or simulated settings (e.g., examination conditions for OSCEs) and workplace-based learning – for example, observation of learners in authentic clinical contexts for long cases, clinical evaluation exercises (CEX), mini-CEX, and direct observation of procedural skills tests (Norcini et al. 2003; Norcini & Zaidi, 2019). Furthermore, clinical skills assessments can include both holistic approaches to patient evaluation and management (e.g., long cases and CEX) and shorter assessments that address partial or focussed evaluations (e.g., mini-CEX, OSCEs, and short cases). The use of OSCEs is particularly widespread in undergraduate health professions education, due to their reliability and perceived fairness (Boursicot et al., 2019).

Like written assessments, the content, structure, and approach to marking of clinical skills assessments can be adjusted to improve realism and integrate uncertainty. For OSCEs, this might include stations dedicated to managing uncertainty (e.g., a communication skills station where candidates need to communicate uncertainty to a simulated patient in a manner that considers their preferences, such as their capacity for understanding technical information and cultural considerations) (Simpkin & Armstrong, 2019). OSCEs might also integrate uncertainty as part of a station assessing another competency (e.g., a clinical reasoning station where a definitive diagnosis is not possible based on the available information, so multiple potential differential diagnoses are rewarded) (Stephens & Lazarus, 2024).

Uncertainty may also be integrated with clinical skills assessments through approaches that require students to evaluate patients and make decisions based on limited information, as they would do in practice (Richmond, 2022). This could include the use of CEX, mini-CEX, or more traditional forms of assessment, such as long and short cases (Richmond, 2022; Wass et al., 2001).

Long cases and CEX involve a candidate evaluating a patient through history and examination, then communicating to the examiner a synthesis of their findings and recommendations about differential diagnoses, further investigations, and management. Long cases require the candidate to evaluate the patient independently, whereas CEX are observed by an examiner. Both typically take multiple hours to complete and are highly authentic and ideal for integrating uncertainty. For instance, the conversation with the examiner might include the candidate describing unknowns about assessment and management and how these could be addressed through relevant investigations or referral to other healthcare professionals (Boursicot et al., 2019). Long cases have fallen out of favour in undergraduate health professions education, due to their resource-intensive nature and challenges with reliability (Norcini, 2001), but interest has been reignited thanks to their educational value in holistically evaluating patients and their potential capacity for developing clinical reasoning under conditions of uncertainty (Richmond, 2022).

By contrast, short cases and mini-CEX are briefer assessments (around 15 minutes) that focus on selected, narrow components of a patient evaluation (e.g., conducting a focussed history and relevant examination based on a presenting complaint). Direct observation of procedural skills assessments provide a variation on mini-CEX focussed on procedural skills. Both typically utilise examiner checklists which can reinforce a certain or single correct approach to the assessment. The alternative is to use global rating scales; however, these can pose challenges for less experienced examiners (e.g., recalling all the relevant aspects that a candidate should be evaluated on) (Norcini & Zaidi, 2019). The approach to marking these assessments may therefore vary depending on the local setting and examiner characteristics, but either could integrate uncertainty in ways similar to those for longer clinical skills assessments; this is perhaps more feasible when managing assessment of large cohorts.

In addition, clinical skills assessments can integrate authenticity and uncertainty by including teamwork. While parts of healthcare and managing uncertainty require decision-making by individuals, teamwork and collaboration can be effective ways of managing uncertainty (e.g., by integrating diverse perspectives). Formative assessment approaches to assessing teamwork and clinical skills in which a group of students in sequences performs part of an evaluation include the team objective structured clinical examination (which uses a standardised patient), and the team objective structured bedside assessment (which uses a real patient) (Deane et al., 2015; Miller et al., 2007). While the objective and structured nature of such approaches could reinforce a single correct answer, the topics selected could integrate uncertainty. In addition to sharing the benefits of integrating uncertainty within OSCEs, the team-based approaches might assist in learning to manage uncertainty by allowing learners to observe their peers’ approaches to uncertainty and by reinforcing how managing uncertainty as a team is an important component of clinical practice.

The extent to which uncertainty is assessed in these approaches depends more on how the assessments are delivered and less on the assessment strategy in isolation. There are multiple ways in which uncertainty can be integrated within existing approaches to clinical skills assessment. Rather than selecting one format for assessment, several formats (e.g., OSCEs, long cases, and direct observation during workplace-based learning) could be used within a programmatic assessment approach to build a picture of how each learner manages uncertainty.

Assessment of Attributes

Attributes such as professionalism, ethics, and cultural literacy are described as required learning outcomes in many health professions programs. The development of these attributes is also known to stimulate uncertainty; for instance, medical students describe uncertainty in integrating their personal and professional identities (Stephens et al., 2023). Therefore, assessments of the attributes could also provide opportunities to gauge learners’ capacity for managing uncertainty.

Evidence suggests that assessments of attributes should ideally be longitudinal and multifaceted (Dogra & Carter-Pokras, 2019; Irby & Hamstra, 2016). For example, assessment of professionalism may involve a combination of reflective learning, multisource feedback (including feedback from patients), self-assessment, and critical incident reports (Irby & Hamstra, 2016). Within these assessment tasks, students’ experiences of and skills for managing uncertainty in relation to the attributes could be highlighted. For instance, multisource feedback proformas might include communication or management of uncertainty as a criterion, and reflective learning could prompt learners to describe and critique their experiences of uncertainty. Reflective learning should ideally be formative, as summative approaches to reflective learning may result in students suppressing uncertainties to save face, which could hinder critical reflection on uncertainty (Sandars, 2009).

Moving towards Assessment of Uncertainty Tolerance

As the field of assessment and uncertainty tolerance continues to evolve, the steps described in the following dialogue cards may help educators identify where their existing assessment strategies integrate uncertainty and where adjustments can be made to influence learning about uncertainty. Turn the dialogue cards to read more about considerations within each step.

Summary

Assessment of core knowledge, skills, and attributes required by health professions learners can include evaluation of uncertainty management, particularly when educators consider how assessment tasks can integrate authentic uncertainties that learners will face in healthcare practice. To effectively build understanding of learners’ capacity for managing uncertainty, multiple different assessment tasks should be used across a health professions program, alongside feedback that helps learners to evaluate their own performance. The discourse around assessment and uncertainty tolerance is likely to change significantly in the near future, particularly regarding the potential impacts of artificial intelligence on assessment strategies. Further research involving the validity evidence for uncertainty tolerance scales and the impact of artificial intelligence on this field is required. For now, educators are encouraged to build bodies of evidence relevant to their health profession and healthcare discipline.

References

Accreditation Council for Graduate Medical Education. (2015). General pediatrics reported milestones. https://www.acgme.org/globalassets/PDFs/Milestones/CrosswalkPediatricsReportingMilestones.pdf

Ajjawi, R., Tai, J., Dollinger, M., Dawson, P., Boud, D., & Bearman, M. (2024). From authentic assessment to authenticity in assessment: Broadening perspectives. Assessment & Evaluation in Higher Education, 49(4), 499–510. https://doi.org/10.1080/02602938.2023.2271193

American Educational Research Association. (2014). Standards for educational and psychological testing.

Ashford-Rowe, K., Herrington, J., & Brown, C. (2014). Establishing the critical elements that determine authentic assessment. Assessment & Evaluation in Higher Education, 39(2), 205–222. https://doi.org/10.1080/02602938.2013.819566

Association of American Medical Colleges. (2021). 2022 AAMC medical school graduation questionnaire. https://www.aamc.org/media/51116/download

Association of American Medical Colleges. (2024). 2024 matriculating student questionnaire (MSQ). https://www.aamc.org/media/33911/download

Boursicot, K. A., Roberts, T. E., & Burdick, W. P. (2019). Structured assessments of clinical competence. In T. Swanwick, K. Forrest, & B. C. O’Brien (Eds.), Understanding medical education: Evidence, theory, and practice (3rd ed., pp. 335–345). Wiley. https://doi.org/10.1002/9781119373780.ch23

Budner, S. (1962). Intolerance of ambiguity as a personality variable. Journal of Personality, 30(1), 29–50. https://doi.org/10.1111/j.1467-6494.1962.tb02303.x

Carnegie Mellon University. (n.d.). What is the difference between formative and summative assessment? https://www.cmu.edu/teaching/assessment/basics/formative-summative.html

Cook, D. A., Brydges, R., Ginsburg, S., & Hatala, R. (2015). A contemporary approach to validity arguments: A practical guide to Kane’s framework. Medical Education, 49(6), 560–575. https://doi.org/10.1111/medu.12678

Deane, R. P., Joyce, P., & Murphy, D. J. (2015). Team objective structured bedside assessment (TOSBA) as formative assessment in undergraduate obstetrics and gynaecology: A cohort study. BMC Medical Education, 15, Article 172. https://doi.org/10.1186/s12909-015-0456-5

Diug, B., Howard, B., & Harvey, K. (2021). Teaching evidence-based medicine using authentic assessment the Whack-a-mole project [Conference presentation abstract]. International Journal of Epidemiology, 50(Suppl. 1), i59–i60. https://doi.org/10.1093/ije/dyab168.163

Dogra, N., & Carter-Pokras, O. (2019). Diversity in medical education. In T. Swanwick, K. Forrest, & B. C. O’Brien (Eds.), Understanding medical education: Evidence, theory, and practice (3rd ed., pp. 513–529). Wiley. https://doi.org/10.1002/9781119373780.ch35

Farlie, M. K., Johnson, C. E., Wilkinson, T. W., & Keating, J. L. (2021). Refining assessment: Rasch analysis in health professional education and research. Focus on Health Professional Education, 22(2), 88–104. https://doi.org/10.11157/fohpe.v22i2.569

Furnham, A., & Marks, J. (2013). Tolerance of ambiguity: A review of the recent literature. Psychology, 4(9), 717–728. https://doi.org/10.4236/psych.2013.49102

García-Pérez, A. M., & Yanes-Estévez, V. (2022). Longitudinal study of perceived environmental uncertainty: An application of Rasch methodology to SMES. Journal of Advances in Management Research, 19(5), 760–780. https://doi.org/10.1108/JAMR-02-2022-0033

Geller, G., Faden, R. R., & Levine, D. M. (1990). Tolerance for ambiguity among medical students: Implications for their selection, training and practice. Social Science & Medicine, 31(5), 619–624. https://doi.org/10.1016/0277-9536(90)90098-D

Geller, G., Tambor, E. S., Chase, G. A., & Holtzman, N. A. (1993). Measuring physicians’ tolerance for ambiguity and its relationship to their reported practices regarding genetic testing. Medical Care, 31(11), 989–1001. https://doi.org/10.1097/00005650-199311000-00002

Gerrity, M. S., DeVellis, R. F., & Earp, J. A. (1990). Physicians’ reactions to uncertainty in patient care: A new measure and new insights. Medical Care, 28(8), 724–736. https://doi.org/10.1097/00005650-199008000-00005

Gerrity, M. S., White, K. P., DeVellis, R. F., & Dittus, R. S. (1995). Physicians’ reactions to uncertainty: Refining the constructs and scales. Motivation and Emotion, 19(3),175–191. https://doi.org/10.1007/BF02250510

Han, P. K., Klein, W. M., & Arora, N. K. (2011). Varieties of uncertainty in health care: A conceptual taxonomy. Medical Decision Making, 31(6), 828–838. https://doi.org/10.1177/0272989×11393976

Hancock, J., & Mattick, K. (2020). Tolerance of ambiguity and psychological well-being in medical training: A systematic review. Medical Education, 54(2), 125–137. https://doi.org/10.1111/medu.14031

Hancock, J., Roberts, M., Monrouxe, L., & Mattick, K. (2015). Medical student and junior doctors’ tolerance of ambiguity: Development of a new scale. Advances in Health Sciences Education, 20(1), 113–130. https://doi.org/10.1007/s10459-014-9510-z

Hillen, M. A., Gutheil, C. M., Strout, T. D., Smets, E. M., & Han, P. K. (2017). Tolerance of uncertainty: Conceptual analysis, integrative model, and implications for healthcare. Social Science & Medicine, 180, 62–75. https://doi.org/10.1016/j.socscimed.2017.03.024

Irby, D. M., & Hamstra, S. J. (2016). Parting the clouds: Three professionalism frameworks in medical education. Academic Medicine, 91(12), 1606–1611. https://doi.org/10.1097/acm.0000000000001190

Johnson, C. E., Keating, J. L., Leech, M., Congdon, P., Kent, F., Farlie, M. K., & Molloy, E. K. (2021). Development of the feedback quality instrument: A guide for health professional educators in fostering learner-centred discussions. BMC Medical Education, 21, Article 382. https://doi.org/10.1186/s12909-021-02722-8

Jolly, B., & Dalton, M. J. (2019). Written assessment. In T. Swanwick, K. Forrest, & B. C. O’Brien (Eds.), Understanding medical education: Evidence, theory, and practice (3rd ed., pp. 291–317). Wiley. https://doi.org/10.1002/9781119373780.ch21

Lazarus, M. D., Gouda-Vossos, A., Ziebell, A., Parasnis, J., Mujumdar, S., & Brand, G. (2024). Mapping educational uncertainty stimuli to support health professions educators’ in developing learner uncertainty tolerance. Advances in Health Sciences Education. Advance online publication. https://doi.org/10.1007/s10459-024-10345-z

Lee, C., Hall, K., Anakin, M., & Pinnock, R. (2023). Medical students’ responses to uncertainty: A cross-sectional study using a new self-efficacy questionnaire in Aotearoa New Zealand. BMJ Open, 13(6), Article e066154. https://doi.org/10.1136/bmjopen-2022-066154

Miller, S. D., Butler, M. W., Meagher, F., Costello, R. W., & McElvaney, N. G. (2007). Team objective structured bedside assessment (TOSBA): A novel and feasible way of providing formative teaching and assessment. Medical Teacher, 29(2–3), 156–159. https://doi.org/10.1080/01421590701287889

Norcini, J. (2001). The validity of long cases. Medical Education, 35(8), 720–721. https://doi.org/10.1046/j.1365-2923.2001.01006.x

Norcini, J. J., Blank, L. L., Duffy, F. D., & Fortna, G. S. (2003). The mini-CEX: A method for assessing clinical skills. Annals of Internal Medicine, 138(6), 476–481. https://doi.org/10.7326/0003-4819-138-6-200303180-00012

Norcini, J. J., & Zaidi, Z. (2019). Workplace assessment. In T. Swanwick, K. Forrest and B. C. O’Brien (Eds.), Understanding medical education: Evidence, theory, and practice (3rd ed., pp. 319–334). Wiley. https://doi.org/10.1002/9781119373780.ch22

Rasch, G. (1966). An item analysis which takes individual differences into account. British Journal of Mathematical and Statistical Psychology, 19(1), 49–57. https://doi.org/10.1111/j.2044-8317.1966.tb00354.x

Raymond, J. E., Homer, C. S., Smith, R., & Gray, J. E. (2013). Learning through authentic assessment: An evaluation of a new development in the undergraduate midwifery curriculum. Nurse Education in Practice, 13(5), 471–476. https://doi.org/10.1016/j.nepr.2012.10.006

Richmond, A. (2022). The chicken and the egg: Clinical reasoning and uncertainty tolerance. Medical Education, 56(7), 696–698. https://doi.org/10.1111/medu.14814

Royal Australian College of General Practitioners. (2016). Curriculum for Australian general practice 2016: CS16 core skills unit. https://www.racgp.org.au/FSDEDEV/media/documents/Education/Curriculum/Curriculum-2016.pdf

Ryan, A. T., & Wilkinson, T. J. (2021). Rethinking assessment design: Evidence-informed strategies to boost educational impact in the anatomical sciences. Anatomical Sciences Education, 14(3), 361–367. https://doi.org/10.1002/ase.2075

Sam, A. H., Wilson, R. K., Lupton, M., Melville, C., Halse, O., Harris, J., & Meeran, K. (2020). Clinical prioritisation questions: A novel assessment tool to encourage tolerance of uncertainty? Medical Teacher, 42(4), 416–421. https://doi.org/10.1080/0142159X.2019.1687864

Sandars, J. (2009). The use of reflection in medical education: AMEE guide no. 44. Medical Teacher, 31(8), 685–695. https://doi.org/10.1080/01421590903050374

Schuwirth, L. W., & van der Vleuten, C. P. (2012). Programmatic assessment and Kane’s validity perspective. Medical Education, 46(1), 38–48. https://doi.org/10.1111/j.1365-2923.2011.04098.x

Schuwirth, L. W., & van der Vleuten, C. P. (2019). How to design a useful test. In T. Swanwick, K. Forrest, & B. C. O’Brien (Eds.), Understanding medical education: Evidence, theory, and practice (3rd ed., pp. 277–289). Wiley. https://doi.org/10.1002/9781119373780.ch20

Scott, I. M. (2020). Beyond ‘driving’: The relationship between assessment, performance and learning. Medical Education, 54(1), 54–59. https://doi.org/10.1111/medu.13935

Simpkin, A. L., & Armstrong, K. A. (2019). Communicating uncertainty: A narrative review and framework for future research. Journal of General Internal Medicine, 34(11), 2586–2591. https://doi.org/10.1007/s11606-019-04860-8

St-Onge, C., Young, M., Eva, K. W., & Hodges, B. (2017). Validity: One word with a plurality of meanings. Advances in Health Sciences Education, 22(4), 853–867. https://doi.org/10.1007/s10459-016-9716-3

Stephens, G. C., Brand, G., & Yahalom, S. (in press). Whose voices are heard in health professions education validity arguments? Medical Education. https://asmepublications.onlinelibrary.wiley.com/journal/13652923

Stephens, G. C., Karim, M. N., Sarkar, M., Wilson, A. B., & Lazarus, M. D. (2022). Reliability of uncertainty tolerance scales implemented among physicians and medical students: A systematic review and meta-analysis. Academic Medicine, 97(9), 1413–1422. https://doi.org/10.1097/ACM.0000000000004641

Stephens, G. C., & Lazarus, M. D. (2024). Twelve tips for developing healthcare learners’ uncertainty tolerance. Medical Teacher, 46(8),1035–1043. https://doi.org/10.1080/0142159X.2024.2307500

Stephens, G. C., Lazarus, M. D., Sarkar, M., Karim, M. N., & Wilson, A. B. (2023  ). Identifying validity evidence for uncertainty tolerance scales: A systematic review. Medical Education, 57(9), 844–856. https://doi.org/10.1111/medu.15014

Stephens, G. C., Sarkar, M., & Lazarus, M. D. (2022a). Medical student experiences of uncertainty tolerance moderators: A longitudinal qualitative study. Frontiers in Medicine, 9, Article 864141. https://doi.org/10.3389/fmed.2022.864141

Stephens, G. C., Sarkar, M., & Lazarus, M. D. (2022b). ‘A whole lot of uncertainty’: A qualitative study exploring clinical medical students’ experiences of uncertainty stimuli. Medical Education, 56(7), 736–746. https://doi.org/10.1111/medu.14743

Stephens, G. C., Sarkar, M., & Lazarus, M. D. (2024). ‘I was uncertain, but I was acting on it’: A longitudinal qualitative study of medical students’ responses to uncertainty. Medical Education, 58(7), 869–879. https://doi.org/10.1111/medu.15269

van der Vleuten, C. P., Schuwirth, L. W., Driessen, E. W., Govaerts, M. J., & Heeneman, S. (2014). Twelve tips for programmatic assessment. Medical Teacher, 37(7), 641–646. https://doi.org/10.3109/0142159X.2014.973388

Villarroel, V., Bloxham, S., Bruna, D., Bruna, C., & Herrera-Seda, C. (2018). Authentic assessment: Creating a blueprint for course design. Assessment & Evaluation in Higher Education, 43(5), 840–854. https://doi.org/10.1080/02602938.2017.1412396

Wass, V., Jones, R., & van der Vleuten, C. (2001). Standardized or real patients to test clinical competence? The long case revisited. Medical Education, 35(4), 321–325. https://doi.org/10.1046/j.1365-2923.2001.00928.x

About the authors

name: Dr. Georgina C. Stephens, MBBS(Hons) PhD FHEA

institution: Monash University

website: https://research.monash.edu/en/persons/georgina-stephens

As a Senior Lecturer in the Centre for Human Anatomy Education at Monash University and medical practitioner by background, Georgina has first hand experience of what it means to manage uncertainty when caring for people seeking healthcare. Dr. Stephens transitioned to a full time academic career in 2017, focussed on clinical anatomy education and health professions education research. During her doctoral studies, she explored how medical students experience uncertainty, and examined the evidence for widely used scales intended to measure the construct of uncertainty tolerance. Her doctoral research led to five peer reviewed publications on uncertainty tolerance, all published in leading health professions education journals, and several winning awards for publication excellence. Georgina is an award winning educator, including being awarded the Dean’s Award for Innovation in Education in 2023.

definition

name: Prof Michelle Lazarus, PhD SFHEA

institution: Monash University

website: https://research.monash.edu/en/persons/michelle-lazarus

As a Professor and Director of the Centre for Human Anatomy Education and the Deputy Director for the Monash Centre for Scholarship in Health Education at Monash University, Michelle has been in the field of medical education for over a decade, leading a research program which explores how to impact learners’ uncertainty tolerance through curriculum design. She has delivered over a dozen related workshops to educators across the globe, and has developed a series of educational artefacts to support learner uncertainty tolerance development, including a pamphlet and webinar, for Education Services Australia for teachers interested in integrating uncertainty tolerance teaching practices in their classrooms. This textbook expands on these initiatives, providing a more holistic and complete source of uncertainty tolerance theory and practice – focusing specifically on health professions educators. Michelle is an award winning educator and author. Notably, she was awarded the Australian Award for University Teaching Excellence in 2021. She is the author of the “The Uncertainty Effect: How to Survive and Thrive through the Unexpected”. Her entire career is a journey into uncertainty.

definition