13 Closing the Loop: Measuring, Evaluating, Reviewing and Improving
The final phase in a safety management system is to ‘close the loop’ by evaluating safety data to determine the cycle’s safety outcomes, before deciding what is reasonably practicable to implement in the next safety management system cycle. This corresponds to the ‘Check’ and ‘Act’ phases of Plan-Do-Check-Act (see figure 6.4).
Learning Objectives
This chapter explains:
- The expected components of a data-driven safety performance review.
- The purpose of a safety performance review.
Measurement of safety should be an ongoing process that is embedded into day-to-day implementation of WHS. Ideally, if the system was planned well, workers are continually collecting data as safety activities are undertaken (input indicators), or as issues arise (output indicators), such as reporting a near miss. Input and output KPIs should then be centralised into a reporting system that can provide up-to-date insights for safety management staff via dashboards. However, evaluation of this data is crucial as data on its own does not generate insights or invoke action.
A principal responsibility of WHS systems specialist staff is to ensure safety performance reviews with senior leadership are scheduled and incorporated into regular business reporting cycles. The purpose of this is to ensure that safety remains a priority for the business, but also to foster the leadership’s commitment to safety and, finally, to secure the resources required to sustain and enhance safety initiatives that, in turn, should lead to improved safety outcomes at the conclusion of the next full cycle of the safety management system.
What comprises a safety review? A data-driven presentation of safety input indicators and output indicators for that reporting period, perhaps complemented by anecdotes that demonstrate that the data reflects impacts on people, culminating in an honest appraisal of the organisation’s performance on hazard control. Safety management staff would then present their proposed safety initiatives for the next reporting cycle, including the resources required to enhance control of current safety challenges, while quickly addressing new issues that have emerged across the last cycle of the system. The safety review would comprise both ‘health’ and ‘safety’ data.
The WHS ‘health’ data would include input indicators, outlining both existing and emerging occupational health preventative measures, and output indicators, updating senior leaders on the occurrence of occupational illness and injury as identified in their worker population. It is a time to brief senior leaders on any emerging occupational disease trends, as identified by public health officials and communicated to the organisation by the WHS regulator.
WHS ‘safety’ data would include input indicators, such as attendance rates at safety training courses or the number of completed emergency evacuation drills, and output indicators, including lost time injuries, and injuries without lost time and near misses (see Chapter 5). Considering this in the context of Reason’s Swiss Cheese Model (see Figure 5.1), the review considers both active failures and latent conditions. If the safety system is functioning well, and no critical incidents occurred, the safety team will explain how inputs into the safety system combined with the safety culture to minimise and prevent losses. This would be an ideal outcome.
One of the greatest challenges of a successful safety management system is securing the resources to sustain and continually improve safety management. It is easy for senior leaders, despite their moral judgement or business risk commitment to safety, to become complacent when safety management is effective in preventing loss within the organisation. As Wieck & Sutcliffe (2001) advise, organisations that are most effective at safety management, such as high reliability organisations, figure out ways to stay ‘mindful’ and preoccupied with safety failure. If this can be achieved, your organisation will stay in the proactive or generative rungs of Hudson’s Safety Culture Ladder (see Figure 6.2). If not, your organisation may find itself slipping towards a calculative or reactive safety culture and, longer term, less effective WHS management.
Notably, organisations that undergo safety system certification evaluate the rigour of their internal safety management system through external, independent, auditing. Some organisations do this to foster their social licence to operate, as it independently demonstrates the organisation’s capacity to manage safety. Other businesses may undertake certification as a governance practice, as part of due diligence, in order to ensure safety staff are accountable for delivery of their process enhancements. Rarely would the auditing cycle of a standards accreditation body be as frequent as an organisation’s internal safety management review cycle. This means that the auditing cycle is overlaid upon the organisation’s safety review cycle and, when these occur, their insights are pivotal and inform system-level improvements.
At the end of your safety review, with your leadership commitment re-affirmed and your budget allocated for the next cycle in your organisation of WHS management, according to the PDCA approach, it is time to “take action based on what you learned” (American Society for Quality, n.d., para. 3). You would examine, based on insights from the review, the adequacy of safety policy and procedures and proceed to enact any required changes. You would then enter the planning phase of the next safety management system cycle, before implementing required changes. Whatever the review timeframe is, continuous improvement can only be achieved by going through the safety management cycle and seeking out opportunities to enhance safety management over and over again (see Figure 6.1), in recognition that internal and external factors are constantly acting on your organisation (see Figure 6.4), potentially creating weaknesses that will develop into holes in your Swiss Cheese (Figure 5.1). Continuous improvement is infinite, however, its discussion in this context of WHS management for HR management must now conclude.
This section of the book, Practice: Establishing, implementing and closing the loop in a safety management system, was designed as a conceptual overview of how organisations can approach the implementation of WHS safety management systems using a standards-based approach, while recognising that the specific tools and techniques that each business will adopt will be based on factors including the scale of the business, the nature of the work (ie. level of risk), and the legislative requirements of the jurisdiction within which the business operates. These chapters were designed to assist you, as an HR practitioner, to identify your moral stance on safety and, most importantly, to determine the level of competency you need, particularly if further professional development is required, to competently fulfil your safety management role within your organisation.
Australia's Model Work Health and Safety Bill defines reasonably practicable as:
"In this Act, reasonably practicable, in relation to a duty to ensure health and safety, means that which is, or was at a particular time, reasonably able to be done in relation to ensuring health and safety, taking into account and weighing up all relevant matters including:
(a) the likelihood of the hazard or the risk concerned occurring; and
(b) the degree of harm that might result from the hazard or the risk; and
(c) what the person concerned knows, or ought reasonably to know, about:
(i) the hazard or the risk; and
(ii) ways of eliminating or minimising the risk; and
(d) the availability and suitability of ways to eliminate or minimise the risk; and
(e) after assessing the extent of the risk and the available ways of eliminating or minimising the risk, the cost associated with available ways of eliminating or minimising the risk, including whether the cost is grossly disproportionate to the risk" (WorkSafe Australia, 2023, Section 18).
“The Plan-do-check-act cycle is a four-step model for carrying out change. Just as a circle has no end, the PDCA cycle should be repeated again and again for continuous improvement” (ASQ, 2023, para. 1) where the different steps are: “Plan: Recognize an opportunity and plan a change, Do: Test the change. Carry out a small-scale study; Check: Review the test, analyze the results, and identify what you’ve learned; Act: Take action based on what you learned in the study step” (ASQ, 2023, para. 3).
"These indicators refer to the resources needed for the implementation of an activity or intervention. Policies, human resources, materials, financial resources are examples of input indicators. Example: inputs to conduct a training course may include facilitators, training materials, funds" (World Health Organization, 2014, para. 1)
Near misses are "unplanned incidents that occurred at the workplace that, although not resulting in any injury or disease, had the potential to do so” (Archer et al., 2015, p. 86). When contextualised in the Swiss Cheese Model of incident causation, they can be understood as "an event that could have potentially resulted in…losses, but the chain of events stopped in time to prevent this” (Wolters Kluwer, n.d., para. 1).
“Key performance indicators (KPIs) are used to measure and monitor whether an organization is on the right track…KPIs play an important role in modern organizations improving performance is key to achieving organizational success” (Madsen and Stenheim, 2022, para. 1).
"A dashboard is a way of displaying various types of visual data in one place. Usually, a dashboard is intended to convey different, but related information in an easy-to-digest form. And oftentimes, this includes things like key performance indicators (KPI)s or other important business metrics that stakeholders need to see and understand at a glance" (Tableau Software, n.d., para. 5).
"Outcome indicators refer more specifically to the objectives of an intervention, that is its ‘results’, its outcome...These indicators, therefore, allow us to know whether the desired outcome has been generated. It may take time before final outcomes can be measured" (World Health Organization, 2014, para. 7).
Occupational health had its modest beginnings in first aid and disease controls for high risk heavy industry workplaces, such as mines, but gained greater recognition in the 1970s when the World Health Organization acknowledged its contribution to the identification of workplace-derived factors causing occupational illness and suggested its remit should be broadened to encompass public health. So, today, many occupational health specialists have more of a public health, and less of an immediate workplace, focus (Schilling, 1989).
In principle, and in accordance with the Australian Model Work Health and Safety Bill, a regulator is a legally established government body whose functions are:
“ (a) to advise and make recommendations to the Minister and report on the operation and effectiveness of this Act;
(b) to monitor and enforce compliance with this Act;
(c) to provide advice and information on work health and safety to duty holders under this Act and to the community;
(d) to collect, analyse and publish statistics relating to work health and safety;
(e) to foster a co-operative, consultative relationship between duty holders and the persons to whom they owe duties and their representatives in relation to work health and safety matters;
(f) to promote and support education and training on matters relating to work health and safety;
(g) to engage in, promote and co-ordinate the sharing of information to achieve the object of this Act, including the sharing of information with a corresponding regulator;
(h) to conduct and defend proceedings under this Act before a court or tribunal;
(i) any other function conferred on the regulator by this Act”
(Safe Work Australia, 2023, section 152).
"James Reason proposed the image of "Swiss cheese" to explain the occurrence of system failures...According to this metaphor, in a complex system, hazards are prevented from causing human losses by a series of barriers. Each barrier has unintended weaknesses, or holes – hence the similarity with Swiss cheese. These weaknesses are inconstant – i.e., the holes open and close at random. When by chance all holes are aligned, the hazard reaches the patient and causes harm" (Perneger, 2005, p. 71).
"Human beings contribute to the breakdown of such [complex technological] systems in two ways. Most obviously, it is by errors and violations committed at the 'shape end' of the system...Such unsafe acts are likely to have a direct impact on the safety of the system and, because of the immediacy of their adverse effects, these acts are termed active failures" (Reason, 1997, p. 10).
"Fallibility is an inescapable part of the human condition, it is now recognized that people working in complex systems make errors or violate procedures for reasons that generally go beyond the scope of individual psychology. These reasons are latent conditions....poor design, gaps in supervision, undetected manufacturing defects or maintenance failures, unworkable procedures, clumsy automation, shortfalls in training, less than adequate tools and equipment - may be present for many years before they combine with local circumstances and active failures to penetrate the system's many layers of defences" (Reason, 1997, p. 10).
A critical incident is “a sudden, unexpected and overwhelming event, that is out of the range of expected experiences” (UNHCR, 2019, para. 1).
Moral judgement is the “evaluation of actions with respect to moral norms and values established in society” (Thoma et. al. 1991 cited in Li et al., 2017, p. 122).
Business risk may be defined as "the exposure a company or organization has to factors that could lower its profits or lead it to fail” (Kenton, 2022, para. 1).
High reliability organisations are distinguished from other organisations as they have a: “…preoccupation with failure, reluctance to simplify interpretations, sensitivity to operations, commitment to resilience and deference to expertise” (Weick & Sutcliffe, 2001, p. 30).
“A culture that supports an organization’s OH&S management system is largely determined by top management and is the product of individual and group values, attitudes, managerial practices, perceptions, competencies and patters of activities that determine commitment to, and the style and proficiency of, its OH&S management system” (Standards Australia and Standards New Zealand, 2018), p. 27).
Certification is the auditing of a business, by an independent accredited entity, to demonstrate (certify) its compliance with standards set by a standards organisation (United Kingdom Accreditation Service, n.d.).
"The social license to operate (SLO), or simply social license, refers to the ongoing acceptance of a company or industry's standard business practices and operating procedures by its employees, stakeholders, and the general public. The concept of social license is closely related to the concept of sustainability and the triple bottom line" (Kenton, 2023, para. 1).
"The care that a reasonable person exercises to avoid harm to other persons or their property" (Merriam-Webster, n.d., para. 1).
Standards are agreed to principles and approaches established by panels of experts (International Organization for Standardization, n.d.).
Accreditation is formal authorisation by an standards organisation permitting the accredited entity (person or business) to certify other businesses against their standards (United Kingdom Accreditation Service, n.d.).
“A law or set of laws suggested by a government and made official by a parliament” (Cambridge Dictionary, n.d., para. 1).
“The power, right, or authority to interpret and apply the law…the limits or territory within which authority may be exercised” (Merriam-Webster, n.d., para. 1).