Measuring Resilience Potentials: A Pilot Program Using the Resilience Assessment Grid

Abstract

Researchers in the resilience engineering space have proposed the notion that organisations operating in complex socio-technical systems cannot ‘be’ resilient but can have the ‘potential for resilient performance’. This theoretical stance also suggests that organisations wanting to enhance their potential for resilience begin by measuring their operational safety performance against four key potentials, these being: the Potential to Anticipate; the Potential to Respond; the Potential to Learn; and the Potential to Monitor. Furthermore, to measure these four key resilience constructs, organisations have been recommended to use a Resilience Assessment Grid (RAG) developed as part of this theory. However, scarce research appears to have been conducted that bridges the theory and practice divide on just how organisations can pragmatically measure their current performance against these four resilience potentials using the RAG. Therefore, this research was interested in undertaking a pilot study using RAG theory in order to examine an organisation’s four resilience potentials, and was conducted within a large road transport organisation in Australia. Results indicated that measuring both the four individual potentials and a combination of the four potentials was possible using a RAG and proved effective in providing a snapshot of operational safety system resilience concepts. Recommendations on how to increase organisational resilience potentials were provided to ensure future safety endeavours would enhance the organisation’s potential to be resilience in the face of system variability and operational demands.

1. Introduction

Within the safety sciences, the call has gone out for those responsible for managing safety to recognise the need to move towards a focus on managing variability in complex safety systems rather than managing occupational health and safety purely focused on paper-based management systems, safety culture and safety rules [1]. This is supported by the notion that the management of safety has moved into the ‘adaptive age of safety’ [1,2] and that organisations now find themselves in the third era of safety, this being the ‘resilience’ era. This is where much more sophisticated approaches in the resilience space are required in order to adjust performance as conditions change, both through more flexibility of work actions and the adoption of an adaptive leadership approach [3,4], where complex approaches are needed in order to manage safety system variability. The notion of measuring resilience in socio-technical work systems is gaining interest as opposed to merely invoking the concept and it is seen as a critical next step for a science of macro-cognitive work systems [5].

Both High-Reliability Organisational (HRO) theory and Resilience Engineering (RE) theory continue to make central contributions to both the safety sciences and organisational resilience fields in particular. These two theories, whilst considered different in some of their theoretical viewpoints and language, have many similarities in their orientations towards ensuring organisations can manage system fluctuations and remain resilient in their operations [6]. Workforce and organisational resilience are seen as critical to ensuring readiness for change and adaption over time with a resilience culture being one that “is built on principles of organisational empowerment, purpose, trust, and accountability” [7] (p. 8).

In attempting to resolve issues around the various meanings and definitions of resilience, Woods [8] has examined four concepts typically implied of resilience being rebound, robustness, extensibility and adaptability. Here, each concept stimulates lines of inquiry and has its own defining features requiring various definitions depending on which lens is being used to look at resilience. A recent paper published in this journal by Pillay and Morel [9], which focused on measuring resilience engineering and bench-marking organisational safety, listed 12 varying definitions and resulted in the authors offering the following description in an attempt to provide a unified definition, “RE is a perspective for organisational safety management which enables organisational members to actively anticipate, respond, monitor and learn; by adapting to operate at the boundary of safe operations by narrowing the gap between work as imagined and work as performed; and manifested in an organisation’s culture, cognition and behaviours” [9] (p. 20).

Importantly, as with any organisational performance strategy, organisations need the means to know (via measurement) what the current situation is, to set a future direction and to know what to target for, and then how to bring about change in order to move the organisation towards its goals. If any of these key elements are missing, organisations find themselves arriving at the wrong place, having a major disaster or falling into uncontrolled drift [10].

Resilience Engineering (RE) theory has developed to the point where it has been able to provide theoretical suggestions on measuring resilience potentials, which are now being tested [11]. Hollnagel’s [10] latest work provides a comprehensive framework from which research endeavouring to measure an organisation’s ‘potential for resilience’ can now take place. In this context the starting point is the application of a Resilience Assessment Grid (RAG) whereby the potential for resilience is measured against four key constructs, these being; the Potential to Respond, the Potential to Monitor, the Potential to Learn and the Potential to Anticipate, which are usually already thought to be embedded somewhere within current safety management endeavours and the socio-technical systems in which work is conducted.

Support for measuring resilience has come from many sources including the Business Continuity Institute [BCI] [12] which has recognised that “More organisations are now focusing on how to measure resilience within their business with the goal to create a culture of resilience … with a view to being able to respond in a coordinated and effective way to disruptions across business …” and it is well recognised that “a successful organisational resilience strategy requires multidisciplinary collaboration, continuous commitment, and long-term investment in the areas of people, processes, technology and infrastructure” (p. 9).

Efforts to apply the RAG as a measurement tool saw its early application in both railway traffic management [13] and air traffic control [14]. By 2017, the Australian Radiation Protection and Nuclear Safety Agency had included the four resilience potentials in its Holistic Safety Guidelines [15] and this was closely followed by the publication of the Australian Organisational Resilience Practitioner Guide for Public Sector Organisations [16]. Since that time studies have continued into specific applications including a comparison of four hospital emergency departments [11] and the production of aircraft components [17]. Some more recent efforts, particularly in the safety sciences, have included the measurement the four resilience potentials within upper and lower-tier enterprises [18] and communication as an aspect of resilient organisational behaviour in emergency situations [19]. However, studies which specifically and pragmatically use the RAG as a starting point for the measurement of an organisation’s overall resilience performance against the four potentials for resilience appear to be both a gap in the literature and a topic gaining wide interest.

Therefore, the research presented here aimed to bridge the theory into practice divide by undertaking a pilot program which would measure the four resilience potentials, for a large commercial transport company based in Queensland, Australia. The research question to be explored was ‘can the measurement of the four resilience potentials, as suggested by RAG theory, give meaningful information about the organisation’s current position as to its potential for resilient performance’? The outcome goal was to be able to provide the transport organisation with an initial starting point and overview of its current resilience potentials position and to be able to provide recommendations for improving safety performance enhancements from a resilience perspective.

To achieve this purpose a RAG was developed specifically for the organisation based on RAG theory [10] and middle managers were surveyed to measure organisational performance against the four resilience potentials. Spider graphs were developed for each of the 4 potentials to identify where resilience efforts should be directed for each potential, as well as examining results as a big picture resilience RAG model, of where the organisation was in relation to the overall management of their potential to be resilience. The pilot program was embedded in the philosophy of pragmatism which is used extensively in the Safety Sciences as a problem-solving research methodology and which points towards a stage 3 approach in model cumulative theory development as proposed by Klockner and Pillay [20] whereby a pilot study is undertaken to advance theory into practice research.

This paper, firstly, presents theoretical discussions in Section 2 by examining the latest understanding of the concept of resilience within Safety I and Safety II theory particular in relation to system variability and adaption. Section 3 introduces how resilience potentials can be measured using the RAG developed for this purpose [10]. Section 4 then sets the scene for the organisational pilot program, which was conducted within a road transport organisation, followed by details of the methodology used in Section 5. Results are then presented in Section 6 and an in-depth discussion of findings presented in Section 7. In conclusion, Section 8 discussed the result of the pilot program in meeting the research aims and objectives and makes recommendations based on both theoretical and practical outcomes from the pilot program.

2. The Resilience Age of Safety—Safety I and Safety II

Resilience within the RE space has been defined as ‘an organisation’s ability to function as required under expected and unexpected conditions alike’ [10] (p. i). Researchers in this space make it clear that it is more about what an organisation does and how it performs that is important, rather than what it has [10]. The focus of RE has also been acknowledged to include the notion that ‘since its inception, those who have developed RE have made it clear that its scope accounts more for what ‘goes right’ in system performance, than what does not. This is in the interest of enabling systems and organisations to continue to operate in the face of unforeseen large-scale demands, as well as to improve their everyday functioning’ [21] (p. 1). The meaning behind this is that reliance on safety management systems, procedures and policies are no longer recognised as being the main area where safety professionals needed to focus, as the limitations with an overemphasis on these systems have been emerging for some time.

The focus on what is now known as Safety 1 concepts, being merely reducing accidents and incidents, has moved to an understanding that “effective safety management strategies in this age involve organisational learning which comes from both failures and successes as people adapted to create safety well before failure and harm occur” [3] (p. 1840). Safety science work in the resilience era, therefore, requires a different kind of thinking, one that has not always been valued by organisations when talking about safety. Safety as the presence of capacity is considered one of the main underlying principles in order to make things ‘go right’ in what has now been coined the Safety II paradigm.

This emphasis has been predominantly espoused by the RE space which has been theoretically moving from protective (Safety-I) to productive safety (Safety-II) where everyday work can be done successfully. In this resilience space, Hollnagel [22] acknowledges that it is not a matter of if an organisation is or is not resilient, but that all organisations have the ‘potential for resilience’ or ‘performing in a manner that can be characterised as resilient’ and that ‘resilience is a characteristic of how a system performs’ with the discussion needing to be around ‘a system’s resilient performance’ [22] (p. 1).

Work in the RE area, therefore, continues to mature and resilience is seen as a sophisticated approach to the management of safety. Hollangel [10], in his review of the development and understanding of the concept of resilience since the 1970s, recognizes that it has moved from a focus on negative connotations to a more positive and people-focused notion, and his latest definition attests to this: “Resilience is an expression of how people, alone or together, cope with everyday situations, large or small, by adjusting their performance to the conditions. An organisation’s performance is resilient if it can function as required under expected and unexpected alike (changes/disturbances/opportunities)” [10] (pp. 14–15).

Safety-I assumptions are effective if a system’s complexity is limited; however, in today’s complex sociotechnical systems, there are numerous highly connected interactions among humans, machines, internal and external organisational procedures and regulations that enable the loss of an effective understanding of system functioning [23]. Normal work activities or ‘work-as-done’ (what the worker is actually doing) are recognised as differing from ‘work-as-imagined’ (the written procedure or instruction for the worker) and this difference necessitates the workers’ adaptation to reconcile the variable working scenarios and safely ensure the system’s productivity [24,25,26]. This adaptation is at the centre of the concept of RE. The resilience concept is linked to terms such as elastic, buoyant, robust and flexible. It may be described as “… a feature of some systems that allows them to respond to an unanticipated disturbance that can lead to failure and then to resume normal operations quickly and with a minimum decrement in their performance” [27] (p. 376).

One of the principal efforts within the RE discipline has been the anchoring of the notion of resilience to some form of clearly describable features or attributes to enable the development of an operational means for the assessment of resilience [8,20,22,28]. There have been a few attempts to provide meanings, features, attributes or properties to the concept of RE and to this end two of the pioneering contributors, considered by many to be the founding fathers of RE, David Woods and Eric Hollnagel, have been endeavouring to provide a measurement and assessment framework.

3. Measuring Resilience Potentials—The Resilience Assessment Grid (RAG)

Woods [8] proposed that there are four basic concepts to resilience with these being; (1) resilience as rebound; (2) resilience as robustness; (3) resilience as graceful extendibility; and (4) resilience as sustained adaptability. The reasoning for developing and highlighting these concepts is that the notion of resilience itself has become overused and confused. By placing these four core concepts of resilience within the RE domain Woods is attempting to enable more targeted empirical research to be conducted to provide a better understanding of the resilience concept as a whole, what Woods [8] (p. 8) describes as “... fundamental findings, foundation theories and engineering techniques”.

Later work by Duchek [29] has identified organisational resilience as a capability model made up of three stages that represent an offence to unexpected events instead of a defensive response and include attempts to anticipate critical developments and potential threats, to cope during critical situations and to adapt, transform and learn after critical situations have occurred as shown in Figure 1 [29] (p. 224).

3.1. The Resilience Assessment Grid

Previously known as the Resilience Analysis Grid, the Resilience Assessment Grid (RAG) has appeared in the literature through a number of articles and conference papers culminating in its full details being outlined in a ‘Safety-II in Practice: Developing the Resilience Potentials’ book by Hollnagel [10]. The book is the most complete text explaining what the RAG is and how the RAG can be utilised in examining an organisation’s potential to be resilient in managing the expected and unexpected when it happens. The key notion is that an organisation can only aspire to be resilient by building its resilience potentials through measuring where it currently is and assessing that against where it wants to be.

The four ‘resilience potentials’ were proposed as being (1) the Potential to Respond, (2) the Potential to Monitor, (3) the Potential to Learn and (4) the Potential to Anticipate with all four potentials playing a role in an organisation’s overarching potential to be resilient [10,22]. This RAG framework uses ‘the potential for resilience’ as the main concept due to the assertion that organisation cannot ‘be’ resilient but can have the potential to become resilient and that resilience is an ever-changing condition of the organisation [10,22]. The RAG is a tool and method to enable an organisation to measure its potential for resilience at a specific time and provide identification as to which potentials can or need to be improved to further develop or enhance the organisation’s resilience potential [10].

The starting point to build a RAG is to develop a set of questions to use as a survey within a subset of the organisational workforce. The survey does not necessarily work from a generic set of questions but is a set of guiding questions for use as a starting point by Hollnagel [10]. It may be possible, however, to take a set of questions developed for one organisation and apply them to one or more other organisations in the same industry with little or minor modification if the basic context of the organisations has similarities. The use of generic questions is not fully detrimental to the assessment of the resilience potentials; however, properly structured and worded questions specific to the organisation/industry under investigation would yield more accurate results. It is believed that the RAG would serve as a helpful measurement tool in identifying areas where the organisation can make improvements towards its potential for resilience and the possible benefits of using a RAG have been explained in some research papers [23], few research papers have tested its usefulness in practice in an attempt to reduce the theory into practice divide.

3.2. The Four Potentials of Resilience

The research presented here used a modified version of the generic questions posed by Hollnagel [10] to develop radar graph/s of the four potentials of resilience which are also defined by Hollnagel [10] as being (1) the Potential to Respond, (2) the Potential to Monitor, (3) the Potential to Learn and (4) the Potential to Anticipate as defined here:

• The Potential to Respond
  Knowing what to do or being able to respond to the irregular and regular. Responding to changes, opportunities and disturbances by activating prepared actions or by adjusting current actions to suit the new conditions and changes to the environment in which it operates.
• The Potential to Monitor
  Knowing what to look for or being able to monitor that which is or could seriously affect the organisation’s performance in the immediate future either positively (Safety II) or negatively (Safety I). Monitoring must cover the organisation’s own performance as well as what happens in the environment in which it operates.
• The Potential to Learn
  Knowing what has happened, being able to learn from all experiences and to learn the right lessons from the expected conditions and unexpected conditions by understanding both what goes right, as well as the right (Safety II) lessons from unexpected conditions and what goes wrong (Safety I).
• The Potential to Anticipate
  Determining or predicting what to expect or being able to anticipate conditions/environment further into the future, such as possible disruptions, unusual demands or constraints, new opportunities, or changing operating conditions.

It is proposed that resilience cannot be directly controlled or managed; however, it can be managed and monitored indirectly through the four potentials [10]. The four potentials should not be managed individually as increasing one without the others may provide benefit to the organisation but would not increase the organisation’s overall potential to be resilient [10].

This research was interested in undertaking a small pilot program in order to develop a RAG to measure and review the resilience potentials in a large commercial transport organisation operating in the Australian road freight industry. The organisation itself was interested in measuring and understanding its current position for its potentials for resilience with a view to any results informing strategic resilience endeavours as well as embedding Safety II theoretical concepts into pragmatic real-world practice.

4. Pilot Program Organisational Setting

In Australia, the road transport industry has capitalised on interactions of an underdeveloped rail network [30] and a vast interconnected network of metropolitan, regional, rural and remote roads [31] to ensure the successful delivery of more than 2 billion tonnes of freight by road each year [32]. The road freight transport industry is, however, a priority industry under the Australian Work Health and Safety Strategy 2012–2022 [33] having the highest fatality rate and highest serious injury claims rate, with truck drivers accounting for the highest number of fatalities within this industry [34]. Safety management systems (SMS) have no recognition status under National Heavy Vehicle Transport Law in Australian, unlike their counterparts in rail and aviation safety [35], and the industry as a whole faces many challenges to enhance its safety record with transport operators working in the industry having varied levels of safety culture sophistication.

The road transport industry in Australia is not immune to business disruption and businesses operating in this space need to be able to respond to rapidly evolving new risks and disruptions. Many of the larger transport operators are working towards enhancing safety practices and are keen to embrace programs and initiatives that can assist them in enriching their operational performance in the safety area in order to remain resilient in the management of their commercial transport operations.

The Australian-owned and nationally based organisation used in this research has transport depots and offices in every state. Employing over 600 staff with hundreds of registered vehicles from standard motor vehicles to multi-combination heavy vehicles travelling tens of thousands of kilometres per year. The organisation operates within the metro, rural and remote areas of Australia some of which are the harshest and most isolated environments for people and machinery. The organisation was interested in the study to help them gain long term improvements in their overall systems and processes, not just those related to safety. It intended to use this as an opportunity to start the journey to build an organisation that continues to build upon its potential to be resilient within the ever-changing modern complex environments in which it functions.

5. Materials and Methods

This research was conducted using participants from this single transport organisation across 3 of its depots based in South Australia and Queensland, Australia. The primary methodology was the use of a voluntary and anonymous paper-based questionnaire with participants being selected from middle management and supervisory positions. Although participation was voluntary, the participants were approached in workgroups via written correspondence (e.g., email or letter request to participate). Since the study was based within a single organisation and for the benefit of the organisation, the organisation encouraged as many people to participate as possible and all subjects gave their informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Ethics Committee of Central Queensland University (Project No. 2018-063).

5.1. Middle-Management Participants

Within the organisational resilience space, the proposed relationship between organisational levels suggested by Vogus and Sutcliffe [36] is one that explains that middle managers are needed as a conduit for the transfer of relevant and timely information. The Vogus and Sutcliffe [36] model has middle managers of an organisation being a continuous loop for moving strategic organisational information to the organising front-line workers and back again. They are also the conduit that top administrators in the organisation direct information to in order to pass it on to front line workers. The Vogus and Sutcliffe [36] model describes middle managers as ‘translators and enablers’ who can reinforce and refine organisational practices. Therefore, it was decided that for the pilot program, middle managers would be a logical place to gauge the initial view of operational processes related to the four resilience potential concepts. Therefore, middle managers based throughout the organisation were targeted for the pilot program.

5.2. RAG Question Set Development

To develop a RAG, there are five guiding steps as advised by the author [10] and the advice is that the development of the RAG is not only organizationally specific but specific to the context that the organisation currently finds itself in [10]. The five important steps are: (1) the development of formative and diagnostic questions set for the organisation, using the experience of the organisation to shape the content and context of the questions; (2) the development of a model of the organisation’s functions with the four potentials included, mapping the functions across the organisation and linking them to the potentials; (3) the application of the RAG to a subset of the workforce as there is no requirement to apply the tool for every worker (test, reapply); (4) work with the same subset of workers for each iteration of the assessment; and (5) the RAG is a longitudinal assessment tool that works best over an extended period and the finished item may be as complicated or as simple as the organisation needs.

The survey questionnaire design was modelled on the questions raised within the book Safety-II in Practice: Developing the Resilience Potentials [10] and only slightly modified to ensure the wording was suitable for the participant organisation, as the book suggested. A 29-question set survey was therefore developed using the generic questions derived from the examples within Hollnagel’s [10] book.

Questions related to the issues relating to the potential to respond are shown in

Table 1. Potential to Respond survey questions.

Potential to Respond
1. There are procedures in place to identify and maintain our capacity to work whenever unexpected events occur
2. We discuss our skills with each other to know who has relevant specialised skills and knowledge within the organization/branch
3. Workers discuss alternative ways of carrying out tasks safely
4. There is access to a variety of resources whenever unexpected events occur
5. Information is shared in a timely manner and communicated to those in the position to influence change

and include examining what procedures are in place; if workers discuss their skills and alternative ways of carrying out tasks safely, if there is access to resources whenever unexpected events occur, and if the information is shared in a timely manner and communicated to those in the position to influence change [10].

The examination of organisational learning considered issues such as planning; the basis for learning; learning style; categories of learning; responsibility; delays in learning processes; adequate resources for effective learning and implementation of a ‘lessons learnt’ perspective. The survey questions for this aspect are shown in

Table 2. Potential to Learn survey questions.

Potential to Learn
6. It is clearly established what should be reported
7. Submitted safety reports are being investigated sufficiently
8. There are good responses on submitted safety reports in an acceptable manner
9. Workers are being motivated and given the time to write reports
10. Lessons are learned from the things that go right
11. Lessons are learned from the things that go wrong
12. We meet with other workers to learn from each other.

[10].

Examining the aspect of monitoring involves considering performance indicators and their validity, delay, sensitivity, frequency, interpretability and organisational support systems to communicate key information on operational performance and the survey questions are shown in

Table 3. Potential to Monitor survey questions.

Potential to Monitor
13. The safety performance indicators are consistent with the organisation
14. Indicators are revised regularly and properly
15. The organisation uses lead indicators
16. The organisation uses lagging indicators
17. The leading indicators are valid
18. The period covered by lagging indicators is appropriate
19. The measurements type (quantitative or qualitative) and frequency is appropriate
20. The delay between the measurement and analysis of results is acceptable

[10].

Lastly, the issues relating to the potential to anticipate become relational to corporate culture; the acceptance of uncertainty; time horizons; frequency of assessment of future threats and opportunities; modelling the future; clear strategic vision; and expertise to look to the future and communication of future unknowns throughout the organisation and the survey questions shown in

Table 4. Potential to Anticipate survey questions.

Potential to Anticipate
21. Systems are in place to look into the future at potential safety and organisational related weaknesses and threats
22. The systems or people who make these forecasts have sufficient expertise, capability and resources to make accurate and relevant forecasts
23. The time scale and frequency of these forecasts are acceptable
24. The criteria used to determine the scope and depth of the forecasted analysis is suitable
25. Are systems in place to ensure it is easy, straightforward and welcoming for staff to raise any issue related to potential or anticipated safety and organisational related weaknesses and threats?
26. Are systems in place to ensure that forecast information is communicated to relevant parts of the organization?
27. Systems are in place to develop and maintain staff skills and competencies to adequately anticipate future safety and organisational weaknesses and threats
28. Are systems in place (where appropriate) to ensure control measures are developed and implemented to address issues raised in forecast analysis?
29. Are staff adequately consulted in the development and implementation process of control measures?

.

The questionnaire was limited to a simple Yes, No or Unknown participant answer for each question. The rationale behind using this closed question set was that middle managers would be forced to consider a ‘yes’ being ‘this is in place and happens’, a ‘no’ indicating ‘this aspect is not in place and/or does not happen’, or ‘unknown’, meaning that ‘they lack information on this organisational practice’. The simple three-question response enables gaps to be easily identified in the organisation’s resilience endeavours and allows a very simple matrix being utilised to assist in easy development of the RAG survey output (spider) radar graphs. The relationships between the four potentials were examined as a percentage of answers across the three responses in order to draw comparisons between the four potentials.

The examination of participant responses enabled the results to be viewed using radar graphs and from the 26 surveys distributed, 15 completed surveys were returned from middle managers resulting in a 57.7% response rate.

6. Results

In relation to the participant’s answers to the organisation’s potential to ‘Respond’, the questions set examined five areas across headings listed as procedures, skills, alternative tasks, resources and information as suggested by RAG theory [10].

6.1. Potential to Respond

Figure 2 shows the radar graph for the ‘yes’ responses to the potential to respond.

A high number of respondents answered Yes to the questions about responding with the question around alternative tasks—‘workers discuss about alternative ways of carrying out tasks safety’ scoring the highest (n = 14). The question ‘there is access to a variety of resources whenever the unexpected events occur’, on the topic of resources, scored the lowest number of yes responses but still represented a quite a high score (n = 9). The radar graph demonstrates that there was a fairly minimum variance across the 5 questions and that overall, this potential scored well.

6.2. Potential to Learn

Higher scores were achieved for the Yes responses for the organisation’s potential to Learn as shown in Figure 3. The 7 survey questions focused on the organisation’s current systems around reporting, investigations, responses, reports, learning lessons of what goes right (Safety II) and what goes wrong (Safety I) and the communication of those lessons across the organisation. Figure 3 shows the radar graph for the resilience potential of learning.

The potential to Learn results showed that the question concerning motivation and time to write reports, ‘workers are being motivated and given time to write reports’ had the lowest Yes response score (n = 9) with the worker’s response suggesting that they may have insufficient time or motivation to complete formal written reports required by the organisation. The question ‘lessons are learned from things that go wrong’ (Safety I) scored the highest number of Yes responses (n = 15) however this was closely followed by the question ‘lessons are learned from things that go right’ (Safety II) (n = 13) with both question responses demonstrating a positive aspect for learning from lessons being conducted within the organisation.

6.3. Potential to Monitor

The potential to Monitor ‘Yes’ results shown in Figure 4 were not as well defined. A larger number of responses for monitoring were from ‘Unknown’ response, indicating that workers may lack information or knowledge as to what the organisation does to monitor the various parts of its operations.

A high percentage (80%) of the participants responded Unknown to the question ‘the period covered by lagging indicators is appropriate’ with only a few Yes responses being given (n = 5). The highest number of Yes responses were equally rated to the safety indicator question ‘the safety performance indicators are consistent within the organisation’ (n = 9) and the indicator revised question ‘indicators are revised regularly and properly’ (n = 9). The lowest score (n = 3) was related to the period of reporting against the question ‘The period covered by lagging indicators is appropriate’.

Overall, the potential to Monitor results indicate that this potential may be the main one where more work is required to instill aspects of continuous monitoring to enhance resilience potentials here within the organisation.

6.4. Potential to Anticipate

The last resilience potential on which participants reported was the potential to Anticipate and radar graph results are displayed in Figure 5.

Two questions scored the highest for Yes responses around issues being able to be raised, ‘systems are in place to ensure it is easy, straightforward and welcoming for staff to raise any issues related to potential or anticipated safety and organisational related weaknesses and threats’ (n = 12) and staff development and skills via the question ‘systems are in place to develop and maintain staff skills and competencies to adequately anticipate future safety and organisational weaknesses and threats’ (n = 12).

6.5. Overall Organisational Resilience Potential

Results were also collated as a percentage of survey questions answered either Yes, No or Unknown and grouped across the four Response, Learn, Monitor and Anticipate (RLMA) resilience potentials to enable a graphic to be displayed as more holistic big picture radar graph. Figure 6 shows the Yes responses for all four RLMA resilience potentials.

The potentials to Learn and Respond have the highest Yes responses (83.3% and 77% respectively) which may indicate that the organisation has these two potentials better managed than the other two potentials, being the potential to Anticipate and Monitor (52.9% and 41.3% respectively) which scored lower. In Figure 7, the RLMA potentials are shown for the No responses across the four potentials.

In this instance information displayed in Figure 7 provides some meaningful information on the No response for the potential to Respond and Learn and is directly correlated to the high Yes answers on these potentials, rather than any Unknown response. The potential to Monitor and Anticipate, however, give a more telling view on the unknown RLMA potentials as shown in Figure 8.

Figure 8 shows the potential to Monitor is the highest potential where participants reported not knowing about these organisational systems (57.8%) followed by the potential to Anticipate (40.3%). Two issues may be in play here with the questions for these potentials either not being well understood by the respondents or the respondents simply ‘do not know’ if the organisation does these activities or actions. As participants from this study were predominately from middle management, which represents the information transmitter in an organisation [36], firstly, the potential to Monitor and, secondly, the potential to Anticipate should be examined further to assess why knowledge about these potentials is lacking. Efforts would need to be directed to increase the knowledge in these areas with a view that future surveys would want to see answers moving towards the ‘Yes’ spectrum to increase the potential for resilience here.

7. Discussion

The results of the participant survey were, firstly, presented showing the ‘Yes’ responses to each of the questions under the four RLMA resilience potentials. Radar graphs for each set of questions were presented which showed the number of Yes responses. The radar graphs enabled areas of strength and weakness to be understood to assist the organisation in focusing efforts to increase their resilience potentials where responses where were lowest. A radar graph was then presented showing the RLMA potentials as a big picture for the organisation on Yes responses and demonstrated that responding and learning actions respectively scored the highest followed by anticipating and monitoring actions (in order of potential highest to lowest this would be RLAM). Hollnagel [10] cautions against merely focusing on all four potentials without a clear strategic plan and suggests that organisations should look at building the four potentials relative to their organisational context and functions, whilst keeping a balance between the four. Consideration also needs to be given to complex systems theory which understands that actions in one area will have a flow-on effect into other areas and the wider organisation as a whole.

Hollnagel suggests that organisations traditionally always put efforts into responding; many put efforts into learning (although very stereotypical); fewer organisations put efforts into monitoring and very few focus on serious efforts to anticipate (ordered as RLMA) [10]. He further suggests that, whilst logically the sequence that could be followed to make organisational resilience performance better is to increase Response, Learning, Monitoring and Anticipation in that order, the balance must be based on knowledge of what the organisation does and needs to do in order to maintain functioning.

The responses on the Monitoring and Anticipation potentials scored highest in the Unknown area. Participants did not report that the organisation did not do these actions (no response) but rather that they did not know the answer to the questions when asked. This may demonstrate that knowledge of efforts put into these areas is not widely known to the participants and organisational efforts could initially be directed at disseminating knowledge of how the organisation is doing its anticipation and monitoring actions before changing what it is doing.

In examining the individual potentials, the responses around the potential to Respond provided a clear picture about two questions in this section around alternative tasks and skills. The responses to this potential provided several definitive answers (Yes or No) which indicated an understanding of what is working and what is not. The information shows good scores overall and there is enough information from the survey and radar graph output to enable the organisation to target future areas for development. In particular, this would be around the access to resources whenever an unexpected event occurs.

The potential to Learn scored the highest overall percentage of Yes answers. It can be considered again that the responses were strong on the Yes response particularly around learning lessons and reporting; however, having the time and motivation to write reports could be enhanced, as these then feed directly into the lessons learned actions.

The potential to Monitor results were not as strong with respect to the yes responses and draws attention to the Unknown responses which were higher. A higher number of responses returned for the Unknown may indicate either the survey questions were not well understood or that the participants (middle managers and supervisors) do not know how monitoring is done and given their position in the organisation this is an area in need of improvement which may require more educational rather than actual organisational change. The possible need for improvement may be in the form of education of what is meant by the questions, understanding of what the organisation may be doing within this potential, or the development of indicators within this potential that they do not already have.

The results for the potential to Anticipate are much more mixed than the previous three potentials. Several questions display high counts in the Unknown response while most of the other areas display a definitive Yes but some No’s. The results provide areas of future improvement similar to the previous three potentials indicating room for organisational involvement and improvement.

Overall, the production of the radar graphs appears to provide the organisation with a simple means of measuring its current position against the four resilience potentials concepts. The organisation is now in a position to develop some strategic endeavours to increase these potentials and at a future time, conduct further surveys as a pre- and post-intervention measuring system. The organisation’s four potentials have been measured as a starting point in time and the output results appear to provide meaningful information which can be used towards strategic safety system enhancement endeavours.

Acknowledged limitations to the study include the low sample size of one workgroup being middle managers. As a pilot program, this research chose to survey middle managers as they are seen as the information conduit within the organisation and a bridge between top administrators and front-line workers. However, future research should consider surveying all three organisational hierarchies including top administrators, middle managers and front-line workers to obtain a better helicopter view of the resilience potentials across the organisation. Furthermore, the question set developed was tailored for the organisation under review and future survey questions should be tailored to ensure relevance to the industry and organisation under study.

8. Conclusions

The RAG enabled measurement of the four resilience potentials of respond, learn, monitor and anticipate. These potentials form the basis of an organisation’s potential to be resilient however the minimalistic impression provide by these four single words hides the expansive work required in building resilience. Overall, the RAG proved a useful method with which to measure the assessment of resilience of an organisation. The research presented here, as a small pilot program, allowed the organisation to obtain and view a current snapshot or starting point for enhancement work, mapping the strengths and weaknesses of the organisation’s resilience potentials to enable strategic planning to be conducted.

This research has shown it is possible to develop a survey to measure an organisation’s potential for resilience as a snapshot at a point in time. It answered, in the positive, the research question which asked if the measurement of the four resilience potentials, as suggested by RAG theory, could give meaningful information about the organisation’s current position as to its resilience potentials.

Meaningful information was obtained by the use of a middle-management survey and provided direct information on each of the four potentials separately and then as a more holistic big picture when all four potentials were considered together. This information has allowed the organisation to pinpoint areas for improvement and set future directive actions which can now be taken in order to enhance the four resilience potentials.

The use of the RAG as proposed by resilience engineering theory has allowed the development of a survey to measure an organisation’s potential for resilience enabling meaningful information about the organisation’s current resilience potential to be shown and, finally, the results from the survey provide the organisation with enough knowledge to further develop their resilience potentials by examining both what is working well from a Safety-II perspective and pointing towards areas in need of improvement.