Evaluation of Occupational Health and Safety Key Performance Indicators Using in Healthcare Sector

Abstract

Keywords: Analytic hierarchy process (AHP); key performance indicators (KPI); multi criteria decision making (MCDM); occupational health and safety (OHS); safety performance measurement.

1. Introduction

Recent improvements and regulations in the field of occupational health and safety have helped to improve working conditions in Turkey. "Occupational Health and Safety Law", which has been valid since June 30, 2012, filled the gaps in this area to a great extent. Measuring the impact of these improvements and regulations on OHS performance is also a major subject. The management of a system depends on the measurement of the system. Unmeasured and unanalyzed systems cannot be improved. Therefore, measuring the performance of the OHS in *

Corresponding Author: e-mail: ykorkusuz@hotmail.com, tel: (216) 280 33 33 / 4057

workplaces is a seri ous topic. In order to protect employees' health and ensure workplace safety, the workplace's OSH performance should be measured at regular intervals by using criteria that best reflect OSH performance. When a company wants to measure the annual change in its OHS performance, or when it wants to learn about the OHS performance of another company to be sure that the other company is trustful on OHS manner, traditionally a limited number of reactive indicators are usually investigated, such as accidents rate, number of experts etc. The information obtained using a few statistics does not reflect actual performance. There is no commonly accepted OHS performance measurement method which can be simply used. In this study, the OHS performance measurement method is presented for closing this gap. Multi-criteria decision-making methods help the decision maker to make the best decision when there are ambiguous, complex, and conflicting goals [1]. One of the most widely used multi-criteria decision making methods is the Analytic Hierarchy Process (AHP). In multi-criteria situations, the AHP elicits the effect of each criterion on the desired goal. In the second part of the article, a literature search was given. Then, the problem is defined, which is followed by the indicators for performance measurement. After that, the AHP method is given and the application is explained. And last part consists of application results and analyze of the work.

2. Literature Review

The AHP method has been used in literature to solve many problems. Özdemir et al. [1] used the AHP method to evaluate the safety risk assessment indicators in nuclear power plants and rank them according to their priorities. Göktolga et al. [2], regarded job choice and alternative job preferences as multi-criteria decision making problem and used AHP method to solve this problem. Examples of use of the AHP method in the field of OHS are also available. Janackovic [3] used the "Fuzzy AHP" method to select and rank OHS indicators. In this study, OHS performance criteria used for road construction studies were determined and prioritized. However, since the OHS indicators used here are not all measurable, there is a danger that the performance measured by this method is subjective. Similarly, in the Law's article [4], 13 OHS performance KPIs have been identified for manufacturing enterprises and these KPIs have been prioritized by AHP method. But here is also KPIs are scored by the expert assessment, they are not measurable quantities. Amir-Heidari [5] identified 20 KPIs for the drilling sector in Iran, separated them into two groups as leading indicators and the delayed indicators, and weighted each group using the AHP method. After that, KPI values are given “0” or “1” by comparing them with minimum acceptable level of each KPI. However, minimum acceptable level does not measure how good the KPI value is. Aminbakhsh [6] identified 9 of the most common risks in the construction field and prioritized these risks using the AHP method. But here, only the order of importance of the risks is made, the order of KPIs is not done. Gürcanlı [7] prioritized the list of things to be done in the construction sector, such as those observed in the field of work, workers wearing appropriate clothing, correcting the markings. Experts scored a scale of 0-10 indicating that these items were fulfilled based on observations. Chang [8] has defined sub-criteria such as "high level commitment to OHS", "level of organization and responsibility" for each element of the PDCA cycle. These sub-criteria were then weighted according to the degree of importance by using AHP and 4 different firms are compared by 4 experts through observation. Podgorski [9] aimed to measure the performance of the occupational health and safety management system in his work. For this reason, SMART (ie Specific, Measurable, Achievable,

Relevant and Time-bound) criteria are applied to the KPI’s. 5 features in SMART criteria is weighted by using AHP method and according to this result, t he safety KPI’s were chosen. Özdemir [10] has identified 60 different criteria for determining occupational health and safety conditions in three different workplaces of an enterprise operating in the textile sector and has analyzed the importance of each of these criteria by using the Analytical Network Process method. Kısa [11] has selected 18 KPI’s and used Analytical Hierarchy Processes method to wieght factors affecting occupational health and safety performance of metal workplaces, those are classified as very dangerous workplace by ministry, and they are valued by experts’ opinions.

3. Problem Definition

Measuring occupational health and safety performance has great importance for companies. The most important step to make this measurement is the selection of performance indicators and the weighting of these indicators with the correct values. If the KPIs cannot adequately represent performance, this leads to unrealistic results. Similarly, if KPIs are not properly weighted or prioritized, then performance is again measured incorrectly. In this study, firstly, KPIs were determined and then weighted by AHP method which is a common multi-criteria decision making method. When applying the AHP method, "Superdecisions v2.6" program, which is often used in multi-criteria decision making problems, was utilized. The screenshot of the problem in "Superdecisions v2.6" is shown in Figure 1.

4. Occupational Health And Safety KEY Performance Indicators

Several indicators are presented in the literature to measure occupational health and safety. The most frequently used indicators are reactive (lagging) indicators that measure past events (number of work accidents, number of lost days, etc.). However, proactive (leading)

indicators such as periodic preventive work (periodic corrective action, given trainings) are also important for measuring OHS performance. In addition, some of the indicators are tangible, measurable (such as the number of controls), while others are more abstract (employee and manager's attitude toward the accident etc.) and are difficult to measure. In this study, the following 3 features were used for the selection of KPIs;

3. KPIs should reflect OHS performance in the most realistic way

In general, two common health and safety indicators for accident statistics can be used; frequency rate and incidence rate. The frequency rate shows how many events occurred over a certain period of time based on a standard number of “working hours”. The incidence rate is the number of events that occurred over a certain period of time with a standard “number of employees” (which is less than working hours). After reviewing literature and consulting with 3 certified occupational health and safety expert, KPIs below are decided to use in this study. 1. Accident Frequency Rate: It refers the number of accidents occurred in 1 year, relative to 1,000,000 working hours [12].

2. Fatal Accident Frequency Rate: It refers to the number of workers died in 1 year,

relative to 1,000,000 working hours [12]. 3. Injured Accident Frequency Rate: It refers the number of injuries occurred in 1 year, relative to 1,000,000 working hours [13]. 4. Lost Time Injury Frequency Rate: It refers the number of lost time injuries in 1 year, relative to 1,000,000 working hours [14].

6. Observed Unsafe Behavior: It refers dividing the number of observed unsafe behaviors

in a calendar year over the number of employees (non-use rate of PPE etc.) [16].

7. The Proportion Of Workers In The High-Risk Group: It refers dividing the number of

employees in the high risk group (Radiation, microbiological hazards, etc.) over the total number of employees [3].

8. Percentage of Remaining Risks After Risk Analysis: It refers dividing the number of

remaining risks after the risk analysis corrections over the total number of risks in the risk analysis [5].

9. Ratio of Remaining Risks With High Importance: It refers dividing the number of

remaining risks with high importance score over the number of remaining risks in the risk analysis after risk analysis corrections[17].

10. Proportion Of Trained Workers: It refers dividing the number of trained workers

(OHS, emergency situations, occupational risks, first aid etc.) over the number of workers required to be trained by legislation [16]. 11. Workspace Controls: It refers dividing the number of inspections carried out in the work place in 1 calendar year over the total number of employees [16]. 12. Observed Unsafe Conditions: It refers dividing the number of average unsafe conditions observed in the work place in 1 calendar year over the total number of employees [16]. 13. Ratio of Periodic Checks: It refers dividing the number of periodic checks (health tests, equipment tests, physical risk factors measurements) over the number of checks required by the legislation [18, 9]. 14. Periodic Corrective Actions: It refers dividing the number of activities carried out in one calendar year in order to eliminate the cause of a determined nonconformity or other undesirable situation over the total number of employees [19].

15. Periodic Preventive Activities: It refers dividing the number of activities carried out in one calendar year for the elimination of a potential non-conformity over the number of employees [9].

5. Analytic Hierarchy Process (AHP)

The Analytic Hierarchy Process (AHP) is a multi-criteria decision-making method that has been widely used since 1970s. It separates a problem into smaller pieces and examines the effect of these parts on each other. As a result of this process, the weight of parts and the importance order of parts are obtained. For this purpose, a benchmark scale was established that quantitatively assesses the effects of parts on each other. Parts of the problem are compared pair wise and effects of each part on the target are quantitatively obtained. The AHP method can be used in both social and physical areas to make measurement [20]. Steps of AHP are given below;

1. Identification of the problem and determination of the desired information,

2. Formation of the hierarchy of decision-making from top to bottom determination of the goal and criteria,

4. Finding weights of criteria.

There is a need for a scale to make comparisons. This scale shows how important an element is compared to the other element. The scale used in AHP can be seen in Table I [21]. Table 1. AHP Scale Importance Value Definitions Values 1 Both factors are equally important

Factor 1 is slightly more important than Experience and judgment shows that Factor 1 Factor 2 is slightly more important than the other.

Factor 1 is more important than Factor 2 Factor 1 is strongly more important than Factor 2

Experience and judgment shows that Factor 1 is more important than the other. Experience and judgment shows that Factor 1 is strongly more important than Factor 2.

Experience and judgment shows that Factor 1 is absolutely more important than the other.

The mathematical realization of AHP will be explained in the following steps [21]: 1. First, the problem and elements (criteria) to be decided are defined. Using these elements, a comparison matrix is constructed. The comparison matrix for "n" elements contains "nxn" elements and the values on the diagonal (where i = j) are 1.

 k11 k12 ... k1n  k   21 k 22 ... k 2 n   . .  K   .   .  . .     k n1 k n 2 ... k nn 

In the comparison matrix there is such relation, between the elements above the diagonal and the elements below the diagonal;

For example, if the third criterion more important than the second criterion, the value of element

2. This matrix shows us the importance of each criterion, but does not allow us to see the weight of each criterion in total. It is needed to get the column vectors for this. Each element is divided by the sum of the values in its column, and if the value is substituted, n column vectors of n elements are obtained.

The above formula is used when the values of the column vector are calculated.

 s11  s   21   .  Si     .   .     s n1 

3. To create column matrix, n column vectors are formed in a matrix. This matrix is as follows

s12 ... s1n  s 22 ... s 2 n  .   .  .   s n 2 ... s nn 

4. Finally, using the S matrix, it is needed to obtain the weight vector to obtain the percentage of the elements. This is obtained by taking the arithmetic mean of the elements in the rows of the column matrix. n

The sum of the elements of the weight vector is 1. The weight vector is as follows;

Consistency analysis is used to measure the consistency after weight results are found. This analysis shows whether there is an error in the work done or the result is consistent within itself. The following steps are taken to calculate the consistency rate [20, 21]. 1. In order to calculate the consistency ratio, firstly the comparison matrix and the weight matrix are multiplied to obtain the T column vector.

 k11 k12 ... k1n   a1   t1  k       21 k 22 ... k 2 n  a2  t 2   . .   .  . T   x     .   .  .  .  . .   .  .       k n1 k n 2 ... k nn  an  t n 

2. After obtaining the T vector, basic value elements are obtained by dividing each element of the T vector by the weight vector A of the T vector.

3. The arithmetic mean of these elements gives the basic value of pair wise comparison of this problem . n

5. The following formula is used to calculate the consistency ratio at the last step.

Here, the RI value is a standard correction indicator called a random indicator, and Table II is used depending on the number of elements compared. Table 2. Random Indicator values [21] N

The result is consistent if the consistency ratio (CR) is less than 0.1. If it exceeds 0.1, either there is a mistake in applying the AHP, or the operation is inconsistent [8].

6. Application

In the above sections, occupational health and safety performance KPIs and the implementation steps of AHP method were given. In the application, the AHP method was used to find the KPI weights. A pair wise comparison of 15 KPIs survey is conducted with the help of 3 certified health and safety specialists working in the healthcare sector. When a pair wise comparison was made, AHP scale "9-7-5-3-1-3-5-7-9" was used, which is described in the AHP. Experts rated KPIs according to the severity of importance. The survey results were analyzed with the help of "Superdecisions v2.6" program. A screenshot of the survey in the "Superdecisions v2.6" can be seen in figure 2.

7. Results And Evaluation

Survey results are given in two different scales; "normalized" and "idealized". Normalized scale indicates the criteria as a percentage, and sums of all criterion is equal to 1. Idealized scale accepts the most important criterion value as "1" and the other values were scaled accordingly. Table III shows KPIs sorted according to their priorities. According to healthcare sector experts, the most important OHS performance measurement KPI is the work accident death rate, which is a reactive indicator. However, 4 other KPIs in the first 5 elements in the list is proactive indicators which shows the significance of proactive indicators. In this study, measurable, number of worker-independent and realistic KPIs are prioritized to measure health and safety performance. These KPIs are described and ranked by occupational health and safety experts in order of importance. In this respect, according to experts of the healthcare sector, the affect of each factor in the performance is obtained. This method provides healthcare sector companies with detecting annual changes in their occupational health and safety performance. Moreover, they can compare their performance with other firms in the same sector. The main contracting firms that will be selecting subcontractors may measure the performance of the subcontractors' OHS performance by using this method. To authors’ knowledge, this is the first study which focuses on measurable safety KPIs and first study which prioritizes KPIs of occupational health and safety in healthcare sector by using AHP method. The method itself is widely used but it was applied to this topic for the first time. For future work, similar study can be conducted by using different multi criteria decision making techniques and KPIs for different sectors can be investigated.

Share and Cite