Risk Assessment and Key Performance Indicator Continuum

I want to continue the discussion related to the relationship between risk assessment and key performance indicators. I have posted about this relationship and other assorted concepts and ideas related to it in several previous blog posts I posted earlier this year. In this post I would like to see if I can tie some of these ideas and concepts together and show how risk and performance are more closely related and how to take advantage of this relationship.

These ideas percolated from a conversation and discussions I have been having with a colleague about a webinar we will be doing together where he suggested the use of a graphic to help to explain the essence of key performance indicators. His graphic was to be an airplane cockpit and all the gauges present on the dashboard that a pilot is looking at. A great deal of data and information to process but s/he focused on about 5-6 gauges that were the most important in flying the plane and really told the pilot if things were ok or not and when s/he needed to check the other gauges because these key performance indicator and risk assessment gauges were telling s/he something was not quite right. I would guess that two of these gauges were the altimeter and speed gauges which I would include as risk assessment gauges and a third gauge would have been the fuel gauge which I would include as a key performance indicator.

Why did I break these gauges down into the two major areas of risk assessment and key performance? Here is my thinking: the altimeter tells the pilot how close to the ground and a potential crash and the speed helps to prevent a stall of the aircraft. Both are high risk factors and things we would want to mitigate. The fuel tank is important to know how much fuel the pilot has left; in, and of itself, not necessarily a risk factor unless it becomes too low but will impact performance because it determines how far the pilot can fly the plane.

A similar scenario could be played out with driving a car. Speed is the risk factor as it increases, while the gas tank gauge is the key performance indicator determining how far we can go and how much we are getting per gallon of gas which is an indicator on many newer models.

Let’s try this out in a totally different industry and scenario, such as the pharmaceutical/drug industry. When finding out if a new drug will work or not, there is a delicate balance of risk-benefit or risk-performance. Same concept, just different terminology being used. For risk assessment, either not taking the drug or taking too much of the drug will not be in the best interest of the patient. Too little or not at all the patient dies because the disease progresses. If the patient takes too much of the drug, given the side effects, the patient dies. The key performance indicator or benefit is finding the right target dosage of the drug which effectively keeps the patient alive and gets better or at least not any worse.

Another example, one that I share somewhat reluctantly because some people may take offense but I think it is an effective example, the Ten Commandments. I actually have posted this earlier in a blog post as an example if one is interested in looking at this in more detail (May 2022). With the Ten Commandments, think of “Thou Shalt not Kill” as a risk assessment rule and “Thou Shall not Steal” as a key performance indicator. Obviously the consequences of the first are much greater than the second where one is literally stealing someone’s life, which is the underlying structure of the relationship between risk assessment and key performance indicators.

So let’s delve into this relationship of performance and risk mitigation based upon the above examples and see how they are all tied together. Risk mitigation (Do No Harm) is sort of the book ends of the relationship, too much or too little is not a good thing, while key performance (Do Good) is somewhere in between balancing effectiveness with efficiency and finding the right balance of rules and recommended standards (The essence of the Theory of Regulatory Compliance). Remember I am addressing regulatory compliance data and not social science data in general although it would be interesting to see how this relationship of performance and risk assessment plays out in the larger context of the social sciences. I have a funny feeling that many relationships of social science variables are more nonlinear than linear in nature.

How are risk assessment and key performance indicators determined? Risk assessment rules are generally determined by expert opinion and group consensus either using or not using a Likert type Scale (Stepping Stones to Caring for Our Children and Caring for Our Children Basics are examples). Key performance indicators are determined from actual data, generally regulatory compliance history utilizing a regulatory compliance statistical methodology that results in the rule’s predictive ability (the statistical methodology is highlighted on this website in the publications section as well as on the National Association for Regulatory Administration’s (NARA) website https://www.naralicensing.org/key-indicators)(ASPE’s Thirteen Quality Indicators and the Early Childhood Program Quality Indicators Scale are examples (see previous blog posts on all these)). From a licensing measurement perspective, risk assessment rules are generally always in regulatory compliance because the rules place clients at such great risk; while key performance indicators do not place clients at high risk as with risk assessment rules, generally have some non-compliance, just enough to distinguish between the high performers and the mediocre performers.

This relationship is made possible because of the regulatory compliance theory of diminishing returns/the ceiling effect between regulatory compliance and program quality where we are really forced to look for a paradigm shift when it comes to licensing and program monitoring. The “One Size Fits All” a very absolute approach needs to be replaced with a more relative approach, such as “Differential Monitoring” and once this paradigm shift is made it naturally leads us to identifying risk assessment rules and key performance indicator rules. It really changes our frame of reference in establishing a proper balance between regulatory compliance and program quality standards.

To summarize, too few or too many rules are not a good outcome, it is finding the proper balance of the “right rules”, finding that balance between effectiveness and efficiency, between risk mitigation and optimun performance. Let me leave you with this statement as an algorithm where TRC = Theory of Regulatory Compliance; RA = Risk Assessment; KI = Key Performance Indicator; RC = Regulatory Compliance; and PQ = Program Quality:

TRC = RA + KI => RC + PQ