The Importance of the Theory of Regulatory Compliance as it relates to Licensing Measurement and Monitoring Systems

This RIKINotes Post will provide the latest thinking and research related to the Regulatory Compliance Theory of Diminishing Returns and how it influences licensing measurement and monitoring systems in the human services, in particular early care and education. Some information has appeared in previous posts over the past couple of years but this post will consolidate these findings with the most recent findings related to the theory.

The theory of regulatory compliance has had a tremendous impact on human services licensing measurement and monitoring systems when taken to its logical conclusion which is that there is no significant difference in the level of quality in programs in substantial versus full compliance with a given set of early care and education rules. However, the theory does provide support for the ability to distinguish levels of program quality in low regulatory compliance performers and those in substantial regulatory compliance. There is now empirical evidence from 5 rather large studies conducted across the USA and Canada both within states and provinces as well at the national level in the USA.

From a public policy point of view, the theory opens up a new way of thinking about how best to monitor which is addressed in the next paragraph by moving from a “one size fits all” to one that is more targeted to the regulatory compliance needs of the provider of services. An approach that focuses on those programs that are struggling to meet all rules in providing them with additional resources and guidance while at the same time doing abbreviated reviews of the top performers and getting out of their way because they have a history of high regulatory compliance with all rules. The theory provides a better balance of “do no harm” and “do good” by infusing quality into rules and by mitigating risk to children while enhancing their program’s performance.

Because of this above relationship between program quality and regulatory compliance, it ushered in differential monitoring, an abbreviated form of program monitoring which led to the risk assessment rule and key indicator rule methodologies. The precursor to differential monitoring and providing the methodology to conduct the regulatory compliance studies was instrument based program monitoring.

A by-product of the studies conducted regarding the theory of regulatory compliance made clear that frequency counts (nominal measurement is a real limitation of the data) were not effective without a weighting component which ushered in the concept of a regulatory compliance scale which placed regulatory compliance into buckets of full, substantial, mediocre, and very low regulatory compliance. This ordinal measurement technique is much more effective than having straight frequency counts of violations and is more consistent with licensing theory in which all rules are not created nor administered equally. There is a need to weigh individual rules in order to take this effect into account. The next logical step for a regulatory compliance scale is to apply it to individual rules and not just to the final aggregated regulatory compliance score. There is also the need to build in an exponential component to the weighting protocol in order to increase the variance in the data and increase our ability to distinguish differences in scoring.

With the introduction of utilizing substantial compliance as an equivalent positive regulatory compliance outcome as full regulatory compliance, a potential analytical problem was created with introducing additional false negatives in making licensing decisions in which regulatory compliance was recorded when in reality other areas of non-compliance were present. This was mitigated by a revision to the 2 x 2 Validation Matrix by cubing (^3) the false negative cell in order to essentially eliminate any rule that had any significant false negative values  (FC* = ((A)(D)) – ((B^3)(C)) / sqrt (WXYZ).  Full regulatory compliance should be able to be used in the majority of cases (the standard 2 x 2 Validation Matrix can be utilized)(FC = ((A)(D)) – ((B)(C)) / sqrt (WXYZ) because of the highly skewed data distribution with very little variance (data dichotomization is warranted in this special case); but in those cases in which substantial compliance comes into play, then the 2 x 2 Validation Matrix revision needs to be used.

The last development is the introduction of a 2 x 2 matrix showing how to combine the use of differential monitoring (DM) and integrated monitoring (IM) into a blended approach to program monitoring (this proposed matrix is highlighted in a previous post earlier this month (September 14th)–the DM x IM Matrix). The ultimate goal is the delicate balancing of regulatory compliance and program quality in improving facilities. This should be done in the most effective and efficient way. By combining differential monitoring (efficiency) with integrated monitoring (effectiveness) it may be possible to reach this blended approach to program monitoring.