Four Approaches to Program Monitoring related to Regulatory Compliance and Program Quality Performance in the Human Services

Over the past 40 years, program monitoring has evolved substantially in the human services related to regulatory compliance and program quality performance measures (rules/regulations/standards). In this post, four approaches will be discussed in this evolution: instrument-based, inferential/differential, integrative, and coordinated. 

Instrument-based program monitoring (IPM) is based upon tools, instruments, and/or checklists. It is quantitative in nature where reliability in the collection of data is increased when a data collection protocol is used along with the respective set of rules/regulations/standards. IPM appeared in the 1980’s and replaced more of a qualitative, anecdotal, clinical case perspective. Human service agencies, in particular, early care and education programs were growing at a tremendous rate and where the case note approach worked well when there were not many facilities, as these facilities increased it became more difficult to keep up with demand and to utilize the data in making comparisons at the macro level. The case note approach is very effective as a micro, point in time measure; but it is not as effective with a large amount of data measured over time where comparisons need to be made at a macro level. 

The IPM approach during the 1980s led to the development of more streamlined and abbreviated methodologies utilizing risk assessment and key predictor indicators as jurisdictions looked for more cost effective and efficient methods. These methodologies ushered in inferential/differential program monitoring in the 1990s in which abbreviated inspections were done with facilities that demonstrated a history of high compliance with rules/regulations/standards. These methodologies and the inferential/differential approach were endorsed by the National Association for Regulatory Administration (NARA) which helped to disseminate and promulgate them. NARA and the Research Institute for Key Indicators (RIKI)(the original developer of the key indicator & risk assessment methodologies and the differential monitoring approach) entered into an exclusive partnership in 2015 for the future development and dissemination of differential monitoring which had taken on increased significance because of its inclusion in the re-authorization of the Child Care and Development Block Grant (CCDBG) legislation in 2014.  Differential monitoring has been highlighted in several federal/national publications.

The key indicator methodology eventually led to the development of quality key indicators in addition to licensing key indicators and with this new development, it ushered in a more integrative program monitoring approach that demonstrated a more balanced monitoring of both regulatory compliance and quality services. This initiative really started with the introduction of Quality Rating and Improvement Systems (QRIS) at the turn of the century but it really got moving as key indicators were being identified in both licensing and quality in the 2020s. Many states and jurisdictions are interested in the approach although it still has a long way to go for full implementation.  IPM and differential monitoring approaches are the dominant program monitoring systems being utilized by most jurisdictions at this point.

Another monitoring approach developed alongside Integrative Program Monitoring is called Coordinated Monitoring. This approach emphasized the need to better coordinate monitoring efforts across the various regulatory and quality initiatives that were springing up in many jurisdictions. This emphasis was very evident at the federal level where the problems of coordination across program areas was most evident. 

This post provides a brief introduction into how human service program monitoring has changed over the past four decades. For those who may be interested in exploring this in greater depth, the following ehandbook should provide additional guidance: Licensing Measurement and Monitoring Systems, available on the NARA website: https://www.naralicensing.org/key-indicators