What is the Relationship between Regulatory Compliance and Complaints

Richard Fiene, Ph.D.

January 2019

Within licensing measurement and the validation of licensing systems it is particularly difficult to have specific outcome metrics that can be measured within a human services licensing system. The purpose of this technical research note is to propose a potential solution to this problem.

Probably the most accurate measures of licensing outcomes focus on improvements in the health and safety of clients within human services licensed facilities, such as: fewer injuries (safety) or higher levels of immunizations (health). Another measure related to client satisfaction is the number of complaints reported about a licensed facility by clients and the general public. The advantage of using complaints is that this form of monitoring is generally always part of an overall licensing system. In other words, the state/provincial licensing agency is already collecting these data. It is just a matter of utilizing these data in comparing the number of complaints to overall regulatory compliance.

The author had the opportunity to have access to these data, complaint and regulatory
compliance data in a mid-Western state which will be reported within this technical research note. There are few empirical demonstrations of this relationship within the licensing research literature. The following results are based upon a very large sample of family child care homes (N = 2000+) over a full year of licensing reviews.

The results of comparing the number of complaints and the respective regulatory compliance levels for specific family child care homes proved to show a rather significant relationship (r = .47; p < .0001). This result is the first step in attempting to understand this relationship as well as developing a methodology and analysis schema since directionality (e.g., did the complaint occur before or after regulatory compliance data collection?) can play a key role in the relationship (this will be developed more fully in a future technical research note). The focus of this research note was to determine if any relationship existed between regulatory compliance and complaint data and if it is worth pursuing.

It appears that looking more closely at the relationship between complaint and regulatory compliance data is warranted. It may provide another means of validating the fourth level of validation studies as proposed by Zellman and Fiene’s OPRE Research Brief (Zellman, G. L. & Fiene, R. (2012). Validation of Quality Rating and Improvement Systems for Early Care and Education and School-Age Care, Research-to-Policy, Research-to-Practice Brief OPRE 2012-29. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services) in which four approaches to validation are delineated for Quality Rating and Improvement Systems (QRIS). This author has taken this framework and applied it to licensing systems (Fiene (2014). Validation of Georgia’s Core Rule Monitoring System, Georgia Department of Early Care and Learning) and more recently proposed as the framework for Washington State’s Research Agenda (Stevens & Fiene (2018). Validation of the Washington State’s Licensing and Monitoring System, Washington
Department of Children, Youth, and Families).

For additional information regarding the above studies, the interested reader should go to RIKInstitute.com.

__________________________________________________________
Richard Fiene, Ph.D., Professor of Psychology (ret), Penn State University; Senior Research Consultant, National Association for Regulatory Administration (NARA); and Research Psychologist, Research Institute for Key Indicators (RIKIllc).