Some Technical Considerations in Using Complaint Data and Regulatory Compliance Data: RIKIllc Technical Research Note #66

Some Technical Considerations in Using Complaint Data and Regulatory
Compliance Data: RIKIllc Technical Research Note #66

Richard Fiene, Ph.D.

January 2019

As promised in RIKIllc Technical Research Note #65, this Note will provide details on the
and analytical considerations when using complaint and regulatory compliance data together.  As pointed out in the previous technical research note, using complaint data as a potential outcome appears to have merit and should be explored in greater detail. However, with that said there are some parameters that the methodology has that should be explored in order to make the analyses more meaningful.

When looking at regulatory compliance and complaint data there are four possibilities: 1) the facility is in full compliance and has no complaints; 2) the facility is in full compliance but has complaint(s); 3) the facility has some non-compliance and has no complaints; and 4) the facility has some non-compliance and has complaint(s). These four possibilities can be depicted in a 2 x 2 matrix:

Cell C = Full Compliance & No Complaints; Cell A = Full Compliance & Complaints (False Negative): Cell B = Non-Compliance & No Complaints; Cell D = Non-Compliance & Complaints. (See the attached Technical Research Note for a clearer picture of the 2 x 2 Matrix).

In the this 2 x 2 matrix, we would want to see cell C and cell D as the predominant cells and cell A and B as the less dominant cells, especially cell A because this represents a false negative result.

However, there are a couple of limitations to the above matrix that need to be taken into account. One, are the complaints substantiated or not. Any complaint must be substantiated to be counted in the model. If it is unsubstantiated, than it is not counted in the matrix. Two, there is the problem with directionality that needs to be addressed. For example, does the complaint occur before or after the full inspection in order to determine regulatory compliance. The 2 x 2 matrix and the modeling for these
analyses is based on the complaint occurring after the full inspection and that is the reason for cell A being labeled a false negative. If the directionality is reversed and the full inspection occurs after a complaint, cell A is no longer a false negative.

RIKI Technical Details


About Dr Fiene

Dr. Rick Fiene has spent his professional career in improving the quality of child care in various states, nationally, and internationally. He has done extensive research and publishing on the key components in improving child care quality through an early childhood program quality indicator model of training, technical assistance, quality rating & improvement systems, professional development, mentoring, licensing, risk assessment, differential program monitoring, and accreditation. Dr. Fiene is a retired professor of human development & psychology (Penn State University) where he was department head and director of the Capital Area Early Childhood Research and Training Institute.
This entry was posted in RIKInstitute. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s