The Relationship between Regulatory Compliance and Quality Initiatives: RIKIllc Technical Research Note #67

The Relationship between Early Care & Education Quality Initiatives and
Regulatory Compliance: RIKIllc Technical Research Note #67
Richard Fiene, Ph.D.
February 2019


Over the past couple of decades there has been many early care and education initiatives, such as Quality Rating and Improvement Systems (QRIS), Professional Development, Training, Technical Assistance, Accreditation, and Pre-K programs to just name a few. Validation and evaluation studies have begun to appear in the research literature, but in these studies there has been few empirical demonstrations of the relationship between these various quality initiatives and their impact on regulatory compliance or a comparison to their respective regulatory compliance. This brief technical research note will provide examples of these comparisons taken from the Early Childhood Program Quality Improvement and Indicator Model (ECPQI2M) Data Base  maintained at the Research Institute for Key Indicators (RIKIllc).

I have written about this back in 2014 (Fiene, 2014) in how the various quality initiatives were having a positive impact on the early care and education delivery system but at that point regulatory compliance data were not available. Today, in 2019, with many changes and developments in state data systems, this is no longer the case. Now it is possible to explore the relationships between data from the various quality initiatives and licensing. Several states in multiple service delivery systems have provided replicable findings in which I feel comfortable reporting out about the relationships across the data systems.

What we now know is that there is a positive and statistically significant relationship between regulatory compliance and moving up the QRIS Quality Levels. In other words, facilities have higher compliance in the higher QRIS Quality Levels and lower compliance in the lower QRIS Levels or if they do not participate in their state’s respective QRIS (F = 5.047 – 8.694; p < .0001).

Other quality initiatives, such as being accredited, shows higher compliance with licensing rules than those facilities that are not accredited (t = 2.799 – 3.853; p < .005 – .0001).

This is a very important result clearly demonstrating the positive relationship between regulatory compliance and quality initiatives. I have some additional state data sets that I will add to the ECPQI2M data base and will continue to analyze these relationships and post additional RIKIllc Technical Research Notes.


Richard Fiene, Ph.D., Senior Research Consultant, National Association for Regulatory Administration; Psychologist, Research Institute for Key Indicators; and Affiliate Professor, Prevention Research Center, Penn State University, Professor of Psychology (ret), Penn State University. (


Posted in | Leave a comment

Dissertation on the Effectiveness CCR&R Services Using a Coaching Model on Licensing Compliance

Here is an interesting Early Care and Education Dissertation completed by a doctoral student at the University of South Carolina, Wenjia Wang.  “The purposes of this study were to investigate the significance of the impact of CCR&R services using a coaching model on licensing compliance outcomes at child care centers and to further our knowledge on the use of coaching to improve health and safety conditions in child care environments.”

A Quasi-Experimental Study on the Effectiveness of CCRR TA Coach

Posted in | Leave a comment

Boston Globe Article

The following article appeared in the Boston Globe this morning:

With New Day-Care Inspection System, High Caseloads and Shorter Visits – The Boston Globe


Posted in | Leave a comment

Some Technical Considerations in Using Complaint Data and Regulatory Compliance Data: RIKIllc Technical Research Note #66

Some Technical Considerations in Using Complaint Data and Regulatory
Compliance Data: RIKIllc Technical Research Note #66

Richard Fiene, Ph.D.

January 2019

As promised in RIKIllc Technical Research Note #65, this Note will provide details on the
and analytical considerations when using complaint and regulatory compliance data together.  As pointed out in the previous technical research note, using complaint data as a potential outcome appears to have merit and should be explored in greater detail. However, with that said there are some parameters that the methodology has that should be explored in order to make the analyses more meaningful.

When looking at regulatory compliance and complaint data there are four possibilities: 1) the facility is in full compliance and has no complaints; 2) the facility is in full compliance but has complaint(s); 3) the facility has some non-compliance and has no complaints; and 4) the facility has some non-compliance and has complaint(s). These four possibilities can be depicted in a 2 x 2 matrix:

Cell C = Full Compliance & No Complaints; Cell A = Full Compliance & Complaints (False Negative): Cell B = Non-Compliance & No Complaints; Cell D = Non-Compliance & Complaints. (See the attached Technical Research Note for a clearer picture of the 2 x 2 Matrix).

In the this 2 x 2 matrix, we would want to see cell C and cell D as the predominant cells and cell A and B as the less dominant cells, especially cell A because this represents a false negative result.

However, there are a couple of limitations to the above matrix that need to be taken into account. One, are the complaints substantiated or not. Any complaint must be substantiated to be counted in the model. If it is unsubstantiated, than it is not counted in the matrix. Two, there is the problem with directionality that needs to be addressed. For example, does the complaint occur before or after the full inspection in order to determine regulatory compliance. The 2 x 2 matrix and the modeling for these
analyses is based on the complaint occurring after the full inspection and that is the reason for cell A being labeled a false negative. If the directionality is reversed and the full inspection occurs after a complaint, cell A is no longer a false negative.

RIKI Technical Details


Posted in | Leave a comment

Relationship between Regulatory Compliance and Complaints in a Human Services Licensing System: RIKIllc Technical Research Note #65

What is the Relationship between Regulatory Compliance and Complaints

Richard Fiene, Ph.D.

January 2019


Within licensing measurement and the validation of licensing systems it is particularly difficult to have specific outcome metrics that can be measured within a human services licensing system. The purpose of this technical research note is to propose a potential solution to this problem.

Probably the most accurate measures of licensing outcomes focus on improvements in the health and safety of clients within human services licensed facilities, such as: fewer injuries (safety) or higher levels of immunizations (health). Another measure related to client satisfaction is the number of complaints reported about a licensed facility by clients and the general public. The advantage of using complaints is that this form of monitoring is generally always part of an overall licensing system. In other words, the state/provincial licensing agency is already collecting these data. It is just a matter of utilizing these data in comparing the number of complaints to overall regulatory compliance.

The author had the opportunity to have access to these data, complaint and regulatory
compliance data in a mid-Western state which will be reported within this technical research note. There are few empirical demonstrations of this relationship within the licensing research literature. The following results are based upon a very large sample of family child care homes (N = 2000+) over a full year of licensing reviews.

The results of comparing the number of complaints and the respective regulatory compliance levels for specific family child care homes proved to show a rather significant relationship (r = .47; p < .0001). This result is the first step in attempting to understand this relationship as well as developing a methodology and analysis schema since directionality (e.g., did the complaint occur before or after regulatory compliance data collection?) can play a key role in the relationship (this will be developed more fully in a future technical research note). The focus of this research note was to determine if any relationship existed between regulatory compliance and complaint data and if it is worth pursuing.

It appears that looking more closely at the relationship between complaint and regulatory compliance data is warranted. It may provide another means of validating the fourth level of validation studies as proposed by Zellman and Fiene’s OPRE Research Brief (Zellman, G. L. & Fiene, R. (2012). Validation of Quality Rating and Improvement Systems for Early Care and Education and School-Age Care, Research-to-Policy, Research-to-Practice Brief OPRE 2012-29. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services) in which four approaches to validation are delineated for Quality Rating and Improvement Systems (QRIS). This author has taken this framework and applied it to licensing systems (Fiene (2014). Validation of Georgia’s Core Rule Monitoring System, Georgia Department of Early Care and Learning) and more recently proposed as the framework for Washington State’s Research Agenda (Stevens & Fiene (2018). Validation of the Washington State’s Licensing and Monitoring System, Washington
Department of Children, Youth, and Families).

For additional information regarding the above studies, the interested reader should go to

Richard Fiene, Ph.D., Professor of Psychology (ret), Penn State University; Senior Research Consultant, National Association for Regulatory Administration (NARA); and Research Psychologist, Research Institute for Key Indicators (RIKIllc).

Logos - RIKI, NARA, PSU-page-001

Posted in | Leave a comment

Data Distributions for the Major ECE Systems: Licensing, QRIS, and ERS

I thought it important to share with researchers who may be doing ECE research on Licensing, QRIS – Quality Rating and Improvement Systems, and ERS – Environmental Rating Scales.  Usually when we are doing research, we find the data to be normally distributed which is the case with ERS data sets.  However, in dealing with Licensing and QRIS data sets, this is not the case.  With Licensing data we find the data distributions to be highly skewed and with QRIS data we find the data distributions to be either bi-modal or highly skewed depending on if only the QRIS sites are used or the full complement of sites statewide.  Attached is a brief technical research note which depicts these data distributions for consideration when doing future research by licensing researchers.

Data Distributions for Licensing QRIS and ERS

Posted in | Leave a comment

Theory of Regulatory Compliance and Quadratic Regression

Here is a RIKIllc brief technical research note on the Theory of Regulatory Compliance and quadratic regressions:

Theory of Regulatory Compliance and Quadratic Regression 


Logos - RIKI, NARA, PSU-page-001

Posted in | Leave a comment

Evolution of Differential Monitoring

Attached please find a Technical Research Note on the Evolution of Differential Monitoring with special emphasis on Key Indicators and Risk Assessment.

Evolution of Differential Monitoring

Logos - RIKI, NARA, PSU-page-001

Posted in | Leave a comment

The Importance of Immunizations

Having children properly immunized is a very important goal within public health.  It helps to protect children’s health.  Within early care and education programs, immunizations are both a standard of care as well as an outcome of that care.  Recently, as I have been doing additional in-depth analyses of the national data base that RIKILLC – Research Institute for Key Indicators maintains, having children properly immunized has been and continues to be a key indicator rule that statistically predicts overall regulatory compliance with all early care and education rules.  This is a result that appeared in the research literature over 40 years ago and is still present in today’s analyses.  It helps to account for approximately 70% of the variance related to statistically predicting regulatory compliance.  These results are across the USA and Canada.

So why is an immunization standard or rule such a good discriminator of high performing early care and education programs.  Keeping track of children’s immunizations is not an easy task.  It is very detailed-oriented which takes a great deal of diligence on the individuals doing the tracking.  One can assume that the best programs have figured this out while the mediocre programs who have difficulty with regulatory compliance have not.

Posted in | Leave a comment

Child Care Licensing Study Trend Analysis

After returning from a stimulating week at the National Association for Regulatory Administration’s (NARA) Licensing Seminar and the Expert Licensing Panel hosted by NARA and the National Center for Early Childhood Quality Assurance (NCECQA), I learned about a new resource made available by the Child Care and Early Education Research Connections (CCEERC).  The resource makes all the data over the past decade from the Child Care Licensing Studies conducted by NARA and NCECQA available as SPSS data files.   I started to mine these data as soon as I got back and plan on posting several blogs on this website over the winter months looking at trends in the data over the past decade.

There are five data points from 2005 – 2014.  The data base provides a national window into child care licensing in both center based and home based care.  I will start with the centers data base and then move to the home data bases.   Here is my first look at the center data base related to licensed capacity, number of centers and average size of centers.  As I said, I will be selecting variables and posting results overtime looking at trends over the five data points.  If anyone has any pressing questions that they are interested in seeing how things have changed over the past decade, please don’t hesitate to get in touch with me at

Child Care Licensing Study CCC Licensed Facilities 2005-2014


Posted in | Leave a comment