Dr. Philip Zimbardo is an influential psychologist best-known for his 1971 Stanford prison experiment. Many psychology students may also be familiar with his psychology textbooks and the Discovering Psychology video series often used in high school and psychology classrooms. Zimbardo is also the author of several notable books including The Lucifer Effect.
I had the distinct honor to be invited to a dinner hosted by the Penn State psychology faculty for Dr. Zimbardo this week. Here is a photo of Dr. Zimbardo and the faculty & guests.
It is exciting to announce a new differential monitoring project in the Province of Saskatchewan, Canada being done by NARA – National Association for Regulatory Administration. This project will assist the Ministry of Education in developing a full blown differential monitoring system with key indicators, risk assessment rules, and quality indicators along with the validation of each. It will be a full evaluation of the ECPQIM – Early Childhood Program Quality Improvement and Indicator Model (please see the following webpage (https://rikinstitute.com/ecpqim/) for additional details about the model. This project will get back to the original purpose of differential monitoring in providing a balance between licensing indicators and quality indicators being used in tandem during abbreviated monitoring reviews. This approach of combining key indicators with risk assessment rules focuses on children’s health, safety and well-being developmentally.
I will be providing updated RIKI Notes as we move along with the project delineating the various phases.
RIKI Technical Research Note #70 – Effectiveness and Efficiency Relationship with resultant Cost Benefit Analysis modeling based upon data from the Theory of Regulatory Compliance. This technical research note depicts a graphic display of the relationship between effectiveness and efficiency and how the intersection of these two can result in cost benefit analysis.
RIKI Effectiveness and Efficiency Relationship1
Dr Richard Fiene, Research Psychologist and Professor of Psychology (ret) at Penn State University is generally regarded as the leading international researcher/scholar on licensing measurement and systems. His theory of regulatory compliance has altered regulatory science and licensing measurement dramatically in thinking about how best to monitor and assess licensing rules and regulations.
Dr Fiene’s measurement and monitoring methodologies have led to targeted or abbreviated inspections in all aspects of human service licensing thru risk assessment, key indicators and differential monitoring approaches. He has maintained an international data base on regulatory compliance for the past 40 years which is housed at the Research Institute for Key Indicators (RIKIllc) and the Pennsylvania State University and has led to the development of statistical techniques for dealing with highly skewed, non-parametric data distributions. His research has led to the following: identification of herding behavior of two year olds, national early care and education quality indicators, mathematical model for determining adult child ratio compliance, Solution to the Trilemma in Child Care Delivery Services, Stepping Stones to Caring for Our Children, Online coaching as a learning platform, Validation framework for licensing systems, and an Early Childhood Program Quality Improvement Model.
He has written extensively on regulatory compliance in the human services and his research has been disseminated all over the world via his website (http://RIKInstitute.com). He presently directs the Research Institute for Key Indicators and is a senior research consultant with the National Association for Regulatory Administration, and is an Affiliate Professor with the Edna Bennett Pierce Prevention Research Center, Consultant to the College of Medicine at Penn State Hershey and the College of Ag. Sciences at Penn State University.
Child Care Aware published a very significant report (Child Care Aware of America’s Child Care Licensing Database: Initial Findings) on state licensing throughout the USA. It builds upon their innovative reports “We Can Do Better“. This new report series utilizes Caring for Our Children Basics as the comparison tool in looking at the state’s licensing rules and monitoring systems. It is an absolutely brilliant approach to being able to look at state’s rules from a national perspective and I applaud Child Care Aware for taking this on. Here is a copy of the report and links to their webpage which contains additional information about the child care licensing data base.
Richard Fiene, Ph.D., Research Psychologist, Research Institute for Key Indicators (RIKIllc); Professor of Psychology (ret), Penn State University; Affiliate Professor, Penn State Prevention Research Center; Senior Research Consultant, National Association for Regulatory Administration (NARA).
There is a major movement within the human services involving big data where rather than selecting samples to do analyses state/provincial agencies have the capability to provide basically population data. For the Theory of Regulatory Compliance as it involves the Licensing Key Indicator Methodology, selection criteria and the dichotomization of data are changing dramatically because of the increased cell sizes in determining and generating the Licensing Key Indicators.
For example, in the past, the Licensing Key Indicator Methodology always utilized a 25/50/25 dichotomization model for segregating high compliance from low compliance facilities. However, with big data being available, cell sizes are much more robust in which this dichotomization model can be increased to 12.5/75/12.5. The move to this model helps to decrease the number of false negatives while at the same time increasing phi coefficients. By doing this, the Licensing Key Indicators generated are very robust and highly predictive.
The following Licensing Key Indicators continue to be identified in state/provincial analyses and results (all these Indicators are from the original ASPE Research Brief: 13 Indicators of Quality Child Care):
- Proper Supervision,
- Children are properly immunized,
- The facility is hazard free,
- Reporting of child abuse, and
- Staff are trained in CPR and first aid.
NARA – National Association for Regulatory Administration conducted a presentation in Indiana on differential monitoring and key indicators (February 14, 2019). Please go to the following Facebook Live link to see the presentation:
Facebook Live Link
The Relationship between Early Care & Education Quality Initiatives and
Regulatory Compliance: RIKIllc Technical Research Note #67
Richard Fiene, Ph.D.
Over the past couple of decades there has been many early care and education initiatives, such as Quality Rating and Improvement Systems (QRIS), Professional Development, Training, Technical Assistance, Accreditation, and Pre-K programs to just name a few. Validation and evaluation studies have begun to appear in the research literature, but in these studies there has been few empirical demonstrations of the relationship between these various quality initiatives and their impact on regulatory compliance or a comparison to their respective regulatory compliance. This brief technical research note will provide examples of these comparisons taken from the Early Childhood Program Quality Improvement and Indicator Model (ECPQI2M) Data Base maintained at the Research Institute for Key Indicators (RIKIllc).
I have written about this back in 2014 (Fiene, 2014) in how the various quality initiatives were having a positive impact on the early care and education delivery system but at that point regulatory compliance data were not available. Today, in 2019, with many changes and developments in state data systems, this is no longer the case. Now it is possible to explore the relationships between data from the various quality initiatives and licensing. Several states in multiple service delivery systems have provided replicable findings in which I feel comfortable reporting out about the relationships across the data systems.
What we now know is that there is a positive and statistically significant relationship between regulatory compliance and moving up the QRIS Quality Levels. In other words, facilities have higher compliance in the higher QRIS Quality Levels and lower compliance in the lower QRIS Levels or if they do not participate in their state’s respective QRIS (F = 5.047 – 8.694; p < .0001).
Other quality initiatives, such as being accredited, shows higher compliance with licensing rules than those facilities that are not accredited (t = 2.799 – 3.853; p < .005 – .0001).
This is a very important result clearly demonstrating the positive relationship between regulatory compliance and quality initiatives. I have some additional state data sets that I will add to the ECPQI2M data base and will continue to analyze these relationships and post additional RIKIllc Technical Research Notes.
Richard Fiene, Ph.D., Senior Research Consultant, National Association for Regulatory Administration; Psychologist, Research Institute for Key Indicators; and Affiliate Professor, Prevention Research Center, Penn State University, Professor of Psychology (ret), Penn State University. (http://rikinstitute.com).
Here is an interesting Early Care and Education Dissertation completed by a doctoral student at the University of South Carolina, Wenjia Wang. “The purposes of this study were to investigate the significance of the impact of CCR&R services using a coaching model on licensing compliance outcomes at child care centers and to further our knowledge on the use of coaching to improve health and safety conditions in child care environments.”
A Quasi-Experimental Study on the Effectiveness of CCRR TA Coach
Some Technical Considerations in Using Complaint Data and Regulatory
Compliance Data: RIKIllc Technical Research Note #66
Richard Fiene, Ph.D.
As promised in RIKIllc Technical Research Note #65, this Note will provide details on the
and analytical considerations when using complaint and regulatory compliance data together. As pointed out in the previous technical research note, using complaint data as a potential outcome appears to have merit and should be explored in greater detail. However, with that said there are some parameters that the methodology has that should be explored in order to make the analyses more meaningful.
When looking at regulatory compliance and complaint data there are four possibilities: 1) the facility is in full compliance and has no complaints; 2) the facility is in full compliance but has complaint(s); 3) the facility has some non-compliance and has no complaints; and 4) the facility has some non-compliance and has complaint(s). These four possibilities can be depicted in a 2 x 2 matrix:
Cell C = Full Compliance & No Complaints; Cell A = Full Compliance & Complaints (False Negative): Cell B = Non-Compliance & No Complaints; Cell D = Non-Compliance & Complaints. (See the attached Technical Research Note for a clearer picture of the 2 x 2 Matrix).
In the this 2 x 2 matrix, we would want to see cell C and cell D as the predominant cells and cell A and B as the less dominant cells, especially cell A because this represents a false negative result.
However, there are a couple of limitations to the above matrix that need to be taken into account. One, are the complaints substantiated or not. Any complaint must be substantiated to be counted in the model. If it is unsubstantiated, than it is not counted in the matrix. Two, there is the problem with directionality that needs to be addressed. For example, does the complaint occur before or after the full inspection in order to determine regulatory compliance. The 2 x 2 matrix and the modeling for these
analyses is based on the complaint occurring after the full inspection and that is the reason for cell A being labeled a false negative. If the directionality is reversed and the full inspection occurs after a complaint, cell A is no longer a false negative.
RIKI Technical Details