RIKI – Research Institute for Key Indicators LLC

Displayed in the attached document is a comparison of regulatory compliance metric principles, prevailing paradigms and a continuum of quality matrix. These principles and key elements have been presented in previous posts but here they are presented side by side depicting where there are common treads. For the interested reader who wants additional information about any of the principles or elements, please consult the previous RIKINotes Posts for additional details.

A technical research note/abstract combining research from several regulatory compliance metrics over the past decade into one abstract emphasizing the public policy implications of this research.

From the introduction to the abstract “This technical research note/abstract provides a data matrix depicting the relationship between regulatory compliance and program quality. The data clearly demonstrate the regulatory compliance theory of diminishing returns which depicts the ceiling or plateau effect in this relationship between regulatory compliance data and program quality data. It also shows the difficulty one will have in distinguishing program quality differences at the full and high regulatory compliance levels but the ease in distinguishing program quality between low regulatory compliance and high regulatory compliance levels.”

“The importance of these studies and the summary matrix is to provide a context in how
licensing and regulatory compliance data should be used in making public policy decisions, for example: is it more effective and efficient to require high or substantial regulatory compliance than full regulatory compliance with all rules and regulations to be granted a full license to operate? It appears prudent to continue with the US emphasis on QRIS as an add on quality initiative, especially in states where rules/regulations are at a minimal level. In Canada their emphasis has been more in line with an integrative monitoring approach in which quality elements are built in or infused within the rules and
regulations themselves. This approach appears to work in a similar fashion and is an effective public policy initiative. Either approach appears to be an effective modality to increasing program quality; but are both equally efficient.”

A short paper is posted proposing ten principles to consider when dealing with regulatory compliance measurement within the regulatory science field. It is based upon 50 years of research diving deep into regulatory compliance data at the state, national, and international levels. These principles are based upon repeated demonstrations in studies conducted across the above three venues.

Here is a link to the Licensing Measurement and Monitoring Systems eHandBook online for easy access via a digital publishing site.

Here is a new report on monitoring practices used in child care and early education licensing published by OPRE’s TRLECE: The Role of Licensing in Early Care and Education Project. This is a wonderful new resource which will help to inform how states are utilizing monitoring to ensure that programs are meeting their state’s respective rules and regulations. This is a highly recommended read for anyone in the ECE field as well as parents and advocates.

Here are the eHandBook and the Powerpoint Slide Deck for Licensing Measurement and Monitoring Systems that can be used by licensing administrators, regulatory scientists, licensing researchers, and licensing program staff. These contain the latest research updates. For those interested, please check back because I will be updating both documents, obviously these are a work in progress as we learn more about licensing measurement.

Below is the second edition of the Licensing Measurement and Monitoring System eHandBook: Regulatory science applied to human services regulatory administration for regulatory scientists, licensing researchers, regulatory administrators and their policy and program staffs.

The purpose of this blog post is to point out the intersections, differences and similarities of integrative, differential/inferential and coordinated monitoring as used in the monitoring of human service programs. Program monitoring has changed over the years in that not only has it grown in the types of monitoring done, such as process, compliance, outcome monitoring, etc.; but also in the functional aspects of monitoring as delineated with integrative, differential, and coordinated monitoring. Much has been written in the research literature about the types of monitoring but not as much regarding the functional aspects of monitoring probably because it is much newer and has grown with the various types of monitoring being used in different contexts.

Coordinated monitoring deals with monitoring across similar service types, for example, in early care and education, monitoring would be done using similar standards in Head Start, child care, preschool, etc. This is an effective and efficient approach which has been demonstrated through the creation and dissemination of Caring for Our Children Basics as a core set of standards for all these various settings. The US Dept of Health and Human Services has advocated this particular approach.

Differential monitoring focusing on the use of abbreviated or targeted inspections of programs that have a history of high regulatory compliance with specific rules or standards. It means spending more time and doing a more comprehensive review of those programs having difficulty complying with specific rules, these can be based upon risk assessment or predictive value of overall compliance. This is a very efficient approach which has been demonstrated to save time in monitoring reviews. Many states in the USA and provinces in Canada use this approach. The US Office of Head Start has experimented with the approach.

Instrument-based program monitoring utilizes instruments, tools, or checklists for recording all data when a review or inspection is completed. It is different from the case review or anecdotal type of record keeping. This approach started in the late 1970’s, early 1980’s when it was introduced by the Children’s Services Monitoring Transfer Consortium, a federally funded research project consisting of California, Michigan, West Virginia, Pennsylvania and New York City. Its development occurred parallel with the development of differential monitoring but with particular emphasis on the metrics or measurement domain when it came to tool development. The Child Development Program Evaluation Scale was a major tool developed from this initiative.

Integrative monitoring is a relatively new approach to monitoring in which the emphasis is on integrating regulatory compliance rules with quality programming standards. Note the emphasis is on the rules and standards and not on who gets applied to those rules and standards nor how they get applied. However, combining integrative monitoring with differential monitoring is an interesting research focus which could be a very effective and efficient approach in combining these two perspectives. In the past, licensing and quality programming have generally been in their own silos when it comes to program monitoring. Integrative monitoring removes them from these silos and suggests building a continuous metric that starts with the health and safety aspects of rules and adds in the quality pieces on top of the rules. Presently, quality initiatives, such as Quality Rating and Improvement Systems, Accreditation, and Professional Development systems are examples of standards that could be used to build upon health and safety licensing rules.

There appears to be interest in pursuing an integrative monitoring approach in several jurisdictions in the early care and education field but this interest extends beyond and has been suggested more broadly by a recent article published in the Journal of Regulatory Science by Freer & Fiene (2023). Regulatory compliance and quality programming: Constraints and opportunities for integration, Volume 11, Number 1, 1-10 (Journal of Regulatory Science). The interested reader may want to take a look at the article, it does provide a unique model for pursuing integrative monitoring. Also, one may be interested in Fiene’s eHandBook on Licensing Measurement and Monitoring Systems: Regulatory science applied to human services regulatory administration available at https://RIKInstitute.com. This eHandBook provides the basics of licensing measurement and program monitoring metrics.

Here is a graphic that has been used to describe a logic model for ECPQIM: Early Childhood Program Quality Improvement and Indicator Model/Differential Monitoring Logic Model and Algorithm (DMLMA) which overlays the monitoring approaches (Coordinated, Instrument-based, Differential/Inferential, and Integrative) with the logic model.

Kenya, in particular researchers from Kisii University, is using the theory of regulatory compliance in a very useful way to impact their regulatory environment and to promulgate effective and efficient regulations related to various industries. This study by Dr Wilfred Ochieng Omollo & George Makua Ogendi is the most recent example: Planning and Conservation of Urban Riparian Reserves.

Here are two articles from the past that highlight early care and education professional development and accreditation and other program quality initiatives that are still pertinent today, that is why I am sharing them today. Take a look at the two articles, they will provide additional support for improving the overall quality of child care.

As we begin a new year and reflect on where early care and education (ECE) is headed, we may need to acknowledge a new normal for the field. I am sure many of my colleagues in ECE will not be happy with what I am about to share but I have always been driven by empirical data and this is what I am observing in the ECE field at this point.

We are all disappointed with the lack of action at the federal level to revamp the ECE system into a much improved and enhanced system. The opportunity was there at the beginning of the pandemic and there was a great deal of debate and discussion but it led nowhere. We are left with an ECE system having difficulty in finding adequately trained staff on a daily basis. If anything, the ECE field looks worse today then it did three years ago and that is saying alot.

So what can we do? I would suggest that we go back to the basics. The original philosophy of licensing and regulatory science is “do no harm”, let’s begin there. We need to make sure that all our children are in healthy and safe environments. We need to revisit the child care trilemma and focus on the availability and affordability side of the equation and put quality on the back burner again. I hate suggesting this but we have no other choice at this point or the system is going to implode. We need to make certain that our children do not lose any additional ground which has been so evident during the pandemic.

Once we have re-established a solid base, then and only then, we can begin to address quality of services via regulatory science, quality rating and improvement systems, and professional development of ECE staff: an Early Childhood Program Quality Improvement and Indicator Model. We do have several excellent examples that I have had the fortune to be part of which should provide some guidance, such as broader adoption of Caring for Our Children Basics as the core set of rules/regulations/standards for the ECE profession. Full implementation of the new Head Start Monitoring System and the full roll out of the iLookOut Learning Platform for ECE staff.

Creating national standards for any industry based upon empirical evidence is possible in analyzing research over the past 50 years in human services. Research completed in human service regulatory science gives a pretty good blueprint in how this can be accomplished. Let’s look at what has been done in early care and education (ECE), and child residential programs.

The key starting point is the unit of analysis which in ECE programs would be the facility: a child care center or home. In other industries, it could be an agency, a factory, a store, a bank, etc…. But back to the ECE example. There will be rules/regulations/standards applied to the facility, i.e., child care center or child care home. These will generally range from health and safety rules/regulations to state of the art program quality standards in most measurement protocols. Assessments will be done with many individual facilities and aggregated appropriately: regulatory compliance with rules and program quality observation tools. It is critical that an instrument based program monitoring system be utilized and not an anecdotal narrative based data collection system. It is too difficult and time consuming to analyze case studies on a large scale. Taking case studies on a sampling basis from the quantitative data base can work and provides a balance between quantitative and qualitative data analysis.

Once these data are aggregated it will be able to determine trends in the data, which rules/regulations are most critical in predicting overall compliance, what are the key quality indicators, which rules or standards that place clients at greatest risk of morbidity and mortality, etc… It is suggested that this be done with multiple samples, these could be done regionally, statewide, nationally, etc. depending on the level of data accessibility. By doing this and utilizing factor analysis it will be able to determine are there any commonalities in the rules/regulations/standards? Generally there is!! Let’s use ECE facilities as an example. In ECE, research went from individual key indicators at the state licensing agency level to generic key indicators (common rules across state licensing agencies) to a national voluntary set of standards (Caring for Our Children Basics). This same blueprint could be used in any industry and it would help to make for more effective and efficient monitoring systems if it were done.

Any industry that is regulated or accredited could follow the above blueprint in moving from individual sites to aggregate data and generating national, international, industry standards to follow based upon empirical evidence. And through factor analyses it would be possible to streamline the rules to a core set of the most predictive key indicators. This is how it was done in the ECE field.

For those individuals who are interested in learning more or pursuing this, take a deeper dive into this blog and the Selected Publications page of this website for details. Also, get in touch with the National Association for Regulatory Administration (NARA) who has consultants who can help design these types of measurement systems (NARA).

An interesting article about ways to improve compliance measures and implementation in Ukraine.

The purpose of the article is to study relatively new phenomenon of compliance (Theory of Regulatory Compliance, Fiene, 2016, 2019) for Ukraine and to identify its priority areas of development that should bring the domestic regulatory framework closer to world practice

There is the need to revise the risk assessment methodology that has been used in the differential monitoring approach. In the past risk assessment has always been listed along with the key indicator methodology as the two approaches for differential monitoring and abbreviated monitoring inspections. However, it has become clear that there is another level of risk within a monitoring system that should be addressed and that involves potential risk. Potential risk is the possibility that because of certain characteristics or factors possessed by a facility or program that may place it at increased risk when you observe its component parts. An example could be that a program is located in a high crime area that places clients at increased risk to harm. It doesn’t guarantee that it will happen but if certain safeguards are not in place it increases the potential risk that something negative could occur.

Potential risk are variables that should be looked upon as separate from actual licensing rules or standards and should be assessed prior to a monitoring review. It would be similar to a pre-audit looking for potential predictor risks. And this is where this new concept of risk combines the previous risk assessment and key indicator methodologies. Risk assessment (RA) is what it purports to be, an assessment of morbidity or mortality risk because of non-compliance with a specific rule or standard. A key indicator (KI) is a predictor rule or standard that predicts overall regulatory compliance or program quality depending on the type of measurement undertaken. The new potential risk (PR) metric is a combination of both. It would look something like this:

RA + KI = PR

The other nuance to potential risk is that it will be very individual to a specific facility or program. No two programs are the same. What may be potential risk indicators for one program may not be for another. It will be very important to determine the proper path for each program so that they can be successful in their implementation efforts. Potential risk assessment is very individual, there probably will not be a one size fits all approach.

This post is for regulatory scientists to think about as they move forward in the further development of differential monitoring approaches, especially those approaches that involve risk assessment.

Here is a letter from over 400 Penn State Faculty members concerned about a leadership decision to not follow through with its commitment to creating a Center for Racial Justice. Please see the following letter for the issues and the faculty who have signed on.

I learned today that I am in the top ten clubs: (top 10% of authors) and top 10,000 downloaded papers (Theory of Regulatory Compliance) on SSRN, the repository for new research in the social sciences. See the two notifications below:

I received this email today:

Congratulations on your nomination and initiation into Sigma Xi, The Scientific Research Honor Society! More information about your membership will be sent to your email soon.

If you have any questions concerning Sigma Xi, please do not hesitate to contact the Society’s Membership Office by calling 919-549-4691.

Once again, congratulations and welcome to Sigma Xi.

Best Regards,
Sigma Xi Membership and Chapter Services

And this arrived today:

Sometimes we have a tendency to push the envelope too far and the contents of the envelope gets lost in the process.

You can tell that there is high inflation because the “money stealer wildflowers” are really bad this year.

Seniors always have something new to talk about because they can’t remember what they talked about the day before.

For seniors every day is a new day because they can’t remember what happened yesterday.

The beginning of life our birthdays are important because we haven’t had many, and at the end of our life they are equally important because we realize we don’t have many left.

Time becomes precious as we age because we begin to realize we don’t have much left.

Doing nothing about anything is the epitome of laziness.

Later is better than sooner is the essence of procrastination.

By thinking we are creating each other.

Object permanence is the basis of all existence. Person permanence is the basis of all relationships.

Today is yesterday’s future.

When the mental meets the physical, one has a singularity; or, Yin + Yang = Singularity.

I always thought that “kicking the can down the road” was not a good thing, until my wife got cancer and her treatments have given us the opportunity to “kick the can down the road” when it comes to longevity.

Forgetting something isn’t a big deal, it is not realizing that you forgot something, That is a big deal.

Living in the present means forgetting about yesterday and tomorrow.

Cars traveling at 60 miles per hour at each other doesn’t make sense; our roads were constructed when buggies went 5 miles per hour.

Rifles were needed back in the 1700s and the second amendment made sense, but assault rifles are not needed for peace or hunting. They are for war and killing the enemy. Second amendment is an 18th century law and doesn’t pertain to the 21st century.

Getting from here to there isn’t a big deal in quantum mechanics because you’re in both places at the same time.

In quantum mechanics, entanglement is looking in the mirror and seeing someone else.

As children we build plastic models, as adults we build logic models.

The most expensive things in life are not really better, we just think they are.

Supply and demand is a group thought process.

Once a decision is made, non-linear reality becomes linear.

Timing is everything in life, from finding the love of your life, to finding a parking spot close to where you need to be.

A line is just a group of connected dots all going in the same direction.

Multiple universes can exist but we can only observe one.

The beginning of life and the end of life are matching bookends to each of our stories. The richness of our life is how many chapters and books we create between the beginning and end.

Senior vision is like star wars on a Monet canvas with the floaters and our less than optimal vision.

Sometimes the line between what is and what could be gets blurred.

We should always think we are better than we are, and sometimes we really are.

A mistress can be a woman/man or an avocation; love knows no boundaries.

Genius is seeing what has always been there but others have missed.

Einstein was only partly right about “God not playing dice with the universe”; God does if you accept quantum mechanics.

For those of you who are interested in the NARA Licensing Measurement Course I wanted to provide a location map on where you can find all the resources for the course because they have grown a great deal over the years.

On the NARA website you will find the following (https://www.naralicensing.org/key-indicators):

Brochure describing the various methodologies

eHandBook

Technical Research Notes

All the NARA Related Reports

Regulatory Compliance/Science Theoretical Papers

Powerpoint/Webinar Slides with Notes

On the RIKI Website you will find the following (https://rikinstitute.com/):

Introduction

RIKI Notes Blog Posts

All the Publications and documents related to Licensing Measurement

Here is the link to a poster presentation with Dr Sonya Stevens, Daniel Blevins, and Amber Salzer entitled: Identifying Predictive Indicators: The State of Washington Foster Care Home Study. The poster presentation was at the National Research Conference on Early Childhood, June 27th – 29th.

https://nrcec2022.ipostersessions.com/Default.aspx?s=69-71-06-39-C6-15-22-0F-7F-C8-75-4F-A1-8E-52-7D

Below is the National Research Conference on Early Childhood Program Book which gives the details of the 2022 conference with all presenters and their sessions. The conference will be held June 27-29, 2022. NRCEC presents the latest research on early childhood programs and the young children and families they serve. The virtual venue will host plenaries, breakout sessions, poster sessions, networking discussions, and more. NRCEC promotes conversations between early childhood researchers, practitioners and policy-makers.

Below is a Licensing Measurement & Program Monitoring Systems eHandBook to accompany the NARA Licensing Measurement and Systems course. It is recommended to be read along with taking the NARA course but it can be read as a stand alone book. It is a short guide to licensing measurement introducing some of the key issues and tenets related to applying regulatory science to human service regulatory administration. It is meant to be read in one sitting but hopefully it will generate a lifetime of questions related to the field of regulatory science.

The attached report is as relevant today as it was 25 years ago; it is a synthesis of major issues, policy questions, available research findings and information needs in child care policy, presented in a form that can provide a framework for ongoing dialogue and action by the research community in partnership with state child care administrators and other key stakeholders.

This report builds upon work at the Child Care Policy Research Symposium, sponsored by the
Child Care Bureau, Administration for Children and Families, US Department of Health and Human Services. The Symposium brought together researchers, child care policymakers and state and federal staff for a unique opportunity to discuss current research efforts and the research needs of state child care administrators.

This is an older report but I thought it was still relevant today, so I wanted to post it for other ECE researchers and regulatory scientists to review and use.

The Better Care for the Babies (BCTB) Project was initiated in April 1989 to help states improve the quality of infant and toddler child care, especially for low-income children whose parents are in the labor force and/or making the transition from welfare to work. The BCTB Project initiated ongoing, negotiated, goal-directed technical assistance with three state interagency teams in Florida, Illinois, and Utah; conducted a national technical assistance forum; and implemented national outreach through the preparation and dissemination of policy papers. The chapters of this case study describe the background and design of the project, the policy context and assumptions, the technical assistance approach and implementation, project actions and policy improvements related to child care quality made by the BCTB states, the project as perceived by key participating state administrators themselves, lessons learned, and recommendations. The recommendations focus on federal mandates that would include incentives, offering states goal-directed technical assistance, coordination of state policies and programs, and conveyance of information to state leaders concerning the influence of child care on child development.

Below is a policy commentary article just published in the Journal of Regulatory Science, Volume 10, Issue 1 on regulatory science monitoring paradigms and the relationship between regulatory compliance and program quality. Eighteen key elements are introduced in a series of dichotomies which help to lay out a blueprint and the parameters when thinking about program monitoring and the continuum between regulatory compliance and program quality.

Below is a brief technical research note providing an example of a data analysis plan utilizing risk assessment indicators. It provides a means for thinking about how best to implement such a plan from initial design to validation of the plan.

Here are five studies that demonstrate validation of the Licensing Key Indicator (LKI), Risk Assessment Methodology (RAM), and Regulatory Compliance Theory (RCT). The studies were done in the states of Georgia (RAM, RCT), Washington (RAM, RCT) national with Head Start (RCT), and internationally in the Provinces of Ontario (LKI) and Saskatchewan (LKI, RAM).

Here is a recently published article on comparing online mandated reporter trainings which highlights the iLookOut Child Abuse Prevention Training program. Very interesting state by state comparisons.

The 2 x 2 matrix format has been used in many different contexts when it comes to decision making. I have found the 2 x 2 matrix very useful in regulatory science especially when it comes to measuring regulatory compliance with rules. In this post, I would like to delineate how the 2 x 2 matrix can be used with nominal measurement of regulatory compliance where it is the essence of regulatory science metrics.

In the 2 x 2 matrix above, the Regulatory Science Metrics Matrix, we are attempting to measure regulatory compliance comparing the measurement by an inspector with what exists in reality. The (+) = a positive response (there is compliance) and a (-) = a negative response (there is non-compliance). The (++) = compliance was recorded/measured and in reality there really was compliance. This is expected and desirable since we want everyone to comply with the respective rules we are measuring. The (–) = there was non-compliance recorded/measured and in reality there really was non-compliance. This is expected but not desirable; obviously we don’t want to find any non-compliance although it is good that the inspector is reliably accurate. The False Positive (+/-) = there was non-compliance recorded/measured but in reality there was compliance. The False Negative (-/+) = compliance was recorded/measured but in reality there was non-compliance.

From a regulatory science point of view and the measurement of regulatory compliance, the (++) and (–) are the two results we want to see; they are expected and desirable. We never want to see a False Negative (-/+), and we would like to minimize False Positives (+/-) whenever possible. In the actual regulatory science world, false positives and negatives do occur and are part of regulatory science. The goal is to minimize them as much as possible. This above Regulatory Science Metrics Matrix has become a useful tool in measuring regulatory compliance and in validation studies related to regulatory science in the human services.

Two of the most widely used early care and education program quality tools used in the field are the CLASS: Classroom Assessment Scoring System and ERS: Environmental Rating Scales. Is there an advantage to using one versus the other. In the state of Washington as part of their QRIS: Quality Rating and Improvement System they happen to utilize both. In a study validating their Licensing Decision Making System, I had the opportunity to see them used side by side and wanted to report the results here. In other separate studies conducted in Head Start, Georgia, and Pennsylvania I saw similar results but wanted to wait to have the CLASS and ERS side by side in a specific study.

Here is what I found in making that comparison. In comparing the CLASS head to head with the ERS the correlation between the two scales was r = .24; p < .0001; n = 385. So both scales had a statistically significant correlation which one would expect since they are both measuring classroom quality, albeit from different perspectives.

Where it becomes interesting is when one begins to compare the two with the Washington state QRIS correlations. The CLASS and QRIS is r = .12; p < .022; n = 385 while the ERS and QRIS is r = .39; p < .0001; n = 385. It appears that the ERS is more sensitive at discriminating differences in QRIS than the CLASS. I further tested this my running one-way ANOVAs: CLASS x QRIS: F = 10.71; p < .0001; n = 385 while the ERS x QRIS: F = 26.534; p < .0001; n = 385. Both are statistically significant but the ERS again shows a much larger F ratio than is the case with the CLASS. To delve more deeply into these differences required looking at some basic descriptive statistics, such as the mean, standard deviation, skewness, and kurtosis. The following chart shows the results.

As one can see from the descriptive statistics there are some major differences between the CLASS and the ERS in how the data distributions play out. The ERS clearly has more variance in their data distribution than the CLASS does. These results are consistent with other studies in analyzing the respective data distributions. I feel that these results are significant for other early care and education researchers, developmental psychologists, and regulatory scientists as they conduct similar studies utilizing these respective tools.

I get asked all the time about what is the difference between the Key Indicator and the Risk Assessment methodologies. Generally I reply with a very academic type of response either explaining the difference given the research literature or the statistical methodologies employed. It hasn’t worked very well and there still is confusion in the field about the differences between what is a key indicator rule and what is a risk assessment rule. So I am going to take a different tack and let’s apply it to one of the most important sets of rules that has ever existed and see if it helps: The Ten Commandments.

Let’s start with the risk assessment methodology and attempt to ascertain which of the Ten Commandments would be a risk assessment rule. What immediately jumps out to me is “Thou Shall Not Kill”. This commandment would definitely fall under the “do no harm” rule of risk assessment in attempting to avoid morbidity and mortality concerns. If I were to send this out to a group of Biblical scholars and ask them for their expert opinion, I am guessing that this would be on the top of their list as well. So I feel pretty confident that we could say that “Thou Shall Not Kill” would meet the criterion of being a Risk Assessment Commandment.

Now, let’s turn our attention to the key indicator methodology and attempt to ascertain which of the Ten Commandments would be a key indicator rule. This gets a bit tricky because key indicator rules usually don’t place individuals at severe morbidity or mortality. But the key indicator rules statistically predictor overall rule compliance. So knowing this one Commandment would help us to know who is most likely to abide by all the other Commandments. That is kind of important from a societal point of view because we would like to have a lot of these people as neighbors; it would be like living in Mr Rogers’ Neighborhood. So what do we think could be a good Key Indicator Commandment? Based upon my 50 years of research in producing key indicator rules I would say that “Thou Shall Not Steal” might be a good candidate. I am guessing that there is a deep structure here where a person who is honest is most likely to abide by all Ten Commandments, so it would be an excellent Key Indicator Predictor Commandment. Of course to be certain, we would have to empirically test this hypothesis out which is the cornerstone of the key indicator rule methodology: data utilization.

I hope I have enlightened those of you who may have been somewhat uncertain about the differences between risk assessment rules and key indicator rules. Hopefully this foray into the Biblical literature via the Ten Commandments has helped to make the distinction more clear.

I have discussed licensing measurement a great deal in this blog. Today, I want to introduce a new term to basically describe what I have been discussing with licensing measurement, called Regulatory Metrics or RegalMetrics for short. The reason for doing this is to be better positioned within the burgeoning new science called Regulatory Science. Licensing may be too delimited in its scope while regulatory science is more all encompassing and I feel will be the new science of rules, regulations, and standards.

The same issues are still present within regulatory metrics as they were in licensing measurement, such as how regulatory compliance data distributions are dramatically skewed with intense kurtosis. How best to deal with nominal measurement data? Do we transform the nominal data to ordinal scales as has been proposed in this blog (January 9th Post) into a Regulatory Compliance Scale to make it more similar to other more normally distributed program quality data distributions? Another way of thinking about this is in having “Licensing Buckets” for “Full, Substantial, Mid, and Low” regulatory compliance levels (see the Post of January 9th). The need for dichotomization of data is warranted because of the skewed data distributions. How best to minimize false positives and false negative decisions regarding the issuing of licenses based upon regulatory compliance scores. And lastly and probably most significant is how to deal with the introduction of mediocrity into fully compliant programs.

This last issue is a major issue for regulatory science regardless of discipline in how best to address the plateau of quality as programs move from substantial to full regulatory compliance. By not addressing this issue will continue to lead to frustration by consumers and the various industries we regulate in not being able to fully reward our outstanding performers because based upon regulatory compliance scores it is difficult to distinguish between these top performers and the mediocre performers. Regulatory science modeling is excellent at distinguishing between fully compliant programs and those that are having real difficulty with regulatory compliance. Where the models break down is distinguishing between programs that are in substantial compliance and full compliance when it comes to any quality dimension. This is what leads to the public wanting deregulation because the rules just don’t seem to make a difference. And then when there is a tragedy, the push for more regulations in order to protect all individuals so that they do not have the same tragedy repeat itself. It is this constant deregulation versus over-regulation mentality that is so counter productive and not driven by good public policy nor empirical data.

Below are a series of resources for the licensing measurement/regulatory science course that are organized as anthologies and summarizing information from the RIKI Publications webpage.

Attached please find an anthology that contains technical research notes from the past decade on regulatory science, differential monitoring and licensing measurement. I thought it would be helpful to regulatory scientists and licensing researchers to have all these various research notes in one location, so I created this anthology.

Here is a policy commentary manuscript that delineates the key elements or alternate program quality paradigms and their subsequent implications for regulatory compliance measurement and program quality that has been submitted to the Journal of Regulatory Science. This manuscript is intended for other licensing researchers and regulatory scientists as they deal with licensing measurement issues in regulatory science. Hopefully it provides some key parameters to consider as the regulatory science field matures into a full-blown science.

A brief comment about the Journal of Regulatory Science. This relatively new journal, started publishing in 2013, is an excellent forum for those researchers and scientists who are doing regulatory science related research. It is open sourced and encourages scientists from all content disciplines who have an interest in regulatory science to submit their research to the journal. I have been involved in research and publishing for 50 years and this journal and its approach is a breath of fresh air in their openness, attention to detail, and creating a peer review process that makes sense and is timely. I encourage any regulatory science researcher or scientist to check this journal out for sharing their research (https://journals.tdl.org/regsci/index.php/regsci/about).

This article was published in 2002 in Child and Youth Care Forum. This article built off several studies in Pennsylvania which clearly demonstrated the lack of an effective professional development system, especially involving infant toddler caregivers. The mentoring/coaching intervention as designed and described in this article was revised and enhanced in several other studies to follow in order to address this major gap in the professional development system in Pennsylvania.

These other studies will be described in subsequent posts in which the coaching intervention was utilized by child care health consultants: ECELS-Early Childhood Education Linkage System, was used online: Better Kid Care, and was used as a micro-learning problem solving approach: iLookOut. This line of research helped to complete the Early Childhood Program Quality Improvement and Indicator Model’s quality initiative sector by adding professional development to Quality Rating and Improvement Systems and Accreditation Systems.

This article was published in 1980 in NARA News as a licensing measurement enhancement. It really grew out of the regulatory compliance need being addressed at the national level with the changes being made in the Federal Interagency Day Care Requirements (FIDCR). There was a concern by many Federal policy makers that the monitoring system was going to be too much of a burden on individual programs in attempting to measure regulatory compliance with the revised FIDCR standards. Interesting this same concern would lead to the development and implementation of the Key Indicator methodology, but more about that in future posts.

For this post, we will just center in on the concerns about how best to measure regulatory compliance with two key rules of the FIDCR: adult child ratios and group size. To measure regulatory compliance with these two rules it was necessary in the past to take painstaking measurements of the number of children and adults at various times during the day in child care programs.

The below article describes a mathematical model “Contact Hours” that can be used as an off-site proxy to determine regulatory compliance without ever stepping foot in a program. There are actually two articles presented here: 1) The original article published in 1980; 2) A 2021 paper based upon the use of the mathematical model in the state of Washington. In this second paper, the Contact Hours mathematical model was enhanced and expanded to deal with potential infection rates in child care programs during the COVID-19 pandemic. State administrators saw it as a solution to determining regulatory compliance without having to make onsite observations which were very restricted during the COVID-19 pandemic in 2020-21. The Contact Hours mathematical model worked very nicely in Washington state determining regulatory compliance but it also helped to target mitigation efforts in programs that were having infection outbreaks based upon particular threshold levels.

This article appeared in the Child Care Information Exchange in the mid 1990s. In the early part of that decade, Gwen Morgan, one of the pioneers of early care and education (ECE) regulatory science and administration, proposed the child care trilemma. The child care trilemma consists of the delicate balancing act of affordability, accessibility and quality. Dr Morgan’s thesis was that you could not change one without impacting the others and the child care field was having difficulty dealing with the trilemma at that point.

The article presents a proposed solution that alters the conventional wisdom of regulatory science and policy by suggesting to not increase adult child ratios but rather decrease it so that one additional child could be cared for by a very highly qualified teacher (BA/MA in ECE) and the additional revenue brought in by the additional child go directly to this highly qualified teacher as a teaching bonus/salary increase. By utilizing such a solution, it addresses all three components of the trilemma of quality, accessibility and affordability without violating any of them.

This article published in the International Journal of Child Care and Educational Policy in 2013 compared the regulatory compliance within the USA with approximately 20 other countries to determine the emphasis placed upon rules and regulations in the respective countries. It is clear from the results that the USA emphasized more structural aspects of rules and regulations dealing with health and safety while the other countries emphasized the professionalization of the teacher in the classroom.

This article also introduced to an international audience the Early Childhood Program Quality Improvement and Indicator Model, now in its 4th edition and its implications with the advent of Quality Rating and Improvement Systems on a large scale in the USA.

This article published in the Journal of Regulatory Science in 2019 has helped to create an interesting heuristic problematic for the regulatory science field. The essence of the treatise is moving regulatory policy from full compliance with all rules to substantial compliance with all rules and full compliance with specific predictor rules. This is a dramatic departure from regulatory policy that has been promulgated within the regulatory field for the past 100 years.

Because of the regulatory compliance theory of diminishing returns, the following approaches and methodologies of differential monitoring, key indicators for licensing and quality, as well as risk assessment rules have been introduced to the regulatory science field. None of this could have occurred without the introduction of this theory. It has really altered how we approach regulatory compliance from a measurement and program monitoring perspective. The implications of this theory will be further explored in an upcoming post dealing with program monitoring paradigms and the relationship between regulatory compliance and program quality.

This is an article written back in 1985 that really tied licensing measurement together into a quantitative approach of instrument based program monitoring rather than case anecdotal records and proposed the use of key indicators/predictor rules, risk assessment/weighting of rules, and the introduction of differential monitoring which back then was called abbreviated inspections or inferential inspections.

The article appeared in Child Care Quarterly and really did begin to usher in a paradigm shift in licensing measurement and with the introduction of the Theory of Regulatory Compliance the movement from issuing full licenses with 100% regulatory compliance to substantial compliance with all regulations. This article also introduced the Early Childhood Program Quality Improvement and Indicator Model as a means for typing regulatory compliance together with quality initiatives, especially technical assistance, training, and professional development which will be addressed in future posts.

I am hoping to do this with several articles that I think are very pertinent to licensing measurement and post summaries of their particular significance for regulatory science and program monitoring. The hope would be that this new series will help to inform future licensing researchers and regulatory scientists regarding the nuances and idiosyncrasies of licensing measurement and regulatory compliance. As one will see, there are many measurement issues with licensing data and how best to analyze licensing data. This new series really started with the post before this one in which Federal, national, and state reports were listed and presented related to licensing and differential monitoring. The subsequent posts will provide a bit more detail of many topics presented in these various reports. These posts will also provide a backdrop to the National Association for Regulatory Administration’s Licensing Measurement course which is part of their Licensing Curriculum.

As one will see, there is a need within regulatory science to get at the key measurement issues and essence of what is meant by regulatory compliance. There are some general principles that need to be dealt with such as the differences between individual rules and rules in the aggregate. Rules in the aggregate are not equal to the sum of all rules because all rules are not created nor administered equally. And lastly, all rules are to be adhered to, but there are certain rules that are more important than others and need to be adhered to all the time. Less important rules can be in substantial compliance most of the time but important rules must be in full compliance all of the time.

Rules are everywhere. They are part of the human services landscape, economics, banking, sports, religion, etc… Where ever one looks we are governed by rules in one form or another. The key is determining an effective and efficient modality for negotiating the path of least resistance in complying with a given set of rules. It is never about more or less rules, it is about which ones are really productive and which are not. Too many rules stifle creativity, but too few rules lead to chaos. Determining the balance of rules is the goal and solution.

Attached are several examples of Federal, national, and state reports on state of the art licensing and differential monitoring initiatives. These reports have helped to shape the research efforts as we move forward with licensing and differential monitoring in early care and education.

Several Federal agencies are well represented, such as the Office of Child Care, the Administration for Children and Families, Health and Human Services, USDA, Assistant Secretary’s Office for Planning and Evaluation, Office of Planning, Research, and Evaluation; National Organizations, such as the National Association for Regulatory Administration, National Women’s Law Center, CLASP, BUILD, Child Care and Early Education Policy and Research Analysis, and Child Trends; and states, such as Ohio, Minnesota and Illinois.

This blog post will attempt to place the Regulatory Compliance Theory of Diminishing Returns into everyday terms addressing its potential implications beyond the human services and suggest how it can be applied anywhere in which standards/regulations/rules are utilized in the public policy domain.

The Regulatory Compliance Theory of Diminishing Returns was first proposed in the 1970’s when several studies were conducted comparing regulatory compliance with program quality in early care and education programs. These studies were expanded to include other child residential programs and similar results occurred in which a plateau or diminishing return in the levels of program quality & child outcomes were observed as regulatory compliance increased from a substantial level to a full (100%) level. Over the past 50 years, this same result was found when these analyses were performed. See the following article published in the Journal of Regulatory Science for additional details: (https://journals.tdl.org/regsci/index.php/regsci/article/view/108).

Why is this important from a public policy perspective? It appears from these results that public policies which demand full (100%) regulatory compliance may not be in the best interest of providers nor clients being served. The Regulatory Compliance Theory of Diminishing Returns has implications for all of regulatory science and would apply to any field in which a closed system of standards/rules/regulations are utilized. Therefore, it is being suggested that the theory be applied to other economic systems involving banking, trade, markets, supply/demand chains, etc… that are heavily regulated. When a more open system of standards/rules/regulations are utilized, the diminishing returns effect is less evident because of the introduction of program quality elements into the equation (see RIKI Technical Research Notes on the balance of regulatory compliance and quality as well as regulatory compliance modeling which clearly demonstrates the differences between open and closed systems).

So what would this look like from a program monitoring perspective? Rather than requiring companies, organizations, or agencies to be in full regulatory compliance, it would focus more on substantial compliance with all standards/rules/regulations and full compliance with key indicator standards/rules/regulations that statistically predict overall regulatory full compliance. This would be a more effective and efficient allocation of monitoring resources that would lead to increased outcomes for clients and better management for providers.

The ultimate goal is to obtain the proper balance of regulatory oversight which is not too stringent nor too lax but rather one that focuses on the right (statistical predictors) standards/rules/regulations producing the greatest impact on clients and providers of service.

The history of licensing measurement and regulatory compliance has actually a rather long lineage but is still in its infancy in terms of development. In the early stages most licensing visits and inspection results were recorded via anecdotal records/case records with the licensing staff recording their results in more social work note taking. It was a qualitative type of measurement with very little quantitative measurement occurring with the exception of basic demographics, number of clients, number of caregiving staff, etc… This qualitative approach worked very well when there were not many programs to be monitored and there were sufficient licensing staff to do the monitoring and conduct the inspections.

This all started to change in the 1980’s when Instrument Based Program Monitoring (IPM) was introduced and started to be adopted by state licensing agencies throughout the United States. Just as a footnote, this brief history is pertinent to the USA and does not include other countries although the Canadian Provinces have followed a similar route as the USA. The reason for the introduction of an IPM approach was the tremendous increase in early care and education programs in the 1960’s and 1970’s. It was difficult for licensing staff to keep up with the increased number of programs in their monitoring efforts. There needed to be a more effective and efficient methodology to be employed to deal with these increases.

A very influential paper was written in 1985 and published in Child Care Quarterly which introduced IPM along with Licensing Key Indicators, Risk Assessment (Weighting), and Differential Monitoring (Abbreviated Inspections). This paper outlined the various methodologies and their use by a consortium of states to test the viability of this new approach to licensing measurement, regulatory compliance, and program monitoring. Also, the terminology has changed over the decades. Back in 1985 weighting was used rather than risk, abbreviated inspections were used rather than differential monitoring, targeted monitoring, or inferential monitoring. All these terms can be used interchangeably as they have been over the years, but the first introduction of them back in 1985 utilized weighting and abbreviated inspections.

In the early 1990’s the risk assessment methodology was used to develop Stepping Stones to Caring for Our Children, the comprehensive national health and safety standards for early care and education (ECE) programs in the USA. This was a major development in attempting to develop national voluntary standards for child care in the USA.

It was during this time that two other very significant discoveries occurred related to licensing data distributions: 1) Licensing data are extremely skewed and do not follow a normal curve distribution. This fact has a significant impact on the statistics that can be used with the data distributions and how data analyses are performed. For example, data dichotomization is warranted with licensing data; 2) Regulatory compliance data are not linear when compared to program quality measures but are more plateaued at the substantial and full regulatory compliance levels. The data appear to follow the Law of Diminishing Returns as compliance moves from substantial to full (100%) regulatory compliance. This finding has been replicated in several studies and has been controversial because it has led to the issuing of licenses to programs with less than full compliance with all rules/regulations/standards. These two discoveries have been very influential in tracking developments in licensing measurement since their discoveries.

In the new century as states began to adopt the various methodologies it became necessary to have a standardized approach to designing and implementing them. The National Association for Regulatory Administration (NARA) took up this role and in 2000 produced a chapter on Licensing Measurement and Systems which helped to guide states/provinces in the valid and reliable means for designing and implementing these methodologies. In 2002 a very important study was conducted by the Assistant Secretary’s Office for Planning and Evaluation (ASPE) in which they published the Thirteen Indicators of Quality Health and Safety and a Parent’s Guide to go along with the research. This publication further helped states as they revised their licensing and program monitoring systems for doing inspections of early care and education facilities based upon the specific indicators identified in this publication. Both publications have been distributed widely throughout the licensing world.

During the first decade of the new century, Stepping Stones for Caring for Our Children went through a second edition. This publication and the ASPE publications were very useful to states as they prepared their Child Care Development Fund (CCDF) plans based upon Child Care Development Block Grant (CCDBG) funding.

From 2010 to the present, there have been many major events that have helped to shape licensing measurements for the future. Caring for Our Children Basics (CFOCB) was published and immediately became the default voluntary early care and education standards for the ECE field. The CFOCB is a combination of the risk assessment and key indicator methodologies. Three major publications by the following Federal agencies: HHS/ACF/USDA: Department of Health and Human Services/Administration for Children and Families/United States Department of Agriculture, OCC: Office of Child Care, and ASPE: Assistant Secretary’s Office for Planning and Evaluation dealing with licensing and program monitoring strategies were published. These publications will guide the field of licensing measurement for years to come. The Office of Head Start developed and implemented their own Head Start Key Indicator (HSKI) methodology. And in 2016, CCDBG was reauthorized and differential monitoring was included in the legislation being recommended as an approach for states to consider.

Most recently, the Office of Head Start is revising their monitoring system that provides a balance between compliance and performance. This system revision will go a long way to enhancing the balance between regulatory compliance and program quality. Also, there has been experimentation with an Early Childhood Program Quality Indicator instrument combining licensing and quality indicators into a single tool. These two developments help with breaking down the silo approach to measurement where licensing and quality initiatives are administered through separate and distinct approaches such as licensing versus professional development systems versus quality rating and improvement systems. A paradigm shift in which an Early Childhood Program Quality Improvement and Indicator Model is proposed. The paradigm shift should help to make licensing measurement more integrated with other quality initiatives.

The licensing field continues to make refinements to its measurement strategies in building a national/international regulatory compliance data base. More and more is being learned about the nuances and idiosyncrasies of licensing data, such as moving from a nominal to an ordinal driven data system. For example, NARA and the Research Institute of Key Indicators (RIKI) have entered into an exclusive agreement for the future development of licensing measurement strategies via differential monitoring, key indicators for licensing and program quality, and risk assessment approaches. Several validation studies have been completed in testing whether the various methodologies work as intended. A significant Office of Program Research and Evaluation (OPRE) Research Brief which developed a framework for conducting validation studies for quality rating and improvement systems has been adapted to be used in licensing measurement.

For additional updates to licensing measurement, please check out and follow these RIKINotes Blog posts. There are and will be many examples of licensing measurement enhancements. Also, although much of the research on licensing measurement has been completed in the ECE field, the methodologies, models, systems, and approaches can be utilized in any human service arena, such as child residential or adult residential services. Also, NARA’s chapter in their Licensing Curriculum has been developed into a full blown course, please go to the following web page for additional information: https://www.naralicensing.org/key-indicator-facilitated-dialogues

A technical research note is provided for other licensing researchers and statisticians who are interested in replicating the methodology through the use of a alternate statistical software package, such as SPSS, Systat or SAS. The research note provides all the background statistics and algorithms for the generation of a Key Indicator Matrix and results.

This technical research note will integrate the Key Indicator Matrix (KIM) and the Risk Assessment Matrix (RAM) into one platform to clearly demonstrate their statistical modeling overlap.  Key Indicators deal with the ability to predict overall compliance or performance based on existing data.  Risk Assessment Indicators do not predict but determine a risk score based upon prevalence and severity measures.  Their purposes are different but when integrated together the two matrices are a powerful tool in determining the health of the measured entity.   

The below matrix integrates the two matrices of KIM and RAM and shows that KIM scores are generally at the lower end of risk but having sufficient prevalence when it comes to non-compliance.  RAM scores have a larger variance and are most concerning at the higher end of the continuum. 

KIM x RAM Matrices 

Here are two documents, one, a technical research note on the latest version of the Early Childhood Program Quality Improvement/Indicator Model (V5)(ECPQIM5a) and two, a powerpoint slide presentation on Licensing Measurement (PPT189).

Attached to this post is a new land use study completed in Kenya utilizing the Theory of Regulatory Compliance. It was published in the International Journal of Human Capital in Urban Management: Planning implication of universities growth on land use: Confirmatory evidence from GIS spatial analysis, by W.O.Omollo, Department of Planning and Development, Kisii University, Kenya.

ABSTRACT

BACKGROUND AND OBJECTIVES: Universities have customarily been seen as agents of development in the regions they serve owing to their roles of teaching, research, innovation and community extension. There is however a dearth of knowledge on how they influence land use change with a specific reference to compliance with planning standards. This paper therefore through a case study investigates the impacts that the growth of Kisii University has on land use change in Nyamage, a neighbourhood where it is situated within Kisii Municipality, Kenya. It subsequently links the observed change to compliance with planning standards.

METHODS: Guided by the theory of regulatory compliance, the study adopted a case study research design with a sample size of 226 drawn from 577 developments in Nyamage. Spatial data on land use change was collected using satellite images from Google Earth covering three epochs of 2005, 2014 and 2021. Analysis was undertaken using GIS. Data investigating compliance with planning standards were conversely collected using an observation checklist, land survey maps and analyzed using a one-sample t-test and paired t-test.

FINDINGS: The study established that in 2005, forest, short vegetation, transitional and built-up areas respectively covered 17%, 39%, 34% and 11%. These by 2021 correspondingly changed by 46%, -10%, -29% and 57% for the forest, short vegetation, transitional and built-up areas. The latter recorded the highest land use change, a condition mainly credited to the hostels built by private developers in an attempt to meet a demand created by students who could not find accommodation within the university. Research findings further disclosed that developments around the university were not complying with the planning standards used in regulating plot sizes, building coverage ratio and road reserves, leading to land use conflicts.

CONCLUSION: The establishment and growth of Kisii University have remarkably influenced land use change, which in the absence of development control contributes to the disregard of planning standards. This is because the government mainly sees universities as an avenue for spurring regional economic growth with less attention on their spatial implications. These findings may enlighten policy-making institutions with critical information for effective planning and development control around universities. The study also fills a gap that hitherto existed on the nexus between land use change and compliance with planning standards as relates to the growth of universities. It additionally enlightens the international audience on how the impacts of universities growth on land use may be evaluated through a triangulation of spatial and statistical approaches.

KEYWORDS: Development control; Kisii Municipality; Land use change; Planning standards; Universities

I have been reading about the Tennessee Study regarding their Pre-K program and their lack of success. Unfortunately, the goals of the program to help very disadvantaged children gain and sustain those gains overtime did not come to fruition according to the study authors. The latest findings are no surprise and have been demonstrated in many other previous studies involving large scale early care and education (ECE) interventions. However, are we designing the wrong interventions and measuring the wrong aspects of development. Play is and has always been the paramount intervention strategy in early care and education programs. But when we design and implement Pre-K we seem to be more concerned about academics and forget about the need for children to play. Curriculum is critical but the curriculum should be based upon developmentally appropriate practices and child development principles, and it should be play based and not academically focused.

When we are thinking about curriculum and assessment, do we need to shift the paradigm in which assessment comes before the curriculum intervention. Shouldn’t the curriculum be driven by each individual child’s specific strengths and areas needing improvement. Having a more individualistic approach based upon the needs of the child which helps us to better solve the “problem of the match”. There needs to be a more synergistic relationship between assessment and curriculum development and implementation.

The next area that is paramount are the overall qualities of the teachers. Teachers need to have a degree in early care and education and not in elementary education or any other degree that is not child development focused. It can be either an AA or BA degree, ideally an MA but that is probably unrealistic and too costly. But it must be in ECE. In the medical profession you don’t want podiatrists doing heart surgery; same thing in ECE, we want ECE teachers teaching in ECE classrooms.

It has become really clear from Quality Rating and Improvement Systems (QRIS) that parent involvement and engagement is a key factor for overall ECE quality and positive child development outcomes. Without parental engagement, 75% of what needs to be accomplished is lost. And the environment that children are spending their days in ECE classrooms needs to be language rich and high quality exchange rates between teachers and children at a verbal level. Real exchange of meaningful dialogue and not commands that are uni-directional from teacher to child; but a real give and take between the child and the teacher. More of a dance rather than regimented marching.

And lastly, Pre-K should not be a separate program but rather one that is integrated with Head Start and child care classrooms. Pre-K classrooms should be part of Head Start classrooms and child care classrooms. We need to break down these structural barriers and have all children fully integrated and not in separate silos based upon funding streams.

In a previous RIKINote (4-8-2018), linear and non-linear models were discussed on a very broad scale. In this note, the Theory of Regulatory Compliance is being suggested as a non-linear paradigm to the dominant neoclassical economic theory and the linear mathematical modeling of econometrics.

The Theory of Regulatory Compliance is based upon several empirical studies conducted in the human services which states that the relationship between regulatory compliance and program quality is not a linear relationship when comparing the upper ends of the compliance x quality continuum. The relationship between regulatory compliance and program quality is linear at the lower end of the continuum when one is looking at non-optimal regulatory compliance up to a mediocre level of regulatory compliance. But once substantial regulatory compliance and full (100%) regulatory compliance are attained, there is a plateau or diminishing return effect when it comes to corresponding program quality levels. In other words, from an outcomes perspective, it is not a worthwhile use of resources to be in full regulatory compliance as versus substantial regulatory compliance. This result has been demonstrated in several studies in the human services field across the USA and Canada.

Why is this an important finding? Because there has always been an assumption that regulatory compliance is a linear variable. But based upon the Theory of Regulatory Compliance, it appears that it is truly a non-linear variable and it would change any mathematical equation within econometrics that introduces regulatory analysis. This could go a long way in explaining many of the disparities in pricing regulations and supply/demand economics where regulations are heavily represented. Could the econometric mathematical modeling be more finely tuned by adding a non-linear paradigm to the formula generation via regulatory compliance?

This blog post will propose a new Regulatory Compliance Scale (RCS)(Fiene, 2022) which should help in making comparisons between regulatory compliance and program quality systems, such as Environmental Rating Scales and Quality Rating & Improvement systems. The proposed scale builds off of a familiar 1-7 Likert scale that has been used a good deal in the early care and education field within program quality instruments/tools. This scale is based upon 40+ years of research into regulatory compliance data distributions which have been reported in this blog (RIKINotes) over the years.

The proposed scale (see RCS Table below) has the following structure of full compliance, substantial compliance, mediocre compliance, and low/non-optimal compliance. Numerically it is proposed that full compliance = 0 no rule violations; substantial compliance = 1-3 rule violations; mediocre compliance = 4-9 rule violations; and low/non-optimal compliance = 10+ rule violations. The transformation to a 1-7 Likert scale is as follows: full compliance = 7; substantial compliance = 5; mediocre compliance = 3; and low/non-optimal compliance = 1.

When the above regulatory compliance scale is utilized it substantially reduces the skewness and kurtosis in the regulatory compliance data distribution which is a major problem with all regulatory compliance data distributions and has been reported repeatedly in the human services licensing research literature. The revised or transformed data distribution begins to approach a more normally distributed data set; albeit, not as normally distributed as the various Environmental Rating Scales but significantly better when straight frequency counts are used in determining regulatory compliance. This has been the preferred means of data recording since the introduction of Instrument-based Program Monitoring (IPM) in the 1980’s. It is being proposed that the above Regulatory Compliance Scale (RCS)(Fiene, 2022) be used in place of this frequency based data system.

This newly proposed scale should go a long way in making future analyses in utilizing regulatory compliance data more useful and meaningful when making comparisons with the various program quality initiatives present in the early care and education field, such as the Environmental Rating Scales and Quality Rating & Improvement Systems.

The major publications surrounding Caring for Our Children dealing with risk assessment and key indicators along with their respective checklists/tools. Each of the publications are listed here for your convenience.

Here is the July 2019 National Resource Center for Health and Safety in Child Care and Early Education Guide for parents in choosing safe and healthy child care. It is a really nice checklist that should help parents in comparing their child care options. There is space for entering key indicator information for three child care programs.

I have also attached a draft of a tool (Early Learning and Child Care Program Quality Key Indicator Instrument) I helped the Ministry of Education in the Province of Saskatchewan develop based upon quality indicators that I thought would be of interest as well to both parents and to ECE professionals. By using both of these guides, one has key indicators drawn from over 40 years of research into ECE licensing and program quality key indicators.

It all started in and around 1965 when the Federal government got into early care and education (ECE) in earnest with Head Start and federally funded day care for low-income families.  It started off slowly but began to pick up momentum with exciting studies and research applying principles from developmental psychology to policy making.  Researchers and policy makers wanted to make sure that these new programs were not detrimental to young children since our frame of reference were children being raised in orphanages and the ultimate outcome for children was not positive.  Would ECE have the same impact? 

Issues around quality, appropriateness of standards, and demonstration programs became the focal point of federal research funding.  The focal point of this essay is on the appropriateness of the ECE standards and the resulting monitoring systems that were to become key to the federal involvement in early care and education.   This essay will be organized by the following 50 years neatly broken out by each decade to get us from this beginning in 1965 until the publication of Caring for Our Children Basics in 2015 by the federal government, the Administration for Children and Families, U.S. Department of Health and Human Services.  A look at the 2020 decade with a future note is also appended to this essay. 

1970s 

During the 1970’s, the federal government became concerned about what were to be the standards for this new national program related to federally funded ECE for low-income families and their children.  Head Start was a separate entity and we will revisit Head Start later but our focus for now is on the federally funded programs which became known back then generically as day care.  This nomenclature changed to child care and to finally early care and education (ECE) during this 50-year history.  The initial standards for day care were the Federal Interagency Day Care Requirements (FIDCR).  A very large appropriateness research study led by Abt Associates to determine what were the most salient standards and their intended impact on children while in day care was conducted during this decade.  These standards were to be federally mandated requirements for any program receiving federal funding.   This is where group size and adult-child ratios standards became such important safeguards and surrogates for children’s health and safety in day care programs. 

It also became of interest for the federal government to design the monitoring system that would determine compliance with the FIDCRs.  But it became clear to the original designers of this new system that the monitoring of the FIDCR was going to be difficult to do across the full USA.  So, the question became, is there a way to monitor the standards in the most effective and efficient manner?  This question and the future of the FIDCR were to be altered and put on hold once we moved into the next decade.  

1980s 

A change in federal administration and a resulting change in philosophy related to the federal role in America altered many things and one of them was the relationship of the federal government and the states.  Rather than the federal government mandating day care requirements, the focus changed with the locus of control moving from the federal level to the state level via block grant funding with very few federal requirements.  This meant a moratorium to FIDCR and its ultimate demise.  The federal government was not going to be in the business of providing day care, this was going to be the jurisdiction of the states.  Head Start did become the exception to this rule with its own standards and monitoring system. 

The focus of federal funding switched from the national to the state level in determining compliance with each state’s respective child care licensing rules and not with an overarching FIDCR.  There was still interest in making these state monitoring systems as effective and efficient but there was no interest in the federal government determining what these requirements would be.  Two monitoring approaches grew out of this need for effectiveness and efficiency:  risk assessment and key indicators.    These two approaches were originally designed and implemented as part of a federally funded project called the Children’s Services Monitoring Transfer Consortium in which a group of five states: New York, Michigan, Pennsylvania, West Virginia, and California teamed up to explore their most effective and efficient monitoring systems and begin transferring these systems to one another and beyond. 

These two monitoring approaches were tested in the above respective states and it was determined that their impact had a positive effect on the children who were in those day care centers.  This was a major finding, similar to the FIDCR appropriateness study, in which these approaches provided safeguards related to the health and safety of children while in day care.   

1990s 

By the 1990s, it became clear that the federal government had pretty much drawn back from any leadership role in having mandated federal requirements when it came to health and safety in child care.  It was left to national ECE advocates who were positioned within the federal government (Administration for Children and Families; Maternal and Child Health Bureau) as well as throughout the USA with national and state agencies and organizations (American Academy of Pediatrics; American Public Health Association, National Resource Center for Health and Safety in Child Care) that saw a need for child care health and safety recommendations at least.  If we could not have requirements, we could at least have recommendations and provide guidance to child care programs throughout the USA. 

This led to the first edition of Caring for Our Children which was a comprehensive set of child care health and safety standards.  It was a major game changer for the ECE field because now there was a universal set of standards based upon the latest research literature for states to use as they considered revising and updating their respective state licensing child care rules.   

But there was a problem.  Caring for Our Children was a comprehensive set of health and safety standards which was their strength but at the same time it was their weakness.  They were so comprehensive (well over 500 well researched standards) that they were intimidating and it was difficult to determine where to begin for the states. 

Several researchers remembered the two approaches to monitoring designed in the previous decade and wondered if they could be helpful in focusing or targeting which of the standards were the most critical/salient standards.  The risk assessment approach to monitoring appeared to have the most immediate applicability and Stepping Stones to Caring for Our Children was born.  This document clearly articulated which of the 500+ Caring for Our Children standards placed children at greatest risk for mortality or morbidity by not being in compliance with the respective standard.  Since the early 1990s, Caring for Our Children and Stepping Stones to Caring for Our Children have gone through three editions and have become very important resources to state licensing agencies as they revise, update and improve their ECE rules. 

2000s 

In this decade several federal and national organizations began to use Caring for Our Children standards in innovative ways to measure how well ECE looked at a national level.  The Assistant Secretary’s Office for Planning and Evaluation in the U.S. Department of Health and Human Services published the Thirteen Indicators of Quality Child Care based upon a core set of predictor standards from Caring for Our Children.  These were standards that predicted overall compliance with all the standards and were seen as an efficient monitoring system.  NACCRRA (National Association for Child Care Resource and Referral Agencies) began publishing a national report card on how well states met specific standards and monitoring protocols based upon similar predictor standards from Caring for Our Children. 

These efforts helped states to make significant changes in their ECE rules in their respective states and in a very voluntary way suggested a means for national standards for the ECE field although we would need to wait until the next decade in order to see such a published document of national ECE health and safety standards for early care and education:  Caring for Our Children Basics. 

2010s 

By the 2010s, ECE had grown into a very large but unwieldly assortment of programs with varying levels of quality.  Again because of major federal funding, the Child Care Development Block Grant, along with changes and enhancements in professional development, accreditation systems, quality rating and improvement systems, the ECE landscape had become more complex and less easy to navigate.  And rather than coming together it was clearly more fragmented than ever. 

We had very minimal requirements for the federal funding and most of these requirements were geared to the state agency using the state’s respective licensing rules as the threshold for standards.  This approach worked well with states with excellent licensing rules, but it wasn’t working as well with states who did not have equally excellent licensing rules.  We still did not have a core set of standards for ECE programs.  Enter Caring for Our Children Basics which took the best aspects of the above two monitoring approaches, risk assessment and key indicators and molded it into this new document.  This work was led by the federal government’s Administration for Children and Families, U.S. Department of Health and Human Services and although the standards are still recommendations and guidance, it is our best attempt at having national standards for early care and education.  It is an attempt to provide guidance to the full ECE field, child care, Head Start, preschool, and center based as well as home-based care.  It would be nice to have Caring for Our Children Basics as the health and safety foundation for early care and education throughout the USA.  I don’t see this happening in my lifetime. 

2020s: Looking to the Future 

As a footnote to this essay, the new decade has been dealt with a major curve ball with COVID19 rearing its ugly head and ECE has been impacted greatly because of this pandemic.  As of this writing we are nowhere closer to a solution to getting ECE programs back on line.  If anything, the pandemic really demonstrated the fragility of the ECE system we have built over the past 50 years and it clearly has not done very well.  My hope is that we can learn from the past 50 years and not continue another 50 years along the same route; although I am guessing that many ECE advocates would be glad to have what we had before the pandemic because what we have right non-sustainable.  We know a lot more today than what we knew back in 1965 when we were worried about would day care hurt children’s development.  We know today that quality ECE benefits children but unfortunately, we are no closer to attaining this today than we were 50 years ago. 

Two programs that have been very successful in avoiding these pitfalls are Head Start and the national Military Child Care program.  Both programs are exemplary examples of quality early care and education being provided with separate funding streams and standards.  Interesting enough when the Administration for Children and Families published Caring for Our Children Basics, both these programs were part of the reach of the published standards.  As we re-invent and re-structure ECE we should be looking to both these very successful programs for guidance. 

Two technical research notes dealing with a paradigm shift related to program monitoring and its subsequent impact on regulatory compliance and quality. These research notes help to develop the key elements, principles, and dimensions when thinking about designing and implementing program monitoring systems.

Attached below are two reports from Saskatchewan and Florida which delineate their respective experiences with developing differential monitoring systems.

Below is a series of technical research notes dealing with licensing measurement paradigm considerations involving performance assessments, regulatory compliance modeling, risk assessment and weighting. It provides some of the latest thinking related to regulatory compliance and performance assessments as a monitoring continuum rather than as two separate assessments systems.

Here are two papers dealing with the licensing key indicator predictor methodology, regulatory compliance and licensing management that help to round out some of the latest research in regulatory science utilizing an international database from the Early Childhood Program Quality Improvement & Indicator Model (ECPQI2M). The reader will find some key metrics/parameters related to licensing measurement, especially in the second paper.

Starting this Fall 2021, NARA – National Association for Regulatory Administration in conjunction with the Research Institute for Key Indicators (RIKI) will be offering a new course, Licensing Measurement and Systems. The course is being developed by Dr Richard Fiene, Penn State Professor of Psychology (ret) & NARA Senior Research Consultant. Here are a couple of summary comments about the course:

This course will provide the learner with the major tenets of licensing measurement. The learner will discover as they go through the course that measurement in licensing is very different than other measurement systems found in many of the various social and human services. It has some very unique and idiosyncratic aspects which will provide us with increasing challenges in coming up with specific metrics in determining regulatory compliance.

The field of regulatory science is a very young field. Although regulations have been kicking around for well over 100 years, the science behind regulations is probably a quarter of this time. So there is not a great deal of empirical evidence to draw upon which is discouraging but it is very encouraging and exciting at the same time because so much needs to be accomplished in establishing regulatory science’s theory.

Check back periodically on this website (https://rikinstitute.com/blog/) or go to NARA’s website at: (https://www.naralicensing.org/key-indicator-facilitated-dialogues).