Latest RIKI Research related to ECPQIM and DMLMA

Posted on October 18, 2016 by Dr Fiene

logos-riki-nara-psu-page-001

Here are the two versions of the RIKI Book of Readings for ECPQIM/DMLMA:

1)  Main Reports and Papers, 486 pages (15 MB).  This research monograph/book of readings/reports/papers contains the basic reports written during 2012-2016 related to ECPQIM/DMLMA now in its 4th edition.

2)  Main Reports, Papers, Technical Notes, Tools, & Powerpoint Slides, 694 pages (21 MB).  This compilation contains all the basic reports but also contains the powerpoint slides, technical notes, tools, etc.  written during 2012-16 related to ECPQIM/DMLMA.

Here are some other videos, webinars, powerpoint presentations, and other resources about program monitoring and ECPQIM:

Here is a webinar (Resources and Tools for Revising and Aligning Early Childhood Program Standards that was held on Wednesday, January 11, 2017) conducted by the National Center for Early Childhood Quality Assurance on health and safety resources that will be useful to state administrators:

I have had the distinct honor to become a member of the National Center for Early Childhood Quality Assurance’s Licensing Expert Panel that they have convened.  It is with great pleasure that I serve on this panel with 25 other national experts (Licensing Expert Panel Members) representing all the various components of an early care and education quality continuum.

I have also had the distinct honor to be asked again to serve as a member of the National Head Start Association’s Task Force on Monitoring (NHSA Monitoring Task Force).  Here is the NHSA Task Force Report on Monitoring released back in 2012 (NHSA 2012 MTF Report), we will be updating it with the new CCDBG/CCDF requirements related to monitoring.

The Federal Departments of Health and Human Services and the the United States Department of Agriculture recently put forth a joint monitoring statement which will have a significant impact on the overall quality of child care.  Here is the overall link to the website and the specific joint policy statement:

The RIKI Blog has posts regarding Caring for Our Children Basics (CFOCB) and its potential impact on the ECE field.  I am taking a look at a few of the standards and why they are so important to the ECE field in establishing a firm foundation to ECE health and safety for all children.  I have geared the blog for parents to think about their own ECE arrangements and if it meets the standards as presented in CFOCB.  I am really curious to see what I hear back from parents.  (#6/7/9/11/12/13)

We can’t underestimate the importance of CFOCB.  I have said this in other venues that CFOCB is as important as Developmentally Appropriate Practices when it was first published.  CFOCB is a game changer for the USA in that now we (ECE) actually have nationally voluntary standards for all ECE programs.  This is a significant event.

Having been a state administrator, policy researcher and analyst for 25 years, I would suggest that present state administrators think about using CFOCB as the basis of any revision to their own state ECE rules/regulations as their core set of rules, and for their basic health & safety standards in the state’s QRIS.  CFOCB is based upon a solid research base developed over the past five decades.  It is one of the best examples of combining the Key Indicator and Risk Assessment methodologies together.

My plan is to think through creative ways that CFOCB can be used by state agencies in helping to improve ECE in their respective jurisdictions.  Those of you who know me, know that I have been at this for over 40 years in figuring out the best ways of improving ECE quality for all children.  CFOCB is a first step for us.  Hopefully, with QRIS we can build upon this solid foundation with CFOCB to really tackle ECE quality.

Please go to RIKInstitute.com to get the latest posts.   All the posts are from Caring for Our Children Basics and Caring for Our Children 3rd Edition.

Here is a new resource from the National Center on Early Childhood Quality Assurance regarding new Briefs on Health and Safety Topics (NCECQA Health & Safety Briefs).  I highly recommend these to parents and providers who are seeking child care or are working in child care.  This is just another excellent example of the high quality, thoughtful resources being produced by the National Center.

Based upon 40+ years of research into identifying key licensing and quality indicators it is possible to distill this list of key indicators into three areas/factors when related to rules/regulations/standards.  These three rule/regulatory/standard areas are the following:

  1. A highly qualified ECE Director with a BA/MA in ECE.
  2. Highly qualified ECE Teachers with AA/BA in ECE.
  3. Parent Engagement similar to what we see in Head Start programs.

These three areas have appeared consistently in key indicator lists when analyzing state licensing regulations and QRIS standards.  In an ECE world with very limited resources, I would recommend that we focus our program monitoring on these three areas in order to efficiently and effectively increase the overall quality of ECE programs.

Another question asked many times is if there is a specific rule/regulation that stands out from all the key indicators, in other words, it shows up on every state’s list or most state’s lists.  There is a rule/regulation that fits this threshold and it has to do with children’s immunizations.  For what ever reason, compliance with this rule/regulation appears to have the ability to consistently discriminate between the highly compliant ECE providers and those that have lower compliance.  This is an area that needs additional exploration to determine in greater detail why this occurs.  Presently a MCHB research project being undertaken by the Pennsylvania Chapter of the American Academy of Pediatrics ECELS (ECELS Report) will help to provide some answers to “why”.

In addition to immunizations, the original thirteen key indicators that were identified  in the 1985 Child Care Quarterly (1985 CCQ) article have not over the past three decades changed alot (STATE KI X 10KIf).  There are fewer of them, 10 rather than 13 with group size and adult child ratio no longer on the list but it is interesting that these key indicators have stayed so constant for such a long time.   And over the past three decades, many states have used the original 13 Key Indicators in designing their abbreviated inspections.  Here is the original list of the 13 key indicators (Parents Guide Checklist) as published by the National Resource Center for Health and Safety in Child Care.  For the convenience of the reader, I have listed the key indicators below, for a more detailed look at these, please use the publications listed above.   Those listed with an asterisk (*) are inclusive of the CCDF health and safety national requirements.  Those that are italicized appear approximately two-thirds of the time on state key indicator lists (Thirteen Key Indicators Technical Research Update).  All ten requirements are contained within Caring for Our Children Basics and Stepping Stones.

  1. Supervision of children
  2. Hand washing and diapering
  3. Director & teacher qualifications
  4. Children’s immunizations*
  5. Toxic substances are innaccessible*
  6. Emergency plan*
  7. Fire drills
  8. Child abuse prevention*
  9. Medication administration*
  10. Staff training/first aid*

One last comment about using the key indicator methodology with different data sets, such as with accreditation or QRIS systems.  The key indicator methodology has been also used with ECERS to see if it was possible to find a similar relationship between scoring very high on individual items and the overall score.  Only one item (Item 16 – Children Communicating)  achieved a perfect correlation (r = +1.00) in which it was always scored very highly with only those ECE programs that scored equally highly on the total ECERS score.

I have maintained a national ECPQIM data base where a portion of these data are available at http://rikinstitute.wikispaces.com in various SPSS data files of the key elements.  For interested researchers, I have the full data base available for further analyses.

Dr. Fiene will continue his work in further developing the professional development, training and technical assistance key elements of ECPQIM through his collaborative work with the Pennsylvania Chapter of the American Academy of Pediatrics, ECELS – Early Childhood Education Linkage System’s, Infant Toddler Program Quality Improvement Project; and the Penn State Hershey, College of Medicine, Center for the Protection of Children’s iLookOut for Child Abuse Prevention Project .

ILOOKOUT Certified

He is also continuing his work in the further development of differential monitoring in Canada with the Province of Ontario’s Ministry of Education (Tiered Licensing).  This project will provide a comprehensive implementation, evaluation, and validation strategy for those jurisdictions planning on undertaking  differential monitoring, risk assessment or key indicator methodologies.

And of course, his continuing collaboration and partnership with NARA – National Association for Regulatory Administration where the further development and dissemination of differential monitoring, risk assessment and key indicator methodologies will continue into the future along with the Validation Studies for each of these methodologies.

The ECELS ITQIP is finishing up its three year MCHB funding and here is an initial draft of the reports that have been produced over the past several years.  It begins with the results from the pre-test in order to establish equivalency of the intervention and control groups.  This is followed by the results from the first post test comparing the intervention group to the control group and looking at change over time.  The third report in the series presents the results from the second post test comparing the cross over effects and latent effects of the intervention and control groups.  And lastly, is the tool/instrument used to collect the data for all three years of the study (ECELS ITQIP Reports).  This study and project is particularly exciting because it clearly demonstrates the effectiveness of a child care health consultant mentoring/coaching model in impacting selected Caring for Our Children standards focused on infant and toddler programs.  It also demonstrated that the intervention is effective in a cross over methodology as well as having latent/lasting effects.  This study builds upon the original mentoring/coaching study conducted at the Penn State Capital Area Early Childhood Research and Training Institute/Prevention Research Center in 2002 (CAECTI/PRC Mentoring/Coaching Article).

This study also demonstrated the effectiveness of monitoring.  Data taken from the number of hours CCHC (Child Care Health Consultants) spent in programs doing mentoring/coaching had a positive impact on improving compliance with the Caring for Our Children standards.  But this result was geared more towards the higher compliant programs and the number of hours in mentoring/coaching was not at the high end of the spectrum.  So it appears that just a little help goes a long way with the highest compliant programs.   This is significant because with the push for differential monitoring and abbreviated inspections, having several short monitoring visits still helps a program to improve (ITQIP JEHC Article).

Speaking of Validation Studies, here are several reports on QRIS Validation that should help to guide the reader with a strategic framework for doing these types of studies and a couple of other interesting studies and presentations:

Letters of support Appropriations Letter, OMB Letter, DOE Letter, DOE Letter1, OMB letter 9.19.17, and Congressional letter 9.19.17 for evidence based programs, regulations, and policies that RIKI – Research Institute for Key Indicators signed on to support.

An excellent presentation done by researchers from ASPE, Child Trends, and Georgia DECAL which presents the future of ECE monitoring.

A New Report from ASPE highlighting 13 compelling models for infant toddler early childhood services in which mentoring/coaching models are highlighted, including CAECTI’s Infant-Toddler Caregiver Mentoring Program.

Interesting article on the impact of quality early care and education services (Child Encyclopedia Article).

Three reports regarding child care licensing in Canada, Accreditation, and good standards improving child care quality that I found very interesting.

These two reports demonstrate support for the Theory of Regulatory Compliance which depicts the relationship between program quality and licensing/regulatory compliance where higher licensing standards show a statistically significant relationship with program quality standards but lower licensing standards do not.   Also, a plateau effect occurs when moving from substantial regulatory compliance to full regulatory compliance as it relates to program quality standards.

Because of this plateau effect, it ushered in the key indicator and risk assessment methodologies which are at the basis of abbreviated inspections and differential monitoring.  The purpose of these methodologies is not to have less standards or rules or regulations but rather to determine what are the “right” standards/rules/regulations that impact services the most because they statistically predict overall regulatory compliance or reduce harm or risk for morbidity or mortality.

Although the reports and examples are from early care and education, these methodologies are applicable to all human services (e.g., child and adult residential services, etc.) and probably to other regulatory areas outside of the human services arena.

Theory of Regulatory Compliance  (DOI: 10.13140/RG.2.2.34971.67360)

Recently Georgia DECAL revised their enforcement and compliance policy which demonstrates one of the better examples of a risk asssessment system.  Here is the link to their work (http://decal.ga.gov/CCS/EnforcementCompliancePolicy.aspx).

Here is a discussion I started within NARA (National Association for Regulatory Administration) about regulatory compliance data limitations and potential solutions:

I’d like to start a discussion about the nature of regulatory compliance/licensing data and the implications related to measurement.  As a research psychologist who has spent his total professional career examining the impact of regulatory compliance policies on children and families, the issues related to measurement and program monitoring have always been at the forefront of my research studies.  I have found regulatory compliance/licensing data to have many limitations when it comes to measurement and analysis because the data are severely skewed.

Why is this important?  Generally in the social sciences, research psychologists deal with data that are more normally distributed with sufficient variance.  However, licensing data are not and probably never will be close to being normally distributed.  Actually, this is a good thing from a public policy point of view.  We don’t want basic health and safety rules to be normally distributed; we want programs (as many as possible) to be in compliance with these basic health and safety rules.  And this is usually what happens.  But from a measurement standpoint, it creates difficulties in analyzing the data.

By having severely skewed data, it is difficult at times to distinguish amongst the data between mediocre programs and either higher performers or lower performers because there isn’t sufficient variance/separation in their scores.  When I first noticed this, I suggested the use of weights attached to each rule in order to increase the variance in the data.  This helps but is not sufficient in increasing the variance in the data.  Unfortunately, this will always be a shortcoming of licensing data.

I point out this above limitation for future researchers who will be dealing with licensing data so that they can be aware of this but also to look at other statistical solutions to this problem and as a discussion point within NARA with other members to be aware.

I started a discussion earlier this morning (the above post) in which I presented some issues with regulatory compliance/licensing data.  I don’t like bringing up issues or problems without at least proposing some solutions.  So here are some solutions to this problem regarding licensing data skewness.

One way is through weighting (I suggested this in my earlier post so let me expand here) which I have advocated for that introduces more variance in the data.  This helps and is the basis for risk assessment systems but it can only go so far because it is really a statistical manipulation where we are saying that all regulations are not created nor administered equally.  There are some regulations/rules that are more important than others; in other words, there are particular regulations/rules which reduce the potential risk of morbidity/mortality to clients if complied with.

Another potential solution, which I have observed in Pre-K programs, is the introduction of higher standards and their resulting influence on licensing compliance in general.  This may be a more effective way to deal with the problem with skewness in data.  If the data become more normally distributed because the standards are more stringent, this is a good thing.  I think with Pre-K standards being utilized in more states and the advent of Caring for Our Children Basics that we may see a change in data distributions.

A complementary issue that probably is a result of the skewness of data has to do with the non-linear relationship between regulatory compliance and program quality.  I have termed this relationship, Theory of Regulatory Compliance.  This relationship I first observed in Pennsylvania in the late 1970’s in early care and education (ECE) programs.  I have continued to find this relationship between regulatory compliance and program quality data which is unsettling from a public policy standpoint.  As a public policy administrator one expects that quality increases with higher levels of regulatory compliance, right.  But this non-linear relationship doesn’t support this conclusion – some of the highest quality programs are in substantial but not full regulatory compliance.  I have suggested that higher licensing standards may eliminate this plateau effect when a high quality Pre-K program is introduced in a state ECE delivery system.

It was because of this non-linear relationship between regulatory compliance and program quality that ushered in the introduction of licensing key indicators and risk assessment systems in attempting to make inspection visits more efficient and effective by balancing program monitoring with quality initiatives.

These results are from the ECE research literature base but I strongly feel that these findings are applicable throughout the human services field and possibly beyond into any regulatory environment, such as banking or environmental regulations, to name a couple of different venues.  This is more about finding the “right” regulations to monitor rather than finding “fewer or more” regulations to monitor.  By utilizing a risk assessment/key indicator approach, this could be a solution to the deregulatory paradox.

For the interested reader, many of my reports which highlight the results above can be found at http://RIKInstitute.com/ecpqim

Here is another discussion question that I have been giving a great deal of thought to in how the key indicator methodology can be used.  Generally, in the past, it has been based upon the compliance history (CI) for a specific provider.  Very high regulatory compliance makes a program eligible for the use of an abbreviated key indicator inspection (KI).  Very low regulatory compliance disqualifies a program for the use of an abbreviated key indicator inspection and generally leads to a more comprehensive full review of all rules (CI).

But there is another way to use the key indicator methodology.  It could be used as a screener where every provider in a state receives the abbreviated key indicator inspection (KI) and based upon the results (compliance with all the key indicators) either the program gets another abbreviated inspection (KI) the following year or it moves to a more comprehensive full review (CI) if non-compliance is found with any of the key indicators.

In summary form, it would look something like this:

Compliance History data (CI)  –>  If high, key indicator inspection (KI), or if low, full comprehensive review (CI).        (CI  –>  KI or CI).

Key Indicator as screener (KI)  –>   If high, key indicator inspection next year (KI), or if low, a full comprehensive review (CI).      (KI  –>  KI or CI).

The advantage with the screener approach is that all providers from the beginning get a chance to be measured via key indicators.   This could be looked upon by providers as initially more equitable in the application of key indicators.  Something to think about as we move forward in the future development of the key indicator methodology.

NARA Newslink Blog of the Month – Key Indicators, by Dr. Rick Fiene.

We often get asked….’What exactly are Key Indicators?’ and ‘Why should my state agency be interested’? This month, Dr. Rick Fiene, the creator of The Key Indicator Methodology has posted a blog to answer those questions. Read today and post your comments. And if your interest has been peaked, join the Key Indicator Circle – a be a part of the NARA community.

NARA has recently created a Key Indicator webpage (http://www.naralicensing.org/key-indicators) that should help state licensing administrators get additional information about differential monitoring, risk assessment, and key indicator systems. I would highly recommend anyone who is interested to check out the new website. It is listed under the NARA Resources Folder on the Menu, just click on Key Indicators.

Here is a pdf of the page which compiles the various reports and studies listed on the NARA webpage (NARA Key Indicator Reports & Studies Examples from Webpage).

RIKI – Research Institute for Key Indicators (http://RIKInstitute.com) has joined a select group of organizations in a strategic partnership with NARA – National Association for Regulatory Administration.  Here is the statement on NARA’s website:

Strategic Partnerships

NARA has developed a broad spectrum of strategic relationships that provide access to the most up-to-date information on child care and child welfare regulations at both the federal and state levels. NARA’s collaborative relationships with agencies and advocacy organizations include:

For more information, please email collaborations@naralicensing.org.

I recently updated the NARA Licensing Curriculum Licensing Measurement and Systems Course.  Here is a brief outline of the Course (Content (Webpage location)):

Licensing Measurement, Regulatory Compliance, and Program Monitoring Systems

Richard Fiene, Ph.D.

Please contact Dr Fiene if you have questions or comments:
Rick Fiene, Affiliate Professor, Penn State Prevention Research Center at rjf8@psu.edu, or riki.institute@gmail.com or RFiene@NARALicensing.org

Big news out of the Province of Ontario:  the Child Care Quality Assurance and Licensing Branch within the Early Years Division won their Ontario’s Ministry of Education Realm Award for Innovation for their Tiered Licensing System.  The REALM Awards recognize excellence and achievement in the Ministry of Education and the Ministry of Training, Colleges and Universities (the Learning Ministries).  Their Tiered Licensing System utilizes the Differential Monitoring, Key Indicator, and Risk Assessment Methodologies.  What is so exciting about the Ontario Tiered Licensing System is that it uses both key indicators and risk assessment approaches for their differential monitoring system.  Most jurisdictions use either key indicators or risk assessment but not both together.  The Ontario approach provides a blueprint for combining the two methodologies together in the most cost effective and efficient differential monitoring approach.  The NARA Press Release:  (narapressrelease-award 002).

Additional information about the award and project:

The Prix REALM Awards program formally recognizes Learning Ministries’ (Ministry of Education and Ministry of Training, Colleges and Universities) staff for exceptional and outstanding contributions to:

  • the services provided to Ontarians and/or,
  • making the Learning Ministries a better place to work

This year outstanding achievements will be recognized in five award categories: Collaboration, Customer Service, Diversity and Inclusion, Innovation, Leadership and Lifetime Achievement

Won for Innovation:

Eligibility: Nominees in the category must have developed a new way of doing or thinking beneficial to a business process, program, initiative, or work environment.

About the project:

As part of the modernization of child care, Ontario’s Ministry of Education has developed an innovative risk-based approach to child care licensing – tiered licensing. Tiered licensing is designed to maximize the effectiveness and efficiency of the licensing process by focusing ministry resources where it matters most – on centres that need help to achieve compliance and areas of highest risk to children’s health and safety and importance to their learning and development – with the goal of improving regulatory compliance in all centres. Tiered licensing is built on best practices from across North America, a robust methodology and a cutting edge IT solution.

More details:

Ontario’s Ministry of Education has developed an original, transformative and innovative risk-based approach to child care licensing called tiered licensing.

The tiered licensing approach has been designed in-house to maximize the effectiveness and efficiency of the licensing process with the goal of improving regulatory compliance and quality in all child care centres.  Under this approach, ministry resources will be targeted to areas where they matter most – on centres that need extra support to come into compliance and on areas of highest risk to children’s health and safety.  At the same time, the approach will free up resources to provide more in-depth support in the important area of child development and wellbeing.

The ministry is transforming how child care licensing is performed in Ontario through tiered licensing by engaging the expertise of front line staff, Municipalities and licensees and integrating best practices from across North America to develop a robust new methodology and a cutting edge IT solution.

Ontario will be the first province in Canada to adopt a comprehensive risk-based approach for child care licensing and is now on the map as a North American leader in innovative regulatory practices. Dr. Richard Fiene, a leader in the field of regulatory administration for over four decades and a consultant on the project, has referred to Ontario’s approach as a “blueprint that other jurisdictions should follow.”

A Canadian Perspective Implementing Tiered Licensing in Ontario

NARA 40 years of Milestones:

NARA Milestones

Maine is in the news for improvements to their child care licensing program.  Please go to the following link (Maine Licensing System) to learn more.

Here is a powerpoint presentation for researchers and statisticians which has all the math and logic modeling for ECPQIM.

Math/Logic Modeling of ECPQIM

Here is a pdf of the latest powerpoint presentation which has an evaluation and validation study of differential monitoring, key indicators and risk assessment methodologies:

RESEARCH INSTITUTE FOR KEY INDICATORS (RIKI) Contributions to the Human Services Field   

  • Early Childhood Program Quality Improvement and Indicator Model
  • Differential Monitoring Logic Model and Algorithm
  • Clustering/Herding Behaviors of Two Year Olds
  • Regulatory Policy based upon Clustering/Herding for Adult Child Ratios
  • Mathematical Model for Computing Adult Child Ratios
  • Child Development Program Evaluation Scale – CDPES
  • Theory of Regulatory Compliance
  • Instrument Based Program Monitoring
  • Human Service Program Differential Monitoring
  • Licensing Weighting/Risk Assessment – CFOCB and Stepping Stones
  • Licensing and Quality Key Indicators – 13 Indicators of Quality, HSKI
  • National Early Childhood Program Accreditation (NECPA)
  • Human Service Provider Mentoring/Coaching – CAECTI/ECELS
  • Pinging linked with Coaching and Individualized Learning

EARLY CHILDHOOD PROGRAM QUALITY IMPROVEMENT & INDICATOR MODEL (ECPQIM) KEY ELEMENTS (Publications)

  • The ECPQIM/DMLMA Model
    • International Regulatory Compliance Comparisons – ICEP
    • Licensing & Monitoring Publications – ASPE, OCC, CCQ, NARA, ZTT
  • Program Compliance
    • Caring for Our Children – NRC/AAP/APHA
    • Instrument based Program Monitoring – CCQ
  • Program Quality
    • Keystone Stars Evaluation – OCD; Validation – OPRE
    • Infant Toddler Mentoring/Coaching Program – CYCF, ASPE
    • Health Consultants Impact Infant Toddler Care – JPHC
    • National Early Childhood Program Accreditation – NECPA
  • Program Compliance x Program Quality
    • Theory of Regulatory Compliance – NEJHS
    • Child Development Program Evaluation Scale – CCQ
  • Risk Assessment
    • Stepping Stones – NRC/AAP/APHA
  • Key Indicators
    • Thirteen Key Indicators of Quality Child Care – ASPE
    • Head Start Key Indicators – OHS
  • Risk Assessment x Key Indicators
    • Caring for Our Children Basics – ACF
  • Child Development Outcomes
    • Quality in Child Care: What Does Research Tell Us? – NAEYC
  1. ECPQIM0:  1972 – 1974.  Regional Model; EMIS (Fiene etal, 1975)*.  This was the original conceptualization when I was a graduate student. (ECPQIM0/EMIS)
  2. ECPQIM1: 1975 – 1994. Qualitative to Quantitative; focus on reliability; data utilization linking monitoring to training/technical assistance systems; distinctions between program monitoring and evaluation; Key Indicators, Weighted Rules, & principles of licensing instrument design introduced. (Fiene, 1981; Fiene & Nixon, 1985).  This is the original article written describing the model and suggesting the use of differential monitoring.  (ECPQIM1/CCQ)
  3. ECPQIM2: 1995 – 1999.  Policy Evaluation and Regulatory Systems Planning added to model. (Griffin & Fiene, 1995).  (ECPQIM2/ZTT)
  4. ECPQIM3: 2000 – 2011.  Inferential Inspections & Risk Assessment terminology added to the model. (Fiene & Kroh, 2000).  (ECPQIM3/NARA)
  5. ECPQIM4/4+: 2012 – present.  Validation with expected Thresholds & Differential Monitoring formally added via a logic model and algorithm; Quality Indicators introduced. (Fiene, 2012, 2013b, 2015).  (ECPQIM4/DMLMA)

*These are the various editions/versions of the Early Childhood Program Quality Improvement and Indicator Models (ECPQIM0–4+) that I developed while a graduate student and then improved upon the original design.  All the citations can be found in the publications webpage which is part of this RIKI website. 

The next section below contains the most recent examples of ECPQIM key elements.  These are all projects actively going on presently (2016) in the Province of Ontario, Pennsylvania Chapter of the American Academy of Pediatrics, and the Penn State College of Medicine.

Here is a very important technical aspect of the Key Indicator Methodology that I want to share with researchers and statisticians.  There are many different cut points or thresholds that can be used to determine the high group from the low group in constructing the 2 x 2 matrix for the phi coefficients (Φ = (a)(d) – (b)(c)/sqrt (w)(x)(y)(z)).  Ideally, (a)(d) should be much higher than (b)(c).  In fact, (b)(c) should be as close to zero as possible.  For example, the high regulatory compliance group (a) could only be those providers who attain 100% regulatory compliance with all rules/regulations.  The low regulatory compliance group (d) could be those providers who attain 99% or lower regulatory compliance with all rules/regulations.  Or the high group could be 100-99% regulatory compliance and the low group could be 95% or less regulatory compliance with all rules/regulations.  In this approach the middle 50% of the data are not used.  I have reported in a previous technical report that a top 25% and a bottom 25% of compliance history for programs was the most optimum cut points.  It appears from two separate studies to test this hypothesis that this approach does appear to be the most effective and efficient dichotomization of the regulatory compliance data.

A study completed in New York bears this out where various cut points/thresholds were used.  Another study going on in Michigan (Centers, Family Homes, Group Homes) where various cut points/thresholds were used with the regulatory compliance data supports this contention as well.

For reaching me online, here are my email and website contacts at RIKI and NARA:

Richard Fiene, Ph.D., Research Psychologist
RIKI – Research Institute for Key Indicators LLC 
Senior Consultant for Licensing Measurement & Systems
NARA – National Association for Regulatory Administration

logos-riki-nara-psu-page-001

RIKI – Research Institute for Key Indicators LLC, is a Pennsylvania Limited Liability Company.

 

About Dr Fiene

Dr. Rick Fiene has spent his professional career in improving the quality of child care in various states, nationally, and internationally. He has done extensive research and publishing on the key components in improving child care quality through an early childhood program quality indicator model of training, technical assistance, quality rating & improvement systems, professional development, mentoring, licensing, risk assessment, differential program monitoring, and accreditation. Dr. Fiene is a retired professor of human development & psychology (Penn State University) where he was department head and director of the Capital Area Early Childhood Research and Training Institute.
This entry was posted in RIKInstitute. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

Gravatar
WordPress.com Logo

You are commenting using your WordPress.com account. ( Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. ( Log Out /  Change )

Cancel

Connecting to %s