Some Technical Considerations in Using Complaint Data and Regulatory
Compliance Data: RIKIllc Technical Research Note #66
Richard Fiene, Ph.D.
January 2019
As promised in RIKIllc Technical Research Note #65, this Note will provide details on the
and analytical considerations when using complaint and regulatory compliance data together. As pointed out in the previous technical research note, using complaint data as a potential outcome appears to have merit and should be explored in greater detail. However, with that said there are some parameters that the methodology has that should be explored in order to make the analyses more meaningful.
When looking at regulatory compliance and complaint data there are four possibilities: 1) the facility is in full compliance and has no complaints; 2) the facility is in full compliance but has complaint(s); 3) the facility has some non-compliance and has no complaints; and 4) the facility has some non-compliance and has complaint(s). These four possibilities can be depicted in a 2 x 2 matrix:
Cell C = Full Compliance & No Complaints; Cell A = Full Compliance & Complaints (False Negative): Cell B = Non-Compliance & No Complaints; Cell D = Non-Compliance & Complaints. (See the attached Technical Research Note for a clearer picture of the 2 x 2 Matrix).
In the this 2 x 2 matrix, we would want to see cell C and cell D as the predominant cells and cell A and B as the less dominant cells, especially cell A because this represents a false negative result.
However, there are a couple of limitations to the above matrix that need to be taken into account. One, are the complaints substantiated or not. Any complaint must be substantiated to be counted in the model. If it is unsubstantiated, than it is not counted in the matrix. Two, there is the problem with directionality that needs to be addressed. For example, does the complaint occur before or after the full inspection in order to determine regulatory compliance. The 2 x 2 matrix and the modeling for these
analyses is based on the complaint occurring after the full inspection and that is the reason for cell A being labeled a false negative. If the directionality is reversed and the full inspection occurs after a complaint, cell A is no longer a false negative.
