Application 1 & 2
Application 1 – Evaluation of Two New Assessment Methods for Selecting Telephone Customer Service Representatives 1. The reliability for both the clerical and the work samples are favorable. We see in the Clerical Test that the co-efficient alpha (at . 85 and . 86) and the test-retest (. 92) are both high so we have an acceptable degree of reliability for the test. For the Work Sample (T) & Work Sample (C) we find the inter-rater agreements to be high at with Work Sample (T) being T1=88% and T2=79% and Work Sample (C) T1=80% and T2=82%, both above the 75% normal requirement for a reliable measure.
Since all three show a decent level of reliability, it would be safe for Phonemin to use them in selecting new job applicants. 2. The correlations between the tests show a low correlation between the Clerical Test and both of the Work Samples; however, there is a high correlation between the two Work Samples. The Work Samples produce very similar outcomes, with error rate and speed being non-significant and complaints significant.
With the Work Samples producing similar results, those who were tactful also show concern for the customer, Phonemin should just choose one of the Work Samples along with the Clerical test to select the best applicants for new job applicants due to the similar results. 3. Current CSRs were chosen to participate in the study, since they are already in the job their effort on the test could be different than a new applicant and they most likely have gained some additional knowledge, especially when handling complaints, having been on the job for a period of time.
Also, are the measures real indicators of performance? It was mentioned that the KSAOs chosen were “likely to be necessary for successful performance as a CSR” and they “their seemingly high impact on job performance” if they are not true indicator, then the test fails to predict the performance that Phonemin is looking for. Application 2 – Conducting Empirical Validation 1. Is the PS score a valid predictor of performance as a store manager? Yes. A correlation (R-Value) of . 407 exceeds the statistical significance threshold of . 7 and indicates a moderately high correlation between the PS scores and performance as a store manager. The following scatter diagram illustrates this correlation visually. While there are 2 outliers, their presence is not significant enough to disprove the correlation and the use of the PS score as a valid predictor. 2. With a cut score of 7 on PS, would its use lead to adverse impact against women? Yes. A cut score of 7 leads to a selection rate in females of 53. 33% versus 56. 67% in the entire sample. Against minorities?
No adverse impact to minorities. Selection rate of minorities is actually higher than non-minorities when a cut score of 7 is used. If there is adverse impact, does the validity evidence justify use of the PS anyway? Because the sample size is relatively small, the adverse impact to women seems insignificant and not enough to justify discontinuation of using the PS scores as predictors of future performance. 3. There are several limitations of this study. The first one would be related to the correlation (R-value).
The correlation does not equate to a percentage. The R-Value calculated above of . 407 does not equate to 40. 7% but rather it equates to 16. 56% (. 4072 X 100) common variation of the scores. A second limitation of the R-Value is that it is affected by the amount of variation in each set of scores. The less variation there is in one or both sets of scores then the smaller the calculated R-Value will be. When there is a lack of variation in scores this leads to what is known as the problem of restriction of range.
A third problem that may arise is that the formula used to calculate correlation assumes a linear relationship between the two sets of scores; however, this assumption may not always be true. A relationship may exist other than a straight line that may capture the nature of the scores better. It is also important to remember that just because there is a correlation between two scores does not mean that there is a causation between them. The R-Value does give you an indication; owever, of the strength of the relationship between the PS score and the performance rating and the direction of the relationship. 4. It is recommend that Yellow Blaze use the PS score in making future promotion decisions. It does seem to be a valid predictor of performance based on the correlation (R-Value) and the scatter diagram. Also, based on the calculations performed using the PS score for making future promotion decisions would not lead to adverse impact on a widespread basis.
