Administration considerations for reliable and valid assessments

Reliability and validity in assessment are of crucial importance.  Issues in administration can have a significant impact on the reliability and validity of assessment results and the inferences that are made about students’ performance based on these results.

It is essential therefore that proper techniques of administration be adopted in ensuring that the integrity of assessments is preserved and that the inference made about students’ performance are indeed reliable and valid.

Administration considerations will differ based on the nature of the assessment, whether it is formal or informal. For formal assessments there are usually pre-established guidelines for the examiner to oversee the administration of the assessment instrument. It is important that the examiner familiarize him/herself with the proper techniques of administering the specific test so that the reliability and validity of the test can be ensured and preserved.

While the assessment may not require formal training to develop competence the test administrator must understand the protocols of the test, understand what is expected of the students, and follow correct and exact procedures during test administration to ensure that equal and fair conditions are provided for all students.

McLoughlin and Lewis (2005) caution that “measures administered by untrained testers produce highly questionable results that must be considered invalid” ( p. 90). It is therefore essential that examiners are comfortable with all exam procedures and follow these explicitly to avoid invalidating the results produced from such tests.

Additionally the tester must also be sensitive to the needs of the students during assessment. The examiner cannot “praise, prompt or give information” (McLoughlin & Lewis, 2005, p. 94). Examiners must understand correct protocol during test administration. Certain practices should be avoided.

The examiner should refrain from confirming or correcting (McLoughlin & Lewis, 2005, p. 95) students’ responses during the test as well as they should avoid coaching students in order to elicit correct responses. Depending on the purpose and nature of the test the examiner may be permitted to repeat oral items for students.

The examiner should also encourage students to complete or respond to all items on the test and may use questioning during tests to encourage students to respond to items. Evidently the ability of the tester to adequately preside over the test is a very critical issue in ensuring that test results are valid and reliable.

The test administrator sets the pace and the outcome of the test pivots around his/her ability to ensure transparency. By ensuring that the tester is adequately prepared for the test the possibility of incorrect administration procedures will be avoided.

The test environment should also be such as provides an atmosphere in which the students can work successfully. As McLoughlin and Lewis (2005) indicate, “this environment can influence test performance” (p. 90).

The test environment should be as comfortable as possible for all students, with good ventilation, adequate and comfortable seating for each student where necessary, adequate space for writing, appropriate lighting, comfortable temperature, minimal external disruptions, minimal distractions such as a view through a window, passersby, other individuals passing freely in and out of the examination room or other visual distractions such as posters.

Additionally the examiner has to ensure that all material required for the test are present including material students will need to complete test items.

While it is ideal that the test environment is free of all these encumbrances, all these conditions do not have to be satisfied to ensure the validity and reliability of the test. In some instances some aspects of the environment, though they are not at their optimal levels, may not affect students’ performance.

However, it is difficult to determine how students’ performance will be affected by the environment and thus the examiner should attempt to ensure that the setting is as close to ideal as possible. The examiner could also exercise discretion during test administration to determine if the location of the exam needs to be changed or the exam discontinued completely because of environment. It is not far fetched that a well-prepared environment becomes inappropriate at some point during the exam.

Based on the reaction of the students the examiner could determine if the distractions or disruptions are sufficient enough to affect student performance. These provisions ensure that, as far as possible, no external factors negatively impact student performance and that students are adequately provided with the right tools with which to reap success.

Another consideration is the preparation of the students for the test. Students should be adequately informed of the time, date, location and purpose of the test as well as be aware of what exactly is required of them.

The time of day is of extreme importance. Researches have been conducted to determine the time of day that will produce the optimal results for students. Recently Hansen, Janssen, Schiff, Zee and Dubocovich (2005) found that students were more vigorous during the evening than in the morning and thus were in a better frame of mind to complete tests.

It is difficult, however, for examiners of standardized test to have any impact on the time of administration since often the examining body has a preset time for completion.

Another administration consideration is the rapport between the students and the examiner. McLoughlin and Lewis (2005) recommend that the examiner familiarizes him/herself with the students and help them  feel at ease with the examiner. The examiner should also explain the purpose of and orient them towards the test. This includes describing test activities, length and difficulty of the test.

The examiner should also inform students of proper exam protocol, informing them how to ask for assistance during the test as well as explain to them any timing signals that may be given. The examiner should encourage students to ask questions about aspects of the test administration that they are uncertain about.

Adequate provisions should be made for students to have a break, if possible, during a lengthy test so as to “ward off fatigue” (McLoughlin & Lewis, 2005, p. 99). These practices would ensure that students know what is expected of them before the test begins so that no time is wasted during the test.

The procedures for administering informal classroom assessment are much the same. The classroom teacher should ensure that students are adequately prepared for the test, are aware of the test objectives, are given adequate time to complete it and that no unfair assistance is provided to any student.

Also the teacher should make sure the environment is conducive to testing with convenient seating and minimal opportunity for cheating. Additionally in administrating certain types of assessment such as observations and interviews, the teacher has to ensure that the objectives to be tested are clearly laid out, with clearly outlined criteria for scoring so that bias is avoided and each student is scored fairly.

The use of rating scales, rubrics and checklists help to ensure that assessment is focused and that the correct behaviors are being observed. However at times a teacher may later find that these instruments do not adequately measure all the attributes and that certain key attitudes are not catered for.

This is one disadvantage with rating scales and checklists, if a student displays a positive behavior that is not outlined in the rating scale no provision is made for it and the child is disadvantaged.

The purpose of assessment is to ensure that each students has an equal chance of performing at their best in the test and thereby demonstrate to the examiner whether or not they possess certain skills, knowledge or attitudes.

Therefore once the examiner adopts these administration considerations the possibilities of measurement error are diminished and test administration conditions will have minimal, if any, negative effect, on student outcome. Students will therefore be able to meaningfully demonstrate their knowledge, skills and attitudes to the teacher.

References

Hansen, M., Janssen, I., Schiff, A., Zee, P. C. & Dubocovich, M. L. (2005). The impact of school daily schedule on adolescent sleep. Pediatrics, 115, 1555–1561.

McLoughlin, J. A. & Lewis, R. B. (2005). Assessing students with special needs. (6th ed.). NJ: Prentice-Hall/Merrill.