Consulting for Business Services

Evaluating Your Current Tests

Test scores are used to make decisions. How do you know that your tests are providing you the information you need to make those decisions? Just because a test appears to have the right content does not mean it measures that content in a meaningful way.

There are standard procedures to evaluate tests. These produce hard evidence showing exactly how a test is working. If you are using tests to make important decisions, you should have this evidence in hand.

Making good tests is extremely challenging, even for trained and experienced testing professionals. As part of the test development process, tests are subjected to formal reliability and validity evaluation procedures that further inform the development process.

The typical requirements of the test development process are:

  1. A coherent plan for test development, with a rationale for how the test will be made and what it is designed to do.
  2. All test items should be written by experienced item writers who know the content
  3. All test items should be reviewed by independent content experts.
  4. All test items need to be piloted on a representative sample of test takers.
  5. The pilot data needs to be analyzed using standard statistical procedures.
  6. Each item needs to be evaluated based on the analysis results, and then accepted, rejected or sent for revision.
  7. Final test versions need to be designed, assembled, equated and converted to a meaningful scoring scale.
  8. Pass levels need to be set by a formal standard-setting process.

Every step of the test development process requires specialized assessment expertise. If your tests have not been through a similar process, it is unlikely they will produce accurate results. Decisions based on such poor tests will be inappropriate and costly.

Lidget Green can carry out evaluation of all tests, regardless of what they are measuring. If the content of the test looks good then, in most cases, a statistical analysis of the test data is sufficient to show whether the test is measuring ability on that content. This is not expensive.

If test content is a concern, we can also carry out a formal content analysis of any test, working with client content experts.

Given the huge cost of poor tests, our test evaluation services are a very prudent and inexpensive investment.