Alignment Studies

Alignment studies evaluate test questions that are being used in high-stakes assessments to help ensure that the test questions actually measure the intended standards. MetaCat's capabilities for online rating and real-time reporting made it the perfect tool for implementing alignment studies more efficiently and cost-effectively.

Key cost-saving and efficiency factors:

1. Alignment-rating teams could enter their alignments online, but without seeing the other teams' ratings.
2. During the alignment rating process, project managers could track the reliability of the raters and rating progress with real-time monitoring. This enables the project managers to intercept more quickly when inter-rater reliability issues occur.
3. Custom reports provided additional alignment analysis, saving time and money in the final reporting process.

The screen shot above shows a partial real-time monitoring report displaying standards ratings
between raters. It also indicates when raters agree and when they do not. These are noted with a red "No" under the Agree column. The interactive reports enable administrators to enter a 3rd rating when the first 2 raters do not agree.