Alignment studies evaluate test questions that are being used in high-stakes assessments to help ensure that the test questions actually measure the intended standards. MetaCat's capabilities for online rating and real-time reporting made it the perfect tool for implementing alignment studies more efficiently and cost-effectively.
Key cost-saving and efficiency factors:
- Alignment-rating teams could enter their alignments online, but without seeing the other teams' ratings.
- During the alignment rating process, project managers could track the reliability of the raters and rating progress with real-time monitoring. This enables the project managers to intercept more quickly when inter-rater reliability issues occur.
- Custom reports provided additional alignment analysis, saving time and money in the final reporting process.
The screen shot above shows a partial real-time monitoring report displaying standards ratings
between raters. It also indicates when raters agree and when they do not. These are noted with a red "No" under the Agree column. The interactive reports enable administrators to enter a 3rd rating when the first 2 raters do not agree.
Examples of Solutions
Powered by MetaCat®