Ongoing collaborative item development provides a continual supply of new items.
163 new items have developed in 2009-2010 that assess higher levels of cognitive complexity.
The item bank covers elementary, middle and high school levels and includes:
Different types of items have been developed for the HEAP:
How the Assessment Items Were Developed
A key goal of the CCSSO~SCASS HEAP was to develop high-quality assessment items. The project developed selected-response items (i.e., multiple-choice), constructed-response items (i.e., short answer and extended response), and performance tasks. Performance tasks are curriculum-embedded projects grounded in authentic student experiences. They support skills-based, standards-based classroom instruction.
To help ensure the quality of the items, the project followed a rigorous process during item development. ACT, Inc., provided support and guidance as a subcontractor to the project. This section will focus on the process to develop approximately 1,400 assessment items in Phase II.
The project developed two types of assessment units: modules and performance tasks. Each module consisted of eight selected-response and three constructed-response (two short answer and one extended response) items. All modules and performance tasks were developed around one of the descriptors, which identify the health content that is linked to specific skills for student assessment in the HEAPĂs Assessment Framework.
Development of Modules and Performance Tasks
Participating states were asked to recruit item writers to develop the modules and performance tasks. The item writers, primarily practicing teachers and curriculum specialists, attended workshops where ACT, Inc., assessment development staff provided training in best practices for item writing. The writers were given the following guidelines:
Following the training workshop, writers were given time to draft materials. The draft materials were then shared in a conference with both a health education specialist and an assessment specialist. In the conference, item writers were provided feedback on the materials they had developed.
Item writers continued their work, completed the first drafts of their assignments, and submitted their materials for review. A critique of the draft materials, consisting of suggestions from both a health consultant and an assessment consultant, was sent to each item writer. The critiques offered advice on how to revise materials further. Once revised drafts were received, staff from ACT, Inc., edited the material in preparation for external reviews.
Item Reviews
The next step required review of all of the assessment items by a large number of professionals. All items were reviewed from both a public health and school health perspective by health educators, representatives from the member states, assessment experts, individuals from CDC/DASH, and members of the HEAP Technical Advisory Group. Reviewers were asked to evaluate items based on a number of criteria, including accuracy, grade appropriateness, and fairness. Staff from ACT, Inc., and the HEAP then reviewed all comments and revised the items as necessary.
Pilot Testing and Scoring
As part of the project, the assessment items were informally pilot tested. Participating states recruited approximately 1,600 classrooms in elementary, middle, and high schools for this pilot test. The following table shows the states and counties that participated in the pilot tests.
States and Counties Involved in the Pilot Tests:
|
|
|
|
Following the pilot tests, all selected-response and constructed-response items were scored. Professionally trained readers scored the studentsĂ responses. Item statistics were prepared and analyzed to serve as the basis for a final review of the materials. Project staff and state representatives reviewed all items again. Following this review, the items and constructed-response scoring criteria were revised and delivered to CCSSO.
As part of the development process, the items were submitted to a multistage review. Reviewers included members of the State Representative Team, Steering Committee, and CDC/DASH. Following the reviews, the HEAP assessment items were pilot tested in classrooms. The selected response and constructed response items were then scored and summary statistics generated for each item. The items, along with the item statistics, were reviewed once more prior to the items being released to the HEAP member states.
The statistics from the pilot tests are available in Table View. The first eight columns in Table View contain information that identifies the item (item number, module number, module title, grade, item type, answer key, core concept, and skill). The statistics are provided in an additional 14 columns to the right of this information, so you may need to scroll right by using the bar near the bottom of the screen to view the item statistics.
Item statistics are available for core concept and/or skill. Not all statistics are appropriate for all items. In the item bank, statistics are differentiated between core concept and/or skill by a prefix of CC for core concept and Skill for skill. For example, the sample size (N) for core concept is presented as CC N and the sample size for skill is presented as Skill N.
This section includes examples of HEAP assessment items to illustrate some of the terms.
Item Pool or Item Resource Bank
Terms used to describe the collection of assessment items developed for the HEAP. The types of items are selected response, constructed response (short and extended answer), and performance tasks.
Selected Response
Commonly called multiple-choice, these items can be scored by machines. These items are also known by the acronym SR.
Selected Response Example Grade Level: Middle School Which of the following is NOT a good example of a refusal skill if someone offers you tobacco?
|
Short Constructed Response
Also known as a short answer (SA) response, the assessment question requires the student to write a short answer. Constructed-response items are performance-based assessments.
Short Constructed Response Example Grade Level: High School Plan a healthful picnic lunch menu for a hot, sunny day. Describe how you will pack the foods for your picnic and explain why you have chosen these packing methods. |
Extended Constructed Response
The assessment question posed requires a longer response. Also written as an acronym ER, for Extended Response, these are performance-based assessments.
Extended Constructed Example Grade Level: High School Family and consumer science students have been asked to create posters about food safety that can be posted in their classrooms. Create a poster that lists five rules that should be followed when food is prepared, stored, and/or served. For each rule, explain why it is important to follow the rule. Be sure to make your poster persuasive. |
Performance Tasks
Performance tasks are curriculum-embedded projects that students complete in or outside of class and over an extended period of time. They are performance-based assessments.
Perfomance Task Example Grade Level: High School Students are asked to demonstrate their understanding of the types of nutrition resources that are available to low-income families in their community. They do so by researching a number of resources, conducting interviews, compiling their information, and presenting that information in a written report. |
Anchor papers are scored student responses used to establish the points for the scoring rubric. (Scoring rubrics are a set of guidelines for scoring performance-based student work.) Anchor papers are the student responses chosen to represent the mid-point of each level of performance described in the rubric. They literally anchor the rubrics and are essential to the scoring system.
Exemplars are previously scored examples of student work used in training to illustrate the four score points. Many different exemplars illustrate the various types of responses students can produce in responding to a stimulus for each of the score points, such as the example that follows.
Exemplar Example Prompt Grade Level: High School Student Challenge Your challenge is to examine a mental health problem, such as depression and suicide, stress, or eating disorders. You will analyze the causes and symptoms of the problem, explore treatment options, and then provide a listing of community resources that might be helpful in treating the problem. Your final product will be a brochure or presentation, including a list that actually can be used as a resource by individuals needing help or information to enhance their mental health. Assessment Criteria You will be assessed on your ability to demonstrate concepts and skills to help cope with the stresses of everyday living. Your project must include appropriate strategies for managing stress. |
The following provides an example of scored student work.
Exemplar Example: Student Response
Exemplar Example: Student Response, continued
Scoring Rubrics
The HEAP has developed a holistic scoring system that applies to all of the performance-based assessment items developed for the project. HEAP uses a four-point scale for scoring both content and skills. For example, a score of four for concepts reflects a complex, accurate, and comprehensive response that shows breadth and depth of information, describes relationships, and draws conclusions. A score of one reflects little or no accurate information about the relationships between health concepts.