Overview
The General Education Assessment Committee (GEAC) developed a process for assessing all goals of the 2010 Revised General Education Curriculum, plus critical thinking (a State Council of Higher Education in Virginia requirement), on a five-year cycle. During the assessment phase of the five-year cycle, student artifacts are collected that align with the General Education and critical thinking student learning outcomes (SLOs). GEAC then convenes an annual Assessment Summit with the goal of training and calibrating faulty raters to assess these student artifacts, rate the artifacts, and draft reports of what is done well and what may need improvement in the competencies.
Faculty members in the General Education disciplines serve as raters of student artifacts for the annual Assessment Summit.
Members of GEAC lead a calibration session with faculty raters to ensure that raters have common definitions of the student learning outcomes and can find where the outcomes are demonstrated. The calibration process takes about three hours to complete. The steps are as follows:
- Review the 2010 Revised General Education Goals and SLOs for the competency being assessed
- Review the rubric, both the scale ("Exceeds Standard," "Meets Standard," "Approaches Standard," and "Needs Attention") and each individual student learning outcome
- Read and rate sample artifacts
- Facilitate group discussion of ratings given on sample artifacts
- Repeat rating and discussing sample artifacts until raters do not differ by more than one point on their assessment of each level of SLO achievement
Raters are instructed to use analytical scoring, whereby they read the artifact in its entirety one time and then score each SLO individually. Two raters read and rate each artifact. Members of GEAC monitor the incoming ratings and identify those artifacts on which raters disagree by more than a point in their assessment of a majority of SLOs. A third rater is then assigned to rate the artifacts identified.
Raters work independently for approximately 6-8 hours over two days. GEAC members monitor the rating sessions and answer questions as they arise. Data tables of their work can be found within the assessment reports.
Faculty members in the General Education disciplines serve as raters of student artifacts for the annual Assessment Summit.
Members of GEAC lead a calibration session with faculty raters to ensure that raters have common definitions of the student learning outcomes and can find where the outcomes are demonstrated. The calibration process takes about three hours to complete. The steps are as follows:
- Review the 2010 Revised General Education Goals and SLOs for the competency being assessed
- Review the rubric, both the scale ("Exceeds Standard," "Meets Standard," "Approaches Standard," and "Needs Attention") and each individual student learning outcome
- Read and rate sample artifacts
- Facilitate group discussion of ratings given on sample artifacts
- Repeat rating and discussing sample artifacts until raters do not differ by more than one point on their assessment of each level of SLO achievement
Raters are instructed to use analytical scoring, whereby they read the artifact in its entirety one time and then score each SLO individually. Two raters read and rate each artifact. Members of GEAC monitor the incoming ratings and identify those artifacts on which raters disagree by more than a point in their assessment of a majority of SLOs. A third rater is then assigned to rate the artifacts identified.
Raters work independently for approximately 6-8 hours over two days. GEAC members monitor the rating sessions and answer questions as they arise. Data tables of their work can be found within the assessment reports.