CAEP Annual Reporting Measures

Measure 1: Completer Effectiveness (Initial Licensure): 1) completer impact in contributing to PreK-12 student learning growth AND 2) completer effectiveness in applying professional knowledge, skills, and dispositions. CAEP Standard R4.1.

Because Virginia does not provide teacher or student performance data to education preparation programs (EPP), Virginia schools and universities of education have collaborated on coordinated data collection from program completers to fill this gap. The Virginia Education Assessment Collaborative (VEAC) was developed to develop instrumentation and coordinate data collection across participating EPPS. The initial licensure completer and employer surveys were developed in alignment with both the Interstate Teacher Assessment and Support Consortium (InTASC) core teaching standards and the Virginia Uniform Performance Standards (VUPS). The surveys use the same 15 items to permit comparison of completer self-report and employer ratings. This survey provides valuable data about the perceptions of completers about their preparation experience and their ability to apply the professional knowledge, skills, and dispositions taught in their preparation programs.

Our response rates have been low (10% in 2020-21, 8% in 2021-22, and 12% in 2022-23). This is in part reflects challenges with access to updated completer contact information, which became more challenging with the departure of a key VDOE employee responsible for providing EPPs with data about where in VA our completers are working. VDOE has now resume providing that data in winter 2023-24. That was not in time to support our data collection for the 2022-23 cycle, but will be helpful to support our data collection efforts in future cycles. Because our response rate is lower than that of our peers, we have reviewed our practices and developed new strategies for increasing response rates. We have created an online portal linked to our licensure verification form that will enable us to collect completer emails and current employment information (including those outside of Virginia). This will better enable us to remain connected to our completers as they enter the workforce. In addition, we continue to communicate the value of completer feedback in guiding our ongoing improvement efforts. 

In addition to data collected through the VEAC consortium, we are in discussion with our school communities to begin collecting data on our completers’ instructional effectiveness.

Completer Impact

Although student-level performance data are not currently available, Item G on the completer and employer satisfaction surveys: “Work results in acceptable, measurable, and appropriate student academic progress” informs the question of completer impact on PreK-12 student learning growth. As summarized in Table 1 below, in 2022-23 completers reported a mean rating of 3.25 (proficient). Their employers rated our completers slightly higher (3.29). ODU’s ratings are comparable to those of the other 31 participating Virginia EPPs, with a VEAC completer mean of 3.28 and VEAC employer mean of 3.24. Although the low n limits extrapolation to completers overall, the high ratings by both completers and their employers indicates that our completers are engaging in practices that are yielding positive, measurable impacts on student learning. We will continue to evaluate these data to monitor trends and identify areas for improvement. By collecting division-level teacher effectiveness data in future cycles we anticipate being able to provide more granular data to inform completer impact.

Table 1: VEAC Completer & Employer Item G: Completer Impact

Rating Scale 1-4 (1: unacceptable; 4: exemplary)

Completer Effectiveness

Completer effectiveness reflects the extent to which the completer has transferred the professional knowledge, skills, and dispositions from their preparation experience into their work in the classroom. The VEAC completer and employer surveys described previously reports on aspects of completer performance aligned with InTASC standards and the Virginia Uniform Performance Standards (VUPS). Completers reported high levels of preparedness across all items A-N, with means for all ratings between 3.06-3.50. Their employers also rated completer effectiveness at proficient levels. Completers reported highest levels of preparedness for professional ethics (3.47), reflective practice (3..42), and instructional technologies (3.41). Completers reported lower levels of preparedness for using data to guide instruction and feedback (3.06) and demonstrating subject matter knowledge and the developmental needs of students (3.16). Employers reported highest levels of satisfaction with professional ethics (3.44) and commitment to equity (3.40).

Table 2: VEAC Completer & Employer Survey: Completer Effectiveness

Rating Scale: 1-4 (1: unacceptable; 4: exemplary)

Note: Items I and K were omitted from the 2022-23 survey.

Employer satisfaction with program completers and stakeholder involvement. CAEP Standards R4.2, RA4.1, R5.3, and RA5.3

Initial Licensure

As demonstrated in the rating of completer effectiveness reported in Measure 1, ODU’s initial licensure employers have rated completers working in their schools at the proficient level across all items. In addition, employers were asked to respond to the following prompt: “based on your experience with this teacher, what best describes the extent to which they were ready to meet the needs of your students in your school?” Employers have rated ODU completers highly, with a mean of 4.41 (mostly ready (able to successfully meet the needs of most students).

Table 3: VEAC Employer Survey: Satisfaction

Rating scale 1-5 (1: not ready (unable to meet the needs of students even with additional supports) - 5: fully ready (able to have an immediate impact on student learning)

Advanced Licensure

In ODU’s four advanced licensure programs (school leadership, reading specialist, math specialist, and school counseling), programs have relied primarily on feedback from advisory board members and key school and division partners. ODU has begun participating in VEAC’s advanced licensure survey of completers and employers, and we anticipate reporting the first cycle of data in 2024-25. Our programs continue to meet regularly with their school and district partner stakeholders to discuss program developments and to seek feedback about their programs. Some examples include:

• School Counseling: a survey asked employers to provide an “overall assessment of the graduate’s ability as a counselor, on a scale ranging from 1 (harmful) to 5 (exceeds). All 5 respondents rated their ODU school counselor completers at the 5 (exceeds) level.
• Math Specialist: This program is cohort-based in partnership with 2 school districts. Both partners rated their satisfaction with the program preparation highly. They noted the convenience and quality of the online instructional environment and noted that the mat specialist completers entered their work ready for their roles. They provided constructive feedback that is informing ongoing programmatic improvement:
  • Provide increased support to prepare secondary teachers to adapt successfully to the elementary environment
  • Add cotaught courses to include a math content professor and teacher education pedagogy expert
  • Add professional interview module to better prepare completers for entering the professional market in their new roles (district offered to provide their administrative personnel to participate in mock interviews)
  • Provide additional information to practicum supervisor to strengthen connections between coursework and field experience.

Stakeholder Involvement

Initial Licensure Programs

ODU actively engages our school and division partners in many aspects of our program design, evaluation, and innovation development. Some notable examples include:

• Paid Teacher Residencies: using funds from the Virginia Department of Education ODU continues to partner with several school divisions in the Hampton Roads region (Southeast VA) to provide rigorous, year-long paid residencies to prepare special education and STEM teachers for high-need schools. In 2022-23 ODU partnered with Chesapeake, Newport News, Norfolk, and Virginia Beach Public Schools. This program, which uses a culturally responsive pedagogical framework, places talented aspiring teachers in a classroom where they are mentored and co-teach alongside trained, skilled mentor teachers. By funding tuition, licensure fees, and providing a $20,000 stipend, the program has been successful in recruiting, preparing, and retaining a diverse cadre of skilled teachers. ODU works closely with division partners in selecting the teacher residents and placing them in schools, training and supporting mentor teachers, and collaborating across the division and university to ensure a close integration of theory and practice.
• Cohort-Based Programs: ODU partners with seven divisions to prepare specialized cohorts of initial licensure candidates (special education general curriculum, STEM education) and advanced licensure candidates (educational leadership, math specialist, reading specialist). These cohorts are developed to meet the specific priorities and needs of each division context.
• National Science Foundation Noyce Grant: ODU is the recipient of a $1.1 million NSF Robert Noyce Teacher Scholarship Program grant that is funding a partnership with Norfolk Public Schools to develop an innovative grow-your-own program for math and science teachers in high-need communities. This program is focused on identifying, recruiting, and support gifted math and science students to persist through high school and college to prepare to be STEM teachers.

Advanced Licensure Programs

Candidate competency at program completion. CAEP Standards R3.3, and RA3.4.

Initial Licensure Programs 2022-23

• Number of completers in program leading to initial teacher licensure: 235
• Completer Licensure Test Pass Rates:
  • Virginia Communication & Literacy Assessment (VCLA): 100%
  • Praxis II subject exams: 100%

Advanced Licensure Programs 2022-23

• Number of completers in programs leading to advanced teacher licensure: 124
• Completer Licensure Test Pass Rates:
  • School Leadership Licensure Assessment (Educational Leadership): 100%
  • Reading for Virginia Educators (Reading Specialist): 100%
  • Comprehensive Professional Counseling Exam (School Counseling): 100%

Ability of completers to be hired in education positions for which they have been prepared.

Old Dominion University has historically tracked completer employment data by drawing on 3-year data provided by the Virginia Department of Education. With changes in staffing, those data have been less consistently available and complete over the past several years. This has significantly impacted our ability to track our initial and advanced completers from year-to-year. Further, limitations in state data management and tracking further constrain this analysis. After a multi-year hiatus, we received these data late this year. We are currently tracking our completers to provide more granular data moving forward. In addition, we have built in a data tracking component to get more information from our completers as they are graduating so that we may maintain contact and stay apprised of where they are teaching.