Educator Preparation Program (EPP) Accreditation

Measure 1: Completer Impact and Effectiveness

As part of our continuous improvement efforts, Mayville State University’s EPP uses multiple instruments to measure completer effectiveness and impact on P-12 learning and development.  Indicators of teaching effectiveness are measured in a variety of ways, including completer surveys, supervisor surveys, and supervisor evaluations.

The EPP’s use of multiple measures provides a more complete description of student growth and completers’ success in the classroom and followed recommendations of best practices to investigate teaching effectiveness.  For further information, please refer to the manuscripts found in the attachments at the end of this measure.

Each spring, findings on completer satisfaction are collected from a survey administered to teachers the academic year following their graduation.  All completers/graduates are invited to complete the Transition to Teaching Survey (TTS), but those who are currently teaching complete an additional section to rate the quality of their preparation.  The TTS asks Mayville State University (MSU) graduates to evaluate their perception of the quality of their instructional practices, their ability to work with diverse learners, ability to establish positive classroom environment, and level of professionalism.  The survey is administered approximately one year after the graduates complete their preparation program.

On the survey, completers’ satisfaction with their preparation being relevant to the responsibilities they confront on the job and their preparation being effective are indicated with tends to agree (3) and agree (4) ratings.  Mayville State University’s Teacher Education Program has set an acceptable target of an average of 3.0 or higher on all indicators (all tends to agree or agree).  EPP results are also compared to North Dakota and super aggregate results to ensure commensurate scores, ultimately aiming for higher scores than both the North Dakota and super aggregate.  Results of the survey are reviewed annually by the Division of Education and Teacher Education Committee faculty.  The most recent results can be found in the table below:

In the spring of 2021, the EPP reached out to all graduates from the previous academic year and asked them to complete the common metrics Transition to Teaching Survey (TTS).  The EPP chose to add several supplemental effectiveness questions to the end of the TTS to gage our completers’ impact on P-12 learning and development.  One of the last questions on the TTS asks participants if they would be interested in sending their supervisor evaluations from their most recent academic year.  Of the 35 respondents to the supplemental effectiveness questions, 28 indicated they were willing to submit supervisor evaluations, and 10 actually submitted when the follow up request was sent. Of those 10, eight were undergraduate students and two were graduate students.  The eight undergraduate students included seven elementary education majors and one secondary math major.  The two graduate students included one secondary science and one secondary social science major. No early childhood or special education graduates sent supervisor evaluations to the EPP for analysis.  All 10 completers who submitted supervisor evaluations received favorable evaluations from their supervisors. A qualitative analysis of results indicated the following:

• Overall Elementary Education Findings: Supervisor survey results indicate that elementary education majors are meeting the expectations of their supervisors.  A common theme reported by most supervisors is that EPP completers are proficient at building relationships with students.  Supervisors also agree that completers design coherent instruction and keep students engaged in a variety of ways.  Supervisors report that completers consistently establish rules and procedures in order to build positive learning environments for their students.  All supervisors shared positive comments regarding EPP completers’ ability to meet the demands that came along with COVID-19 and distance learning.
  
  
• Overall Secondary Education Findings: Supervisor survey results indicate that secondary education majors (BSEd and MAT) are meeting the expectations of their supervisors.  A common theme reported by all supervisors is that EPP completers are proficient managing classroom procedures and student behaviors.  Supervisors also agree that completers deliver instruction in engaging ways.  Supervisors report that completers are proficient at family and community outreach and are capable of meeting professional responsibilities they will confront on the job.

One final, comprehensive measure related to completer effectiveness is the case study research that the EPP has engaged in.  As a strategy to assess the effectiveness of completers in the classroom and impact on P-12 student learning, case study was used as an alternative to value-added measures (unavailable at the state level in North Dakota) to holistically examine complex attributes of quality teaching (Merriam, 1998; Yin, 2014).  The Reflective Experiential Teacher framework, developed by the EPP, was used to design methods and guide analysis.  The constructivist framework incorporates the knowledge, skills, and dispositions of the ten core teaching standards of the Interstate Teacher Assessment and Support Consortium (InTASC) (CCSSO, 2013) divided into the four CAEP Standard 1 categories of: Learner and Learning, Content, Instructional Practice, and Professional Responsibility. 

This case study was part of a larger investigation of the role EPPs play in graduates’ teaching effectiveness, among a myriad of other complex factors, and the role graduates’ effectiveness has on student learning.  Data was collected to answer the questions:

• Do P-12 students in the classrooms of EPP completers demonstrate expected levels of learning and development?
  
  
• What factors do graduates perceive impact students’ learning and development? and
  
  
• How can a EPP improve program quality based on results of P-12 student learning in classrooms in which graduates are teaching?

Mixed methods data collection in the descriptive study included: graduate and supervisor surveys, self and supervisor evaluation of skills and dispositions, interviews, document review, student engagement surveys, achievement data, and pre and post assessment data. Descriptive statistics were used to describe data for each case.  Within each case, the constant comparative method of qualitative data analysis was used (Glaser & Strauss, 1967) to construct codes, categories, subcategories or themes through continuous comparison of data (Merriam, 1998, p. 159).  Validity and reliability of measures within the case study are described in instrumentation in the manuscripts below.

Results indicated most students from participants’ classrooms demonstrated learning growth, and participants viewed their teaching as effective.  Implications for programs include pre-service opportunities to identify and respond to authentic student engagement, collaboration with colleagues, the impact of reflection on teaching practices, and opportunities to build collective teacher efficacy.  Additional insights describe pre-service teacher training to measure student progress and measure impact based on classroom assessments. The investigation provides a replicable case study design for teacher educators to examine relationships between teacher preparation, program graduates, and P-12 student outcomes.  Findings and results are presented in the finished manuscripts available on this webpage as attached files; these include detailed accounts of recruitment, methodology, research questions, and analysis.

Data cycles started with protocol development in 2016 followed by purposeful sampling of participants, implementation, refinement, and further data collection. A pilot study in Spring 2017 included two elementary completers. In Spring 2018, two completers from elementary education and two from secondary science education participated.  To represent all EPP program areas, participants in Spring 2019 were recruited from the remaining areas of early childhood, special education (SPED), and MAT. Recruitment scripts were sent to 11 eligible completers (early childhood n = 6; SPED n = 3; MAT n = 2); none agreed to participate. Of the three SPED graduates, only one was currently teaching in a special education position. All others were employed in a regular education setting. The one SPED completer teaching in a Special Education classroom did not consent to participate. The completers in MAT and EC noted their reasons for not participating: limited time due to other school-related obligations (i.e., coaching, assessment), personal responsibilities, and need for supervisor participation/consent. Further complications for participant recruitment came in March 2020 when the COVID19 pandemic shut down schools across the state and nation.  Because of the stress teachers were under and challenges they were experiencing (and continue to experience), the EPP decided to suspend the case study research and collect data that would regularly be collected from graduates including the Transition to Teaching Survey (TTS) and Supervisor Survey (SS). The TTS includes a question inquiring whether first year teachers would be interested in participating in further data collection to measure their impact on P-12 learning and development.  Of the 44 participants who completed the TTS, 15 first year teachers agreed they would be willing to send the EPP data manager their two most recent supervisor evaluations.  Again, due to the COVID19 pandemic in spring 2020, the EPP decided to postpone data collection from participants.   Beginning in the Spring of 2021, the EPP will continue previous efforts to collect annual measures examining completer performance in different grade levels and content areas.

Also of note, the EPP typically receives a “First Year Teacher Effect on Student Outcomes” report from State Longitudinal Data System (SLDS) in early spring.  This report shares information related to SLDS data for demonstrated growth in math, reading, and language arts of students in Mayville State’s completers’ classrooms.  The data is compared to North Dakota non-first year teachers and other North Dakota first year teachers and is shared visually.  Due to the COVID19 pandemic and closing of schools, SLDS did not collect comparative data for the 19-20 school year and therefore, the report was unavailable this spring and last spring.  Either way, the EPP continues to find the state-provided data as inadequate for evaluating program impact due to a limited sample, the inability of state data to be disaggregated by program or directly linked to completers, and test scores limited to reading, language arts and math which accounts for only a portion of EPP completers. In addition, only completers who remained in ND to teach have student data available.

Overall, the EPP continues to determine that state provided, teacher-linked student achievement data does little to inform continuous improvement efforts, and thus the EPP continues to pursue other sources of evidence (i.e., teaching effectiveness survey questions, supervisor evaluations, and case study research).

In conclusion, multiple measures support the fact that Mayville State University Educator Preparation Program completers' students did indeed demonstrate expected levels of learning. As a descriptive case study, the EPP did not attempt to infer direct causality of teacher preparation or effectiveness of teaching on the results of student outcomes; however, it appears completers contributed to positive student outcomes amid multiple influencing factors. Foremost, results from learner outcomes reaffirmed the EPP’s commitment to ensure every learner has a competent, effective, engaging, and reflective educator.

File Attachments:

1. Anderson, S.K., Hagen, B., Whitsel, C., Dulski-Bucholz, A., Smith, K., & Willeson, A. (2019). Leveraging case study research: A mechanism for measuring teaching effectiveness. Mid-Western Educational
    
2. Anderson, S.K., Hagen, B., Whitsel. C., Smith, K.D., & Duffield, S. (2020). Graduate impact on student learning: A descriptive case study.
    
3. Madler, A., Anderson, S. K., & Smith, K. (2020). Perceptions of teacher preparation for classroom diversity: A secondary data analysis.
    

Measure 2: Satisfaction of Employers and Stakeholder Involvement

Each spring, findings on employer satisfaction are collected from a survey administered to supervisors of first year teachers who graduated from Mayville State University (MSU).  MSU graduates complete a Transition to Teacher Survey (TTS) where contact information for their supervisor is collected and used to administer the Supervisor Survey (SS).  The SS asks supervisors to assess the quality of graduates’ instructional practices, abilities to work with diverse learners, abilities to establish positive classroom environment, and levels of professionalism. The SS is administered to direct supervisors, mainly principals, of EPP graduates employed in schools approximately one year after the completion of their preparation program at Mayville State.

On the survey, employers’ satisfaction with completers’ preparation for their assigned responsibilities in working with P-12 students is indicated with tends to agree (3) and agree (4) ratings.  Mayville State University’s Teacher Education Program has set an acceptable target of an average of 3.0 or higher on all indicators (all tends to agree or agree).  Results of the survey are reviewed annually by the Division of Education and Teacher Education Committee faculty.  The most recent results can be found in the table below:

Stakeholder involvement is an integral part of the success of Mayville State Teacher Education graduates.  With over 340 MOUs with schools in the region, state, nation, and even international connections, the EPP involves stakeholders in program design, evaluation, and continuous improvement processes.  This is done in several ways including displaying CAEP accountability measures on our website, meeting bi-annually with our Teacher Education Advisory Committee, and consistent communication with administrators and teachers when designing mutually beneficial partnerships and field experience placements.  Over the last academic year, the Teacher Education Advisory Council met and discussed feedback on a variety of topics.  Evidence to support stakeholder involvement, including TEAC agenda, presentation, and summary (minutes available upon request), can be found below. 

TEAC May 2021-Agenda

TEAC May 2021-Presentation

TEAC May 2021-Feedback Summary

Communication and collaboration with partners occur in both structured and organic ways.  Mayville State University’s Teacher Preparation Program implements a variety of field experiences where our teacher candidates can apply theory to practice.  We nurture our partnerships and have consistent relationships with our partners each year while allowing room to develop new partnerships that are mutually beneficial to the partner and our teacher candidates.  Below, you will find a list of several partnerships implemented over the 2020-2021 academic year, a brief description of each, and how stakeholders were involved in the design and implementation:

• Block 1 Field Experience – Early Childhood and Elementary Education majors participate in this field experience as part of their first block of methods classes.  Third graders from the local elementary school in Mayville, ND are invited to campus where mini-classrooms are created to welcome them for 2.5 weeks of learning.  Teacher candidates apply methods of instruction for reading, math, and science aligned to state standards selected by clinical partners, the principal and third grade teachers.  The faculty members who teach in these methods classes work closely with the school administrators and teachers to design a field experience that is mutually beneficial for both the school district and Mayville State.
• Block 2 Field Experience – Early Childhood and Elementary Education majors participate in a 5 ½ field experience in our local elementary schools. Each Block 2 student is paired with a cooperating teacher that they work from 8:00-11:30. On Mondays, Tuesdays, and Wednesdays, our students work with their cooperating teachers to pull small groups, design whole-group lessons, provide interventions, etc. Since our Block 2 students have their North Dakota Substitute Teaching License, they are provided the opportunity to sub in their field experience classrooms. This gives our students opportunities to design and implement sub plans. This also gives the classroom teachers opportunities to work in their PLCs, collaborate with other teachers, and plan future lessons. On the Thursday and Friday substitute teaching days, a university faculty member is at each of the schools to provide guidance and feedback to each of the Block 2 students.
• General Methods Field Experience – General Method students are paired with a cooperating teacher in an area school to complete a 30-hour field experience. During this field experience, they are to implement strategies they are learning during their General Methods course. They are required to write lessons plans and teach two lessons. They record these lessons and debrief with their General Methods instructor and the Field Mentors also provide direction on what lessons to teach and feedback on how the lessons went.
• Secondary Methods Field Experience – Secondary Methods students participated in an immersive week long observation/field experience learning opportunity at various schools across the state and region. Students were actively involved in student and faculty life during their time in their school placement. As a result of full day observations, student feedback was very positive. The teacher candidates had time to build relationships with both students and teachers. The experience of spending five consecutive days allowed our candidates to be immersed into the world of full-time teaching.
• Learning Disabilities Practicum – Students seeking a Special Education major or Special Education minor participate in this field experience as part of their requirement for their program of study. Students are required to complete 60 hours of field experience in a special education classroom serving students with learning disabilities. Teacher candidates apply methods of instruction provided in their Learning Disabilities course. The faculty members that teach the course work closely with the school administrators and teachers to design field experiences that are mutually beneficial for both the school district and Mayville State teacher candidates.

Measure 3: Candidate Competency at Program Completion

The table below identifies academic measures that provide data on candidate competency.  These measures include Praxis exam data and GPA.

The table below identifies non-academic measures that provide data on candidate competency.  These measures include Skills of Teaching Observation Tool (STOT) and Disposition Evaluation results from the student teacher cohorts during the 2020-2021 academic year.

Measure 4: Ability of Completers to be Hired in Education Positions for Which They Have Prepared

In the spring of 2021, the EPP reached out to all graduates from the previous academic year and asked them to complete the common metrics Transition to Teaching Survey (TTS).  In addition to addressing completer impact, the TTS asked licensure and job status questions.  The survey indicated that 95% of completers had applied for a professional teaching license.  Of those who had applied for a professional license, 87% applied for a North Dakota license, and 23% applied for a Minnesota license.  When asked in which state(s) do you hold a teaching license, 82% indicated North Dakota and 23% indicated Minnesota.  One completer applied for a job outside of teaching and stated “family or personal” as the reason.

When asked if they sought employment as a licensed teacher, 87% of completers indicated “yes”.  Most students (58%) indicated they had one job interview while 33% of students had 2-3 job interviews.  When asked how well prepared they think they were for their teaching job interviews 94% of students felt prepared.  For those that completed an interview, 100% reported that they were offered teaching positions and accepted them.

The following table depicts EPP completers’ current employment situation:

Additional data related to the ability of EPP completers to be hired in education positions for which they have prepared is collected through the First Destination Survey.  This survey is administered through the Career Services department on campus and not through the EPP.  A summary of results can be found below.