Annual Reporting Measures (CAEP Component 5.4 | A.5.4)
Hollins University program completers have demonstrated a positive impact on P-12 student learning as measured by improvements in their student test scores. The Virginia Department of Education does not make available to the public a specific set of P-12 scores associated with respective teachers nor have Virginia school districts adopted value-added measures (VAM). Instead in Virginia, most teachers prepare SMART goals to measure student progress and teacher impact. Virginia Code requires that student academic progress be included as an element of evaluating instructional staff performance. In 2011, guidelines for teacher evaluation were released and they included the use of SMART goals as a means of tying teacher performance and student progress together. These goals are S-pecific, M-easurable, A-chievable, R-ealistic, and T-imely and spell out the SMART goal acronym. A Smart goal incorporates all of these criteria and increases the likelihood of tracking progress and better ensuring success.
Eleven out of the thirteen teachers reporting SMART Goals had measured improvement in their students’ scores. While some of the instruments differed, the same goal process was achieved in each of these goal settings: identifying specific, measureable, achievable, realistic, and timely goals were set with teacher interventions, lessons, and strategies for optimal student learning. As evidenced by these improvements in test scores and by the program completers as a whole reaching their SMART goals, Hollins University program completers have a positive impact on student learning and development.
Teachers cannot make an impact on student learning if they do not embody the knowledge, skills, and dispositions needed to be successful. Through select coursework and gateways for teacher progression and monitoring, the Hollins University education department faculty mentor teacher candidates to be effective in each of these areas: planning and designing lessons, reaching learners where they are, selecting a variety of instructional methods, creating a supportive learning environment, and maintaining levels of teacher professionalism. In examining CAEP standard 4 for teacher effectiveness of our program completers, the Hollins University Education department selected to use classroom observation reports principals make during a program completer’s teaching assignment. This primarily qualitative instrument provides a window into the classroom of our program completers and a rich narrative about the effectiveness of our program completers in an authentic setting.
This group of principals indicated program completers had used multiple instructional strategies in teaching a lesson. Not one principal mentioned the program completer had only used one primary method of instruction such as lecture. In our teacher training program, we emphasize in every course that teacher candidates must build, mold, and shape lessons to include reinforcement activities, assessment opportunities, and higher level thinking skills. This is only achieved by connecting activities, building activities over a period of time, and varying instructional methods over the continuum of teacher-centered to student-centered instruction. Not only are our program completers striving for excellence, they set high standards for their learners, reach them where they are, and give them the tools and materials.
All four program completers from various grade levels and subject matters successfully have met each of the Virginia Professional Performance Standards (PPS) and, in turn, the ten InTASC standards. The results provided have included principal quotes based on first hand observations of teacher effectiveness for each of these standards: professional knowledge, instructional planning, instructional delivery, assessment, learning environments, professionalism, and student academic progress.
Program completers are effective in their instruction because they have solid content knowledge and the flexibility to look at content from different angles and tie in relevance. They can plan lessons that differentiate instruction, that reach higher level thinking, and that use a variety of instructional strategies. Our program completers assess frequently and meaningfully using variation in formative assessments and provide the opportunity for P-12 students to discuss and put concepts into their own words. Program completers work to create a supportive learning environment for all learners while modeling and maintaining professional conduct and dispositions. Program completers also monitor student progress and meet students where they are in their learning. All four program completers have met and excelled in each of these standards.
One of the strengths of the Hollins University training program is the emphasis on instructional planning. In every course, teacher candidates build multi-dimensional and multi-faceted units. They do not submit one single isolated lesson plan; they connect lesson plans into a unit of several weeks, and see how content builds on each other, how to engage learners, and, more importantly, how to reach higher level thinking skills needed for the 21st century learner.
The Satisfaction of Employers Survey asked principals to assess program completers’ performance in each of the Interstate Teacher Assessment and Support Consortium (InTASC) categories.
The ten InTASC competencies are:
- Learner Development
- Learning Differences
- Learning Environments
- Content Knowledge
- Application of Content
- Planning for Instruction
- Instructional Strategies
- Professional Learning and Ethical Practice
- Leadership and Collaboration
Over the three cycles of data collection (2018, 2019, and 2020), the average response rate was 53.36%, well above the minimum response rate of 20% for surveys CAEP recommends.
Composite scores were created by averaging the ten competency item scores. Possible average scores range from 1.0, indicating teachers were rated as “Needs Improvement,” to 4.0, indicating teachers were rated as “Exemplary.” An average score of 3 for a competency item would indicate that, on average, teachers were perceived as “Proficient” on the InTASC competency. Thus, a heuristic of 3.0 has been used as an indicator of effectiveness for CAEP Standard 5.
Average scores ranged from 3.30 (Assessment) to 3.45 (Application of Content and Professional Learning and Ethical Practice). All competency items had average scores above the heuristic of 3.0, which indicates that for all competencies surveyed, on average, employers were satisfied with Hollins University program teachers.
The results show teachers completing the Hollins teacher training program are successful. In each of the InTASC standards, the average scores for the program completer for a given year met the minimum proficiency level of 3.0. It should also be noted that the most recent year Hollins teachers were evaluated (2019-2020), principals in this group rated each InTASC category higher than any other year. This would indicate that the improvements the Hollins University education program are making are translating into direct results for teacher effectiveness in the P-12 classroom. Thus, we can be reassured the Hollins University training program is headed in the right direction.
Survey of Employer Satisfaction by Year
In the same Satisfaction of Employers survey after quantitatively rating each InTASC category, principals were asked the open-ended question- Are there any additional comments you would like to make? Consistent with the quantitative survey results, the comments were also very positive. Nine principals left comments in this section that included:
-“Mr. C is extremely strong in content knowledge, and he continues growing in pedagogy and classroom management.”
– “We believe that Ms. D will become an exceptional teacher, role model and building leader in time. With more experience and time to practice her craft, Ms. D will be great.”
-“Ms. B in her first year built the French program from 3 sections by providing engaging activities that promoted interest in the subject.”
-“Ms. K was an outstanding first year teacher. It was amazing to watch her grow. She will have many successful years teaching.”
-“Ms. K continues to do a nice job in her role as a science teacher. She has a good rapport with students and staff and creates lessons to promote student learning.”
-“Ms. L is an exceptional educator. When she served as a TA for our school, it was evident that her skill set was advanced for her position. Now, as a classroom teacher, her talents and dedication are amplified. Ms. L demonstrates skills, knowledge, and performance beyond her years. She is keenly aware of students’ learning needs and styles; she possesses exceptional classroom management techniques, and is sincerely dedicated to her students’ personal and academic needs. She is a problem solver. She is an effective instructor. She is student-centered. Ms. L is a sincere asset to our school and a direct factor for our students’ success.”
“Mrs. E is a joy to have within our school building. She has built strong relationships with students. She is well prepared and delivers effective art instruction daily.”
-“Ms. G is the best Social Studies teacher I have ever worked with. She is by far the most dedicated and exceptional teacher I have seen in my 35 years of working in Education.”
“Mrs. A is one of the best teachers in our school. She is shared between two schools. We would love to have her all to ourselves.”
-“Mrs. A is consistently performing above the benchmarks in review of her student assessment data. Her classroom is well organized. She harnesses the power of creativity within her classroom. She has great relationships with her students and parents.”
Based on multiple measures of teacher effectiveness from teachers identifying and reporting SMART goals (CAEP 4.1), to principal classroom observation reports of four program completers (CAEP 4.1), to the Employers Satisfaction survey (CAEP 4.3), Hollins University teachers have risen to the occasion and are succeeding in their P-12 classrooms. Teacher effectiveness in each InTASC area and positive teacher impact on student learning will better ensure P-12 students are receiving the tools they need to succeed and, in turn, have the best chance for reaching their potential.
The purpose of this report is to provide evidence of the success of the Hollins University Education department for CAEP Standard 4: Satisfaction of Completers. The Hollins University education faculty administered a Qualtrics survey called Satisfaction of Completer Survey to program completers over the course of three cycles after their first, second, and third year teaching respectively (2018, 2019, and 2020).
Over the three cycles of data collection (2018, 2019, and 2020), the average response rate was 44.65%, well above the minimum response rate of 20% for surveys CAEP requires for survey results integrity.
Composite scores were created by averaging the eleven competency item scores. Possible average scores range from 1.0, indicating respondents were “Extremely Dissatisfied,” to 4.0, indicating respondents were “Extremely Satisfied.” An average score of 3 for a competency item would indicate that, on average, students were “Satisfied” with their experience in the Hollins University program. Thus, a heuristic of 3.0 has been used as an indicator of effectiveness for CAEP Standard 4. The Hollins University Education department generated this list based on key areas of teacher preparation and the goals we set out in our courses and in their student teaching.
The eleven competencies rated by respondents are:
- Varying Instructional Strategies
- Instructional Planning
- Addressing Student Diversity
- Sequencing Content
- Incorporating Instructional Technology
- Classroom Management
- Differentiating Instruction
- Engaging Learners
- Field Experience and Internships
- Reading Strategies
Survey of Completers by Year
The results show the Hollins teacher training program is meeting the needs and expectations of our teacher graduates and that they are satisfied with their training. In each category of preparation, the average scores for the satisfaction levels of program completer for a given year met the minimum proficiency level of 3.0. The only exceptions were for the category of “sequencing content” in 2018 and 2019 and instructional technology in 2018. It should also be noted that the most recent year Hollins teachers were evaluated (2019-2020), program completers rated each category higher than any other year. This would indicate improvements the Hollins education program are translating into direct results for teacher satisfaction levels with their preparation.
Program completers on the Satisfaction of Completers survey were able to state what areas Hollins did well in their preparation. The following are quotes compiled by program completers about their level of satisfaction. Each quote represents a different program completer. These comments correspond and enhance the quantitative results of program satisfaction discussed earlier in this report. Below are comments to the question in the survey please identify any areas in which you think Hollins prepared you well:
“Lesson Planning, assessment, and field experiences were all very helpful”.
“Planning and technology”.
“Instructional planning and delivery; assessing student success using various products; maintaining cooperative relationships with colleagues, students and families”.
“Designing lesson plans.”
“Instructional and reading strategies”.
“Hollins emphasized field experience through volunteering at Boys and Girls Club, student internships (month long), classroom observations, and student teaching.”
“With everything, gosh! I treasure the things I learned during my time in the MAT program and still regularly refer to and use work that I did during that time.”
“Classroom management, lesson planning, and data evaluation”.
“Classroom management, lesson planning, differentiation, technology incorporation”.
“Assessing students, differentiation, and field experience and internships.”
“Rigor of the teaching profession, experiential learning ideas to use in the classroom, varied classroom setting experience.”
“Classroom management and field experience. Hollins made it very easy to student teach- they set it all up and it went so smoothly. I couldn’t have had a better experience.”
“I think Hollins emphasized observations, internships, and student teaching. This is the best way to learn as a teacher.”
“Lesson planning and creating evidence for a portfolio.”
“Varying instructional strategies, incorporating technology into the classroom, and lesson planning were well taught and used throughout the program repeatedly.”
“The faculty and coursework at Hollins prepared me to go into a classroom and have a myriad of instructional strategies and techniques that would allow me to engage many students. They also did a wonderful job of modeling how to use technology effectively in the classroom and incorporating it into different types of lessons.”
“I felt like Hollins covered the small things very well. [The professors] especially helped me learn ways to really connect with students and meet them on their level. I also felt, personally, that my own writing improved while attending Hollins, which is a great help in coaching students to improve theirs. The field work (internships and student teaching placement, as well as required observation hours and time at CYP) were incredibly helpful to me.”
“Hollins did well setting us up for planning. I am by nature an organized person, but Hollins educators emphasized all of the -other- things you need to plan for in teaching careers. My lesson plan format is not nearly as detailed, but it contains most elements from the coursework. The student internship and teaching placements put me in an unfamiliar situation with an expert teacher, and that experience had an enormous impact on my success.”
“Student teaching prepared me most.”
“Rigorous Lesson Planning”.
“Classroom management, field experience, Incorporating Instructional Technology.”
“Hollins prepared me well for instructional planning, field experience, and student diversity.”
“Hollins University really opened my eyes to what the education/teaching profession is and what is required to be successful in the profession. The instructors are very knowledgeable and most of them seemed to be interested in the student’s success as well.”
“Hollins prepared me well in terms of managing a classroom as a whole. Hollins gave me multiple opportunities and settings to observe and teach, and it helped me find what worked for me as an educator. I observed at the Community School, student taught at a city school, and student taught at a county school. The combination of different methods that teachers used really helped me in the future.”
“All areas of diversity, differential learning, instructional planning, and classroom management.”
“I think my education at Hollins was exceptional and I am so glad I chose HU for my Master’s.
“I loved my time at Hollins and I felt very prepared to become the best version of myself as an educator. Though I have been teaching for just 3 years, I feel Hollins has provided me with a very strong base to prepare me for the rigor of teaching in the 21st century!”
“Thank you so much for everything you taught me!”
Though these results are encouraging and demonstrate program completer satisfaction in key preparation areas, the Hollins Education program preparation must continually reflect the demands of the 21st century classroom and stay ahead of P-12 student needs. We will not be complacent or satisfied in our current preparation programs knowing each teacher candidate and their students will have unique needs and challenges. We can grow in what we offer to teacher candidates by this continuous feedback loop of self-analysis, flexibility, and high standards for ourselves and for our teacher candidates.
Hollins University Education Department defines a cohort as a group of teacher candidates beginning student teaching together.
Undergraduate Rate after 4 years: 100%
Graduate Rate after 4 years: 100%
Total Program Completers for 2019-20
Elementary Education: 7
English/Language Arts: 1
Visual Art (preK-12): 1
History/Social Sciences (6-12): 1
Praxis II Scores for Elementary Education Content Knowledge and Praxis II Scores for Secondary Licensure areas:
100% pass rate for program completers 2019-2020
Virginia Commonwealth Literacy Assessment (VCLA)
100% pass rate for program completers 2019-2020
Graduates of the Hollins University Education program have tremendous success in securing employment.
- From fall 2014 – spring 2020 Hollins University graduated 68 teacher candidates. Of the graduates seeking employment, 60 teacher candidates out of 65 have secured employment, a success rate of 92% employment
The Fiscal Year 2017 official 3-year cohort default rate according to the U.S. Department of Education is 4.1%.
The national average (regardless of type of college/borrower) is 11.5%.