Why Are We Looking at This Data?!?

Percentage of Students at Level 3 and 4
EQAO School Report September 18, 2013

Last year was my first year back in the classroom after working as an Instructional Coach for two years.  I really tried to put into practice all of the strategies I had read and learned about.  It was a wonderful year, the best year of my teaching career thus far.  My students and I had a blast.  In comparing assessments from September to June, I truly believed that my teaching had had a positive impact on my students' learning. 

Then the EQAO scores came out.  I was crushed! I fully recognized that the Junior EQAO Literacy and Numeracy Assessments really are just one type of assessment and can't possibly validly assess all that my students had learned over the past year.  My students had learned to work collaboratively to solve problems, share their thinking, and connect with others outside of their classroom to learn authentically from the world around them.  How can EQAO possibly assess that?  But still, I believed that all of those experiences would have had a positive impact on their ability to be successful on the paper/pencil EQAO test. 

That's probably because one of my very favourite books is "Hooray for Diffendoofer Day" by Dr. Seuss (with the help of Jack Prelutsky and Lane Smith).  In fact, I have always read it to my Grade Six students on the first day of school.  In case you are unfamiliar with it, I've included a YouTube video of the story.  Basically, at Diffendoofer School, the students learn differently, and while they never prepare for the high-stakes test, they are more than ready for it because of their unconventional learning.  Oh - you've just got to watch it - it's still one of my very favourites!

So in my new role as a Curriculum Consultant I was recently at one of the schools I support conducting an Item Analysis of the EQAO data for the Primary and Junior Divisions.  Together with the School Improvement Team, we analyze the EQAO data to determine our students' needs because our student needs tell us where we, as teachers, need to focus our learning.  It is very public knowledge that the scores for Math in the Junior Division across the province are dismally low.  Why?  One of the teachers conducting the Item Analysis with me said "We've been asked to use the 3-Part Lesson in Math and to teach through Problem Solving.  When are we going to accept defeat and acknowledge that it doesn't work?"

And for the first time I had doubts.  I doubted the efficacy of teaching Math through collaborative problem-solving because of the low scores in my own school in Junior Math.  Had I been wrong?  Could everything I had been espousing been incorrect?  I couldn't answer that teacher's question; I couldn't blog - I felt like a fraud.

I needed to take a hard look at the data from the school in which I had taught. What could that data tell me?  What couldn't it tell me?  When we look at the data, we don't only look at the Achievement or Outcome data.  We also have to look at the Contextual or Demographic Data as well as the Perceptual Data.  That is why we shouldn't ever rank schools.  We have a Community Living Classroom in our school and those students are not physically able to participate in EQAO but they are still counted in the overall scores.  We also have a very high ELL population and sometimes have to exempt students who arrive from non-English speaking countries only weeks before testing.  When we looked at the percentages of participating students, our results were a bit better.  When I looked at the scores for my class alone the data looked even better still.  78% of the students in my class achieved benchmark levels in Math. Of course, when you are talking about only 22 students, it is difficult to talk in percentages, but I was relieved none the less. 

I am tired of hearing "these kids can't" and want to prove that these kids most definitely CAN! Last year I became convinced that Carol Dweck was correct in her theory on mindsets.  Basically, she says we can have a growth mindset, where we believe accomplishments are a product of hard work and dedication, or we can have a fixed mindset where we believe intelligence is a fixed entity that can't be changed.  Dweck's research demonstrated that teachers who have a growth mindset are better able to motivate and engage their students.   

Why do I pour over EQAO data?  Because it is one of the best tools we have in this province to reflect on our impact on student learning.  I think my class' EQAO data proves Dweck is correct.  Here's a concrete example.  I had one student who struggled all year in Math.  Early on in the year, she told me she didn't like Math (Perceptual Data).  When I asked her why not, she explained that when she had been in Grade One her teacher had told her she didn't have a brain for Math.  I told her her Grade One teacher was wrong, and I would prove it.  A couple of days ago, I called her at home to celebrate her level 3 on the Math EQAO scores.  She said, "So does that mean I am good at Math?"  I answered, "It means you can be good at anything you want to be!"

So why are the Junior Math scores so low in our province?

I have a theory about the Math.  Traditionally, teachers have always taught the Math lesson to the whole class, then assign a set of questions from the text, and then take up the questions with the whole class.  This teaching strategy has a certain level of effectiveness.  I believe teaching Math through collaborative Problem-Solving is more effective BUT ONLY IF THE TEACHER IS GOOD AT FORMATIVE ASSESSMENT.  If the teacher is teaching through Problem-Solving but does not begin with Assessment FOR Learning, the impact on achievement is lower than the impact of teaching with the text book.  I was able to positively impact my students' learning because I started by finding out what they already knew and what misconceptions they had; then I worked toward closing gaps and correcting misconceptions.  This style of teaching does not work if you don't first teach your students how to communicate their thinking in Math. You have to ask the right questions to elicit their understandings.  You have to know the significance of what they are saying.  If you only ask for an answer, you have no idea how they got there!

During one of the Item Analysis meetings at one of my schools, two of the primary teachers were discussing the following exemplar from the released EQAO Math scoring guides.  

They were having an excellent discussion about whether or not this should be considered a level 3 response.  (By the EQAO scoring system, is was considered a level 30 response).  We ended up discussing what the work told us about the child's understanding of the Math concepts.  Did the child in fact use critical and creative thinking to solve this problem?  What does this child's solution tell us about his/her understanding of Math concepts?  Are we just looking for the right answer?  Or are we assessing the child's level of understanding?

We need to know the kids that we are teaching.  We need to know what they know and what they can do.  We have to give them multiple opportunities to explore Math concepts until they develop deep understanding of these concepts.  

As long as EQAO is out there, I will be pouring over that data, analyzing it, trying to determine what pieces we are missing and how we can do it better.  The current scores in Junior Math tell me that we still need to learn a lot more about how students learn Math.  

I have a fantasy that one day we will live in a world where no one ever says "Oh, I'm not a Math person, I've never been good at Math."