Barish Golland

Learning Eco-Systems Management| Ed Tech | Analytics

Formative Evaluation in Higher Education

Posted by on Apr 21, 2016

Formative Evaluation in Higher Education

Among Hattie’s most impactful influences on student achievement descibed in his book Visible Learning (Hattie 2013), is formative evaluation. It has an effect ranking of 0.9, so highly impactful on student achievement. Formative evaluation refers to “any activity used as an assessment of learning progress before or during the learning process itself” (Hattie 2012 and Visible Learning Website). Formative evaluation is arguably more important than summative evaluation, because it gives a chance for the students to get relevant feedback on where they are and where they need to go. Watch this video that best explains the difference between formative and summative evaluation:

This is also a great video on formative evaluation based on William & Black 1998:

I love the analogy in the above video about how formative assessment is really like taking a car for air inspection. They don’t just tell you, “Your car failed the test. Now go away.” Instead they say, “Your car failed. These are the reasons (brakes, radiator, muffler, etc.). Go and fix those and come back for another inspection.” That’s exactly what we need to do with students! A summative assessment may tell a student “you failed this course” but then provides no pathway to success. It leaves the student demoralized and discouraged.

Response/Reflection:

My immediate thought is that formative evaluation is not being done enough in the Higher Education classroom. Summative assessment (large end of term final exams worth %40) are given much more focus. But summative assessment that will not affect student learning in the way that formative assessment can, because formative learning allows the student to change course after given constructive feedback from their instructor.  Summative assessment is easier to deploy because no detailed feedback is necessary because now the course is over. It’s therefore less work for the instructor. They’re less involved in the student learning process because they don’t have to work with the student from then on. However Hattie’s research of over 800 meta-analyses proves that formative evaluation is much more effective in improving the quality of student learning.

Some instructors, especially those teaching large courses on campus of 200 or more student, might cringe at the thought of having to give formative assessment feedback to every single student. This process can be simplified through providing rubrics to students that show them how they measure up in the grading process, and some feedback can be automated or pre-written to cover different ranges of student abilities. All that to say that large classrooms should not be a reason to avoid formative assessment considering it is so effective.

For Teacher Training / Professional Development

In order to help change the culture of instructors avoiding formative evaluation, required reading should be Hattie (2012, 2013), Brookfield (2015) and Black & William (2006). Workshops on how to do effective formative evaluation should be set up, guiding instructors step by step in the process. And advocating what Brookfield (2015) calls “critical reflection” should be communicated by administration to faculty, along with tools for classroom research into how the students are experiencing the learning and perceiving the teaching.

References:

Black, P., & Wiliam, D. (2006). Inside the black box: Raising standards through classroom assessment. Granada Learning.

Brookfield, S. (2015). The Skillful Teacher: On Technique, Trust, and Responsiveness in the Classroom. San Francisco, CA: Jossey-Bass.

Hattie, J. (2012). Visible learning for teachers: Maximizing impact on learning. Routledge.

Hattie, J. (2013). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge.

Read More

Wise Crowds Liberating Structure – a supplement to CIQ?

Posted by on Apr 18, 2016

Wise Crowds Liberating Structure – a supplement to CIQ?

The Liberating Structure known as “Wise Crowds” seeks to “tap the wisdom of the whole group in rapid cycles” (see LS 13 – Wise Crowds). I believe it can be used as a different approach to what Brookfield recommends in his The Skillsful Teacher as the CIQ or Critical Incident Questionnaire (Brookfield 2015, p. 34).

Normally in an anonymous CIQ, students are asked to describe their most engaging, confusing, helpful or surprising moments in the class that week. Their comments are then considered, tabulated and analyzed by instructors, who then respond by adjusting their teaching, lecturing, or assessment instruments for the following week.

I believe the CIQ is an incredibly powerful and sadly underused (at least at my institution) classroom research technique that has many merits. It’s an excellent way to get a snapshot of how students are doing with the learning at any point in the course. The challenge comes when you try to address the issues raised in the CIQ in order to help students better learn. Often you may not have time to address every single question or every confusing concept students raised in the anonymous survey. That’s where the Wise Crowds Liberating Structure comes in.

Rather than have the teacher trying to solve all of the students problems with the learning, she can use the  Liberating Structure (LS) known as “Wise Crowds” (Lipmanowicz & McCandless 2013, pg. 217) to allow students to take control of their learning and consult their peers, the Wise Crowds, for answers to their questions. The Wise Crowds LS allows students to  “gain more clarity and increase their capacity for self-correction and self-understanding” by consulting the expertise and inventiveness of everyone in a group (p. 217). A spinoff benefit of Wise Crowds is that students grow in supportive relationships and “deepen inquiry and consulting skills”. This is because student alternate in being a “client” and also a “consultant” to others. Here’s how you would structure it and set it up in real time.

 Applying Wise Crowds to the Classroom

The following is an adaption to a higher education classroom setting of the “Five Structural Elements – Minimum Specifications for a small Wise Crowds” in Lipmanowicz & McCandless 2013, pg. 217.

  1. Structuring Invitation
    1. Inform students they will be tapping the wisdom of their peers to help them clarify the most confusing  aspects of the course experienced this past week. They will also get the opportunity to be consultants to other students with their own specific needs for clarity.
    2. Each student briefly describes his or her challenge and asks their peers for help. Peers are consultants who ask clarifying questions and offer ways for student to be more clear about the concept raised.
  2. How Space is arranged
    1. Students get into groups of 4 or 5 facing each other, adapting the classroom chairs and setup as necessary
  3. How Participation is Distributed
    1. All students allowed to raise their most confusing concept they had last week
    2. All students given equal amount of time to ask for or get help
    3. All students have an equal opportunity to offer help to other students
  4. How Groups are configured
    1. 4-5 students
    2. Can either be randomly organized to allow for mix of students, or grouped by team based on the teams that are already set up to work on specific projects
  5. Sequence of Steps and Time Allocation
    1. Every student given 5 minutes, broken down as follows: 1 minute to explain most confusing concept, 1 minutes for peers to ask clarifying questions,  2 minutes for advice given by consultants, 1 minute for client to provide feedback to consultants on how useful the advice was.

References:  

Brookfield, S. (2015). The Skillful Teacher: On Technique, Trust, and Responsiveness in the Classroom. San Francisco, CA: Jossey-Bass.

Lipmanowicz, H., & McCandless, K. (2013). The surprising power of liberating structures. New York: Liberating Structures Press.

Liberating Structures content, including the image used in this blog post, is licensed under a Creative Commons License.Creative Commons License

Read More

Reflections on Experiencing Teaching – 1st Chapter in The Skillful Teacher

Posted by on Apr 11, 2016

Reflections on Experiencing Teaching – 1st Chapter in The Skillful Teacher

Some really great quotes from this chapter in The Skillful Teacher: On Technique, Trust, and Responsiveness in the Classroom:

“The truth is teaching is a gloriously messy pursuit in which shock, contradiction, and risk are endemic” (p. 1)

“Our lives as teachers often boil down to our best attempts to muddle through the complex contexts and configurations that our classrooms represent” (p. 1)

“…teaching is riddled with irresolvable dilemmas and complex uncertainties”  (p. 2)

I like how Brookfield is just very upfront with the realities of teaching and “muddling” through otherwise highly complex and unpredictable situations. I have often found myself in situations where I’ve had to “muddle through” – A student stabs another student in the back with a pencil (this is in Grade 1), I lose complete control of the class because students are restless and bored, I’m playing a video to students and out pops a terrible swear word that has been unexpectedly dubbed over the original in the YouTube video. Those are just a few of the examples where I’ve had to muddle through, improvise, change course suddenly and throw out my otherwise perfectly planned lesson.

‘Growing into the Truth of Teaching’

I also appreciate Brookfield’s honesty and in stating some of the truths he has come to in his teaching career. Here are the ones that stood out to me:

“I will never be able to initiate activities that keep all students engaged all the time” (p. 9)

“Making full disclosure of my expectations and agendas is necessary if I am to establish an authentic presence in the classroom” (p. 9)

“I always have power in the classroom” and can’t make it so that students don’t notice me in the room (p. 9)

That kind of gut level honesty is highly appreciated and makes me reflect more deeply on my own convictions in teaching. It also helps me to know where my role starts and ends, and where the student is responsible for their own learning. After all, I cannot force learning onto a student. They must decide whether they want to learn or not. But I can also not be completely self negating and show no confidence in my teaching skills, or disregard the power and authority I hold as a teacher in the classroom.

Liberating Structures

Bumbershoot+wksp+seriesIn reading through Chapter 1, I find that I disagree with Brookfield on one point. He gives an example on pages 4-5 of how he was confronted by a silent classroom where none of the students wanted to speak, despite several successive questions asked by the teacher to the whole classroom. He then goes into a long speech about how no one was obliged to speak and that he wasn’t presuming failure on their part if they didn’t respond. It was a nice speech and I do agree that it probably alleviated some students’ minds to actually want to speak but. However, I think he could have solved this in a much easier, less dramatic way by simply using a “Liberating Structures” technique such as 1-2-4-All. In this simple group configuration activity, the instructor gets students to spend 1 minute thinking about the question, then 2 minutes talking in pairs with another person, then get into groups of 4 to come up with the best response worthy of sharing with the class, then in the final “all” stage, the teacher calls on all groups to share their discussions with the whole class. If Brookfield would have deployed this simple structure for promoting discussion, he would have avoided all the blank stares and the subsequent dramatic speech he had to make, which in the end amounted to simply more “teacher talk time” or TTT.

References:  

Brookfield, S. (2015). The Skillful Teacher: On Technique, Trust, and Responsiveness in the Classroom. San Francisco, CA: Jossey-Bass.

Lipmanowicz, H., & McCandless, K. (2013). The surprising power of liberating structures. New York: Liberating Structures Press.

Teacher Talking Time. British Council. Available at: https://www.teachingenglish.org.uk/article/teacher-talking-time.

Read More