Comparing Proficiency and Growth Based Assessment

With the advent of No Child Left Behind (NCLB) and the current policies of the Every Student Succeeds Act (ESSA), the debate on a standards-based and growth-model-based assessment began and rages on.  It is clear that both tactics have pros and cons though having the same goal that students meet or exceed a level of proficiency on state level assessments (Lachlan-Haché & Castro, 2015).  Most teachers, schools, and districts can agree that standards are a good way to set basic goals for students, but how to measure achievement can get messy.  Let’s discuss the pros and cons of both a standards-based, or what I will call a proficiency-based model, and the growth-based model at the teacher and district levels.

First, it should be noted that the terminology of standard-based and proficiency-based are being used interchangeably here.  Standard-based grading is the process of teachers evaluating specific skills or criteria met by students (Bennett, 2019) in contrast to simply assigning a letter grade.  This is the system of using terms like “meets standard,”  “exceeds standard,” etc.  Proficiency-based models use standards-based grading to report on how well a student has met a specific standard (Bennett, 2019) and therefore inherently encapsulates the concept and term using standards-based grading as a means of measuring proficiency.

Growth based assessment on the other hand, is “‘A collection of definitions, calculations, or rules that summarizes student performance over two or more time points and supports interpretations about students, their classrooms, their educators, or their schools’” (Bennett, 2019, para. 19).  A simple example can be used to illustrate the difference between these two models.

• Proficiency Target:  All students will get a score of at least 75 on the end of unit assessment.

• Growth Target: All students will increase their scores by 40 points from pre assessment to post assessment (Lachlan-Haché & Castro, 2015).

Proficiency-based models have many pros and cons at the teacher and district level.  For teachers this model asks them to think about the minimum expectation of student performance (Lachlan-Haché & Castro, 2015) knowing that this is the baseline for all students in a class.  This is also a helpful concept for districts whose populations may be diverse and widespread as is the case in some large counties.  A minimum expectation can be helpful with wide ranges in economics and demographics.

This model also focuses on narrowing achievement gaps with proficiency targets (Lachlan-Haché & Castro, 2015).  This refers to the disparity in performance between different groups of students in various success measures (Ansell, 2011).  These disparities are often between cultural, racial, socio-economic, and gender lines.  For teachers and districts the standard nature of targets can lessen disparity between diverse groups of students, in theory.

Finally the proficiency model has three big advantages for teachers in particular, but districts as well.  The model is more familiar to teachers, it simplifies scoring for teachers, and it does not require pre assessment or any other baseline data (Lachlan-Haché & Castro, 2015).  These aspects of the model may save precious time for teachers and districts.  Teachers can score assessments quickly and with less complications.  With no need for pretesting and baseline data, districts can accept new students easily and teachers can just start where they are without trying to find previous data on students.   Teachers can avoid extensive training or professional development, which districts like based on cost and resources.

The model is not without its downfalls however.  To start, just measuring proficiency may not meet requirements of states or at the federal level (Lachlan-Haché & Castro, 2015).  Districts and teachers will not be meeting requirements if states or the federal government require evidence of growth in addition to meeting proficiency standards.  Both districts and teachers should know what is expected at state and federal levels to avoid noncompliance, which may take up some of the time spared by the simplicity of the model.

More importantly, the model may support the unrealistic expectation that all students will achieve proficiency in an academic year (Lachlan-Haché & Castro, 2015).  Differences in development, previous skills, and starting levels that may be deficient can all have an impact on student proficiency.  Not to mention outside conditions for students like trauma, illnesses, and more.  This highlights the important points that the model may not accurately show a teacher’s impact on student achievement and that it may gloss over the highest and lowest performing students (Lachlan-Haché & Castro, 2015).  The all-in-one format of the model can overlook the many factors that teachers take into account when teaching and a focus on the minimum level may leave out the students needing more help or wanting more.  

The proficiency model allows teachers and districts to have a arguably accurate account of the status of each student.  However, as Colette Bennett (2019) says “Status is not growth…” (para. 15).  It is for this reason that many teachers and districts prefer the growth-based model.  Of course, this model has its own peaks and pitfalls for both to consider.

Many of the drawbacks associated with this model fall on the teachers and target creators, perhaps even at the district level.  The most glaring of which is that creating rigorous and realistic growth targets is particularly challenging (Lachlan-Haché & Castro, 2015).  If teachers or district level target creators are off, the lowest achieving students may not ever reach proficiency and growth targets may create even more challenges comparing data across teachers or grades (Bennett, 2019).  

In addition, poorly constructed pre and post assessments can devalue growth targets (Lachlan-Haché & Castro, 2015).  Although this is the case in any model, if baseline data is not good due to poorly constructed tests the targets may be invalid from the start.  Careful thought will be needed from teachers or test creators to ensure the creation of valid student learning targets.  In correlation with this, growth target scoring can become much more complex than proficiency-based versions (Lachlan-Haché & Castro, 2015).  Again, time and care will need to be employed in scoring growth based assessments, spending precious teacher and district capitals of time and effort.

Pitfalls aside, there are some advantages of the growth-based system.  Teachers especially are recognized for their efforts with all students and that their impact will look different from student to student (Lachlan-Haché & Castro, 2015).  At a district level this recognition could keep teacher retention at higher levels and make the district a more hospitable place.   

This model also guides critical discussions on achievement gaps by addressing individual students rather than a class as a whole.  This helps teachers to identify the needs of students at the ends of the achievement spectrum, increase growth for higher achieving students, and support lower performing students (Bennett, 2019).  This intimate knowledge of every student is nothing but a benefit to teachers.  Districts also benefit from attention to individual students as, in theory, student achievement will go up.

There are many aspects of each model that must be considered.  Neither is perfect or totally flawed in its essence.  It is important to note that there are several states that are using a hybrid model of the two (Lachlan-Haché & Castro, 2015), perhaps the best option.  It is important to remember that the ultimate goal is the success of the student and whatever option helps to achieve that goal is the right one.

References

Ansell, S. (2011, July 7). Achievement Gap. Education Week. Retrieved from https://www.edweek.org/ew/issues/achievement-gap/index.html

Bennett, C. (2019, November 4). Contrasting Growth and Proficiency Models for Student Achievement. ThoughtCo. Retrieved from https://www.thoughtco.com/growth-model-vs-proficiency-model-4126775

Lachlan-Haché, L., Ed. D., & Castro, M., Ed. D. (2015, April). Proficiency or Growth? An Exploration of Two Approaches for Writing Student Learning Targets (Publication). American Institutes of Research. Retrieved from https://www.air.org/sites/default/files/Exploration-of-Two-Approaches-Student-Learning-Targets-April-2015.pdf

Shifting Assessment

I grew up in the small town of Laramie Wyoming and I don’t specifically remember taking a standardized assessment until I was in high school.  Of course, in my classes before that many teachers had used the standard method of a high-stakes summative assessment at the end of a unit or lesson. The process was nearly always the same.  The occasional pre assessment was followed by the lesson or unit with the looming test at the end.  This was a model that I was able to master.  Not the material mind you, but the model.  The priority was put on the grade that I would get and because of this I worried less about the learning goal and more about outperforming others.  This is what research confirms about giving grades in general (Ryan & Deci, 2020).

As I mentioned before, the first standardized test I remember taking was in high school.  This would have been around the year 2002 or my Junior year.  This makes sense with the consideration that 2002 was the year that No Child Left Behind (NCLB) effectively began in schools.  NCLB mandated that states had to give statewide tests once in grades 10 through 12 (Lee, n.d.).  The test I remember must have been part of this mandate.  At the time I took the test I was very anxious because I had put so much importance on the outcome.  However, when we got the test, it was very simplistic.  I clearly remember a test question that asked me, a high school junior, to read and write down the time shown on a clock.  The whole experience was unnerving and somewhat patronizing.

It wasn’t until I got to college that I came into contact with rubrics.  I had chosen to go to school for theater arts and not only had summative assessments in subjects like theater history, but was also being assessed in actual performances.  In order for these performance projects to be effectively assessed, several of my professors had created rubrics. It was a game-changer.  I loved knowing what was expected of me up front and conversely exactly what I had done or not at the end.

Unfortunately, some of the less helpful pieces of my experience are still around.  According to Ryan and Deci in their 2020 report grading is still pervasive in schools around the world and is almost synonymous with school in general, even though there is little evidence of its positive effect.  However, many strides have been made by educators to make grades less important as a concept by using many different formative assessments as well as rubrics for student learning.  

Standardized tests like the one I took in highschool are still very present in our system today.  Some High Stakes Testing (HST) has been very ineffective (Ryan &Deci, 2020), but still is a necessity for large scale data.  Although standardized testing is still hotly debated, sources like the Ohio Department of Education (ODOE) (2016) give actionable and practical guidelines for creating and choosing assessments that are better formulated for students.  The Every Student Succeeds Act (ESSA) brought about many changes from NCLB in testing and school accountability that are well organized in the infographic below from Educators for Excellence (2016) as well.

Overall, in my experience, the purpose and process of assessment has become more well defined since my time in school.  Even in just the last 10 years teaching in my elementary school, the landscape of assessments has leaned to the positive, hopefully leading more students to mastery and understanding.

References

Educators for Excellence. (2016, October 4). Infographic: No Child Left Behind v. The Every Student Succeeds Act [Web log post]. Retrieved from https://e4e.org/blog-news/blog/infographic-no-child-left-behind-v-every-student-succeeds-act

Lee, A. M., JD. (n.d.). No Child Left Behind (NCLB): What You Need to Know [Web log post]. Retrieved from https://www.understood.org/en/school-learning/your-childs-rights/basics-about-childs-rights/no-child-left-behind-nclb-what-you-need-to-know

Ohio Department of Education (ODOE). (2016). A Guide to Using SLOs as a Locally-Determined Measure of Student Growth (Rep. No. Guidebook). Retrieved from https://education.ohio.gov/getattachment/Topics/Teaching/Educator-Evaluation-System/Ohio-s-Teacher-Evaluation-System/Student-Growth-Measures/Student-Learning-Objective-Examples/SLO-Guidebook-041516.pdf.aspx

Ryan, R. M., & Deci, E. L. (2020). Intrinsic and extrinsic motivation from a self-determination theory perspective: Definitions, theory, practices, and future directions. Contemporary Educational Psychology, 61. Retrieved from https://selfdeterminationtheory.org/wp-content/uploads/2020/04/2020_RyanDeci_CEP_PrePrint.pdf

Frequent Checks for Understanding

“Does that make sense?” has passed over my lips so many times in my classroom.  I often get nods or grunts, the occasional question.  I have been guilty of this half-hearted check for understanding too many times.  Oftentimes students will simply stay quiet, because they are too confused, too embarrassed to ask, or they think they get it, but they don’t (Fisher & Frey, 2014).  This doesn’t give me a real look at what my students understand and may lead to gaps in understanding that will show up much later in instruction or even life.  Checking for understanding can enhance student learning and teach good study skills in the process, showing students that as I am checking their understanding, they should check their own (Fisher & Frey, 2014).  Goodwin and Hubble (2013) suggest that these checks be done every 10 to 15 minutes in class to enhance learning.

So what quick checks can I start using today to check for student understanding?  Here are my top five picks for the 2020 school year:

1. Hand Signals

   Students respond to a concept or question with signals of understanding like a thumbs up or down, waving for a question (Regier, 2012).  Teachers can also create a scale of understanding for students, ie. one finger means “I don’t understand;” five fingers means “I am an expert and could teach this to others” (TeachingChannel, 2011).

2. Exit/Entrance Tickets

   These are written responses to questions about learning that happened that day, or the day before (Regier, 2012).  This will help in planning the direction of the next class to close any gaps in understanding.  This can be done on paper, but could be digital too taking advantage of a digital platform like Padlet or Polleverywhere.

3. Four Corners/Move to Where You Are

   Students move to the corner of the room that represents their level of understanding, ie. corner one is “need’s help” and corner four is “I got this” (Regier, 2012).  This movement could also represent a physical quiz of sorts.  If each corner is an option for a multiple choice question it will be a much more active way of assessing knowledge.  Adding movement into class strengthens learning, improves retrieval of information and memory, and improves learner morale and motivation (Jensen, 2005). 

4. Think, Pair/Group, Share

   Students think of their individual response to a question or idea then share with a partner or group.  Several groups are called on to say what they shared out loud (Regier, 2012).  Research has shown that educational experiences that are active, social and student-owned lead to deeper learning (Center for Teaching Innovation, n.d.).  Allowing students to share ideas then share back is a great check of understanding or misconceptions.

5. Journal/Blog/Vlog

   Students record in a journal an entry surrounding what they understand about a topic (Regier, 2012).  This has long been a standard in classrooms, but with the advent of technology, these entries can be made via blog or vlog.  This is a great way for differentiating for students that are not as strong as writers as well.  Using video or technology can also motivate students that are not as excited about pencil and paper.

These formative assessments should allow me to check understanding in my students.  So, then what?  What if all, most, a few, or even one student does not understand?  Here is the important step of feed-forward.  Taking the information gleaned from the checks and doing something with it is of utmost importance.  Here's what can be done for each instance:

• Everyone in the class doesn’t understand.

Reteach.  It is that simple.  Consider what went wrong the first time or how you can chunk the material differently so that students can understand.  For reteaching to be effective, it must focus on the omissions and errors in student thinking (Marzano, 2010).  Don’t be embarrassed to have to reteach, remember the goal is mastery and understanding.

• Most in the class don’t understand.

This could be an opportunity to reteach to the whole class.  If most of the class doesn’t understand it may not be a bad idea to go back.  However, collaboration can help here too.  Consider grouping students that understand the material with those who don’t, making the process beneficial to both.

• A few in the class don’t understand.

Here we can use the power of small group instruction.  Perhaps I as a teacher could take the group that doesn’t understand and do some guided instruction with them.  Guided instruction is the strategic use of questions and prompts for students (Fisher & Frey, 2014) and would be a perfect tool for small group work on content for a few students.

• One student doesn’t understand.

It is time for a one on one interview and work.  I can look at what the student doesn’t understand and work on differentiating the material so that they do. Content, process, or products can be differentiated for student learning (Fisher & Frey 2014).  Perhaps they need the information in a graphic form or something else.  Only knowing your student and finding out what is being missed will solve the problem.

References

Center for Teaching Innovation. (n.d.). Collaborative Learning [Web log post]. Retrieved from https://teaching.cornell.edu/teaching-resources/engaging-students/collaborative-learning

Fisher, D., & Frey, N. (2014). Ch. 1: Why check for understanding? In Checking for understanding: Formative assessment techniques for your classroom (2nd ed.).  ASCD. Retrieved from http://www.ascd.org/publications/books/115011/chapters/Why-Check-for-Understanding%C2%A2.aspx

Goodwin, B., & Hubble, E. R. (2013). The 12 touchstones of good teaching: a checklist for staying focused every day. ASCD, McREL International.

Jensen, E. (2005). Chapter 4. Movement and Learning. In Teaching with the Brain in Mind, 2nd Edition. ASCD. Retrieved from http://www.ascd.org/publications/books/104013/chapters/Movement-and-Learning.aspx

Marzano, R. J. (2010). Art and Science of Teaching / Reviving Reteaching. Educational Leadership, 68, 2nd ser., 82-83. Retrieved from http://www.ascd.org/publications/educational-leadership/oct10/vol68/num02/Reviving-Reteaching.aspx#:~:text=At a basic level, reteaching,content the teacher must reteach.

Regier, N. (2012). Book two: 60 formative assessment strategies. Regier Educational Resources. Retrieved from https://stjschools.instructure.com/courses/26673/files/105219

TeachingChannel (2011). Talking About Teaching with Jim Knight [video]. TeachingChannel. Retrieved from https://library.teachingchannel.org/landing-page?mediaid=0KHOLCMi&playerid=7Ftxba59

Processing Feedback: Stop, Collaborate, and Listen

I have done theater for a long time now.  I started in middle school, got my bachelor’s degree in it and continue to do it.  I run my own theater company, Arts in the Open,  here in Denver and continue to direct and be in shows.  One thing that always happens in theater, whether you are the director, choreographer, music director, or an actor in the show is notes.  Basically, for any non-theater folk, at the end of each rehearsal you all sit together and the director or choreographer gives notes on what went well or not, what looked good or not, basically how to make the show better for the next rehearsal or for opening.  

This is a process that happens in every show.  There is always time set aside for it either as a group or sometimes in writing if time is short.  It is an expectation that this time or method will always be included in the process.  The ins and outs of how notes are given and received is a whole other post’s worth of material, but it is always a part of collaborating to create a show.  The Performer Learning Objective is the show after all.  So why in education do we not specifically make time for this important piece of growth?

It is fair to mention here that time is a valuable currency for teachers.  According to Goodwin and Hubble (2013) teachers only spend around 68% of teaching time actually teaching core curriculum, with other time swept up in other tasks.  I am sure we have all felt the crunch for time while teaching even when planning to every minute, making routines effective for students, and using technology to our advantage.  However, I would argue that taking time to receive and process feedback is worth the time.  Teachers often worry that feedback goes ignored, not understood, or goes unheeded by students (Price et al, 2010).  Therefore we must make the time to process this feedback if we don’t want these fears to be realized.

I was inspired in how to frame this time by Chris Kaundart (2019) in an article about processing feedback without stress when he used a lyric from a great song.  Well, let's say “great” with quotation marks.  He asserts that we should Stop, Collaborate, and Listen. Now I’m back with a brand new invention to use as a ten minute feedback processing exercise in my classroom.  Each time I return written, oral, audio or video feedback to students or after they deliver feedback to each other I would like to have students take the following steps.

• STOP - 3 mins 30 sec

In this step students will stop and re-read, or listen to the feedback again on their own. Or simply take it in and think about it.  Maybe even calm down.

• COLLABORATE - 3 mins 30 sec

In this step they can think of questions to ask the teacher or the student accessor about the feedback.  They can have peers read, listen, or watch the feedback to get an outside view of it, maybe even collect questions they might have.  They should choose the most important questions to ask.

• LISTEN - 3 mins 30 sec

In this step students can ask the teacher or accessor the most important questions that they had about the feedback.  They only have three and a half minutes after all.

So why these steps?  “Stop” comes from the instant anxiety, defensiveness, or stress that can occur when you get feedback.  Especially feedback that students feel is negative.  In truth, receiving either good or bad feedback can make you anxious or stressed (Hayes, 2018).  Any feedback can make you anxious because it is a situation in which you are being judged, usually by someone in authority like the teacher, but also just as scary maybe your friend or peer (Hayes, 2018).  This stop gives students time to take a deep breath and actually process what the feedback is before reacting.  A student’s first reaction may be stress, anger, or embarrassment and that can’t really be stopped, but we can give students the opportunity to attempt to change or stop that response (Kaundart, 2019).  

Once we have taken the three and half minutes to get calm and actually know what the feedback is, we can collaborate on what it means and what we need to know more about the feedback.  Group-thinking about how to ask questions about the feedback can help us clarify.  Getting the opinion of  someone outside the group you are in or even a good friend can help you see the feedback from a different angle.  This can also be time to choose which questions you want to ask that will be the most important to help you absorb the feedback and grow from it.

Finally, listen.  Once we have calmed down and picked the important questions, we are much more likely to be able to actually listen to the explanation of the feedback.  One of the reasons getting feedback is so hard sometimes is that we equate feedback with judgement of us personally (Hayes, 2018).  I remember an opera singer telling me that all notes in theater have to be written down by performers.  When I asked why, his simple answer was that getting feedback, in our case notes, always feels like an attack on who you are.  So his solution like “Stop” was to just write it down.  Of course this isn’t true.  My acting doesn’t make me a good or bad person, just like a lackluster paragraph about penguins doesn’t make a student good or bad.  An important item to teach before using this system will be the difference between constructive feedback and just judging.

I plan to start with that first.  I hope to create a lesson on what feedback is and how to give/get it.  Once that has happened this 10 minute process can be used in class.  I think it would be most effective after more high-stakes assessments or projects.  Hopefully, after practice, students will be able to give and get feedback and actually use it after really analyzing what it meant and what should be done next.

Yo man, let's get out of here.

Word to your mother.

References

Goodwin, B., & Hubble, E. R. (2013). The 12 touchstones of good teaching: a checklist for staying focused every day. ASCD, McREL International.

Hayes, L. N. (2018, November 27). Why Feedback Makes Us Anxious. TalkSpace Voice. Retrieved from https://www.talkspace.com/blog/receiving-feedback-anxiety/

Kaundart, C. (2019, February 18). 5 Ways To Process Feedback At Work Without Triggering A Stress Response. Trello Blog. Retrieved from https://blog.trello.com/process-feedback-at-work-stress

Price, M., Handley, K., Millar, J., & O’Donovan, B. (2010). Feedback: All that effort, but what is the effect? Assessment & Evaluation in Higher Education, 35(3), 277-289.  Retrieved from http://www.library.uwa.edu.au/__data/assets/pdf_file/0006/1888530/Price.pdf

Providing Choice in Student Learning Goals

I recently started my training to get my license as a teacher.  In the orientation we were asked to write about our favorite teacher and why they were our favorite.  I chose my sixth grade teacher Mr. Walsh mostly because he was a teacher that gave us choices in his class.  For one project he asked us to research one of the lyrics in his favorite song We Didn’t Start the Fire by Billy Joel.  If you have never heard the song before, it basically lists important historical events in a catchy succession.  Looking back now, he had the foresight to let us choose, within a reasonable framework, what to research to reach our learning goal of writing a research paper.  This choice not only was one I remember to this very day, but set his place as one of my favorite teachers.

Choice is a powerful tool in the classroom and can help us overcome two major challenges in setting learning goals.  In his book Learning to Choose, Choosing to Learn Mike Anderson (2016) lays out those two major challenges: differentiation and apathy.  Let’s take a look at setting learning goals through these two lenses with the idea that students can and should set their own.

First let’s talk about differentiation when setting learning goals.  I think that we can all agree that as a teacher, the idea of sitting and writing a specific learning goal for each student in our classes accounting for differentiation is literally a nightmare.  Like actually a nightmare. I envision me sitting at a desk in a dark room, writing and writing and never getting to see my students.  Then I wake in a cold sweat wondering if I will ever see the kids again.  Of course that is hyperbole, but it has some parts that definitely ring true.  So why not let the students take on some of the responsibility?  One of the main reasons to use choice in your class is to provide options and  have students self-differentiate (Anderson, 2016).  They need to find their Goldilocks zone for their goals; not too hard as to be frustrating and not too easy as to be boring.  This zone is also called the zone of proximal development.  When students find this “just right” zone for their goals, it makes the process of achieving that goal more enjoyable, thereby increasing intrinsic motivation and engagement (Anderson, 2016).  You can see a great visual of this in the figure borrowed from Anderson's book below.

Note. Zone of Proximal Development and Engagement reprinted from Anderson, M. (2016). Chapter 1. The Key Benefits of Choice. In Learning to Choose, Choosing to Learn. Alexandria, Virginia: ASCD. Retrieved from http://www.ascd.org/publications/books/116015/chapters/The-Key-Benefits-of-Choice.aspx

Teaching students how to find this zone is one of our most important jobs.  The first thing we have to do is to provide students with choices in the first place.  Most good choice making comes from practice (Brenner, 2015), so we need to provide opportunities for practice in class.  Dr. Abigail Brenner (2015) also suggests five steps to teach good decision making:

• Define the decision, including the reasons to make it.

• Brainstorm the possible outcomes.

• Discuss the options and narrow them down to no more than three.

• Pick one of the three, formulate the goal and make a plan.

• Evaluate the outcome.

We can also teach the SMART Goal system by breaking it down and teaching about each step as Genia Connell (2016) suggests.  By working on each part of the goal students will be able to get in the habit of making informed choices about their own learning goals.

Let’s take a look at that second challenge, apathy.  We can see from above that when students get to choose their goals, they have a more joyful experience of learning.  The brain is more available to learning when learning is joyful and we can avoid the fight, flight, or freeze response from students when goals are boring or frustrating (Anderson, 2016).  If students make their own goals they are more likely to take ownership for them and take on the responsibility of achieving them.  Just like Mr. Walsh and his memorable research project those positive emotions associated with choice are long lasting.  Giving students opportunities to make choices about their learning goals is a great way to help them feel good about their work and thereby stay motivated in the face of apathy (Anderson, 2016).

By teaching students that they have the ability to make their own choice of goals, we are empowering them.  Empowering them not only in our classroom, but in the wider world where they will make choices everyday.  Perhaps if we provide good teaching around decision-making and goal creation, someday our students will look back at us as one of their favorite teachers.  Or much more importantly be successful in continued learning and choice making.

References

Anderson, M. (2016). Chapter 1. The Key Benefits of Choice. In Learning to Choose, Choosing to Learn. Alexandria, Virginia: ASCD. Retrieved from http://www.ascd.org/publications/books/116015/chapters/The-Key-Benefits-of-Choice.aspx

Brenner, A., M.D. (2015, May 30). The Importance of Learning How to Make Decisions. Psychology Today. Retrieved from https://www.psychologytoday.com/us/blog/in-flux/201505/the-importance-learning-how-make-decisions

Connell, G. (2016).  Setting (almost) SMART goals with my students. Scholastic. Retrieved from https://www.scholastic.com/teachers/blog-posts/genia-connell/setting-almost-smart-goals-my-students/

The Philosophical Battle of Creating Assessments

Who should be in charge of creating assessments for Student Learning Objectives (SLO)?  Assessments are at the heart of the SLO process and are the main gauge of student performance evaluated by educators (RSN, 2014).  Therefore, this question is of the utmost importance. There are two main schools of thought on the answer.  One is that standard assessments that have been pre approved should be used by teachers and districts to streamline validity.  The other, a more flexible approach, is that an assessment can be “...any measure that allows students to effectively demonstrate what they know and can do…” (RSN, 2014, p. 14) and can be created by teacher teams and include standardized assessments.  

Let’s look at both of these options in the framework of three important criteria for assessment as laid out by the Ohio Department of Education (2016).  These criteria are (a) Alignment to Standards, (b) Stretch, and (c) Validity and Reliability.  

First, how well do these two schools of thought fit into the need of assessments to align to standards?  This alignment to standards in an assessment would mean that items on the assessment would cover all the standards for that grade or subject, not cover standards outside of the scope of the course or grade, and distribute questions relative to the time spent on each standard (ODOE, 2016).  Pre approved or commercially constructed assessments may have the advantage here as they are inherently based on standards and have content directly based on standards either common core or state level.  However, these pre-created tests may break the last caveat in assessment selection noted above.  Since these tests are produced at a national or state level, they may include or be missing questions pertaining to standards not covered in a course or grade, thereby detracting from the validity of the test itself.

Teacher team or educator created assessments may lack the streamline connection to subject or grade as provided by pre approved tests.  In addition, if teacher teams from different districts and schools all create assessments it will be more complex to measure student growth on the macro scale at a district or state level since there will be no standardized format or list of questions.  This could also be seen as an advantage, however.  Especially in the current landscape of pandemic reconstruction, teachers will know what standards have been prioritized at district and school levels and may be able to create assessments that will better represent the actual standards that have been addressed.  Even without the facet of a pandemic this would be true.

How do these approaches align with the idea of stretch?  Stretch refers to the ability of an assessment to show the growth of the lowest and highest achieving students (ODOE, 2016).  Although many pre approved tests have this in mind, they may be less flexible in this area.  Since they are by definition standard, they may not have the questions that will show the achievement of both the highest and lowest learners.  Another consideration is that often content is not changed for students with different needs, but rather accommodations are created for those students to take the same test.  

In contrast, a teacher team built assessment may have input from teachers of those high and low students, giving the opportunity to create sections and questions that will rigorously examine growth of both groups.  The downfall to avoid here is to create a test specifically geared toward low or high learners as it skews the validity and reliability of the data collected.  Teachers input can also create an assessment that is not simply loaded with accommodations for students, but created with students in mind.

Finally, how do these models stand up to the criteria of reliability and validity?  In other terms, assessments should create consistent results and measure what they are intended to measure (ODOE, 2016).  Assessments can be evaluated for their validity and reliability on four important guideposts.  

• The assessment should not have overly complicated vocabulary, unless testing reading skills, or have overly complicated language.

• Test items should be written clearly and concisely.  Performance assessments must have clear steps.

• Clear rubrics and scoring guides should be provided, especially for performance assessments.

• Testing conditions should be consistent across classes. (ODOE, 2016).

Pre approved assessments may or may not follow these guideposts.  Many times these assessments have been created by teams that think through the wording of questions endlessly.  This does not mean that there have not been many examples of questions that were not understandable to students because of wording or even the choice of activity listed in a story problem.  Standardized assessments are often created based on majority groups that can leave some minority groups confused by wording or topic (Kim & Zabelina, 2015).  These tests can also have confusing scoring guides or may not even have rubrics since the use of standardized performance assessment is still relatively young.  They do have the distinct advantage of consistency in conditions and instructions since those are often strictly laid out.

Teacher created assessments may solve some of these problems.  Since teachers know their students, they may be able to create assessments that are manageable for students as well as avoiding culturally biased pitfalls of topics.  Rubrics can be created by teacher teams for specific performance assessment when those assessments are created.  However, the problems that may arise are lack of validity and reliability from poorly constructed tests or rubrics and guides.  Consistency of test conditions would be on the shoulders of teachers, schools, and districts to ensure that they were as standardized as possible.  Avoiding large incongruities would definitely be more work for the educators on the ground.

Overall, there are both pros and cons to each system.  A basic philosophical difference lies in the two different schools of thought on assessing for SLOs.  In the corner of pre approved assessments the philosophy is one of consistency and uniformity to gather data.  Its logic and practicality are obvious.  In the other corner, the philosophy is that of shifting power to the teachers and educators that work with students every day. Trading uniformity for the hope of deeper understanding.  John R. Troutman McCrann (2018) sums up this philosophy in one pithy comment. He says, “...it starts and ends with a very simple idea: I, the students' teacher, have expert knowledge about my students and...content standards, so I ought to have the power to assess those students' growth on those standards” (para. 5).  The issue is a complex one, but one that demands debate to ensure the students that represent that “S” in SLO achieve the most that they can.

References

Kim, K. H., & Zabelina, D. (2015). Cultural Bias in Assessment: Can Creativity Assessment Help? International Journal of Critical Pedagogy, 6, 129–147. Retrieved from http://libjournal.uncg.edu/ijcp/article/viewFile/301/856#:~:text=Standardized tests intend to measure,language backgrounds, socioeconomic status, and

Ohio Department of Education (ODOE). (2016). A Guide to Using SLOs as a Locally-Determined Measure of Student Growth (Rep. No. Guidebook). Retrieved from https://education.ohio.gov/getattachment/Topics/Teaching/Educator-Evaluation-System/Ohio-s-Teacher-Evaluation-System/Student-Growth-Measures/Student-Learning-Objective-Examples/SLO-Guidebook-041516.pdf.aspx

Reform Support Network (RSN). (2014). A toolkit for implementing high-quality student learning objectives 2.0. Retrieved from https://www2.ed.gov/about/inits/ed/implementation-support-unit/tech-assist/toolkit-implementing-learning-objectives-2-0.pdf

Troutman McCrann, J. R. (2018). Putting Assessment Back in the Hands of Teachers. Educational Leadership, 75(5), 41-45. Retrieved from http://www.ascd.org/publications/educational-leadership/feb18/vol75/num05/Putting-Assessment-Back-in-the-Hands-of-Teachers.aspx

Creating Higher Level Study Guides

Goodwin and Hubbell (2013) remind us that we as educators should peel back the curtain to make our expectations clear.  Wiggins and McTighe (2006) suggest that we should build our lessons with the end in mind.  Does this mean that students should be able to see the end of year/unit summative assessment to use as a guide? The simple answer; yes, but no.  Let me explain.

Being clear about what is being assessed is very important.  Teachers should, in precise ways, clearly articulate expectations to students and what it looks like when those objectives are met (Goodwin & Hubbell, 2013).  So it would stand to reason that if the ultimate summative assessment of those objectives was a test, that to give the students the test up front in the form of a study guide would do just that.  Students could see exactly what information was needed and how it should be regurgitated on the test.  This would, in fact, be very useful for the student that is needing to complete work on the bottom of Bloom’s taxonomy.  In the graphic below from Vanderbuilt (2016), in the base of the pyramid we see the tasks that would best be suited by this plan.  Students could duplicate, list and memorize using the test study guide as a template.

There are many situations in which this would be a handy tool, especially in the field of elementary education.  In an elementary setting, students often need to repeat and memorize information since it forms the basis of moving to understanding in this taxonomy.  The question remains then, are these study guides only for that basic level of the taxonomy?  Or can a guide based on the summative assessment be valuable for a higher order level of understanding?

Study guides themselves are useful tools.  Study guides “...enable students to draw upon their existing knowledge to assist them in formulating meaning from the text, are constructive, dynamic (affective and cognitive), and interactive tools” (Hollingsead et al., 2004, p. 57). The use of study guides has been linked to percentile growth in several studies with some gains in the 70th and 80th percentiles  (Hollingsead et al., 2004).  However, simply giving students the answers to the test in a different format, in this case a study guide, could be argued only to be at the bottom level of the taxonomy regardless of the style.  So this couldn’t be the type of study guide that is described in the quote above.  Simply memorizing an answer can’t truly be a dynamic interactive tool. Not to mention that effective study guides are personal to students and their learning styles (Thurler, n.d.).

Kasey Short (2019) gives a great metacognitive method for students to create their own study guides.  

1. First students answer questions about the content that will be on the assessment or review questions created by the teacher.

2. Students then individually write down or draw everything that they know about the topic from memory.  This should be done in regular pencil. A digital version of this would be typing everything they know in black text.

3. Students are then put into groups and discuss what they wrote down or drew on their papers.  As the group discusses, students write down information from peers that they didn’t have in new colors.  Again, digitally, this could look like adding new ideas in new text colors to your doc or sharing docs as a group to get the new info onto your study guide.  During this time, the teacher walks the room looking for gaps or misunderstandings.

4. Ask the students what they should be studying for the assessment.  Many will say that they should study the information in black, but remind them that this is info that they already knew.  You can celebrate that fact.  Tell them that the info on their page that is in different colors are things they need to spend more time on when preparing for the assessment.

5. Students take one more look at the review questions and determine what they need to add to their study guide.

In this way students will be able to have the use of a study guide that will be more like the dynamic and interactive one mentioned by Hollingsead et al (2004) and also get the content that you want them to see without handing them a copy of the test to memorize.  Hopefully, this leads them up the pyramid to understanding and not just remembering. 

References

Goodwin, B., & Hubble, E. R. (2013). The 12 touchstones of good teaching: a checklist for staying focused every day. ASCD, McREL International.

Hollingsead, C. C., Ostrander, R. J., & Schilling, J. (2004). Study Guides: Teacher Tips: A Review of Literature with Practical Implications. Language Arts Journal of Michigan, 20(2), 57–64. Retrieved from https://scholarworks.gvsu.edu/cgi/viewcontent.cgi?article=1234&context=lajm

Short, K. (2019, July 17). Study Strategies Beyond Memorization. Edutopia. Retrieved from https://www.edutopia.org/article/study-strategies-beyond-memorization

Thurler, P. (n.d.). How to Create a Successful Study Guide. Herzing University Blog. Retrieved from https://www.herzing.edu/blog/how-create-successful-study-guide

Vanderbilt University Center for Teaching. (2016, Sept. 6). Bloom’s taxonomy [Image file]. Retrieved from https://www.flickr.com/photos/vandycft/29428436431

Wiggins, G., & McTighe, J. (2006). Understanding by design (2nd ed.). Upper Saddle River, NJ: Pearson Prentice Hall.

Are Summative Tests Working?

Even though the landscape of education has changed dramatically in the last 70 years the format of standardized tests have remained oddly the same since the 1950’s (Bryant, 2018).  Here in Colorado, students prepare to take the Colorado Measures of Academic Success (CMAS) assessments each spring.  Having been a proctor and technology assistant for the testing, I can attest that it looks much like the way a standardized test has always looked, except with computers.  Students must sit in a defined space, making no noise, with strict time limits to answer mostly multiple choice questions. The format very much adheres to the values of the Industrialized 20th century: cost efficiency, quantifiability, uniformity, speed, and mass production (Bryant, 2018).  So then our questions should be, does that age-old format work for modern learners and if not what should replace the current system?  

Let’s take a look at the first question.  Other Organisation for Economic Co-operation and Development (OECD) countries have focused their energy toward test validity while the U.S. has stayed on the track of test reliability (Vander Ark, 2019).  In research from the PISA given in 2015 the U.S. ranks in the middle of the pack, well behind many other advanced industrial nations (DeSilver, 2017)  In this table from the Pew Research Center, we can see that several other nations are far ahead of the U.S. in math, science, and reading.  Of course, these numbers are derived from a standardized test and full systems of education are much more complex than just summative tests.  However, it can be gleaned from these numbers that something isn’t quite right with the current system.  Bryant (2018) asserts that “...the fixed content and rigid testing conditions [used in the U.S.] severely constrain the skills and knowledge that can be assessed” (para. 3).  This includes skills created by substantive and authentic learning experiences and essential non-cognitive skills like resilience or collaboration (Bryant, 2018).  On a more simple level, standardized tests don’t take into account all the knowledge teachers have of their students (Vander Ark, 2019).

So what should replace this system if it isn’t working?  We need and will continue to need some sort of large scale testing.  Just classroom-based assessments are not enough for districts, states or even the country to gain data. Without this data we wouldn’t be able to see what is working and what is not, where resources are needed or differentiate between groups or subgroups (Bryant, 2018).  Since this is the case we need to be able to make the tests more valuable by creating them to encompass a wider range of important academic skills.  In his book The Promise of Next Generation Assessment David Conely outlines 10 Principles for Better Assessment (Getting Smart Staff, 2018) that include the idea of cumulative validity.  This idea takes advantage of many points of data about a student.  The data can include classroom-based evidence, continuous assessment, real-word and performance based assessments, the addition of diploma networks (like IB), and the use of AI grading of digital portfolios.  All these can be utilized to make the large-scale summative test work toward validity and ultimately the benefit of students.

References

Bryant, W. (2018, February 20). The Future of Testing. Getting Smart. Retrieved from https://www.gettingsmart.com/2018/02/the-future-of-testing/

DeSilver, D. (2017, February 15). U.S. students’ academic achievement still lags that of their peers in many other countries. Pew Research Center. Retrieved from https://www.pewresearch.org/fact-tank/2017/02/15/u-s-students-internationally-math-science/

Getting Smart Staff. (2018, September 20). David Conley on Next Generation Assessment. Getting Smart. Retrieved from https://www.gettingsmart.com/2018/09/david-conley-on-next-generation-assessment/

Vander Ark, T. (2019, April 1). A Proposal For The End Of Standardized Testing. Forbes. Retrieved from https://www.forbes.com/sites/tomvanderark/2019/04/01/a-proposal-for-the-end-of-standardized-testing/#7860494621d8