Assessment
My goals for teaching next year are as follows:
1. Work on differentiating instruction. To do this effectively, I need to collect more accurate and detailed information about exactly where each student stands in terms of understanding a particular concept. I also need to know a bit more about my students' preferences in terms of presentation formats - while it may be true that all children can benefit from a mix of visual, auditory, and kinesthetic teaching styles, it is also true that some children do a really good job showing that they learned by writing an essay, while others may do better by drawing a cartoon strip or making a poster. I would like to balancne assignments where everyone creates the same type of product with assignments where the students have some choice in the type of product they create.
2. Be calmer and more patient. I think this one's pretty self-explanatory!
3. Create better systems for helping students get organized and for keeping track of things such as make-up work for absentees. My grade level team decided that each teacher will have a manila folder for each child in his or her homeroom. Anytime a child is absent, his or her folder will travel from class to class that day in the hands of a student monitor. That student will put a copy of every hand out into the folder, so that at the end of the day, the homeroom teacher has a folder with all the missed work in it. If we get really ambitious, we may have a student who is really good at taking notes photocopy his or her notes at the end of each day, and add those to the absentee folders as well. We are also developing a regular notebook checking policy. As for helping the students get organized, I am consciously building in one period at the end of each unit for the student to look through his or her folder, throw out stuff that is not needed anymore, select his or her best work (and all quizzes) to go in a portfolio, and choose any reference hand-outs to keep for the next unit.
4. Finally, in order to make #1 happen, I need to change the type of assessment data that I collect and the method I use to record it. This is where Transforming Classroom Grading by Robert J. Marzano, comes in.
I wish that schools of education included at least one really comprehensive, rigorous, and hands-on class on assessment for all teachers. (Are you listening, Jenny?). Think about it. It could include a couple of classes on the history of assessment - how did our current grading systems come to be? What about standardized tests? It could include enough statistics for teachers to understand the advantages and pitfalls of different kinds of grading systems (for example, what does it really mean to "curve" a grade?). And then it could take a close look at different methods of grading student work, from day-to-day classwork to final exams and standardized tests. Teachers could practice developing high-quality tests, useful rubrics, etc., and perhaps even look at examples of student work brought in by in-service teachers and practice grading them using different systems.
Marzano's book could be one of the texts.
He argues that the traditional A-F or percent systems don't really tell students, parents, or teachers much about the student's strengths and weaknesses. He argues for and demonstrates a grading system using a 0-4 rubric that is designed to give useful feedback to both students and teachers about what skills the student has mastered and how they are doing on other skills. New York State uses a 0-4 rubric on standardized tests, so the book is particularly useful to teachers here. My school is in the process of changing all of our grading to that same rubric, so that information is presented in a consistent format at all levels. We then convert the rubric score to a percentage and report both in order to keep high schools happy.
Last year, I began discovering my own ways of making my assessment fit within the rubric system. If I gave a quiz, I graded each question on the 0-4 rubric, and if the quiz had matching or multiple choice sections, I graded those as a whole by deciding what number correct would equal what rubric score. Then I averaged the rubric scores to get a final rubric score for the quiz. In the end I found that this slightly raised students' grades, but not in a way that I found unreasonable. Projects are not too hard to score on a rubric, and it is in fact faster and feels more fair than trying to figure out a percentage grade for a poster or presentation. I make a little grading slip with all the rubric criteria and the possible scores, and then just check off the score for each aspect of the project; I think this very clearly shows the student where he or she lost points.
Marzano would take this further. Instead of computing a total score for the project or quiz, he suggests that you find a rubric score for each important topic that you are assessing using this assignment. For example, a lab report grade might include experiment design, prediction, data collection, analysis, and professional presentation. The teacher would give a separate rubric score for each of these, and record them in her grade book as such. What makes the book so helpful is that he gives examples of how this works on a quiz, including a copy of the quiz and two students' answers and grades assigned by teachers along with their rationale for each grade.
He also includes an example of what a gradebook looks like if it is set up to accomodate topic-based rubric scores in this way. Essentially, you have a larger section for each student. You list the topics that you are assessing across the top (columns) and the assignments along the side (rows). One assignment might have scores for several topics, so you enter the scores in the appropriate sections of the grid. You have a section like this for each student. It's easier to follow if you see it.
Okay, up to that point I am with him completely. As the marking period progresses, you collect really specific data about which topics that student is strong in, and where they are struggling. SO USEFUL!!! If a kid isn't doing well because they understand the topics but they don't present their ideas well, you see that. If they did great at understanding simple machines but their mathematical skills are week, you see that. Awesome. But what do you do at the end of the term?
This is where Marzano gets a little controversial. Actually, his argument might not be controversial at all to statisticians (I don't know) and isn't controversial among teachers ('cause we've mostly never heard of it before), but I predict that it WOULD BE controversial if proposed on a widespread basis to educators. He suggests using the Power Law to compute final grades. That means, instead of taking an average of the student's rubric scores for each topic, you apply a formula which fits their grades to a curve. He argues that many experiments have shown that learning closely approximate a power law curve over time, and this reduces small errors in assessment made by teachers or due to various kinds of (statistical) bias. Hmmmm. I don't exactly disagree, I mean, I've heard of the Power Law before and have even read Linked, which is ALL ABOUT that curve and how it applies in an astonishing variety of situations.... but it feels a little nebulous to me.
I guess this is the same discomfort people felt when the Census Bureau tried to use sampling rather than actually counting every darn person. The census people probably had the math right, but it's a hard one to explain to the average person.
I mean, how the heck would I explain this computation to parents? I guess if the grades seem about right, most parents will not question them at least on a mathematical level, but I have had one family compute all their son's grades and challenge me on an issue of rounding, so you can't rule that out entirely.
Marzano does include examples of a student's grades, what the average score would be, and what a power curve would look like for those grades. And guess what? The power curve does approximate the scores more closely than the average! That is, the curve really closely tracks the trend in the student's scores over time, while the average is just a line drawn straight across. So, the curve seems to do a better job taking into account learning over time, compared to the more static average.
I am definitely adopting the new gradebook design, and when it comes time to compute grades, I will experiment with both the power law and the average and see which ones feel more accurate and fair. I need more empirical evidence, I guess.
I'm really curious what YOU think about this one.
*****
Here's some on-line reading if you want to know more (or if my explanation made absolutely no sense).
Power Law of Practice - states it simply along with some of the areas of learning in which it has been found to apply.
Serial Learning (and Savings) - provides two easy try-this-at-home lab activities to test statistical models of learning. I wish this guy had taught my stats class!
Distinguishing Qualitatively Different Kinds of Learning Using Log Files and Learning Curves - An academic but readable paper looking at learning curves as applied to students using tutoring software (LISP) to learn to use a spreadsheet program. Here's an excerpt:
I think the quote above sums up why more people - particularly teachers - have never noticed or heard of the power law of learning: it shows up when you look at learning of one particular skill compared to time spent practicing that particular skill. Traditional grades where scores for different topics and skills are all mixed up together wouldn't necessarily show this kind of trend. That's why I'm going to wait until I've collected data by topic and then test this theory.
1. Work on differentiating instruction. To do this effectively, I need to collect more accurate and detailed information about exactly where each student stands in terms of understanding a particular concept. I also need to know a bit more about my students' preferences in terms of presentation formats - while it may be true that all children can benefit from a mix of visual, auditory, and kinesthetic teaching styles, it is also true that some children do a really good job showing that they learned by writing an essay, while others may do better by drawing a cartoon strip or making a poster. I would like to balancne assignments where everyone creates the same type of product with assignments where the students have some choice in the type of product they create.
2. Be calmer and more patient. I think this one's pretty self-explanatory!
3. Create better systems for helping students get organized and for keeping track of things such as make-up work for absentees. My grade level team decided that each teacher will have a manila folder for each child in his or her homeroom. Anytime a child is absent, his or her folder will travel from class to class that day in the hands of a student monitor. That student will put a copy of every hand out into the folder, so that at the end of the day, the homeroom teacher has a folder with all the missed work in it. If we get really ambitious, we may have a student who is really good at taking notes photocopy his or her notes at the end of each day, and add those to the absentee folders as well. We are also developing a regular notebook checking policy. As for helping the students get organized, I am consciously building in one period at the end of each unit for the student to look through his or her folder, throw out stuff that is not needed anymore, select his or her best work (and all quizzes) to go in a portfolio, and choose any reference hand-outs to keep for the next unit.
4. Finally, in order to make #1 happen, I need to change the type of assessment data that I collect and the method I use to record it. This is where Transforming Classroom Grading by Robert J. Marzano, comes in.
I wish that schools of education included at least one really comprehensive, rigorous, and hands-on class on assessment for all teachers. (Are you listening, Jenny?). Think about it. It could include a couple of classes on the history of assessment - how did our current grading systems come to be? What about standardized tests? It could include enough statistics for teachers to understand the advantages and pitfalls of different kinds of grading systems (for example, what does it really mean to "curve" a grade?). And then it could take a close look at different methods of grading student work, from day-to-day classwork to final exams and standardized tests. Teachers could practice developing high-quality tests, useful rubrics, etc., and perhaps even look at examples of student work brought in by in-service teachers and practice grading them using different systems.
Marzano's book could be one of the texts.
He argues that the traditional A-F or percent systems don't really tell students, parents, or teachers much about the student's strengths and weaknesses. He argues for and demonstrates a grading system using a 0-4 rubric that is designed to give useful feedback to both students and teachers about what skills the student has mastered and how they are doing on other skills. New York State uses a 0-4 rubric on standardized tests, so the book is particularly useful to teachers here. My school is in the process of changing all of our grading to that same rubric, so that information is presented in a consistent format at all levels. We then convert the rubric score to a percentage and report both in order to keep high schools happy.
Last year, I began discovering my own ways of making my assessment fit within the rubric system. If I gave a quiz, I graded each question on the 0-4 rubric, and if the quiz had matching or multiple choice sections, I graded those as a whole by deciding what number correct would equal what rubric score. Then I averaged the rubric scores to get a final rubric score for the quiz. In the end I found that this slightly raised students' grades, but not in a way that I found unreasonable. Projects are not too hard to score on a rubric, and it is in fact faster and feels more fair than trying to figure out a percentage grade for a poster or presentation. I make a little grading slip with all the rubric criteria and the possible scores, and then just check off the score for each aspect of the project; I think this very clearly shows the student where he or she lost points.
Marzano would take this further. Instead of computing a total score for the project or quiz, he suggests that you find a rubric score for each important topic that you are assessing using this assignment. For example, a lab report grade might include experiment design, prediction, data collection, analysis, and professional presentation. The teacher would give a separate rubric score for each of these, and record them in her grade book as such. What makes the book so helpful is that he gives examples of how this works on a quiz, including a copy of the quiz and two students' answers and grades assigned by teachers along with their rationale for each grade.
He also includes an example of what a gradebook looks like if it is set up to accomodate topic-based rubric scores in this way. Essentially, you have a larger section for each student. You list the topics that you are assessing across the top (columns) and the assignments along the side (rows). One assignment might have scores for several topics, so you enter the scores in the appropriate sections of the grid. You have a section like this for each student. It's easier to follow if you see it.
Okay, up to that point I am with him completely. As the marking period progresses, you collect really specific data about which topics that student is strong in, and where they are struggling. SO USEFUL!!! If a kid isn't doing well because they understand the topics but they don't present their ideas well, you see that. If they did great at understanding simple machines but their mathematical skills are week, you see that. Awesome. But what do you do at the end of the term?
This is where Marzano gets a little controversial. Actually, his argument might not be controversial at all to statisticians (I don't know) and isn't controversial among teachers ('cause we've mostly never heard of it before), but I predict that it WOULD BE controversial if proposed on a widespread basis to educators. He suggests using the Power Law to compute final grades. That means, instead of taking an average of the student's rubric scores for each topic, you apply a formula which fits their grades to a curve. He argues that many experiments have shown that learning closely approximate a power law curve over time, and this reduces small errors in assessment made by teachers or due to various kinds of (statistical) bias. Hmmmm. I don't exactly disagree, I mean, I've heard of the Power Law before and have even read Linked, which is ALL ABOUT that curve and how it applies in an astonishing variety of situations.... but it feels a little nebulous to me.
I guess this is the same discomfort people felt when the Census Bureau tried to use sampling rather than actually counting every darn person. The census people probably had the math right, but it's a hard one to explain to the average person.
I mean, how the heck would I explain this computation to parents? I guess if the grades seem about right, most parents will not question them at least on a mathematical level, but I have had one family compute all their son's grades and challenge me on an issue of rounding, so you can't rule that out entirely.
Marzano does include examples of a student's grades, what the average score would be, and what a power curve would look like for those grades. And guess what? The power curve does approximate the scores more closely than the average! That is, the curve really closely tracks the trend in the student's scores over time, while the average is just a line drawn straight across. So, the curve seems to do a better job taking into account learning over time, compared to the more static average.
I am definitely adopting the new gradebook design, and when it comes time to compute grades, I will experiment with both the power law and the average and see which ones feel more accurate and fair. I need more empirical evidence, I guess.
I'm really curious what YOU think about this one.
*****
Here's some on-line reading if you want to know more (or if my explanation made absolutely no sense).
Power Law of Practice - states it simply along with some of the areas of learning in which it has been found to apply.
Serial Learning (and Savings) - provides two easy try-this-at-home lab activities to test statistical models of learning. I wish this guy had taught my stats class!
Distinguishing Qualitatively Different Kinds of Learning Using Log Files and Learning Curves - An academic but readable paper looking at learning curves as applied to students using tutoring software (LISP) to learn to use a spreadsheet program. Here's an excerpt:
Anderson, Conrad, and Corbett have demonstrated that the power law relationship between practice and
performance may not always be readily evident in the practice of complex skills. For instance student
performance over the course of individual steps in each practice problem in the LISP tutor showed a rather
chaotic pattern. However, by decomposing the skill required to program in LISP in terms of production
rules, and examining performance as a function of the opportunity to practice each of the underlying rules
Anderson and colleagues observed a clear power law relationship between practice and performance.
I think the quote above sums up why more people - particularly teachers - have never noticed or heard of the power law of learning: it shows up when you look at learning of one particular skill compared to time spent practicing that particular skill. Traditional grades where scores for different topics and skills are all mixed up together wouldn't necessarily show this kind of trend. That's why I'm going to wait until I've collected data by topic and then test this theory.
posted by ms. v. at 9:10 AM
4 Comments:
A gift basket is one of the most underrated gifts, ever!
Hello,
Your site is the nicest one I have seen for a long time. I would be interested if could you send me a newsletter or some further information.
Regards,
cash back
Do not miss this beautiful Gourmet Groom Bride Gift.
Your blogs is very good and nice content,very use full also keep it up.my website is Niches given to all products information.
niches
Post a Comment
<< Home