The current conventional wisdom seems to be that “spit back” questions on tests are bad and archaic and that “thought” questions are the ideal of what a test ought to be. I would like to question this line of thought on several levels (I am only discussing from a high school teacher’s perspective, since that’s what I’m qualified to discuss; I understand that it would be a very different discussion if we are talking about elementary school or post high school teaching). 1] I certainly agree that thought questions are a wonderful method of taking the material being taught to another level, but I think that it is better used as a homework or classroom discussion than as a test question, for the following reasons: A] Thought questions ought to be just that, questions that the students have time to think about. Giving it as homework gives the student time to think it over in a meaningful way, whereas a test is about how quickly they can think on the spot. Usually they give the thought part just a brief amount of time, since there are many other questions that need to be answered and the test time has very definite limits. I feel like teachers will often spend somewhere in the range of an hour thinking of the thought question, but then expect the student to answer it in about 5 minutes. B] As much as we want the test to be a learning experience, at the end of the day, we need to give a grade and have a fair system for what that grade ought to be. Thought questions are often open ended and can be analyzed in several ways. I feel like we often fall into one of 2 traps 1] the “can you guess how the teacher thinks” way of grading instead of having an objective standard (if you ask your thought question to another teacher doing the same material, I dare say that you will often get a different take than the one that you had and expect your students to come up with) or 2] giving credit for pretty much anything somewhat intelligent that they write, which, I think, undermines the value of the question as part of a test. I know that people will say that they use rubrics and I think that rubrics that are given to a student before they do an assignment are fair guidelines. However, I don’t think it’s so fair to have a thought question where the students need to “guess” what your rubrics are. C]I think it is a truism for high school homework that anything that can copied, will be copied and that thought questions for homework are more meaningful and less likely to be just passed around during breakfast. Now, one could reasonably argue that giving an oral bechina instead of a written one would deal with many of these points, but I don’t think that the average high school set up allows for doing that in a meaningful way. 2] I am sympathetic to the fear that when there are “spit back” questions on tests, the kids are just memorizing their notes and not really understanding them at all. However, I feel like the “spit back is all bad” way of thinking is overly simplistic. At the end of the day, our students should walk away with a baseline of knowledge of what they’ve learned. If we did a machloket ibn ezra/ramban in chumash or rashi/tosfot in gemara, they should be able to “spit it back.” First of all, any subsequent meaningful discussion requires, as a prerequisite, thorough prior knowledge of the material. Second of all, I don’t think our goal is just to enable future discussion. Our students knowing the material is a value in its own right and just knowing what rashi versus tosfot said is important in and of itself. I believe that raising the level of Jewish literacy of our students ought to be one of our major goals as teachers. I also think, and realize that now I’m on shakier ground, that thought questions often involve taking out the basic overall concept in a topic and don’t necessarily show a thorough knowledge of the overall subject. So for example, I don’t think that the take away from learning the basic rules of Shabbat ought to just be the ideas of davar sheino mitkavan, pesik reisha etc, I think that the kids ought to know the gemarot about each one and which cases the rishonim debated (assuming that that’s what you learned in class). Giving just application questions to those concepts (which is, I think, a good thought question and much less open ended than the average) is, I think, not enough. I am sure that many of the readers will disagree what parts (if not all) of what I am saying and certainly respect the other points of view. I have seen thought questions on tests that were brilliant, but have all too often seen ones that are, I think, open to my aforementioned critiques. Yaakov Blau http://lookstein.org/lookjed/read.php?1,20340,20340#msg-20340 Tue, 21 May 2013 03:39:42 -0600 Phorum 5.2.19 http://lookstein.org/lookjed/read.php?1,20340,20377#msg-20377 http://lookstein.org/lookjed/read.php?1,20340,20377#msg-20377
If we look at the learning experience that we are trying to convey, there is a combination of content to be learned and critical thinking that we seek to impart. To our benefit and detriment, many Lookstein contributors have been the recipients of more of a memorization type of approach in our own educations. Cultivating good memory skills is an necessary (yet insufficient)asset in terms of amassing a body of information and as a general life skill. But, it is also an asset in conjunction with critical thinking of allowing for setting the stage to being “medameh davar l’davar”.

I think that we have to all come to grips with what has sort of emerged as a “new normal” in terms of how people take in information, as a function of reliance on technology primarily. We have calculators and GPS’s which provide shortcuts to arriving at a final answer or a final destination. In the short terms that might work. But, it comes at the expense of young people developing life skills like simple computation and navigation. If the technology is somehow not available, people are lost. Furthermore, communication is quick and short, but often sloppy, much to the chagrin of us older folks. It is hard to say whether the emerging generation will realize that having these skills at least as a "contingency plan" (for technology inaccessibility) is of value.

As far as structuring a course, evaluation opportunities should include a combination of assignments, tests, and perhaps some form of “presentation”. While some students might excel in all three forms of evaluation, some students do better on tests, and some do better on assignments and presentations. So, that's a good thing.

In terms of testing, there is a pragmatic advantage to say a short-answer or multiple-choice over essay. That is the former tests are easier to score, as well as it s easier to justify to students why they were marked right or wrong. While these tests are more challenging to develop, they are honestly easier to score, not to mention contributing to fairness and objectivity. The former can and should probably have elements of memorization of material as well. When I develop my exams, I try to include some multiple-choice items which call on memory and some which require critical thinking to answer correctly.

On an exam, there might also be room for a few essay questions, which will draw on more critical thinking. (On the other hand, given that the aforementioned communication trend has been more towards quick and short, we then return to the short-answer and multiple choice as a reflection of the current reality.)

Of course, the challenge is developing a course evaluation system which supports the learning objectives of the course in these areas.

Elly D. Lasson, Ph.D.
Baltimore, MD]]> Elly Lasson Lookjed List Archive Sun, 05 Feb 2012 09:50:35 -0700 http://lookstein.org/lookjed/read.php?1,20340,20360#msg-20360 http://lookstein.org/lookjed/read.php?1,20340,20360#msg-20360
Before I weigh in with my $0.02, I beg forgiveness of the audience with the length of my post, but offer the following personal vignette as a caveat to my opinion.

Attending the Yeshiva of Flatbush HS in the 80's, I was required to memorize psukim from chumash and navi, nivim, and a plethora of other passages on a weekly or biweekly basis. But there was a catch, I cannot memorize. (This often comes as a shock to people who discuss Torah, halacha, and educational matters with me because I often quote chapter and verse, though not verbatim. I will reference this later) No matter what method I tried I could not succeed and time after time was handed back a paper with a big zero on it and another communication home to my parents. What I quickly learned was that I could cheat if I learned the teachers' styles of testing. I would spend the first three weeks of the term learning the style and then prepare the instrument of deception that I needed to succeed. In college I prevailed upon professors to allow me to learn the passages rather than memorize them. By then, I found that if I studied the linguistic aspects of the text, and formed meaning from it, I was able to recall most of it, though not verbatim (hence my earlier parenthetical reference).

For centuries, mastery of learning was demonstrated by memory. For years, Chumash, Mishna, and Talmud were mastered by shinun, with a select few developing the skills to take their memorized passages into the next realm of understanding and higher order thinking skills. Think of Rabbi Meir uprooting mountains and crushing to dust. The reason was that this was always the method of Talmud instruction because Talmud or Chumash, or Mishna, were taught by those who had mastered the subject through the same method. But that method evolved for a self selected group, not the nation as a whole or even a larger more heterogeneous group.

But reacting to years of disaffected students and the advances in technology including but not limited to such programs as Bonayich or Gemara berura, has made learning Gemara, learning, and not simply memorizing vocabulary and shakla v'tarya to be spit back at a faher.

While those who can memorize will have an advantage, memory is not really a skill to be honed in school. Educational research is focused on meaning. As mechanchim, meaning and value are the keys to developing lifelong learners. And even in the study of vocabulary, memory is not the mode of learning.

Look at most vocabulary books. Back when, students were given a list of words and were expected to memorize them. At some levels students were expected to spit them back with definitions and at the more advanced level they were expected to have a sentence that demonstrated their knowledge of the usage of the word. But in our modern day, words are given with many of their nuanced meanings, and pages of exercises that develop an appreciation for the different uses of the word are offered. Then the more creative teachers assign writing exercises that promote further long term learning of the vocabulary list. That is learning. For the majority of students, the words are organically learned and stored in long-term memory. Any memorization for a test or quiz is minor and is simply review of a set of information already learned and assigned storage with value.

Studying lists of gemara vocabulary will not make a student learn more Gemara or Chumash or Navi. Meaning and comprehension allow for more text study. In my Talmud classes, I had the students construct phrases with the vocabulary and had a nonsensical exercise where they inserted the appropriate Talmud term into an English sentence. I found that using techniques similar to the vocabulary exercises increased my students' appreciation for Talmud and helped them retain the vocabulary for later learning.

Learning that results in long term memory is organic and simply memorizing is artificial (unless it comes naturally).]]> tzvi klugerman Lookjed List Archive Thu, 26 Jan 2012 00:44:12 -0700 http://lookstein.org/lookjed/read.php?1,20340,20358#msg-20358 http://lookstein.org/lookjed/read.php?1,20340,20358#msg-20358 I’d like to take the discussion one step further, but I believe that it’s the same basic topic. One of the terms that get used a good deal these days is “essential question.” Now, the term means many different things to different people (just google it and you will see what I mean), but there’s an extreme application that I disagree with strongly. I have heard it been suggested that before every lesson, the teacher should ask him/herself what the big picture take away is and why am I using this particular material to get to that take away. So, one should even question why they are teaching about the Civil War in History of Ein Mukdam U’meuchar Batorah in Chumash. Now, the straw man that often comes up in these discussions (I’ve seen it in several blogs) is the “old school” rebbe who just wants to teach the next daf and that’s the extent of his planning. I certainly agree that teachers should ask themselves what are the key words, important structures of the shakla v’tarya, key concepts etc in the given sugya that they are teaching(just to use gemara as an example, this would be true in every subject in a way that is appropriate for that subject). I also agree that a thoughtful teacher will determine which sugyot he/she does during the year depending on whether the kids will be able to understand it , will be engaged etc, although these latter considerations are highly subjective questions which very much depend on the particular teacher and class. However, I don’t think there’s any reason to question why a teacher should teach the Civil War or Ein Mukdam U’meuchar Batorah, the only justification needed is that if you don’t, your students will be ignorant of the basics. Sometimes you’ll do a sugya that isn’t so exciting, because it’s something that they ought to know. Yes, we should consider the big picture take away, but our students need to know things and that’s a value all by itself. (I will admit to having only skimmed “Understanding By Design”, but don’t think the authors meant to go to such extremes as the position that I am disagreeing with; they have a category of topical understanding and questions besides just overall understanding and questions.) I think that one of the essential questions we should be asking ourselves (and I use high school as an example, because that’s what I know) is will our students sound like a bunch of am aratzim when they get to yeshiva in Israel and how can we prevent that.
Yaakov Blau]]> Yaakov Blau Lookjed List Archive Tue, 24 Jan 2012 20:27:21 -0700 http://lookstein.org/lookjed/read.php?1,20340,20356#msg-20356 http://lookstein.org/lookjed/read.php?1,20340,20356#msg-20356
Regarding the query by my colleague at The Frisch School, Rabbi Yaakov
Blau on Thought questions vs "spit back", I believe that the best
answer, as is often the case, is there is a place for both on these
types of questions on well written assessments.

My philosophy towards testing is greatly influenced by a seminal class
that I took a number of years ago with Dr. Scott Goldberg in the
Azrieli Graduate School for Jewish Education and Administration. We
used the textbook, Classroom Assessment: Principles and Practice for
Effective Standards-Based Instruction by JH McMillan
(http://www.amazon.com/Classroom-Assessment-Principles-Standards-Based-Instruction/dp/0205485847)
which I would highly recommend. In the course, Dr. Goldberg advised
that the teacher first identify Learning Targets for their course and
then make sure that all assessments address at least some of these
goals. This is similar to the Understanding By Design approach where
you "Begin With the End in Mind", as Stephen Covey would say, or put
another way, סוף מעשה במחשבה תחילה, as we describe the Shabbat in לכה
דודי. The learning targets should include the areas of Knowledge and
Simple Understanding, Deep Understanding and Reasoning, Skills,
Products, and Affective Targets.

Obviously some of these targets would require "spit back" types of
questions while others would call for more "thought" based questions.
Even in reference to "spit back" questions, it is important to
recognize that there are different types of "spit back". One can ask
questions for simple knowledge like basic translation and information
and one can ask more sophisticated deeper understanding "spit back"
that requires the learner to follow the various stages in a logical
progression like the back and forth arguments in a long Tosfot. This
is the type of high-level "spit back" that I believe Rabbi Blau
recommends for formal exams. However, if one's targets include skills
like using keywords to read a Talmudic sugya or applying knowledge to
new situations then "thought" type questions are also warranted
whether in a formal test or using alternative forms of assessments as
Rabbi Aaron Ross advocated.

Technology can greatly assist in constructing these alternative
skills-based assessments for Talmud. For example, students can
indicate their mastery of the Shakla Vetarya of a sugya by breaking
down the stages of the Gemara and classifying them using a computer
program like Gemara Berura (www.gemaraberura.com). They can also read
the sugya for the teacher to listen to later using Voicethread
(www.voicethread.com), a web-based app that allows students to record
their voices using a computer's microphone . This is much more
practical than the oral testing that Rabbi Blau advances but points
out cannot easily be done in a Yeshiva Day School schedule. Since the
teacher can listen to the Voicethreads at his/her leisure, these types
of assignments can be assigned regularly. They can also include any
Hebrew text including Talmud, Tanach, Rishonim, or Acharonim so they
can be easly adapted for many levels of learners.

At the same time, when I have taught Gemara in the past, I also
included skills-based "thought" questions on my formal tests as well
by giving my students "unseen" Gemara texts containing the same
keyword structures studied in class. My students were naturally
worried about these "unseen" texts but soon realized that they were
eminently doable since I only asked them to replicate exactly the
skill learned in class; to use the keywords to explain what a new
Gemara is doing in terms of unlocking the Shakla Vetarya, the back and
forth of the debate, rather than decipher what an unfamiliar Gemara is
saying in terms of the content of the "unseen" Gemara. This is an
important skill for students to master so they can learn to "make a
laining" on a new Gemara as we would say in Yeshiva, to gain the
ability to independently read an unfamiliar Gemara. Since this is an
important learning target, naturally I included this on my exams. I
also included application questions on my tests as well, although I
usually limited them to one or two questions at the end for many of
the same reasons Rabbi Blau mentioned.

The bottom line is that one should test to the learning targets that
one teaches. Therefore, I believe that a good assessment should
include simple knowledge and deeper understanding "spit back"
questions, skills based questions, and some application type questions
as well. If a test is balanced and closely aligned to the clearly
communicated learning targets then students will adapt to them and use
these assessments to show their knowledge and grow in their
understanding.

I welcome continued feedback on this most fruitful discussion.

Kol Tuv,
Rabbi Tzvi Pittinsky
Director of Educational Technology
The Frisch School]]> Tzvi Pittinsky Lookjed List Archive Tue, 24 Jan 2012 10:43:19 -0700 http://lookstein.org/lookjed/read.php?1,20340,20355#msg-20355 http://lookstein.org/lookjed/read.php?1,20340,20355#msg-20355
Thinking questions often tend to demand from students thinking skills
that we have not necessarily taught them in class. Therefore, while
some students can answer these questions "correctly" (however that is
defined) without instruction and practice, many students cannot
because we have not worked on it during class. Assuming that the goal
of our assessments is to assess whether the students have mastered
what we have taught, these questions do not really achieve that goal.
Although I have found that students actually enjoy taking tests that
require them to think and apply their knowledge and not just "spit
back" information, I think that critical thinking should remain an
in-class or at-home activity unless we can honestly say that we have
focused on these skills in class enough such that it would fair to
assess our students on them.

--
Tova]]> Tova W. Sinensky Lookjed List Archive Tue, 24 Jan 2012 10:41:52 -0700 http://lookstein.org/lookjed/read.php?1,20340,20349#msg-20349 http://lookstein.org/lookjed/read.php?1,20340,20349#msg-20349
1. In the end it's about mastering detailed material and not just about general ideas or improvisation. It all runs through the details, even the ones we don't expect students to remember a year later.

2. The sense of achievement in memorizing material or a train of reasoning is a healthy one.

3. In real life, we are often called upon to present what we know clearly: with the logic transparent, with all the details set down in a comprehensible order, without extraneous details. This is a communication skill. It is very important if you are a physician who interacts with people, or an administrator, or a person using the services of such people etc etc. etc.]]> Shalom Carmy Lookjed List Archive Mon, 23 Jan 2012 14:09:53 -0700 http://lookstein.org/lookjed/read.php?1,20340,20347#msg-20347 http://lookstein.org/lookjed/read.php?1,20340,20347#msg-20347
1) He creates a straw man by claiming to argue against those who hold that spit back questions are awful education and we should only be giving thought questions to our students. I do not know too many educators who actually take such a position. My sense is that many educators value the more thoughtful and thought-provoking questions, yet realize that there is a certain degree of spit back necessary in order to ensure that students know the basics. And, of course, even spit back questions can be phrased in ways that force the students to think a bit and not simply spit back (without being the lengthy essays that Rabbi Blau posits should be done at home).

2) Rabbi Blau seems to feel that thought questions are too time-consuming (and perhaps deserve that amount of time) to be done in the context of a one period test. That position leads him to say that such questions should therefore not be done in class. However, why does Rabbi Blau not consider that perhaps conventional tests are not the best method of assessment for such classes? There is no shortage of ways in which to assess student understanding of material, and if standard tests do not allow a teacher to assess in the best way possible, then perhaps tests should be replaced by some other form of assessment. Rabbi Blau does allude to this by saying that thought questions should be homeworks - but why stop at homework? Perhaps give a short answer test accompanied by a more thoughtful and lengthy take-home assignment?

Rabbi Blau acknowledges this point in his discussion of rubrics. However, he seems to discount rubrics by writing "However, I don’t think it’s so fair to have a thought question where the students need to “guess” what your rubrics are." - No problem. If you use an alternative form of assessment, you have to make sure to tell the students the rubric in advance. I am not sure why Rabbi Blau discounts a widely-used method of grading simply because there are those people who do not know how to use them.

Bivracha,
Aaron Ross]]> Aaron Ross Lookjed List Archive Mon, 23 Jan 2012 01:48:04 -0700 http://lookstein.org/lookjed/read.php?1,20340,20340#msg-20340 http://lookstein.org/lookjed/read.php?1,20340,20340#msg-20340 1] I certainly agree that thought questions are a wonderful method of taking the material being taught to another level, but I think that it is better used as a homework or classroom discussion than as a test question, for the following reasons:
A] Thought questions ought to be just that, questions that the students have time to think about. Giving it as homework gives the student time to think it over in a meaningful way, whereas a test is about how quickly they can think on the spot. Usually they give the thought part just a brief amount of time, since there are many other questions that need to be answered and the test time has very definite limits. I feel like teachers will often spend somewhere in the range of an hour thinking of the thought question, but then expect the student to answer it in about 5 minutes.
B] As much as we want the test to be a learning experience, at the end of the day, we need to give a grade and have a fair system for what that grade ought to be. Thought questions are often open ended and can be analyzed in several ways. I feel like we often fall into one of 2 traps 1] the “can you guess how the teacher thinks” way of grading instead of having an objective standard (if you ask your thought question to another teacher doing the same material, I dare say that you will often get a different take than the one that you had and expect your students to come up with) or 2] giving credit for pretty much anything somewhat intelligent that they write, which, I think, undermines the value of the question as part of a test. I know that people will say that they use rubrics and I think that rubrics that are given to a student before they do an assignment are fair guidelines. However, I don’t think it’s so fair to have a thought question where the students need to “guess” what your rubrics are.
C]I think it is a truism for high school homework that anything that can copied, will be copied and that thought questions for homework are more meaningful and less likely to be just passed around during breakfast.
Now, one could reasonably argue that giving an oral bechina instead of a written one would deal with many of these points, but I don’t think that the average high school set up allows for doing that in a meaningful way.
2] I am sympathetic to the fear that when there are “spit back” questions on tests, the kids are just memorizing their notes and not really understanding them at all. However, I feel like the “spit back is all bad” way of thinking is overly simplistic. At the end of the day, our students should walk away with a baseline of knowledge of what they’ve learned. If we did a machloket ibn ezra/ramban in chumash or rashi/tosfot in gemara, they should be able to “spit it back.” First of all, any subsequent meaningful discussion requires, as a prerequisite, thorough prior knowledge of the material. Second of all, I don’t think our goal is just to enable future discussion. Our students knowing the material is a value in its own right and just knowing what rashi versus tosfot said is important in and of itself. I believe that raising the level of Jewish literacy of our students ought to be one of our major goals as teachers. I also think, and realize that now I’m on shakier ground, that thought questions often involve taking out the basic overall concept in a topic and don’t necessarily show a thorough knowledge of the overall subject. So for example, I don’t think that the take away from learning the basic rules of Shabbat ought to just be the ideas of davar sheino mitkavan, pesik reisha etc, I think that the kids ought to know the gemarot about each one and which cases the rishonim debated (assuming that that’s what you learned in class). Giving just application questions to those concepts (which is, I think, a good thought question and much less open ended than the average) is, I think, not enough.
I am sure that many of the readers will disagree what parts (if not all) of what I am saying and certainly respect the other points of view. I have seen thought questions on tests that were brilliant, but have all too often seen ones that are, I think, open to my aforementioned critiques.
Yaakov Blau]]> Yaakov Blau Lookjed List Archive Wed, 18 Jan 2012 09:53:22 -0700