Tuesday, December 12, 2006

Exam Design in the Physical Sciences

We are in the deepest, darkest part of our exam schedule and there is a fairly constant stream of students that come to my office with really one question on their mind and they cannot bring themselves to ask it directly so they ask a number of oblique questions that circumnavigate what they really want to ask.

What is on the exam?

In the Physical Sciences there is an expectation of content mastery that typically involves "problem solving" (now there is a term that by itself is a problem). The issue is always how much time do you as a professor allot for students to stare at the ceiling and think during a chemistry exam?

There are a number of aspects of this issue of exam design. Here in a small liberal arts and science college we have the luxury of being able to ask essay questions. This is because the numbers are small enough that we can cope with the marking. In larger universities, even if they don't just resort to multiple choice questions, quite often the questions are choped up into sub-questions so that the marking becomes a series of right / wrong decisions.

At its simplest level the expectation is that the professors will set an exam that the professors themselves can sit down and physically right out the complete, correct solution set in one third of the time allotted. Theoretically, this means that the average student (see below) will be able to write the exam in two thirds of the time allotted and still have one third of the time for thinking / correction / addition.

The real issue comes down to choice. In my exams there is typically choice, especially for the high value questions. Choice is also a luxury of the small liberal arts and science college. The issue with choice however is that there are really two kinds of high value choice questions: 1) the very long death-march question that wrings all the information out of the student that they have learned or 2) the shorter, thinking question that tests what the student understands. If a student decides to answer three death march questions they are going to run out of time and many a tear-stained final exam has ended with the ink trailing off the last page scrawling "ran out of time, exam too long". On the other hand if you have a student that is smarter than the average bear a canny choice of questions can have them out of the exam with a high mark even though they only wrote out one an a half pages of material.

No one ever said that life, or exams, were fair. A University is not a democracy, it is a meritocracy. Perversely, things get easier as you get smarter and harder as you get weaker. Almost Darwinian.

4 comments:

Mike said...

At its simplest level the expectation is that the professors will set an exam that the professors themselves can sit down and physically right out the complete, correct solution set in one third of the time allotted. Theoretically, this means that the average student (see below) will be able to write the exam in two thirds of the time allotted and still have one third of the time for thinking / correction / addition.

Wow, that means you are either a stenographer, or just extremely skilled with a quill. In one of my last essay exams (4 out of 7, 3hours), I wrote 22 pages and just finished on time - score was a high first class (UK). So you're saying a lecturer could shake more out of his/her wrist in a thrid of the time?

If this sounds arrogant or presumptuous please accept my apologies.

Liberal Arts Chemist said...

Well, first of all, in the physical sciences there are few courses that are ONLY tested with essay questions. I assume you would agree that mathematical solutions to even extended questions require physically less writing than essays. It is also true that in essay writing the length of the essay often bears little relationship to the content of the essay. Some people take a long time to make the five points required for full value. If you are going to argue that the final mark was an assessment of your writing style then you are on your own. If what I am saying is correct, the persons that set the test that you wrote did not assume three hours of writing would be required. Therefore the difference must have been in your approach to answering the questions or a general failure of the test design.

Mike said...

Some people take a long time to make the five points required for full value.
This is of course true, but I'm a concise writer. However, explaining complex matters such as the control of smooth muscle contraction in less than 5 pages including some diagrams may turn out to be an impossible task. All 4 essay questions carried equal marks and had similar complexity.

or a general failure of the test design.
Well, it definitely distinguished between average and other-than-that-students ;)

I really like your approach of setting exams, but I doubt that british professors follow your concept.

In my oppinion, the major problem about essay questions revolves around detailed required in the answer. I've sat exams where I thought "hell that's easy" and eventually my essay was too superficial about the basis.

renan said...

i agree with liberal arts chemist that the amount of time alloted to students to answer a chemistry exam should be 3 times the amount it takes the professor or another colleague to answer it. this is probably because (from what i've seen), chemistry exams are mostly of the multiple choice, short answers/essay, or problem solving type.

if an essay exam requires the student to write an average of 6 pages per question, it's probably better to give it as an oral exam. it's not an exercise in creative writing after all.

About Me

My Photo
For a while it was all about research and then it was all about teaching and now it's all about trying to find a balance while teaching at a small liberal arts and science university.