Shelf Exams – Fair Assessments of Clinical Knowledge?

After taking clinical shelf exams in Internal Medicine, Psychiatry, OB/GYN, and Surgery, I’ve arrived at a conclusion which may be shared by others who, like me, aren’t gifted test takers – multiple choice exams cannot accurately assess a student’s clinical knowledge.

Let me construct a hypothetical vignette:

42 year old G3P2103 overweight Caucasian female presents to the emergency center with excruciating right upper quadrant abdominal pain beginning a few hours after eating dinner.

As an astute individual, biliary colic secondary to cholelithiasis (gallstones) is probably at the top of your differential. Okay, so what’s the big deal?

Medicine is a team-oriented field which places a patient’s overall wellness and health at the forefront of our objectives. We can’t address clinical vignettes without knowing other details about a patient’s labs, imaging, vitals, physical exam, etc. While shelf exams often provide the aforementioned information in question stems, in my opinion, the most important part of an encounter is completely lost  – what’s your first impression upon seeing the patient? Is the patient above constantly writhing in pain unable to find a comfortable position? Are there physical symptoms even she is not even aware of? Subtle clues often go a long way in diagnostics.

In addition, I think the process of arriving at a diagnosis is lost in shelf exams. If we think the gallbladder is involved and decide to do a right upper quadrant (RUQ) ultrasound, what’s the next step if there’s no sign of acute cholecystitis or cholelithiasis? CT abdomen? Maybe the patient is malingering? How do we justify the radiation exposure inherent to CT imaging or the implications of labeling someone as a malingerer? These are things we have to consider!

Furthermore, finding out “what it is” is just as important as finding out “what it is NOT.” Being able to quickly construct a differential with strengths and weaknesses supporting each diagnosis is a pivotal part of clinical knowledge which is difficult to test on multiple choice exams. Knowing which diagnoses are more likely will effectively guide the work-up and save tremendous amounts of resources, money, and time.

Don’t get me wrong – I’ll be the first to say that the test writers do a fantastic job in constructing questions (or rather, selecting from a circulating pool) which require higher order reasoning, but our hands are often tied when given a finite set of answer choices. Why should I be forced to select answer when I know the updated literature has either a.) completely refuted a prior tenet in the workup and/or therapy or b.) there are newer protocols available? Multiple choice exams are better suited for the objective medical knowledge we see in the first two years of training in the basic sciences.

The focus should now be on standardized patient encounters, having the opportunity to respond with short answers, creating a differential diagnosis with the ability to prioritize which items are more likely given the results of imaging and labs we order, etc. Maybe a free response section would be prudent to include on shelf exams? Of course it’s not as easy to grade, but I strongly feel our knowledge regarding the updated literature and thinking beyond the scope of the patient’s chart would be more thoroughly tested.

It’s undeniable that “textbook” medicine is different than “clinical” medicine. I’m a huge proponent of free response tests which will adequately assess the latter – the same clinical medicine we’ll actually be using as practicing physicians.

What are your thoughts?