A news flash paper published recently in JASN showcased the down trending test scores of nephrology fellows in certification exams. The authors analyzed the data from 2010-2019 and found that the pass rate has been falling below the bench marks. Interestingly, they found that the factors associated with this decline were lower internal medicine exam scores, older age and training in a smaller program. In addition, female sex and being IMG were also associated with a lower board score.
The IM board score as a predictor can make sense as both exams evaluate knowledge and skills of reasoning. Age over 33 performed less well than younger candidates is interesting. This could be because of non medical factors. Even since 2009 when I took my boards, the knowledge level has changed. There is more and more to read and more diseases to understand in medicine. Residency has not changed, Fellowship years have not changed. While knowledge and science has advanced, we have not changed our ways to teach and perhaps even consider changing the timeline of residency and fellowship. Fellows have family and other commitments as well and a well balanced life-work-training is critical for our trainees.
The fact that graduates of the least competitive nephrology fellowship programs(smaller programs) performed worse after regression adjustment indicates there might be a peer effect, or advantages of a structured program at a larger academic center.
IMGs were less likely to score high. The field of Nephrology has seen an increase in IMG applicants. In 2019, IMGs comprised nearly 70% of those taking the nephrology exam for the first time, an increase of more than eight percentage points from 2010. We keep forgetting that everyone learns differently- not everyone has a structure of learning in multiple choice questions in rest of the world; there are language barriers and other factors that play a role as well. Fellowship programs need to explore non ppt format of teaching and novel ways to teach the same material for varied type of learners.
Finally, women were found to have lesser scores. To my knowledge, not sure of any published papers showing this difference in test taking strategies. I don't think we need to take any stake in these findings as these might be not of any significance. The editorial nicely reminds us to not take this finding seriously.
What should be done?
Why can't we test the fellows on what we really encounter rather than esoteric rare and confusing diseases. Why can't the tests really mirror the life of a renal fellow and attending?
Institutions need to take ownership on better techniques and strategies to help their fellows. Many residencies may not be training them in proper test taking techniques. Institutional and program resources must support trainees’ needs, protect their time, and ensure education is prioritized.
I can say from my personal example of few fellow I trained- had trouble passing the boards due to their test taking abilities. Their patient satisfaction scores as attendings are off the roof and their overall understanding of both patient care and medicine is excellent. They may not be a good test taker, but they can manage a good census, take care of patients and call for help when needed and effectively communicate with other doctors. They win patient trusts, they do well with following up and most important of all- they care! and want to be Nephrologists that matter.
While test scores are important, failures sometimes teach us to be better and improve our abilities to be the best at what we do. But regardless, this is a wake up call for our field to improve as instructors and teachers and not disappoint our students.
You are right, Dr. Jhaveri! Institutions need to protect time for teaching (faculty) and learning (fellows). That learning should be focused less on test taking skills, and more on nephrology content. We need to focus of better assessment strategies--not just one type that is optimized for the "testers", not the "testees"!
ReplyDelete