How a tool sparked at McMaster is helping select medical school applicants worldwide

While measuring a medical school applicant’s intelligence is easy enough, assessing their bedside manner as a potential physician is a question that has long perplexed medical schools the world over.

Enter Casper, an online assessment of personal characteristics, first developed at McMaster University’s Faculty of Health Sciences to improve the Michael G. DeGroote School of Medicine’s admission process.

The tool is now helping shape the next generation of health-care professionals who possess both a caring nature and sound medical skills.

Casper, which stands for Computer-Based Assessment for Sampling Personal Characteristics, is an open-response situational judgment test. It poses timed scenario-based questions to applicants to evaluate aspects of their social intelligence and professionalism such as ethics, empathy, problem-solving and collaboration. The scenarios ask not only what the respondent would do in the tough situation, but why. The test is taken online and rated by humans.

The assessment is meant to complement existing measures such as grades or standardized academic tests such as the Medical College Admissions Test (MCAT) to ensure medical schools have a clearer picture of the applicant, rather than just their intelligence.

“These are the attributes that will become even more important in the future of healthcare. I use the analogy of adding more pixels to the picture to allow schools to get a truly holistic sense of who the applicant is as a person,” said Kelly Dore, McMaster adjunct associate professor of medicine and one of Casper’s developers.

“Doctors are incredibly smart, so they don’t often run into problems where their domain knowledge is concerned,” said Harold Reiter, a McMaster professor of oncology and Casper co-creator.  “When some do run into problems, you’ll find that it’s more likely tied back to resiliency and conduct.

“If we can identify these individuals who are more likely to have those lapses in judgment, we can make great strides in identifying remedial issues, providing applicants with more support in the program, and overall improving the general people skills that will help improve our health-care system.”

communication, collaboration, professionalism and patient advocacy

For Kelly Dore, co-founder of Altus Assessments which has licensed Casper from McMaster University, the key to a more unbiased application process is to incorporate more concrete data points on these important personal and professional characteristics  and balance them with other important measures that are focused on their technical knowledge like GPA and MCAT scores.

Relying solely on tools like personal statements to evaluate these difficult-to-measure qualities is problematic, she said, because they’re loaded with buzzwords and have major input from applicants’ friends or relatives. Such statements cannot predict what a candidate will achieve in medical school nor a medical residency program.

“Who do I want to send my parents or kids to for the best care? To a physician who not only has the scientific aptitude or knowledge base, but is also an empathetic, strong advocate, and collaborative professional who sees my loved ones as people, not just a set of symptoms,” said Dore.

The Royal College of Physicians and Surgeons of Canada describes communication and collaboration skills, professionalism and patient advocacy as core competencies, alongside ‘hard’ medical skills for physicians.

Scenario-based testing

As someone who used Casper to apply for medical school in 2018, McMaster graduate Bryce Bogie says the platform is ideal for supporting programs to assess applicants’ professionalism, simply by posing scenarios that have relevance in everyday clinical practice.

These may include an example of how the candidate solves the challenges of working in a team, or the importance of workplace collaboration. In these scenarios, applicants are asked not just what they would do, but why they think that course of action is appropriate, allowing diverse perspectives rather than a single correct answer.

In posing such questions, Casper forces an applicant to display sound judgment and critical thinking to come up with a solution that is both ethical and practical.

“When someone displays unprofessional thinking or unethical judgment within a Casper test scenario, that is obviously a huge red flag about their future clinical performance,” said Bogie, who obtained both his BHSc and MSc at McMaster and is completing his MD-PhD in Ottawa.

“Based on the questions I answered, I believe Casper would definitely flag people who lack professionalism within the scenario-based testing it provides.”

Capturing the personal and professional

The Casper evaluation system started as an idea around 2004 as Dore and Reiter, together with professors Geoff Norman and Kevin Eva, innovated a new pre-interview process to capture the personal and professional attributes of medical school applicants.

After six years of development and testing, Casper was officially launched at McMaster’s Michael G. DeGroote School of Medicine in 2010. This work was funded through major grants from Medical Council of Canada and National Board of Medical Examiners, as well as substantial internal funding and resources from the undergraduate program of the medical school and the computer services department of the Faculty of Health Sciences.

Those within the Faculty who played a key role in the administrative and software implementation of CASPer included Susan Birnie, then director of education services; Wendy Edge, admissions officer for the medical school, and several members of the Faculty’s computer services department including then-director Rocco Piro and technical staff Todd Murray, James Dietrich and Kevin Reed.

Based on the success of its first few years of use at McMaster, Casper gained the interest from other universities in Canada and around the world. With the support from John Kelton, then Faculty of Health Sciences’ dean and vice-president; Alan Neville, then the Faculty’s vice-dean, education, Rob Whyte, then associate dean of undergraduate medical education and Gay Yuyitung, executive director of the McMaster Industry Liaison Office, along with the creators, a new company was created to commercialize this unique platform.

Selecting the very best

Casper was licensed to Altus Assessments, and it is now one of the major assessment tools for medical and other professional schools in North America.

“McMaster’s Michael G. DeGroote School of Medicine has pioneered successful creative educational innovation since its founders developed small group problem-based learning more than 50 years ago,” said Kelton.

“Finding a way to select the very best medical students who are not only smart but truly understand people, was, of course, our logical next step.”

The platform took on its current form in 2015 with upgrades by Altus Assessments under CEO Rich Emrich. The types of questions and scenarios it poses are often changed to avoid candidates tipping each other off.

Today, Casper is being used at nearly 400 university medical schools and health sciences programs in Canada, the United States and Australia, with plans for roll out in the United Kingdom and in other types of programs. At McMaster Casper is used at both the School of Nursing and the Michael G. DeGroote School of Medicine.

“Casper has been hugely successful and accounts for over 95% of applicants applying to Canadian or US medical schools. It addresses an ongoing challenge as there are thousands of applicants to the schools and only a few hundred interview spots,” said Reiter.

In addition, a different version of Casper is used to assess over 80% of applicants to Canadian medical residency programs.

Admission of more diverse applicants

Dore and her colleagues hope that Casper and broadly a more holistic review of applicants will encourage more medical school applicants from under-represented or marginalized backgrounds to apply, including Indigenous peoples or recent immigrants.

Unlike personal statements, Casper aims to reduce personal, community, gender, and racial biases to encourage the admissions of more diverse applicants, and continues to improve with new outcomes data, changes to the content, and additions to the Casper formula.

Dore said Casper is continuously evolving, as assessors combine data and add new assessments.

For Dore, an expert in assessment, medical education, psychometrics and cognitive psychology, Casper’s success was a meaningful moment in her career.

“Seeing this realized at an international level is so exciting and is the start of a journey to creating a fairer and more equitable selection process that measures broadly all the things we know are important in our health care professionals,” she said.

Tony Vlismas, director of marketing at Altus, has the following advice for applicants using Casper.

“I will say be honest, be open, be your unique, authentic self and use your genuine answers. This really lets your personal qualities and other capabilities shine through,” he said.

Applicants can learn more at TakeAltus.com, and other schools can reach Altus at altusassessments.com.