The University Clinical Aptitude Test, which has replaced the UMAT, appears to further disadvantage women.
A change in the entrance test for undergraduate medicine has done nothing to improve the gender gap in results, and has even made it worse, despite more females applying for medical degrees.
A cross-sectional study published today in the Medical Journal of Australia found that the mean scores for the University Clinical Aptitude Test (UCAT), introduced for the 2020 undergraduate medicine intake, were higher for men than for women.
The research team analysed the data from about 4100 students who sat the 2019 Undergraduate Medical and Health Sciences Admissions Test (UMAT), and about 4200 who sat the 2020 UCAT across three undergraduate training programs in NSW.
The UCAT last year replaced the UMAT, which had been a part of the undergraduate medicine selection process for the previous 28 years.
The mean scores across both the UMAT and UCAT cognitive subtests were found to be significantly higher for men than for women.
The performance of women compared to men was not attributed to a difference in applicant ability, but rather a limitation in the testing instrument, the study authors said, noting that “the academic performance of girls in high school is equivalent to or better than that of boys, and they typically achieve higher academic grades than men in medical school”.
Although the UCAT is not the only means of selecting students, the apparent bias is not unimportant, as “the proportion of women selected for interview following the UCAT was significantly lower than that of applicants (47% v 56%)”.
Interestingly, women outperformed men when it came to assessing situational judgement in both tests, but men were found to outperform women in verbal and non-verbal reasoning, decision making, quantitative reasoning and abstract reasoning.
“Our findings indicate that the difference in overall test scores between male and female applicants is greater with the UCAT than it was with the UMAT, while the influence of socioeconomic status areas and remoteness of residential address were similar for the two tests,” the study authors said.
All mean UMAT and UCAT subtest scores for applicants with home addresses in rural areas were also lower than those from major cities.
Professor Barbara Griffin, co-author and professor in the department of psychology at Macquarie University, said that while no test was perfect, it was not uncommon for medical entrance tests to introduce a bias toward male applicants.
“This is why it’s important that the UCAT is used as just one of a battery of tests such as the ATAR, an interview and some universities even do personality testing too,” she said.
“More women are applying for medicine, that’s been the case for quite a few years now. But the point is that if you have 55% of women applying, then one would expect that in the shortlist you would also have 55% of women, but this study shows you don’t.”
So, does this mean the UCAT should be scrapped for a different entrance test?
Professor Griffin said not necessarily.
“Publishers are always revising tests and they look at studies like ours and do their own analysis. It’s a constant development process,” she said.
Results across both tests show lower scores in lower-socioeconomic-status students.
But this study shows that with the new exam format, wealth is no advantage at least when it comes to paying for professional coaching.
Half of the applicants in the 2020 intake from higher socio-economic areas had engaged professional coaching services, as compared to only 40% of students from lower socio-economic areas.
But it didn’t pay off, with no marked difference found in UCAT results between coached and non-coached students.
“With the UMAT, professional coaching companies were able to help with the pattern matching part of the test and helping students practise, and practise, and practise, so they got fast at it and being able to do that part under speed helped them,” Professor Griffin said.
But now with the UCAT, spending money on excessive coaching wasn’t necessarily going to help you improve your overall score, Professor Griffin said.
“And at some point in life, whether it be at university, in medical school, or when you are with a patient, you won’t have a coach next to you.”
MJA 2020, 14 December
