|Year : 2021 | Volume
| Issue : 2 | Page : 74-78
Multiple mini-interview for undergraduate allied health programs in Singapore
Meredith T. L. Yeung
Singapore Institute of Technology, Singapore
|Date of Submission||11-May-2021|
|Date of Acceptance||02-Jun-2021|
|Date of Web Publication||15-Sep-2021|
Dr. Meredith T. L. Yeung
Singapore Institute of Technology, 10 Dover Drive.
Source of Support: None, Conflict of Interest: None
Aims: To investigate the acceptability and feasibility of a four-station multiple mini-interview (MMI) model for admission selection into the four allied health undergraduate programs in Singapore. Methods: From 2015 to 2017, applicants to the four allied health undergraduate programs (physical therapy, occupational therapy, diagnostic radiography, and radiation therapy) at a university in Singapore underwent a four-station MMI. In 2017, interviewers of the MMI were polled using a 7-point Likert scale survey. Descriptive statistics were performed to analyze the quantitative portion of the survey, while content analysis and thematic description were applied to the qualitative data. Results: Data was collected from 79 interviewers. The majority agreed that the MMI was a fair process for applicants (83.5%), a good way of assessing interest and potential to become healthcare professionals (78.5%), and tested more aspects of the applicants than the traditional interviews (72%). They concluded that the MMI format was structured, objective, rigorous, unbiased, and assessed a valid range of competencies (67%). Conclusion: This study assessed and established the acceptability and feasibility of the MMI for undergraduate admissions to the allied health programs in Singapore.
Keywords: Acceptability, admission, allied health education, diagnostic radiography, multiple mini-interview, occupational therapy, physical therapy, radiation therapy
|How to cite this article:|
Yeung MT. Multiple mini-interview for undergraduate allied health programs in Singapore. Educ Health Prof 2021;4:74-8
| Key Messages:|| |
This study showed:
this is the first study on the use of multiple mini-interview in undergraduate admission of allied health programs in Singapore.
good acceptability and feasibility for the role of multiple mini-interview as an interview method to undergraduate allied health programs in Singapore.
this four-station construct provided a good insight of its deployment to programs with relative resource constraints.
| Introduction|| |
Admission to health profession training programs is a high-stake decision. The single-station panel interview or board interview is commonly used for undergraduate and postgraduate healthcare program admission. However, this method carries significant problems of context specificity and interviewer bias., It also provides limited predictive values while carrying the risk of dissuading potential applicants. Conversely, structuring the interview has been reported to enhance the acceptability and reliability of the interview process. The multiple mini-interview (MMI) was, therefore, conceptualized to overcome such problems.
MMI is well-established as an alternative process for admission interviews; it is a highly structured student selection method designed to resemble the Objective Structured Clinical Examination (OSCE). MMI aims to assess a broad array of the applicants’ characteristics. This is made possible via multifacet behavioral assessment in an OSCE-like rotational approach. MMI was introduced by Eva et al. to answer the need for an interview process with robust psychometric properties. By testing a larger content sample through multiple independent interviewers, MMI can offer a more accurate picture of the applicants’ behavior. MMI has been adopted as the preferred student selection method in health professions, such as medicine, dentistry, pharmacy, and veterinary science. However, few reports described using MMI as an interview process for admission to the allied health programs; the single-station panel interview format is dominantly adopted despite lacking clear empirical evidence to support and show its suitability. The university adopted MMI as the admission process for the four allied health undergraduate programs, namely physical therapy, occupational therapy, diagnostic radiography, and radiation therapy, since the inaugural admission exercise in 2015. The MMI structure was developed to tackle challenges associated with traditional interviews and defensibly assess nonclinical attributes (e.g., ethical and moral reasoning, critical thinking, and general awareness of the contemporary social system). Evidence supporting the MMI structure is primarily found in Western societies, across health professions. Evidence involving its use in Asia is limited. The MMI, which was developed to suit the Western contexts, values, experiences, and expectations, leaves some uncertainty regarding its use in Asian societies. The distinct cultural differences warrant studies on its acceptability of MMI in Asia. To address these issues, the purpose of this study was
- to create and describe the MMI process, and
- to investigate the acceptability and feasibility of a four-station MMI model for admission selection to the university’s four allied health undergraduate programs.
| Materials and Methods|| |
Overview of the allied health programs in the university
Our university offers entry-level honors degree programs in physical therapy, occupational therapy, diagnostic radiography, and radiation therapy. The programs are cocreated with clinical partners and stakeholders to meet societal needs. The strength of the curriculum is rooted in its strength in interprofessional collaborations.
Overview of the multiple mini-interview model
In January of 2015, our program committee proposed implementing the MMI model to address our challenges in recruiting students with nonclinical attributes that we believed were critical to allied health professions. We developed different scenarios that allowed for assessing ethical and moral reasoning, critical thinking, and general awareness of the contemporary social system. A traditional interview with the program director or a senior professional member was maintained, though given the equal value in creating the rank order list, to assuage concerns about the loss of some of the program-specific information that could be exchanged in that setting. The university established a four-station MMI structure, with station one on ethical and moral reasoning, station two on critical thinking, station three on general awareness to the contemporary social issues, and station four as the traditional interview with a senior member from the same profession of the program the applicant was applying.
Multiple mini-interview calibration
All interviewers received one session of 2-h information cum calibration training in MMI concepts and logistics within 2 months before the start of the MMI. The MMI calibration included practice on two preproduced case vignettes with responses in the video format. During the calibration, interviewers watched the videos and rated the responses with the MMI scoresheet indivdiually. The researchers captured all the scores from the interviewers. Interviewers were then asked to form small discussion groups to discuss and reconcile their scores, achieving the calibration purpose. The interview panel comprised of faculty members and senior members from the professions of physical therapy, occupational therapy, diagnostic radiography, and radiation therapy. Our MMI adopted the interprofessional approach. Interviewers from various backgrounds were randomly assigned to the MMI stations one, two, and three, whereas only the respective senior professional members served as the interviewers in station four.
All applicants must complete the four 10-min MMI stations. MMI interviewers from the first three stations were blinded to the applicant’s background information, previous academic attainment, and the program they applied to minimize biases incurred by advanced knowledge of the applicants. At each MMI station, except station four, applicants had 2 min to prepare for the scenario and 8 min to address the scenario with the interviewer. Interviewees spent a total of 40 min for the whole MMI interview process, whereas interviewers spent a total of 160 min interviewing 16 applicants on each interview session, excluding short break intervals. Individual rooms were used and assigned with a single interviewer to each station. At the end of each mini-interview, applicants were rated on their overall performance. The element of communication skills was embedded within each station. The ratings were done on a ten-point ordinal scale between 1 and 10 with 1 representing the most inadequate performance and 10 the best that one could be. Also, the score between 1 and 3 was categorized as poor, 4 to 7 as average, and 8 to 10 as good. Interviewers were allowed to note down comments and any “red flag” issues in a free-text commentary field.
Acceptability and feasibility
Upon the completion of the MMI session, interviewers were invited to complete an acceptability questionnaire adopted from Campagna-Vaillancourt et al. in which they rated their perception of various aspects of MMI using a 7-point Likert scale with 1 indicating strongly disagree and 7 indicating strongly agree. Minor modification on item 9, “The MMI is a good way of assessing interest and potential in the SG-AH programs” to fit into the context of this study. Written qualitative comments were recorded and thematically categorized by the researchers.
The university’s Institutional Review Board granted the ethical approval of this project (IRB number: 20170055)
The statistical analysis of the data used Graphpad Prism 8.4.3 (686). Descriptive statistics were used to analyze the calibrated scores between interviewers during the calibration session and the quantitative portion of the exit survey. Content analysis and thematic description were applied to qualitative data.
| Results|| |
A total of 79 interviewers (n = 79) returned the acceptability questionnaire (100% response rate). Interviewers who attended more than one interview session were required to submit only one evaluation form. The majority of the interviewers (87%) had previous experience conducting the single-station panel interview or board interview related to work or professions matters. More than half (58%) of interviewers had previous experience with MMI from previous interviews with local or overseas medical programs, undergraduate and/or postgraduate programs.
Multiple mini-interview calibrations
Of a total of 79 interviewers who attended, 75 and 72 sets of completed scores from vignette one and two were received, respectively, at the end of the calibration workshop series. [Table 1] presents the interviewers’ scores on the scenarios practice during the calibration exercise. It depicts the minimum, maximum, mean, and standard deviation (SD) of interviewers’ score values to the two scenarios presented. The majority of the interviewers have good agreements in the rating of the performances based on the scenarios. The mean score of scenario one falls within the average band, whereas scenario two falls in the poor band.
[Table 2] presents the interviewers’ mean response values (minimum, maximum, mean, and SD) to the acceptability questionnaire. In the question-directed comments section, most interviewers indicated that they preferred the MMI over the traditional interview for admission to the undergraduate programs. The qualitative comments were grouped according to the three main themes: (1) the opinions of MMI as an interview process for allied health undergraduate programs, (2) what they liked most about the MMI process, and (3) what they liked least about the MMI Process? The comments were grouped according to the themes [Table 3]. Some examples are “MMI assess on various attributes and I am sure it is very useful to select the right candidates from so many” from the question on MMI as an interview process for allied health undergraduate programs; “can be quite stressful for applicants” and “it is very time and resource-intensive” from the question on what they liked least from the MMI process.
| Discussion|| |
This study finds the implementation of the MMI in the context of the undergraduate admission process of the allied health programs in Singapore to be a success. Adaptation of the MMI model developed by Eva et al. allowed the local programs to meld several qualitative nonclinical attributes into their undergraduate admission process. Station construction was guided by the original conception that fit the initial curriculum planning team’s specifications for the desirable qualities in applicants for the allied health undergraduate program. The university faculty members and senior clinicians of the four professions had the first-hand experience in identifying the presence or absence of such qualities and attributes in applicants and evaluating behaviors associated with professionalism potential in a very heterogeneous group of program applicants.
The general quantitative and qualitative results are consistent with the finding reported in medical programs,,,,, and health sciences programs.,, Interviewers showed an overall high level of satisfaction with the MMI, stating that they preferred it over traditional interviews. Interviewers indicated a high level of satisfaction with their training and the material provided to them for their stations. Although some interviewers reported end-of-day interview fatigue, most interviewers found value in the MMI methodology for having sufficient time to assess the candidates at each station. Interviewers agreed that the MMI process was a fair assessment of the applicants. Despite a time limit of 10 min per station, which is a shorter interval than the time spent on the traditional interview structure, the interviewer recognized MMI as more objective, unbiased, providing a more global and holistic picture of the applicants in this high-stake setting. Interviewers appreciated the MMI for the bandwidth to include both professional and interprofessional collaborations.
Our study illustrated the acceptability of a four-station MMI model for the local admission process in the undergraduate allied health programs. We introduced interprofessional collaborations and provided a leaner approach to MMI interviews, which can potentially benefit similar programs with less available personnel and resources. We also showed that the MMI process was acceptable to interviewers who are experienced in standard interview formats without introducing more stress. Interviewers found the process to be fairer and more effective in evaluating the noncognitive traits of the applicants. Several studies have confirmed this finding of MMI experience.,,,
The intensity of labor and resource requirements may cast doubt on the feasibility of the MMI process. However, we found that escalation in resource requirements is mostly limited to the planning and developmental stages. Once established, resources and documentation developed (including station objectives, evaluation forms, and question-based scenarios) can be readily reused or modified for subsequent use. Ideally, individual rooms with one-way mirrors should be available as the setting for MMI. We were, however, able to adapt and use individual office rooms.
Limitations of this study include the small sample size of study participants from a single institution. Also, there is no direct comparison of the MMI to traditional interviews, even though most interviewers (87%) reported previous exposure to traditional interviews through school admission, scholarship selections, or staff hiring processes. In addition, interviewers were informed of the limitations of traditional interview methods during their MMI training.
| Conclusion|| |
To our knowledge, this is the first study to describe the role of MMI for admissions to undergraduate allied health programs with a vital essence of interprofessional collaborations in Asia. It has shown good acceptability for the major stakeholders. The four-station construct of our MMI process provides a good insight into its feasibility and deployment to programs with relative resource constraints. Future directions will include examining the predictive validity of the MMI about the applicant’s future performance throughout training and beyond.
Ethical policy and institutional review board statement
Ethical approval for this study was received from the Institutional Review Board of the university (application number: 20170055).
The author wishes to acknowledge Mr Jeremy Yeung, Drs Andy Lee, and Bernard Leung for useful input.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Edwards JC, Johnson EK, Molidor JB. The interview in the admission process. Acad Med 1990;65:167-77.
Eva KW. On the generality of specificity. Med Educ 2003;37: 587-8.
Eva KW, Rosenfeld J, Reiter HI, Norman GR. An admissions OSCE: The multiple mini-interview. Med Educ 2004;38:314-26.
Wilkinson D, Zhang J, Byrne GJ, Luke H, Ozolins IZ, Parker MH, et al
. Medical school selection criteria and the prediction of academic performance. Med J Aust 2008;188:349-54.
Patrick LE, Altmaier EM, Kuperman S, Ugolini K. A structured interview for medical school admission, phase 1: Initial procedures and results. Acad Med 2001;76:66-71.
Roberts C, Walton M, Rothnie I, Crossley J, Lyon P, Kumar K, et al
. Factors affecting the utility of the multiple mini-interview in selecting candidates for graduate-entry medical school. Med Educ 2008;42:396-404.
Pau A, Jeevaratnam K, Chen YS, Fall AA, Khoo C, Nadarajah VD. The multiple mini-interview (MMI) for student selection in health professions training - a systematic review. Med Teach 2013;35:1027-41.
Campagna-Vaillancourt M, Manoukian J, Razack S, Nguyen LH. Acceptability and reliability of multiple mini interviews for admission to otolaryngology residency. Laryngoscope 2014;124:91-6.
Brownell K, Lockyer J, Collin T, Lemay JF. Introduction of the multiple mini interview into the admissions process at the university of calgary: Acceptability and feasibility. Med Teach 2007;29:394-6.
Kumar K, Roberts C, Rothnie I, du Fresne C, Walton M. Experiences of the multiple mini-interview: A qualitative analysis. Med Educ 2009;43:360-7.
Razack S, Faremo S, Drolet F, Snell L, Wiseman J, Pickering J. Multiple mini-interviews versus traditional interviews: Stakeholder acceptability comparison. Med Educ 2009;43:993-1000.
Cameron AJ, Mackeigan LD. Development and pilot testing of a multiple mini-interview for admission to a pharmacy degree program. Am J Pharm Educ 2012;76:10.
Thomas A, Young ME, Mazer BL, Lubarsky SE, Razack SI. Candidates’ and interviewers’ perceptions of multiple-mini interviews for admission to an occupational therapy professional program. Occup Ther Health Care 2015;29:186-200.
Fraga JD, Oluwasanjo A, Wasser T, Donato A, Alweis R. Reliability and acceptability of a five-station multiple mini-interview model for residency program recruitment. J Community Hosp Intern Med Perspect 2013;3:1-4.
Hofmeister M, Lockyer J, Crutcher R. The acceptability of the multiple mini interview for resident selection. Fam Med 2008;40:734-40.
Humphrey S, Dowson S, Wall D, Diwakar V, Goodyear HM. Multiple mini-interviews: Opinions of candidates and interviewers. Med Educ 2008;42:207-13.
[Table 1], [Table 2], [Table 3]