ChatGPT Passes Medical License Exam, Bar Exam After Top Performance On Wharton MBA Final

Ben ZeisloftJanuary 24, 2023

ChatGPT is a mass-market platform artificial intelligence OpenAI launched a chatbot last year. It passed both the bar exam as well as the medical licensing exam, which typically requires humans. students It took years of intensive learning and postsecondary education before you were able to graduate.

Over the last few weeks, the language processing tool has been widely recognized by knowledge workers who use it to quickly complete tasks like writing emails or debugging code. Academics have successfully applied the system to exams often considered difficult by even the world’s brightest students.

ChatGPT was performed “at or near the passing threshold” For all three parts of the United States Medical Licensing Exam (a test that doctors with Doctor of Medicine degrees must pass in order to be eligible for medical licensure), without “any specialized training or reinforcement,” One research paper. The system also showed “a high level of concordance and insight in its explanations,” It means that “large language models may have the potential to assist with medical education, and potentially, clinical decision-making.”

ChatGPT was asked open-ended multiple choice questions. The researchers gave ChatGPT forced explanations and two physicians adjudicators scored the answers in respect to accuracy, concordance, insight. ChatGPT scored significantly higher than scores obtained by other artificial Intelligence systems a few months prior. ChatGPT also performed better than PubMedGPT. “trained exclusively on biomedical domain literature,” And landed “comfortably within the passing range” Scores

Another research found that the system scored passing marks on the multiple-choice section of the Bar Exam. paper. Humans with seven years of postsecondary education and exam-specific training only answered 68% of questions correctly; ChatGPT achieved a correct rate of 50.3%, while the model’s top two and top three choices were right 71% and 88% of the time, far exceeding the baseline guessing rate.

ChatGPT was the conclusion of the researchers. “significantly exceeds our expectations for performance on this task” It was also noted that the rank-ordering possible choices confirms “general understanding of the legal domain” The system reflects this.

Although the discussion about technological unemployment has been centered on the loss of blue-collar employees to automated robotics solutions over the past decades, ChatGPT has brought up similar questions for white-collar professionals. ChatGPT has been used by many knowledge workers in a variety of capacities. A majority (27%) of the professionals working at prominent technology and consulting firms have used it. survey Fishbowl

Similar studies are being done in the areas of difficult legal and medical licensure exams. project which examined the performance of ChatGPT on a graduate-level operations management test at the University of Pennsylvania’s Wharton School. Professor Christian Terwiesch stated ChatGPT scored between B and B on a final exam that is usually given to MBA students.

“It does an amazing job at basic operations management and process analysis questions including those that are based on case studies,” He wrote. “Not only are the answers correct, but the explanations are excellent.”

Terwiesch stated that ChatGPT’s performance still had some shortcomings. The system was able to make “surprising mistakes in relatively simple calculations” At the sixth grade level, these were frequently “massive in magnitude,” While the current version is available “is not capable of handling more advanced process analysis questions, even when they are based on fairly standard templates.”