You are the auditor: A ChatGPT-based multiple choice exam
Abstract
This case study examines the use of ChatGPT, a chatbot that utilises generative artificial intelligence (GenAI) technology, to amplify the efficacy of multiple-choice examinations in a financial accounting course. The primary objective is to introduce students to the advantages and limitations of GenAI tools such as ChatGPT while simultaneously fostering the development of critical thinking skills in their application. By demonstrating the tool’s ability to give both correct and incorrect responses, students will deepen their understanding of its appropriate use and develop a professional scepticism towards AI-generated responses, which can ‘hallucinate’ and yield incorrect answers to AI prompts. Furthermore, this study seeks to explore the role of AI-supported tests in fostering students’ problem-solving abilities and analytical acumen. The research examines student performance by giving exam questions, an AI-generated response and additional alternative answers in a multiple-choice format. On average, ChatGPT 3.5 responds with correct answers to accounting questions about 50 per cent of the time. To succeed in such examinations, students must evaluate if the ChatGPT answer is correct or if another of the given answers is more accurate. The case study also highlights the advantages and challenges of incorporating AI into accounting pedagogy, addressing concerns about the assimilation of this advanced technology.
The full article is available to subscribers to this journal (subscription is free).
Author's Biography
Scott Dell is an award-winning academic, recovering CPA, lifelong learner, enthusiastic teacher, experienced entrepreneur, motivating coach, successful author and proud Navy veteran. His artificial intelligence (AI) workshops, presentations and keynotes are highly reputed. He is the author of the definitive LinkedIn resource Your COMPLETE Guide to SUCCEEDING with LinkedIn and co-developer and author of the Living Your DREAM CAREER online course, book and workbook. Scott has been published in practitioner journals such as Strategic Finance and the Journal of Accountancy, along with contributing peer-reviewed research in AI and nonprofit financial health. His co-authored article ‘ChatGPT and AI in Accounting Education and Research’ was selected as the no. 2 best article in the IMA’s Strategic Finance for 2023. His co-authored article ‘The ChatGPT Artificial Intelligence Chatbot: How Well Does It Answer Accounting Assessment Questions?’, as published in the AAA’s Issues in Accounting Education, has been nominated for their prestigious 2023 Best Paper Award. He is a former Big 4 auditor holding multiple degrees, including a UW Doctorate (DBA), Wharton MBA, Boston University Bachelor’s in accounting, and Education Masters (MEd) in ed. tech.
Mfon Akpan is a distinguished academic and an Assistant Professor at Methodist University in Fayetteville, NC. His academic journey is marked by a Doctorate in Business Administration from Indiana Wesleyan University and a profound involvement in research focusing on the intersection of technology and accountancy. Mfon has contributed extensively to the literature on artificial intelligence’s (AI) role in financial disciplines through notable publications in journals such as Corporate Ownership & Control and Risk Governance and Control: Financial Markets & Institutions. His research often explores emerging technologies such as AI and non-fungible tokens (NFTs), assessing their impact and integration into accounting practices. Mfon is an active participant at conferences and seminars, where he presents findings and engages in discussions that bridge technology with traditional accounting.
Citation
Dell, Scott and Akpan, Mfon (2024, December 1). You are the auditor: A ChatGPT-based multiple choice exam. In the Advances in Online Education: A Peer-Reviewed Journal, Volume 3, Issue 2. https://doi.org/10.69554/EINF1743.
Publications LLP