figshare
Browse
ChatGPT_and_C_Programming.pdf (554.13 kB)

Conversations with ChatGPT about C Programming: An Ongoing Study

Download (554.13 kB)
Version 3 2023-03-11, 23:17
Version 2 2023-03-11, 23:16
Version 1 2023-03-11, 17:49
preprint
posted on 2023-03-11, 23:17 authored by James C. DavisJames C. Davis, Yung-Hsiang LuYung-Hsiang Lu, George K. ThiruvathukalGeorge K. Thiruvathukal

AI (Artificial Intelligence) Generative Models have attracted great attention in recent years. Generative models can be used to create new articles, visual arts, music composition, even computer programs from English specifications. Among all generative models, ChatGPT is becoming one of the most well-known since its public announcement in November 2022. GPT means {\it Generative Pre-trained Transformer}. ChatGPT is an online program that can interact with human users in text formats and is able to answer questions  in many topics, including computer programming. Many computer programmers, including students and professionals, are considering the use of ChatGPT as an aid. The quality of ChatGPT's aid is therefore of interest. To shed some light on this quality, this article presents conversations between the authors and ChatGPT. These conversations are analyzed to understand ChatGPT's ability to answer questions related to computer programming. We consider questions that may appear in C programming at the introductory levels. These questions are classified into different categories: (1) facts that can be looked up from documentations, (2) extension and derivation of facts, (3) simple computer programs, (4) extension of simple computer programs, (5) debugging, and (6) integration of discrete mathematics and C programming.  This study discovers that ChatGPT can provide correct answers in many cases, but also make mistakes in the others. For a sequence of related questions, ChatGPT may provide inconsistent answers and exhibits self contradiction.  It is also possible that ChatGPT gives a program with {\it security vulnerability}. ChatGPT may give different answers when the same questions are asked again. This article reports ChatGPT's answers captured in February and March 2023.


History