Coding with a Creative Twist: Investigating the Link Between Creativity Scores and problem-solving Strategies

Creative ideation and its adaptive value in reacting to new events are critical to advancing scientific and technological innovation. Software development, at its core, is a problem-solving endeavor that inherently demands creativity. Yet, the available research on creativity in software engineering is fragmented and limited. We present the first empirical evaluation of the role of creativity in software engineering (SE) tasks. We conducted an empirical study with 77 participants to objectively assess the effect of creativity— quantified via established psychometric tests—on developers’ performance and behavior through the problem-solving strategies they used during programming. We find that participants with higher creativity scores employed diverse strategies with significant variations, adding extra game features and multiple animations. Additionally, we report a notable correlation between task time and divergent creativity scores, as participants with higher DAT scores exhibit extended task times. Our findings can inform educational and training strategies in SE, fostering innovative approaches and boosting developer performance and engagement.

The recorded data for reproducibility of our results is organized in the following files:

1. users_valid.csv:This file contains a formatted version of our recorded data, which includes details such as time, accuracy, and relevant mechanism usage for each participant. Detailed descriptions of all columns are available in the file "columns_guide_final.txt."
2. columns_guide_final.txt: This file provides comprehensive information about each column featured in the "users_valid.csv" file.
3. raw_info.xls: Contains the raw data of users retrieved from our database.

For reusability and replication of our study, the following experiment materials are available:

1. Experiment-setup-screenshot.png: A screenshot displaying the study stimulus, including three code panes and the game interface at the bottom, offering an overview of the experiment setup.
2. Developed Website: The website for the study was developed using Wordpress and hosted on our private server. You can explore the website here: https://sense-lab.ca/
3. Creativity Tests: We utilized two standard tests to assess divergent and convergent creativity:
4. • Divergent Association Task (DAT): Participants generate a list of 10 words with maximal distinctness in meaning and usage. Results are submitted via a web form to our co-author for scoring. Comprehensive details and integration instructions are available at [https://www.datcreativity.com/].
   • • Jay A Olson, Johnny Nahas, Denis Chmoulevitch, Simon J Cropper, and Margaret E Webb. 2021. Naming unrelated words predicts creativity. Proceedings of the National Academy of Sciences 118, 25 (2021), e2022340118.
   • Remote Associates Test (RAT): Participants find a common associative word for each set of three words. Below are the 10 sets used in our study, with the solution and difficulty level provided: (References: Sarnoff A Mednick. 1968. The remote associates test. The Journal of Creative Behavior , 1968):
   • • rocking / wheel / high => chair => very easy
     • cracker / fly / fighter => fire => very easy
     • aid / rubber / wagon => band => easy
     • hound / pressure / shot => blood => easy
     • opera / hand / dish => soap => medium
     • print / berry / bird => blue => easy
     • boot / summer / ground => camp => medium
     • sleeping / bean / trash => bag => very easy
     • palm / shoe / house => tree => medium
     • pile / market / room => stock => hard
5. Whack-a-Mole Game Code: Implemented in JavaScript, the game is available in the folder [Code]. The folder contains HTML, JavaScript, and CSS files. We have also added methods to implement various visual and auditory elements, such as:
6. • [increaseScore()] : This function increases the user’s score by one score whenever it is called.
   • [playClapsAnimation()] : This functions displays a clapping animation in the middle of the screen whenever it is called.
   • [changeBackgroundColor()] : This function changed the background of the game to green for one second whenever it is called.
   • [playWinSound()] : This function plays a winning sound each time it is triggered.
   • [playLoseSound()] : This function plays a game-over sound each time it is triggered.
   • [playAnimation()] : This function enlarges the game board for a couple of seconds each time it is triggered.
7. Task specification: The specs provided to the participants are available inside the instructions.docx file.
8. Questionnaires: all the questionnaires are available in the pre-questionnaire.docx and post-questionnaire files.