Computer Quiz Games in General Chemistry for Engineering Majors in an English as a Second Language Environment

Computer quiz games are introduced to improve teaching and learning in a freshman engineering chemistry course in an English-as-a-Second-Language (ESL) environment. 15 These quiz games are developed and implemented as a supplemental and augmentative tool to enhance traditionally delivered lectures. The paper shows an increase in students’ motivation and compare the performance between students who participated in computer quiz games, a paper-based quiz or neither activity. Assessment of the effectiveness of quiz games in learning is conducted via a proposed novel chemistry achievement test, 20 Freshman Engineering Chemistry Aptitude Test and an attitude questionnaire. The findings contribute to our understanding of the role of game-based learning in students’ achievement in chemistry and their motivation and attitudes towards learning general chemistry at a university within an ESL environment, while the computer games developed are useful in all English based Chemistry classes. 25

The CQGs are weakly incentivized as follows. The Introductory General Chemistry for Engineering Majors (GenCHEM) course grade comprises contributions, from lab (25%), mid-semester exams (20%), traditional quizzes (25%) and final exam (30%). The overall grade from traditional quizzes could be improved through CQGs by performing better than ~67% of the class on quiz games, with a maximum possible improvement of 70 2.5 points towards 100 points for the overall course grade.

CHEMISTRY LEARNING IN AN ESL ENVIRONMENT
Basic language proficiency plays an important role in course delivery and learning in science [38][39][40][41][42], as poor language proficiency increases chance of conceptual misunderstanding [41]. Despite having nearly all university instruction in English, 75 Journal of Chemical Education 9/30/21 Page 3 of 22 difficulties in English proficiency are considered a very common issue for college education in the UAE and MENA (Middle East and North Africa) [41,42]. This is affected in unpredictable ways by a large international multi-generational migrant population, which creates an increasingly heterogeneous population in secondary and university education that does not map onto other well-studied populations (e.g. USA, Europe, 80 Australia, or Asian Far East). The educational environment in the UAE differs from that encountered by ESL students studying in English-speaking countries, in that education may be the only context in which English is the predominant, day-to-day language. The population of students majoring in engineering disciplines at Khalifa University is unique in several ways: >98% are ESL speakers, and are of diverse socio-economic backgrounds; 85 students are either UAE nationals (~85%) or come from a diverse group of predominantly MENA-region expatriates (~15%). Within the overall set of ESL issues encountered elsewhere, reading comprehension is the skill lagging most. Simultaneously, there exist substantial gaps in several areas of elementary (pre-calculus) mathematics. These skill deficits hamper learning from both the students' and instructors' perspectives, which 90 motivates development of teaching and learning strategies that enhance, or augment, language skills alongside chemistry content [41,43].

RESEARCH QUESTIONS
The following research questions have been explored in this report:  In addition, selected students representing all demographic categories and 135 performance levels were interviewed after the end of the semester. In addition to our own motivation to use control groups, a need for varied control groups is felt in some previous studies [43][44][45].
A. Chemistry Quiz Game (CQG) 140 Upon starting the CQG, the student is presented with a dialog frame and asked to confirm their identity (for record-keeping), after which the game rules are displayed.
The student starts the quiz by clicking the button labelled "Start when ready!" on the frame displaying the rules. The student sees a matrix of 9 questions. Simultaneously, a countdown timer begins and is displayed in a frame titled "Referee" alongside the updated Each button deactivates after one click, to prevent the same question being 170 selected twice. Clicking a question button prompts the opening of a question frame, which is organized as follows: at the top is the question, containing text and/or graphics; underneath are 4 possible answers (also text and/or graphics based), each of which is a clickable button. In the current implementation, there is only one correct answer.
Answers are randomly organized in the frame on each run, to avoid bias or recognizable 175 patterns, and to reduce the likelihood of cheating in the classroom environment. At the bottom of the question frame are three additional buttons, "hint", "pass", and "help".
"Hint" provides a clue, "pass" leaves the question without incurring a point penalty, and "help" removes two wrong answers. Clicking the "help" or "hint" buttons incurs a point Within standard game classifications [35,36], our CQGs are quantitative games; the score is prominently displayed and responds to student entries throughout the game.
They are also finite games, (a limit of five choices out of nine possible questions). Also, 190 winning is the goal; students learn the anonymized top class score and the distribution of scores between finishing one game and playing the next. One difference with traditional computer games is that students in this study could not choose when to play the game.
Future faculty users could allow students to play at individually chosen times. The students are free to define their own strategy which "easy", "medium" and "hard" including analysis of individual categories is provided in a separate report [47].  [49,50]. We expect that further applications and some improvements of the test will lead both coefficients to increase toward more desirable values of 0.85 or 0.9, respectively, which would make the reliability "very good" or "excellent". Normality tests for each 220 question showed that skewness and kurtosis of 30 questions is within the recommended range, given α = 0.01, so the critical z value is ±2.58 [50]. Questions  and other results in this report have included these questions, since their impact on increase or decrease on the gain is much smaller than the uncertainty.

C. Students' Attitudes Survey
The attitudes survey is based on selected sets of questions about "enjoyment of 230 chemistry lessons", adapted from the Test Of Science-Related Attitudes questionnaire (TOSRA) [51]; and "motivation" questions, based on Science Motivation Questionnaire (SMQ) [52], [43]. A five-point Likert-type scale with Strongly Agree (SA) to Strongly Disagree (SD) was used as the response format of the survey.

D. Interview Protocol
Interviews were conducted after the end of the semester. Interviewees (n = 29) were selected from a range of backgrounds (i.e. gender, nationality, course performance).
Interviews comprised of two parts: questions answered on a Likert scale and several openended questions. Interviews were recorded, transcribed, and viewed and checked by 240 interviewees for clarification.

A. Demographic Results
Demographic details, including number of students per semester and per group, 245 gender and nationality, are shown in Table 1. Note that groups X and Y were initially equal in size, but incomplete datasets for some students were removed from analysis (e.g. if a student completed less than 7 of the 10 CQGs or PQs, or was absent for the post-test and/or attitude survey). Due to a lack of available information before the study began regarding English literacy and math skills, students were simply randomized to groups X 250 and Y, resulting in an uneven distribution of student abilities across these three categories. In addition, there were noticeable changes in the male-female ratio between the three semesters, that were beyond experimenters' control.

B. Hake's gain on FECAT
All three groups (X, Y and Z) showed learning gains in the FECAT post-test vs pretest methodology, as shown in Fig. 3. In Sem I, the Hake gain for group X (with CQGs) 260 was slightly larger compared with group Y (with PQs), whereas groups X and Y both had similar positive Hake gain in Sem II. Although the differences were small, it must be noted that as a result of imperfect randomization, students in group Y had higher overall GPA, math skills and IELTS scores (English skills) compared with students in group X. In comparison, group Z, from Sem III (with neither CQGs nor PQs) showed lower Hake gain 265 compared with groups X and Y, while having nearly identical math and IELTS scores and the overall GPA. Next, we discuss the FECAT Hake gain as a function of two independent variables: cumulative IELTS (comprising contributions from reading, writing, listening and speaking) and Final Exam scores. There is increase in the gain as the cumulative IELTS score increases, up to the IELTS value of 6.5. This is understandable in that higher 270 IELTS score is frequently correlated with increased performance and GPA [55]. Also, students with IELTS = 7.0 and higher, often had such high pretest scores that they were unable to have significant Hake gain. were actively preparing for through biweekly and mid-term assessments throughout the semester (unlike FECAT). We make the following observations: B.1 Group Z students, who did not use either CQG or PQ at any point in the semester, 305 show lower Hake gain, demonstrating the activity itself (whether it is CQG or PQ) helps students achieve higher gain regardless of the type of activity. future use is not just as a research instrument, but also as a standardized freshmen college readiness assessment instrument.

C. Students' Attitudes Survey
Responses for one of the motivation questions/statements, "I find learning  Data for a number of other questions are available upon request, and are not shown here only due to brevity (article length) constraints.

D. Analysis of interviews
Finally, we will complete the triangulation of data collection (FECAT results, 360 attitude surveys, interviews) with a discussion of the students' interviews. The questions/statements asked are listed below.
1. I enjoyed chemistry lectures more if they had CQGs.
2. I think that playing CQGs helped me in learning some new chemistry concepts.
3. I think that playing CQGs increased my motivation to learn chemistry. For interview question 1 (Fig 7a) all interviewees report either "A" or "SA" with the statement that playing CQGs increased their enjoyment. Similarly, in interview question Journal of Chemical Education 9/30/21 Page 12 of 22 2 (also Fig. 7a), students report that their grasp of concepts in chemistry was helped by 370 the CQGs. We wish to note here, however, that it is hard for students to objectively distinguish between the increase in conceptual knowledge due to the regular traditional coursework and due to CQGs.

385
Moving to the interview questions 3 and 4 (still Fig. 7a), the majority of students (~73%) report that they feel strongly motivated to study chemistry thanks to, at least in part, CQGs. Answers to the fourth question indicate that, while no students reported decreases in interest to study chemistry due to the CQGs, fewer students reported strong positive effect, compared with the motivation.

390
In a pair of interview questions where students need to rely on their self-reported skills, students with lower English skills (typically below or at cumulative IELTS score of 6.0), agree that their desire (interest) to looking forward to playing the CQGs, was in part, driven by the lower English skills. Fig. 7(b) indicates that slightly over 50% of students agree or strongly agree that their lower level of English led to their desire to play CQGs 395 as a compensatory learning mechanism. This last finding points towards the anticipated channel of introducing the computer games in the learning process in ESL science classroom, and in many third world countries, latter being relevant wherever a portion of higher education teaching and learning processes is conducted in English.

SIGNIFICANCE AND CONTRIBUTION TO PRACTICE OF CHEMISTRY EDUCATION
To the best of our knowledge, this is the first multi-semester, two different control groups study on the effectiveness of CQGs in an ESL environment for engineering majors, possibly overall and particularly in the broader MENA context. The study has several specific contributions, which may be considered independently, or as mutually  as a working tool in collaborative learning, or as an initial discussion point in the studio method.

CONCLUSION
The answers to our research questions are:

3.
We submit that the test, tentatively named FECAT is a useful general chemistry test, given its acceptable values of KR20 and Cronbach Alpha [42].

4.
CQGs and PQs both helped improve students' learning gains compared with students who had neither activity (Semester III). There is a statistically significant difference between chemistry achievement for students in group X (with CQGs) and group Y (with PQ) only for subsets of the overall population (either in separate semesters or for one gender). However, there is a statistically significant difference 470 between group X (CQG) and group Z (without CQG or PQ). The four stages of selfdirected learning model (dependence, interest, involvement, self-direction) proposed by Grow [55] help explain how students were motivated and transformed from low to high self-directed learning.

5.
There is an increase in positive attitude among the CQGs group compared with 475 the PQs group. Interview data also indicated that students' taking CQGs had positive attitudes toward, and motivation and interest to learn chemistry. This is supported by discussion in literature [56][57][58], which highlights that the greatest educational benefit of computer-assisted instruction could be increased student motivation and improved attitudes, together with [59], where it was reported that 480 games can be used to increase both intrinsic motivation and cognitive growth.