An undergraduate research experience in earth science education that benefits pre-service teachers and in-service earth science teachers

Abstract Three cohorts of six pre-service Earth Science teachers (undergraduate majors in Earth Science Education) participated in summer research experiences focused on developing dynamic physical models of Earth processes to help middle and high school students understand complex concepts and confront misconceptions. The pre-service teachers used published criteria for evaluating models. Participants deepened their understanding of specific Earth Science concepts and broadened their perceptions of effective, student-centered, constructivist pedagogical practices through the use of models and model-based learning. Our pre-service Earth Science teachers achieved the same benefits that STEM majors report from their undergraduate research experiences, including better understanding of the nature of science, gains in problem-solving and communication skills, increased confidence, collaborative skills and comfort in working independently. Evaluation of the research experience via the Undergraduate Research Student Self-Assessment indicated that pre-service teachers reported higher gains than STEM majors in nearly all categories. The pre-service teachers presented the results of their projects to in-service teachers in professional development workshops at a science teachers’ conference. In-service teachers’ responses to these workshops were uniformly positive (98.2%; n = 57). Unlike most professional development activities in which participants benefit, but presenters may not, these professional development activities benefited participants and presenters alike.


Introduction
The benefits of undergraduate student research experiences (UREs) (e.g., increased understanding of the nature of research, improved problem-solving skills, gains in confidence, patience, and creativity) are well documented (Eagan et al., 2013;Guertin, 2014;Hunter et al., 2007;Lopatto, 2004Lopatto, , 2010Periera and Neves, 2014;Russell et al., 2007;Seymour et al., 2004;SERC Starting Point, n.d.). The benefits of UREs also include long-term impacts, such as clarification of graduate study and STEM career choices, deep insight into the process of research, envisioning one's self as a scientist, development of transferrable skills and development of professional networks (Trott et al., 2020).
UREs are commonly available to geoscience majors, occasionally for in-service teachers (e.g., RET programs), but appear to be less common for majors in Earth Science Education (ESE). Abolins (2014Abolins ( , 2015Abolins et al., 2015;Bomar & Abolins, 2015;Comacho & Young, 2014;Mims, 2014) engaged ESE majors in geologic research rather than in topics directly related to their careers as teachers. Ebert and Downey (2013) argued that the same benefits that accrue to undergraduate researchers in geoscience also accrue to pre-service teachers that engage in Earth Science education projects, particularly in developing and evaluating physical models of geologic processes. Such research experiences can assist pre-service teachers in developing pedagogical content knowledge (Shulman, 1986).
With the advent of the Next Generation Science Standards (NGSS) (NGSS Lead States, 2013) and derivative state standards, K-12 teachers are expected to educate their students using disciplinary core ideas with emphasis on cross-cutting concepts along with science and engineering practices -so called "three dimensional" teaching. Disciplinary core ideas and the broader crosscutting concepts are familiar content in college science courses. However, in our experience, encounters with scientific research and with any aspect of engineering are less than common in science classes and science education curricula. Thus, pre-service teachers have had few, if any, opportunities to gain these experiences. Given these circumstances, Ebert and Downey (2013) questioned how this type of teaching could take place in K-12 classrooms.
In this paper we report the results of a cohort-based, summer URE for pre-service Earth Science teachers (ESE majors) that was designed to provide firsthand experiences in scientific inquiry and engineering design through the design and construction of physical models of Earth processes. We also report on the reactions of in-service teachers to professional development provided by our pre-service participants.

Literature context
Models are simplified representations of systems that enable better understanding of the model's target (Gilbert, 1995;Gilbert & Ireton, 2003;Gobert & Buckley, 2000;Ingham & Gilbert, 1991). Models are especially necessary in Earth Science education because so many processes and concepts in the geosciences are abstract or operate on spatial and temporal scales that are difficult for students to grasp (Cheek, 2010;Dodick & Orion, 2003a, 2003bEbert & Downey, 2013). Bryce et al. (2016) suggested that concrete (physical) models are most effective for student learning if they provide clear visual representations of the target process, are context-rich and revisable. Although they dealt primarily in the field of biology, Bryce et al. (2016) cited the utility of models in elucidating processes that are not easily observed because of inaccessibility or because of their spatial or temporal scale such as tectonic movement of continents, Earth's core and geologic time. Other investigators have shown that novices have great difficulty developing useful understandings of many of geology's core concepts, such as plate tectonics (Clark et al., 2011), Earth's interior (Libarkin & Anderson, 2005) and geologic time (Dodick & Orion, 2003a, 2003b. Mayer (1989) and Louca and Zacharia (2012) showed that model-based learning (MBL) is pedagogically effective in enhancing students' conceptual understanding. In a review article that included 31 tests of MBL from 20 studies, Mayer (1989) reported substantial gains in conceptual recall and creative transferrable problem solving, but not students' verbatim retention of vocabulary. Louca and Zacharia (2012) reported on the cognitive, metacognitive, social, material and epistemological contributions to science education by MBL. In particular, Louca and Zacharia (2012) cite the facilitation of student learning through the use of physical models, and Dede et al. (1999) noted that three-dimensional models are more effective than two-dimensional representations (i.e., inscriptions).
Models and model building are important components of the process of science as practiced by scientists (Kastens & Rivit, 2008;Manduca & Kastens, 2012) in addition to their utility in the process of teaching science by science educators. Citing Windschitl (2013), Bryce et al. (2016) argued that modeling is a metacognitive process used in testing hypotheses and is, therefore, integral to the process of science and science instruction. Numerous authors have stressed the importance of students' understanding of the nature of science (NOS) (Allchin, 2012;Lederman, 2007;Matthews, 2015) and models have been shown to be effective in helping students understand NOS (Clement, 2008;Coll et al., 2005;Khan, 2011).

Purpose and learning goals
The purpose of this project was to immerse pre-service Earth Science teachers (undergraduate majors in Adolescence Education Earth Science) in a research experience directly related to their chosen profession. Our project was designed to provide participants with a URE in Earth Science education that focused on models and model-based learning, such that participants would gain the benefits associated with UREs and would gain facility in engineering practices through the design and construction of physical models. The focus of this experience was on the development and testing of physical models to represent complex or abstract concepts in the geosciences. We intended our participants to achieve the following learning goals: (i) strengthen their content knowledge regarding specific geoscience concepts, (ii) use that knowledge to conceive of and build physical models to represent the target concepts/processes, (iii) broaden their perspectives of what constitutes effective pedagogy, (iv) provide participants with experience in professional presentation, and (v) through these presentations to provide professional development to in-service teachers by sharing the models that were developed along with lesson plans for the use of the models. With explicit knowledge of models and model-based learning, our participants could then utilize models effectively to improve K-12 students' understanding of complex Earth Science concepts. Our learning goals for the in-service teachers were to gain knowledge of the newly created models and to think more explicitly about the use of models in the classroom.

Study population and setting
In the summers of 2017-19, three cohorts of six undergraduate pre-service teachers (ESE majors) participated in month-long Earth Science Model Research and Design Institutes (ESMRDI) at the State University of New York College at Oneonta (SUNY Oneonta), supported by an NSF grant (GEO/ICER 1701048). Participants received stipends for their participation and were provided with room and board during the Institutes. Seventeen pre-service teachers completed the program. Of these, fourteen were SUNY Oneonta students and three were from sister institutions in the SUNY system. All participants were rising juniors or rising seniors. Demographic data from the URSSA/SALG instrument indicated that 76% of our participants self-reported as female and 12% self-identified as Hispanic/ Latina/Latino. The majority of our participants are now practicing Earth Science teachers. SUNY Oneonta's Institutional Review Board examined the human subjects research aspects of the project and determined that exempt status was appropriate.
In these Summer Institutes, ESE majors were engaged in research experiences similar to an NSF-sponsored Research Experience for Undergraduates (REU) but with a focus on developing three-dimensional physical dynamic models of Earth processes and features that could help K-12 students understand complex and/or abstract concepts. These experiences also familiarized participants with the NGSS thread of engineering design. The ESMRDI were scheduled immediately following the end of the spring semester so that students would have time for summer jobs following the Institute.
Models developed by our pre-service participants were disseminated to in-service teachers via workshops at state science teacher conferences. Participants in these workshops were predominantly experienced teachers of Earth Science. Several workshop participants were teachers certified in other sciences, but had been assigned to teach Earth Science.

Materials and Implementation: Earth Science Model Research and Design Institutes
Concrete experiences are important for helping students understand abstract and counterintuitive concepts in the geosciences, and models can help facilitate student understanding (Niebert et al., 2012;Niebert & Gropengiesser, 2015). Unfortunately, for many concepts in the geosciences, there are no suitable physical/concrete models to help students develop the intended understandings of concepts and/ or confront their misconceptions. To address this, our participants spent four weeks in an intensive, summer research experience, the ESMRDI. During these Institutes, pre-service teachers designed, built, tested, and evaluated prototypes of concrete dynamic (Boulter & Buckley, 2000) models ( Figure 1).
ESMRDI participants spent four weeks immersed in Earth Science content, examining existing models of Earth processes and in the fundamentals of model-based learning, a theoretical framework for understanding students' mental models (Clement, 2008;Núñez-Oviedo & Clement, 2008;Núñez-Oviedo et al., 2008;Rae-Ramirez, 2008). During the first two weeks, mornings were spent reviewing the nature of science, exploration of models, metaphors and making meaning, the methods of geoscience (Kastens & Rivit, 2008;Manduca & Kastens, 2012), types of models and the properties of models (Gilbert & Ireton, 2003), working with existing physical models (e.g., convection, earthquakes and others), and criteria for evaluating models (Table 1; Gilbert & Ireton, 2003). Students were divided into teams of two or three and spent afternoons during the first two weeks brainstorming ideas for new models, designing prototypes, testing and evaluating the prototypes and modifying them as needed (Figure 1). Short field trips and team-building activities were scheduled on one to three afternoons, depending on the progress that participants were making with their models. Participant teams were reconfigured for weeks three and four and the new teams brainstormed additional new models, designed and built them and tested and evaluated these models.
Participants engaged in self-and peer-evaluation of their models using the criteria of Gilbert and Ireton (2003;also Table 1). Following these initial tests and evaluations, participants made improvements to their models. In this fashion, they gained firsthand experience with the iterative process of engineering design and NOS (NGSS Lead States, 2013). Analyzing the strengths and limitations of the models that they developed helped participants understand modeling as an integral part of the scientific process. Participants also composed lesson plans for the use of their new models.
Pre-service teachers presented their models to panels of New York State Master Teachers, who offered helpful suggestions for improvements and suggestions for classroom use. Our participants also presented the models that they designed and built in middle and high school classrooms and in two workshops (2018 and 2019) at the annual conference of the Science Teachers Association of New York State (STANYS). Pre-service participants from the summers of 2017 and 2018 presented at the 2018 STANYS conference and the 2019 participants presented their models at the 2019 STANYS conference. After brief introductions by the project directors (JE, PB and GD), the pre-service teachers presented the majority of each workshop by demonstrating their models and describing the process of designing and constructing their models. Pre-service presenters were also responsible for fielding questions from in-service teachers.
Although still under construction, examples of student-designed and built models are available on the project website (https://suny.oneonta.edu/earth-and-atmosphericsciences/earth-science-models). The experience of researching, designing and building their models, with collegial dialog throughout these processes, combined with presenting their models in a professional setting provided our pre-service teachers with a unique opportunity for induction into the profession prior to student teaching (practicum).

Structure of the Earth Science Model Research and Design Institutes
The authors of this paper organized and presented the ESMRDI. During the first few days of the summer institutes, participants were introduced to the use of models in teaching Earth Science through a series of lectures and hands-on activities led by author GD. These early presentations approached models, modeling, and MBL from many different contexts. These included history of science, where the history of geological concepts was traced from their earlier explanations to more modern ones (Dolphin & Benoit, 2016), and case studies that highlight the role of new data and new points of view on changing conceptions on the development of scientific knowledge (Allchin, 2011(Allchin, , 2014Dolphin et al., 2018). GD also presented some of the current understandings of learning from the cognitive sciences centered on how the mind makes sense of reality through the senses and mental process such as accommodation and projection (Indurkhya, 1992). GD led a discussion on the nature of language with the theme of metaphors as models in terms of how teachers teach science and how scientists actually do science (Nersessian, 2008;Thagard, 2012). In all, these presentations drew together the role of models in the development of scientific knowledge in the minds of experts and in the minds of novices, culminating with the implications for science teaching.  Participants were shown models that are commonly used in Earth Science classrooms such as the "earthquake machine" (Figure 2; Hubenthal et al., 2008;IRIS, n.d.), phases of the moon, block models of geologic structures, topographic maps, etc. Author JE presented multiple models for some concepts such as convection (Ebert et al., 2004). Participants practiced evaluation of these existing models using the criteria of Gilbert and Ireton (2003; See also  Table 1).
In formative evaluations of the project, participants in the 2017 ESMRDI indicated that the introductory sessions of the first week were quite intense and they recommended that more time be devoted to these sessions. Therefore, the project directors changed the opening structure of the 2018 and 2019 ESMRDI in response to this feedback. The foundational information on models in science and science education were spread over the first two weeks instead of just in the first week, as was the case in the 2017 ESMRDI.
Early in each ESMRDI, author PB familiarized participants with the New York State Science Learning Standards (NYSSLS), which are modeled on the NGSS. After gaining familiarity with the content and structure of the NYSSLS standards that deal with Earth and space science, participants identified content with which they felt uncomfortable. The NYSSLS reading was supplemented with a list of topics for which there are no dynamic physical models or for which existing models are inadequate. This list was generated from remarks collected from in-service teachers via the Earth Science Resource for Improved Teaching (ESPRIT) listserv (Ebert, 2016;2021) and the rich literature on geoscience misconceptions/alternative conceptions/naïve conceptions (Cheek, 2010;Francek, 2013;Kirkby, 2014). Participants discussed these topics and then were divided into teams of two in which both students identified similar topics. Each pair then began deepening their understanding of the target concepts through research via the Internet and a collection of middle and high school Earth Science textbooks.
With improved understanding of specific concepts, each team of pre-service Earth Science teachers began brainstorming ideas for new dynamic physical (functional, three-dimensional) models that could help K-12 students understand each target concept. Teams designed and built prototype models ( Figure 3). Project leaders assisted participants in obtaining building materials and in troubleshooting problems with models. Prototypes were tested and, in some cases, abandoned in favor of more effective models derived from the failure of the initial model. For example, one team of pre-service researchers built a prototype model intended to demonstrate planetary retrograde motion through the use of plexiglass rings. The researchers noticed that this prototype suffered from a problem with the observer's frame of reference and that the retrograde motion was less than apparent. This prototype was abandoned and a new model (Figure 4) was constructed that avoided the problems noted with the first prototype.
Throughout the brainstorming, designing and building of prototypes, participants profited immensely from PB's input as he modeled questioning and scientific thought processes. As a non-geoscientist (microbiologist and science educator), his questions often mimicked the types of   questions that K-12 students might have. This questioning provoked deeper thinking by the participants and enabled them to consider ways that their models might be used in classrooms to aid students' understanding of the model's target concept.
When teams were reasonably satisfied with their models, they evaluated their new models (Table 1). They presented their models to the other teams of participants for evaluation using the same criteria. Each team improved their prototypes based on feedback from the other groups. Teams then created lesson plans for using the models in middle and high school Earth Science classrooms. Teams also produced short videos describing each model, how they were constructed, and how these new models could be used in K-12 classrooms (See Models Project website).
For the second half of each Institute, new teams were constituted, and the processes described above were repeated.
These new teams constructed and tested a second set of models. In this fashion, five to six new models were constructed during each iteration of the ESMRDI.
Several short field trips and team-building activities such as caving and canoeing on a local lake were interspersed throughout each ESMRDI. These activities served to refresh participants and to allow informal time to ponder their models and problems associated with the development of models. Near the end of each Institute, participants presented and described their models for groups of New York State Master Teachers. In one summer, participants also presented their models to middle and high school students in the classroom of one of the Master Teachers.

Presentation to in-service teachers
During the fall semesters after the 2018 and 2019 ESMRDI, participants reconvened periodically to plan workshops for in-service teachers at the annual conference of the Science Teachers Association of New York State (STANYS). Outlines of each workshop were constructed and ESMRDI participants practiced brief demonstrations of their models in preparation for the STANYS conference. In early November of 2018, participants in the 2017 and 2018 ESMRDI presented models at the STANYS conference in a workshop entitled "New Models for Earth Science Concepts and Processes" (Figure 5). Similarly, participants in the 2019 ESMRDI presented a workshop the following fall entitled "New Models for Earth Science Concepts and Processes − 2019 Edition." Responses of in-service teachers to these workshops are presented in a later section of this paper.

Pre-service teachers
The research experience of our pre-service teachers was assessed by several means. The project's external evaluator interviewed the participants and provided deidentified transcripts of those interviews to the PIs as a formative assessment of the research experience. The external evaluator also generated a post experience survey completed by the 2017 participants to assess the need for any changes in the following Institutes. Likert-type participant responses to this survey appear in Table 2. The survey also included three free response items. Selected excerpts from these responses are included in the Results section. All responses are available in the online Supplemental Material.
Participants engaged in reflective writing in response to writing prompts provided by the principal investigators and these artifacts were also used as formative assessments of the research experience. Participants in the 2017 and 2018 ESMRDI engaged in reflective writing exercises that were intended as formative assessments to help the PIs make changes to the program if necessary. Two reflective writing exercises were scheduled at convenient points during the ESMRDI. Students responded to prompts such as "Were you familiar with the concepts of models and model-based teaching before this week? If so, describe what you think about the pedagogical utility of using models in teaching earth science" and "What has been your greatest accomplishment this far in the Model-Building Institute? Why?" Late in the summer of 2019, seventeen of our participants completed the Undergraduate Research Student Self-Assessment (URSSA; Weston & Laursen, 2015), a validated instrument that measures Student Assessment of Learning Gains (SALG). This occurred within a few months of completion of the summer institute for our 2019 participants. One to two years elapsed between participation in the ESMRDI and completion of the URSSA instrument for our 2017 and 2018 student researchers. The URSSA is a survey instrument that addresses students' opinions of their gains through their research experience. The instrument is comprised of 116 items divided among 26 headings/clusters. Our use of the URSSA focused on students' perceptions of their gains in Thinking and Working like a Scientist, Personal Gains Related to Research, Skills, and affective components of the research experience. Most sections of the URSSA use Likert-type responses but some items require written responses.

In-service teachers
Our participants presented their models in workshops at two annual conferences of the Science Teachers Association of New York State (STANYS). At the end of each STANYS workshop, we distributed a seven-item questionnaire to the participants to evaluate their reactions to the workshops presented by our students, but most especially how they felt about relatively inexperienced pre-service teachers providing professional development. The questionnaires were also designed to elicit the perspectives of in-service teachers on the efficacy of the models that our students presented in terms of helping middle and high school students' understanding of complex and/or abstract concepts. Half of the items on the questionnaire were keyed specifically to the designed models and their potential use in middle and high school classrooms. The questions in the workshop evaluation instrument are listed in Table 3.
Items 1, 2, and 6 were targeted at the perceptions of the in-service teachers regarding professional development provided by pre-service teachers. Item 3 was intended to assess the teachers' thoughts on the efficacy of the models presented in terms of assisting students in understanding specific concepts in Earth Science. Items 4 and 5 were intended to assess the likelihood of teachers using the models presented in their classrooms. Lastly, item 7 asked teachers to  1) Please comment on the overall presentation/format of this workshop.
2) Have any of the models presented changed your thinking about the content related to that model? if so, please describe how and explain why.
3) Based on your experience in the classroom, do you think that the models presented in this workshop will help students understand concepts better or abandon their misconceptions regarding specific concepts? 4) Please estimate the likelihood that you will use at least one of the models presented with your students (0% = i won't use it; 90% = i'll definitely use it; 100% = i will definitely use it this year; or any percentage in between). 5) Which model or models do you feel you are most likely to use and why? 6) Please share your thoughts on the concept of pre-service students conducting research that benefits in-service teachers. 7) can you suggest other earth concepts/topics/processes for which pre-service teachers might develop new models to help students better understand these phenomena?
suggest other concepts for which pre-service teachers might develop models to help students better understand the concepts or which could help students confront their misconceptions. The responses of the in-service teachers on the questionnaire are described in the next section, following the results for the pre-service teachers.

Pre-service teachers responses on the Undergraduate Research Student Self-Assessment (URSSA) of Student Learning Gains (SALG)
All seventeen participants responded to most items on the URSSA (n = 17). However, not all students answered all questions, so some items have an n ranging between 12 and 16. The most pertinent URSSA data are displayed in Tables 4-7. All URSSA data, including written responses to some items are available in the online Supplemental Material.
In the categories of Thinking and Working like a Scientist (Table 4) and Personal Gains related to Research (Table 5), mean results were all between 4 and 5 (Maximum possible is 5.) representing good to great gains. Grand means for these categories were 4.7 and 4.6 respectively. (Table 6) ranged from 3.5 (between moderate and good gains) to 4.6 (between good and great gains). The grand mean for this cluster of items is 4.0 (good gain). It should be noted that several items in the Skills cluster are more pertinent to scientific research (e.g., calibration of instruments) as opposed to the research in scientific pedagogy through models in which our participants engaged. All participants agreed or strongly agreed that the research experience helped prepare them for graduate school or entry into the workforce (Table 7).

Pre-service teachers responses on the reflective writing prompts, post-institute survey and semi-structured interviews
It became apparent through the reflective writing that some participants were engaged in metacognition and were gaining greater understanding of effective pedagogy. Four students out of eleven responses wrote about metacognitive gains and increased awareness of their thought processes (e.g., "develop my cognitive thinking process"). Nine of eleven participants that responded wrote that their perceptions of the processes of teaching and learning had changed from teacher-centered lecturing to more inquiry-based, student-centered pedagogies. The following quote is an example of how one participant changed her perception of teaching and learning: "I started to reevaluate the way I was taught science in high school. None of my previous teachers taught me to question what I was learning or teach it in a way in which I am discovering answers (before entering college at least); it has always been presented to me in a clear-cut and dry fashion where there was no room for exploration and development of ideas or concepts. " (Audrey -name changed) Similar awareness of changing views of teaching was voiced repeatedly in conversations with participants throughout each summer Institute. Clearly, the ESMRDI process of researching and constructing models expanded participants' views of effective pedagogy.
We observed that the 2019 ESMRDI cohort struggled more with model development than participants in the first   Similarly, another pre-service teacher's response reflects increased content knowledge and understanding of pedagogy: "I've been through high school and part of college and only now, after seeing these models, do I truly understand certain concept(s) in the geosciences that I had thought I knew well prior, but in actuality, I didn't. Models engage students and help them to discover things they might not have been able to discover before. They're a useful tool, so much better than words on a board." Item 26 provided participants an opportunity to evaluate the ESMRDI overall. All responses to these three items are archived in the online Supplemental Material.
Our external evaluator also conducted semi-structured interviews with the 2017 cohort. These interviews were intended as formative evaluations to assist the project leaders in adjusting the program as needed. From these interviews, it was apparent that participants increased their knowledge of the use of models in science and in the teaching of science. The following quotation taken from the transcript of one semi-structured interview is illustrative: I think that's why inquiry or experiments is just so important because you're not forcing something down their throat, they are coming to it through questioning, and they see it and do it a hands-on experiment or something like that, and they are coming to an answer on their own and discovering that "oh maybe it's not what I thought before, " they are experiencing something different … I'm learning that there is (sic) multiple ways or different ways…. (unidentified participant). A verbatim transcript of the 2017 semi-structured interview is available in the online Supplemental Material.

In-service teachers responses on conference workshop evaluation questionnaire
Fifty-seven in-service teachers participated in the professional development workshops presented by the pre-service Earth Science teachers at the annual STANYS conferences. Twenty-seven teachers attended the 2018 workshop and 29 attended the 2019 workshop.
In-service teachers responses to the workshops presented by the pre-service teachers were overwhelmingly positive (98.2% on item 1). The most commonly cited positive aspect of the workshops was that students were the presenters (26.3% of responses). Selected responses to Item 1 are given in Table 8.
In response to item 2, 49.1% of participants reported that the models presented in the workshop changed their thinking about the target concepts. This was especially the case for two biology teachers in the 2018 workshop that had been assigned to teach Earth Science for the first time (Ebert, Dolphin, & Bischoff, 2019). Over one quarter of the participants (26.3%) indicated that the workshop did not change their thinking about the target concepts, but the models prompted them to think of new ways to teach the target concepts to their students. Example responses to item 2 are given in Table 8.
In-service teacher responses to item 3 were also highly positive (96.5%). Nearly all of the participants indicated that the models presented would help their students understand the target concepts. Over ninety percent (91.2%) of the teachers reported that there was a greater than 75% chance that they would use some of the models in their classrooms and 42.1% indicated that they would definitely use at least one of the models with their students.
The concept of pre-service teachers providing professional development for in-service teachers was viewed positively by 66.7% of the in-service teachers. Further, 14% indicated that this was a definite positive for the pre-service teachers and an additional 10.5% commented on the fresh/new ideas that pre-service teachers brought. Some of the in-service teachers (7%) were pleased that undergraduate students were developing models because in-service teachers lack the time to do so. There were no negative responses to Item 6. Selected responses are listed in Table 8. Table 8. example responses of in-service teachers to three items on stanYs Workshop evaluation Questionnaire.
Example responses to item 1 by in-service teachers "I am very impressed by the quality of both the models and the student presenters." "The students were excellent at explaining how they constructed their models." "great ideas for hard to teach concepts" Example responses to item 2 by in-service teachers "Great material and content. I had a lot of "ah-ha" moments" "Mars retrograde. I had no concept of how this worked (I'm a biologist.) until I saw this." "I found the scale model of Earth/Troposphere to be particularly eye-opening. We tend not to realize just how thin the atmosphere is." Example responses to item 6 by in-service teachers "This is awesome for teachers and I wish I had this opportunity as a pre-service teacher." "Great idea! Often in-service teachers could like to do more but find it difficult to find the time to research/implement ideas they may have." "I love it… students that are very knowledgeable in content but not far-removed from learning it have a very unique perspective that more seasoned teachers lack." "LOVE IT. Wish I had been able to do it when I was pre-service, so helpful, kinda want to have these kids be a student teacher in my class!" In response to item 7, 51 topics were listed by the in-service teachers as areas where new models would be especially helpful in aiding the understanding of middle and high school students. Three of the concepts that were mentioned on the 2018 workshop evaluation were the inspiration for some of the models that were devised by participants in the 2019 ESMRDI.

Interpretations and discussion
Participants in this project deepened and enriched their understanding of geoscience content, designed and built physical models, developed facility with nature of science and engineering practices and attained the benefits that are typically associated with UREs (Bauer & Bennett, 2003;Eagan et al., 2013;Guertin, 2014;Hunter et al., 2007;Lopatto, 2004Lopatto, , 2010Periera and Neves, 2014;Russell et al., 2007;Seymour et al., 2004;Trott et al., 2020). Our pre-service Earth Science teachers also gained confidence and a sense of professionalism by presenting their work at science teacher conferences.
Pre-service teachers reported strong gains in understanding and appreciation of the process/nature of science. ESMRDI participants developed their facility with inquiry and now know the joys and frustrations that are inherent in the process of science (See also Ebert & Downey, 2013). Of equal importance, participants changed their perceptions of the processes of teaching and learning and increased their openness to pedagogies that are more constructivist and involve authentic research. Pre-service teacher/researchers improved their problem solving and communication skills and reported gains in confidence, patience, creativity, time management, ability to answer questions, collaborative skills, and comfort in working independently.
Our evaluation data show that pre-ser vice teacher-researchers benefited in numerous ways from the research experience (e.g., gains in problem-solving ability, communication skills, increased confidence, patience, creativity, time management, ability to answer questions, collaborative skills, and comfort in working independently, etc.). How do these benefits compare to those gained by STEM or other majors that engage in undergraduate research? The use of the URSSA instrument facilitates a direct comparison of our results with other projects that used the URSSA to assess the URE. Several of these other studies utilized a CURE model of undergraduate research (Course-based Research Experiences; Auchincloss et al., 2014). Some studies evaluated individual mentored apprenticeships (REU model). In Table 9, we compare the URSSA results of our cohort-based URE with a CURE-based study involving geoscience majors (Kinner & Lord, 2018), three years of a mentored apprenticeship REU program at the Lamont Doherty Earth Observatory of Columbia University (Ebert, unpublished data), a field-based, conservation-focused CURE (Sorensen et al., 2018), an interdisciplinary study of human impacts and ecology that focused on underrepresented minority students (White, 2017), a large scale freshman CURE (Sandquist et al., 2019) and a very large, summer research experience for both STEM and non-STEM majors (Stanford et al., 2017). Table 9 compares the mean scores of the first four URSSA clusters: 1) Gains in Thinking and Working like a Scientist: Application of Knowledge to Research Work, 2) Personal Gains Related to Research Work, 3) Gains in Skills, and 4) Overall Research Experience and Any Changes in Attitudes or Behaviors as a Researcher between this study and published cohort-based, course-based and apprenticeship models of URE. In all categories, the Earth Science Education majors reported higher gains than participants in another cohort-based study in which the research topic was interdisciplinary in nature (White, 2017). The ESE majors also reported higher gains in all categories compared to a CURE study in the geosciences for both lower and upper-level geoscience courses (Kinner & Lord, 2018) and CURE studies of a broader nature (Sandquist et al., 2019;Sorensen et al., 2018). ESE majors also reported higher average gains in all categories compared to studies of apprenticeship models (Stanford et al., 2017;Ebert unpublished data) with the exception of Gains in Skills in which one group of REU apprentices reported slightly greater gains (4.14 vs. 4.0). The data in Table 9 are presented for comparison purposes only. The significance, if any, of the differences presented were not evaluated statistically owing to the incomplete nature of much of the previously published data.
In addition to exceeding the gains in nearly all categories compared to other studies that used the URSSA, the mean gains reported by ESE majors are considerably higher than the means reported in other URE studies. There are several potential explanations for this difference. The sample size of this study (n = 17) is rather small, though similar to the other cohort-based study in Table 9 (n = 18; White, 2017) and to one of the CURE studies (n = 23; Sorensen et al., 2018). The other studies presented in Table 9 evaluated much larger numbers of participants. This difference in sample size may account for some of the differences, but not necessarily the magnitude of the differences.
There is an important demographic difference between the ESE majors in this study (12% minority) compared to White's (2017), which focused on underrepresented minority students. Further, the ESE majors in our study are predominantly female (76%), similar to the demographic data reported by White (2017; 70% female) and Stanford et al. (2017;83% female) but contrasts with the demographic profiles reported by Kinner and Lord (2018; 13-38% female for mid-and upper-level courses) and Sorensen et al. (2018;approximately 50% female). From this, we conclude that gender did not play a role in the differences in the means presented in Table 9.
The magnitude of the gains reported by ESE majors may be attributable to differences in prior background/coursework between these science/education dual majors and other students, particularly STEM majors. In general STEM majors may have greater background knowledge prior to a URE than ESE majors. In other words, the potential for gains among ESE majors may have been greater than for the STEM majors that comprised most of the other studies listed in Table 9. However, it is also likely that students that choose to enter the teaching profession (our participants) may possess different personality traits (Göncz, 2017;Harris et al., 2006) than other majors and that these traits lead to more positive/optimistic reports of gains than students that choose other majors. Schermer (written communication; Harris et al., 2006;Schermer, 2012) has noted that teaching is positively correlated with the personality trait of agreeableness, whereas STEM-interested individuals typically show a negative correlation. Further, she notes that teaching is linked to Holland's (1985) social dimension. Therefore, the personality traits inherent to those that choose a career in education may explain, at least in part, the magnitude of gains reported by our pre-service Earth Science teachers on the URSSA. We believe that the ESE majors in this study reported such high gains because the research in which they participated was directly related to their career goals. The concrete models generated via the ESMRDI research are applicable immediately to classroom use. In the other studies reported in Table 9, the research topics may (e.g., Kinner & Lord, 2018;Sorensen et al., 2018) or may not (e.g., Sandquist et al., 2019;Stanford et al., 2017;Ebert, unpublished data) be directly related to the participants' majors or career goals.
Like their counterparts in geology UREs, the pre-service teachers that participated in the summer research benefited immensely from the experience with gains in content knowledge but they also increased their pedagogical content knowledge (Shulman, 1986). Further, they benefited from the experience of preparing for and presenting their models in the STANYS workshops. The collegial reception of these efforts by the in-service teachers at STANYS and by the New York State Master Teachers increased the confidence of our pre-service teachers. Presenting at a conference was also a form of early induction into the teaching profession and fueled a growing sense of professionalism in these future teachers. Time will tell, but we believe that the success of this early professional contribution may be an important factor in developing the leadership potential of these nascent teachers of Earth Science.
Professional development for in-service teachers with respect to model design and construction is a critical need. Most Earth Science teachers make extensive use of models in their teaching but lack formal training in model design and construction (Van Driel & Verloop, 1999). Moreover, in-service teachers seldom have time available to create new models even when they recognize that such models would aid student understanding. Therefore, the models developed by our pre-service teachers fill an important gap, but only if disseminated to in-service teachers.
In-service teachers have many forms of professional development available including conferences, workshops, and webinars. In most professional development venues, the participants are the beneficiaries of the expertise of the presenters. Here, we introduce a novel form of professional development for in-service teachers (See also Ebert, 2006;Ebert & Downey, 2013): workshops presented by pre-service teachers (ESE majors). In these workshops, the in-service participants benefit, but the pre-service presenters do as well.
The professional development workshops conducted by undergraduate students were well received by the in-service teachers, who treated the pre-service presenters with professional courtesy and genuinely appreciated the professional development that was provided. Results from the STANYS workshop evaluations indicate that in-service teachers are receptive to and benefit from professional development provided by pre-service teachers. The success of the STANYS workshops demonstrates that pre-service teachers are not only capable of developing instructional materials and strategies that can benefit their in-service colleagues, but that they are also capable of delivering unique, high-quality professional development for in-service teachers. With opportunities and mentoring undergraduate students represent a vast, untapped resource capable of providing professional development for in-service teachers.

Limitations
Our study has several limitations. Our sample size of pre-service Earth Science teachers is small (n = 17). Participants were drawn from a single state university system (SUNY) and were recruited predominantly from the host institution (14 of 17 participants). Pre-service teachers from a wider array of institutional types may have yielded differing results.
It would have been instructive if all three cohorts of our pre-service teachers could have field tested their models with 8th and 9th grade Earth Science students. However, this was only possible for one cohort (2017) owing to the timing of the Summer Institutes which concluded near the end of the K-12 school year in close proximity to the high stakes Regents Examination in the Physical Setting: Earth Science. Therefore, it was extremely difficult to have middle and high school teachers sacrifice instructional and review time at the end of the year.
In order to compare the perceptions of the learning gains of our participants with participants of other UREs, we used the URSSA instrument, which was designed to evaluate traditional STEM research experiences rather than the geoscience education research, engineering design and providing of professional development activities in which our participants engaged. Consequently, some items on the URSSA (e.g., calibrating instruments) were not applicable to our study. We are not aware of any instrument that would be more suitable for evaluating the type of activities in which our pre-service teachers participated.
Lastly, we relied on our external evaluator almost exclusively for formative evaluations to aid the PIs in identifying ways to improve the program. In retrospect, the external evaluator could have played a larger role in summative evaluation of the project.

Implications
Our pre-service Earth Science teachers successfully designed and constructed dynamic concrete models of specific Earth Science processes and concepts. Through these experiences, they gained deeper insights into earth science content, NOS, and gained facility with the processes of engineering design. These activities enabled our participants to strengthen their understanding of important concepts and build pedagogical content knowledge, which should translate into their teaching practice. Our evaluation data support the assertion that the participants achieved the benefits typically associated with more traditional STEM UREs, with pre-service participants reporting greater gains than reported by STEM majors in other studies. Traditional UREs tend to focus on specific areas of content, whereas our participants developed skills that are readily transferable to researching, developing and implementing models across the earth science curriculum.
Our pre-service teachers shared the fruits of their URE with in-service teachers, broadening the content knowledge of these veteran teachers and inspiring new ways of teaching specific concepts. In-service teachers that participated in the professional development workshops presented by our undergraduates were overwhelmingly positive in their evaluation of the experience. The success of these conference workshops demonstrates that pre-service teachers are eminently capable of providing professional development for in-service teachers, thereby increasing the pool of potential providers of professional development. The experience of presenting a workshop to experienced teachers, in turn, helped increase the confidence of the pre-service teachers and provided them with a collegial reception into professional practice. Wilson (2018) reported that only 3% of secondary teachers hold degrees in the geosciences. Full implementation of NGSS requires a dramatic increase in addressing concepts in the earth sciences. Even in districts where there is no dedicated course in the earth sciences, these concepts are to be embedded in other science courses. Therefore, it is essential that STEM teachers be well versed in geoscience concepts. Models are a pivotal way for secondary teachers to develop these conceptual understandings for themselves and for their students. The models developed by our pre-service teacher are one step in fulfilling this enormous need.