Professional Development

Ethnography & Evaluation Research (E&ER) has studied several programs that address professional development for STEM educators and scientists at various career stages. In higher education, these include faculty leadership development, women’s career advancement, and STEM teaching. Our work in K-12 education has focused on teacher professional development. Also see our work onÌýprofessional development of scientistsÌýfor their education outreach roles.

We have worked to develop and understand a variety of descriptive and evaluative measures to characterize teaching practices and change in these practices over time.

IBL Workshops in Mathematics

We have written extensively about the outcomes and processes for a decade-long series of intensive workshops on teaching college mathematics with inquiry-based learning. An overview of this work is available in our project summaryÌýfor the last of these projects (PRODUCT), and many more details for PRODUCT and prior workshop projects are provided in the Research and Evaluation sections below.Ìý

Laursen, S. L., Archie, T., & Hayward, C. N. (2021).ÌýCollaborative Research: PROfessional Development and Uptake through Collaborative Teams (PRODUCT) Supporting Inquiry Based Learning in Undergraduate Mathematics. Project Outcomes Report.Ìý

Evaluation of the IBL Centers workshops was supported by the National Science Foundation under award Ìýand by the Educational Advancement Foundation. Evaluation of the SPIGOT workshops was supported under NSF award .ÌýEvaluation of the PRODUCT project and cumulative analysis of the workshop dataÌýwere supported under . Any opinions, findings, conclusions, or recommendations expressed in these reports are those of the researchers, and do not necessarily represent the official views, opinions, or policy of the funders.

  • Archie, T., Daly, D., & Laursen, S. (2021). How much is enough professional development? Outcomes of short and extended workshops on inquiry-based learning in college mathematics. 2021 Joint Mathematics Meeting, January 6-9.
  • Archie, T., Daly, D.,ÌýLaursen, S. L., Hayward, C. N., & Yoshinobu, S. (2022).ÌýInvestigating the effectiveness of short-duration workshops onÌýuptake of inquiry-based learning. InÌýKarunakaran, S. S., & Higgins, A. (Eds.),ÌýProceedings of the 24thÌýAnnual ConferenceÌýon Research in Undergraduate Mathematics EducationÌý(pp. 36-45). Boston, MA.ÌýÌý
  • Archie, T., Hayward, C. N., Yoshinobu, S., & Laursen, S. L. (2022). Investigating the linkage between professional development and mathematics instructors’ use of teaching practices using the theory of planned behavior.ÌýPLoS ONE 17(4),Ìýe0267097.ÌýÌý(open access)
  • Archie, T., Laursen, S. L., Hayward, C. N., Daly, D., & Yoshinobu, S. (2021).ÌýInvestigating the linkage between professional development and mathematics instructors’ adoption of IBL teaching practices. In S. S. Karunakaran & A. Higgins (Eds.),ÌýÌý(pp. 1-10).Ìý Author accepted manuscript.
  • Archie, T., Laursen, S., Hayward, C. N., Yoshinobu, S., & Daly, D. (2020, November 5-7).ÌýFindings from 10 years of math instructor teaching professional developmentÌý[poster]. This Changes Everything,ÌýAAC&U Virtual Conference on Transforming STEM Higher Education.
  • Daly, D., Laursen, S., & Archie, T. (2021). Choosing the right tool for the job: The role of traveling workshops in engaging math faculty in active teaching strategies [poster]. Transforming Institutions 2021 Virtual Conference, online June 9-11, 2021.
  • Hayward, C. N., Archie, T., Daly, D., Weston, T. J., & Laursen, S. L. (2022). The cycle of inquiry: Building effective evaluation relationships to support continuous improvement of faculty development initiatives. In S. Linder, C. Lee, K. High (Eds.),ÌýHandbook of STEM Faculty Development, Information Age Publishing, Charlotte, NC.ÌýAuthor approved manuscript. ÌýHandbookÌý.
  • Hayward, C. N., Kogan, M., & Laursen, S. L. (2016).Ìý .Ìý International Journal of Research in Undergraduate Mathematics EducationÌý2(1), 59-82. Published online November 25, 2015. DOI 10.1007/s40753-015-0021-y [Abstract]
  • Hayward, C. & Laursen, S. (2014). Evaluating professional development workshops quickly and effectively.Ìý. Denver, CO, February 27-March 1.ÌýConference paper
  • Hayward, C. N., & Laursen, S. L. (2018). ÌýInternational Journal of STEM Education, 5:28. Ìý[Open access]
  • Hayward, C., & Laursen, S. (2017). Supporting instructional change in mathematics: The role of online and in-person communities. 20thÌýAnnual Conference on Research in Undergraduate Mathematics Education, San Diego, February 23-25.Ìý
  • Kogan, M., & Laursen, S. (2012). Role of faculty professional development in improving undergraduate mathematics education: The case of IBL workshops. In (Eds.) S. Brown, S. Larsen, K. Marrongelle, and M. Oehrtman,Ìý, pp. 80-87, Portland, Oregon.
  • Laursen, S. L., Archie, T., Weston, T. J., Hayward, C. N., & Yoshinobu, S. (2023). ÌýA measurement hat trick: Evidence from a survey and two observation protocols about instructional change after intensive professional development.Ìý25th Annual Conference onÌýResearch in Undergraduate Mathematics Education, Omaha, NE, Feb. 23-25. [Conference paper]
  • Yoshinobu, S., Jones, M. G., Hayward, C. N., Schumacher, C., & Laursen, S. L. (2023). A broad doorway to the big tent: A four-strand model for discipline-based academic development on inquiry-based learning.ÌýPRIMUS, 33(4),Ìý329-354.ÌýÌýÌý[Preprint,Ìý2022]

IBL Centers Workshops (2010-2013)Ìý

As evaluators for four universities who have offered intensive week-long workshops on inquiry-based learning in college mathematics, we are documenting immediate and longer-term changes in faculty knowledge, beliefs, and teaching practices as a result of participating in a workshop. The results highlight the impact of multi-day, interactive workshops that help faculty learn teaching methods, think through problems, and plan their own course.

SPIGOT Workshops (2013-2015)Ìý

Evaluation of a second series of multi-day, interactive workshops on inquiry-based learning (IBL) in college mathematics shows that a high proportion of participating instructors are implementing IBL approaches in their own classrooms. Their implementation is supported by active discussion on a cohort-based e-mail list.

  • Hayward, C., & Laursen, S. (2016, March).Ìý Collaborative Research: Supporting Pedagogical Innovation for a Generation of Transformation via Inquiry-Based Learning in Mathematics (SPIGOT), Cumulative Report: Workshops 1-4.Ìý [Report to the National Science Foundation] Â鶹ӰԺ, CO:Ìý Ethnography & Evaluation Research.ÌýReportÌýAppendix - InstrumentsÌý ÌýCombined set of all SPIGOT formative & summativeÌýevaluation reports
  • Hayward, C., & Laursen, S. (2013, October).Ìý Collaborative Research: Supporting Pedagogical Innovation for a Generation of Transformation via Inquiry-Based Learning in Mathematics (SPIGOT).Ìý Evaluation Report:Ìý Workshop 1 at California Polytechnic State University, San Luis Obispo, June 24-27, 2013.Ìý [Report to SPIGOT]Ìý Ethnography & Evaluation Research, Â鶹ӰԺ.
  • See also Research Products (above) led by Chuck Hayward, 2017-2018, on workshop follow-up mechanisms.

PRODUCTÌýWorkshops (2015-2020)Ìý

Using the same tools developed for the IBL Centers and SPIGOT workshops, we studied a third series ofÌýmulti-day, interactive workshops on inquiry-based learning (IBL) in college mathematics.ÌýWe provide formative and summative evaluation feedback to the Ìýoffered by the .

We developed a structural model of workshop participant outcomes that isÌýbased on Ajzen's (1991)Ìýtheory of planned behavior. We can do this using the large, combined data set from the three workshop series combined (IBL Centers, SPIGOT, PRODUCT)Ìýbecause they are broadly similar in design and implementation (see Research Products, above).ÌýThe structural model suggests that the workshops affect instructors' skills, knowledge and attitudes. These in turnÌýinfluence instructors' intention to use IBL and their reportedÌýintensity of use. Their work contexts matter too, especiallyÌýspecific features of their local course context that simplify and thus support implementationÌýof IBL.

In addition, we are exploring questions of how workshops of different types (face to face or online, longer or shorter duration)Ìýplay a role in supporting instructors to learn, develop interest, and implement IBL approaches. Ìý

Summer intensive workshops - examples of formative reports
  • Archie, T., Daly, D., & Laursen, S. (2020, March).ÌýCollaborative Research: PROfessional Development and Uptake through Collaborative Teams (PRODUCT) Supporting Inquiry Based Learning in Undergraduate Mathematics.ÌýCombined Evaluation Reports: 2019 Workshops.Ìý[Report to AIBL] Â鶹ӰԺ, CO: Ethnography & Evaluation Research, Â鶹ӰԺ.
  • Archie, T., Hayward, C., Daly, D., & Laursen, S. (2021, April).ÌýCollaborative Research: PROfessional Development and Uptake through Collaborative Teams (PRODUCT) Supporting Inquiry Based Learning in Undergraduate Mathematics.ÌýFollow-Up Report: 2019 Workshops. [Report to AIBL] Â鶹ӰԺ, CO: Ethnography & Evaluation Research, Â鶹ӰԺ.
Summer intensive workshops - summative reports
  • Archie, T., Hayward, C., & Laursen, S. (2021, July). CollaborativeÌýResearch: PROfessional Development and Uptake through Collaborative Teams (PRODUCT) Supporting Inquiry Based Learning in Undergraduate Mathematics. Cumulative Evaluation Report: 2016-2020.Ìý[Report to AIBL] Â鶹ӰԺ, CO: Ethnography & Evaluation Research, Â鶹ӰԺ. Ìý
  • Laursen, S., Hayward, C., Archie, T., & Daly, D.Ìý(2021, August).ÌýCollaborative Research: PROfessional Development and Uptake through Collaborative Teams (PRODUCT) Supporting Inquiry Based Learning in Undergraduate Mathematics.ÌýSynthesis and Study Methods: Final Version.Ìý[Report to AIBL] Â鶹ӰԺ, CO: Ethnography & Evaluation Research, Â鶹ӰԺ.
  • See also Research Products (above) led by Tim Archie, 2020-2021.
Online intensive workshops
  • Daly, D., Ethnography & Evaluation Research, & the Academy of Inquiry Based LearningÌý(2021, June). AIBL Handbook for Online Professional Development: Lessons Learned from PRODUCT Workshops. Â鶹ӰԺ, CO, and San Luis Obispo, CA: Â鶹ӰԺ, Ethnography & Evaluation Research; and Academy of Inquiry Based Learning.
  • Archie, T., Daly, D., & Laursen, S. (2021, April).ÌýCollaborative Research: PROfessional Development and Uptake through Collaborative Teams (PRODUCT) Supporting Inquiry Based Learning in Undergraduate Mathematics.ÌýEvaluation Report: 2020 Online Workshops. [Report to AIBL] Â鶹ӰԺ, CO: Ethnography & Evaluation Research, Â鶹ӰԺ.
Traveling workshop series - formative and summative reports

Two-year college biology teaching

E&ER conducted a needs assessment about teaching and professional development in two-year college biology, including a survey and literature review, and studied the workshops that were developed in response to these needs.

Archie, T., Wise, S. B., Robalino, J., & Laursen, S. (2024). . CBE—Life Sciences Education, 23(4), ar43. [open access]

Wise, S. B., Archie, T., & Laursen, S. (2022). . CBE—Life Sciences Education, 21(2), ar39. [open access]

This work was supported by the HHMI BioInteractive program of the Howard Hughes Medical Institute, under grant numbers GT16734 and GT14866. The opinions, findings, conclusions, and recommendations expressed in this work are those of the authors and do not necessarily reflect the views of the Howard Hughes Medical Institute.

Open software in mathematicsÌý

We served as evaluators for UTMOST, Undergraduate Teaching in Mathematics with Open Software and Textbooks, which supported faculty development and implementation for mathematics instructors to use Sage open-source software and Sage-based open-source textbook materials in their classroom teaching. Results highlight the outcomes of Sage EDU Days and the team's working process.

Evaluation of UTMOST was supported under NSF award .ÌýAny opinions, findings, conclusions, or recommendations expressed in these reports are those of the researchers, and do not necessarily represent the official views, opinions, or policy of the funders.

Professional development for women scientists

We evaluated career development workshops for women scientists.ÌýÌýTo learn about this work, visitÌýour page on ADVANCE Partnerships, Workshops for Women Faculty.

As part of our evaluation work for theÌýBiological Sciences Initiative (BSI)Ìýat the University of Colorado at Â鶹ӰԺ, E&ER studied a classroom outreach program in which graduate student scientists visit K-12 classrooms to present inquiry-based science lessons. Benefits to students and teachers suggest the potential and limitations of short-term classroom interventions. Graduate student scientists experienced powerful professional growth as effective teachers and reported significant impacts on their career paths.

Research publications

Evaluation reports

Drawing on data from three evaluation studies of innovations in STEM undergraduate education in chemistry and astronomy, this synthetic analysis portrays the work of teaching assistants in undergraduate courses and identifies how they can help—or hinder—educational innovation. Interview data from TAs offers insights about the sources of student resistance and identifies TAs’ professional development needs.

  • Seymour, E, with Melton, G., Wiese, D. J., & Pedersen-Gallegos, L. (2005).ÌýPartners in Innovation: Teaching Assistants in College Science Courses. Lanham, MD: Rowman & Littlefield.

As evaluators for , the College Mathematics Instructors Development Source, we gathered data to assess needs and improve programming for people who lead teaching professional development for graduate TAs in mathematics departments.Ìý We also developed profiles of different types of TAPD programs to help TAPD leaders choose and refine their own program model.ÌýSee "Preparation of Graduate Students for Teaching" for more about this work.

We evaluated a Massively Open Online Course (MOOC) on evidence-based STEM teaching, developed by members of the Center for the Integration of Research, Teaching, and Learning (CIRTL Network). The evaluation focused on MOOC-centered learning communities and how facilitators prepared for and led them.ÌýThe formative reports provided here are data sources that informed later publications.

  • Goldberg, B. B., Bruff, D. O., Greenler, R. McC., Barnicle, K., Green, N. H., Campbell, L. E. P., Laursen, S. L., Ford, M. J., Serafini, A., Mack, C., Carley, T. L., Maimone, C., & Campa III, H. (2023).ÌýPreparing future STEM faculty through flexible teaching professional development.ÌýPLoS ONE 18(10), e0275349.ÌýÌý[open access].
  • Laursen, S. (2016, May). Summary of MCLC facilitator survey data. [Report to CIRTL MOOC team] Â鶹ӰԺ, CO: Ethnography & Evaluation Research, Â鶹ӰԺ.
  • Laursen, S. (2017, November). Evaluation of CIRTL MOOC-Centered Learning Communities. [Report to CIRTL MOOC team] Â鶹ӰԺ, CO: Ethnography & Evaluation Research, Â鶹ӰԺ.

Evaluation of CoMInDS was supported by the National Science Foundation under award DUE-. Evaluation of the BSI was supported by the Howard Hughes Medical Institute. Evaluation of the CIRTL MOOC was supported by the National Science Foundation under award DUE-. ÌýAny opinions, findings, conclusions, or recommendations expressed in these reports are those of the researchers, and do not necessarily represent the official views, opinions, or policy of the funder.

Evaluation work forÌý, an ADVANCE Institutional Transformation project at the University of Colorado at Â鶹ӰԺ, identified faculty gains from participating in multi-day leadership workshops. Other analyses identified faculty development needs at the individual, departmental, and institutional levels.

Research publications

  • Laursen, S. L., & Rocque, B. (2009). .ÌýChangeÌý(March/April), 18-26.

Evaluation reports

Studies for LEAP were supported by the National Science Foundation under award . Any opinions, findings, conclusions, or recommendations expressed in these reports are those of the researchers, and do not necessarily represent the official views, opinions, or policy of the NSF. Ìý

E&ER has worked with CU Â鶹ӰԺ’sÌýBiological Sciences InitiativeÌýon its teacher professional development programs. We have examined immediate post-workshop and longer-term outcomes from the BSI’s one- to three-day workshops focused on basic science concepts, cutting-edge scientific developments, or new technologies in the life sciences. Evaluation findings from surveys and interviews indicate teachers’ growth in content understanding, confidence in their ability to teach these ideas, and a sense of being supported by a network of colleagues and the BSI staff. Many teachers reported making classroom use of new teaching materials. Relatively few teachers reported gains in pedagogical content knowledge, and teachers did not always perceive ways to adjust workshop materials for their own students, curriculum, or other constraints.

These resources assist professional developers in conceptualizing and preparing for their role in helping faculty understand, select and adopt strategies for more effective teaching.Ìý

  • Daly, D., Ethnography & Evaluation Research, & the Academy of Inquiry Based Learning. (2021, June). AIBL Handbook for Online Professional Development: Lessons Learned from PRODUCT Workshops. Â鶹ӰԺ, CO, and San Luis Obispo, CA: Â鶹ӰԺ, Ethnography & Evaluation Research; and Academy of Inquiry Based Learning.
  • from MAA Professional Development Leadership Symposium, 2022
    • ÌýSandra Laursen, 9/21/22
    • , Stan Yoshinobu, 9/28/22

We are interested in questions about how to measure change in teaching, as a result ofÌýprofessional development or departmental change work. Our researchÌýon developing, evaluating and applying measures of teaching isÌýongoing.

Development and applications of survey measures

  • Archie, T., Hayward, C. N., Yoshinobu, S., & Laursen, S. L. (2022). Investigating the linkage between professional development and mathematics instructors’ use of teaching practices using the theory of planned behavior.ÌýPLoS ONE 17(4),Ìýe0267097.ÌýÌý(open access)
  • Hayward, C. N., Kogan, M.,Ìý& Laursen, S. L. (2016). . International Journal of Research in Undergraduate Mathematics Education, 2(1), 59-82. Published online 11/25/2015. ÌýDOI 10.1007/s40753-015-0021-y [Author accepted MS]
  • Weston, T. J.; Laursen, S. L.; & Archie, T. (2024). Exploring the adoption of research-based instructional strategies in undergraduate mathematics with the Teacher-Centered Systematic Reform model. In Cook S., Katz, B., & Moore-Russo, D. (Eds.), forthcoming.ÌýProceedings of the 26th Annual Conference on Research in Undergraduate Mathematics Education. Omaha, NE. Ìý[Conference paper]

Development and applications of observation measures

  • Hayward, C. N., Laursen, S. L., & Weston, T. J. (2017).ÌýTAMI-OP: Toolkit for Assessing Mathematics Instruction – Observation Protocol. Â鶹ӰԺ, CO: Ethnography & Evaluation Research, Â鶹ӰԺ.
  • Hayward, C. N., Weston, T. J., & Laursen, S. L. (2018). First results from a validation study of TAMI: Toolkit for Assessing Mathematics Instruction.ÌýIn A. Weinberg, C. Rasmussen, J. Rabin, M. Wawro, & S. Brown (Eds.),ÌýÌý(pp. 727-735). San Diego, CA: Mathematical Association of America, SIGMAA on RUME.
  • Weston, T. J., Hayward, C. N., & Laursen, S. L. (2021). When seeing is believing: Generalizability and decision studies for observational data in evaluation and research on teaching.ÌýAmerican Journal of Evaluation, . Ìý[Author accepted MS]
  • Weston, T. J., Laursen, S. L., & Hayward, C. N. (2023). Measures of success: Characterizing teaching and teaching change with segmented and holistic observation data.ÌýInternational Journal of STEM Education 10​, 24. []
  • Weston, T. J., Laursen, S. L., & Hayward, C. N. (2023). Using latent profile analysis to assess teaching change. Ìý25th Annual Conference onÌýResearch in Undergraduate Mathematics Education, Omaha, NE, Feb. 23-25. [Conference paper]
  • Wise, S., Gallion, K., & Laursen, S. (2024). Exploration of TAMI-OP as a professional development tool for mathematics instructors.ÌýIn Cook S., Katz, B., & Moore-Russo, D. (Eds.), forthcoming.ÌýProceedings of the 26th Annual Conference on Research in Undergraduate Mathematics Education. Omaha, NE. [Conference paper]

Other measures

  • Katz, B., & Laursen, S. (2018). Adapting an exam classification framework beyond calculus.ÌýIn A. Weinberg, C. Rasmussen, J. Rabin, M. Wawro, & S. Brown (Eds.),ÌýÌý(pp. 980-987). San Diego, CA: Mathematical Association of America, SIGMAA on RUME.
  • Laursen, S., & Archie, T.Ìý(2018). ÌýHow do we teach thee? ÌýLet me count the ways. ÌýA syllabus rubric with practical promise for characterizing mathematics teaching. ÌýIn A. Weinberg, C. Rasmussen, J. Rabin, M. Wawro, & S. Brown (Eds.),ÌýÌý(pp. 862-870). San Diego, CA: Mathematical Association of America, SIGMAA on RUME.

Multiple measures

  • American Association for the Advancement of Science [AAAS] (2013).ÌýDescribing & Measuring Undergraduate STEM Teaching Practices. Washington, DC: AAAS. ÌýNote: E&ER personnel contributed but did not lead this report. The report is useful and difficult to find online, so it is archived here.
  • Hayward, C. N., Archie, T., Daly, D., Weston, T. J., & Laursen, S. L. (2022). The cycle of inquiry: Building effective evaluation relationships to support continuous improvement of faculty development initiatives. In S. Linder, C. Lee, K. High (Eds.),ÌýHandbook of STEM Faculty Development, Information Age Publishing, Charlotte, NC. Author approved manuscript. ÌýHandbook .
  • Laursen, S., Weston, T., Archie, T., Hayward, C., Gallion, K.Ìý(2023). Measuring what matters: Characterizing change in instructor practice in college mathematics. Session on "Highlights from Research on Instructors' Learning about Teaching,"ÌýJoint Mathematics Meetings, Boston, MA, January 6. [, 22 minutes]
  • Laursen, S. L., Archie, T., Weston, T. J., Hayward, C. N., & Yoshinobu, S. (2023). ÌýA measurement hat trick: Evidence from a survey and two observation protocols about instructional change after intensive professional development.Ìý25th Annual Conference onÌýResearch in Undergraduate Mathematics Education, Omaha, NE, Feb. 23-25. [Conference paper]

These studies were supported by the National Science Foundation under awardsÌýÌýand and by the .ÌýAny opinions, findings, conclusions, or recommendations expressed in these reports are those of the researchers, and do not necessarily represent the official views, opinions, or policy of the funder.