Evaluating your inquiry

Evaluation is an important part of SoTL, and is one of our areas of expertise at ISoTL.  Evaluating your project asks that you regularly “check in” on your project, and assess how it is helping you answer the question / curiosity you had set out in the early stages of your project. Evaluation also helps you determine whether your stated objectives/goal of your inquiry are being met, and informs how you can improve your project design and implementation. In this way, it is essential to evaluate your work as you go because it helps you to document what may or may not be working for you and your students.

Questions to ask yourself as you evaluate your project: What seems to be working well? What are some areas that may need revision/improvement/adaptation? How are students engaging in this work? What feedback (if any) from the students might inform what happens next? 

Common approaches to evaluation in SoTL 

The method that you employ in your SoTL inquiry will inform how you evaluate this work. At ISoTL, we often encourage instructors to approach their evaluation through a mixed-methods approach – that is, through approaches that will generate both qualitative and quantitative data. Qualitative approaches are excellent for generating rich data through a small sample size, while quantitative approaches are excellent for getting a broader understanding of a topic through larger sample sizes.

Surveys & Questionnaires

Surveys and questionnaires are an excellent way of gathering large data sets in courses that may have high enrolment. UBC tools such as Qualtrics make the creation of surveys simple and accessible to both novice and expert researchers, and make data analysis manageable.

In your SoTL practice you may be investigating a well-researched outcome (e.g., motivation) and want to examine existing surveys/ questionnaires to see if there are items that you could use in your own evaluation. The questionnaires included in our SoTL Questionnaire database are just a sample of items that you may be interested in using, or to help inspire your own survey creation.

Explore SoTL projects employing surveys and questionnaires 

Interviews & focus groups 

For smaller classes, interviews and focus groups may be a great way to evaluate your project. Also, depending on the question you have set out to explore, interviews (1-1) with students can help you gain a deeper understanding of students’ experiences with a particular pedagogy, classroom assignment / experience, and so on. Focus groups are also commonly used in large classes, where a few groups of students are invited to discuss particular issues in a more intimate and collaborative setting.

Explore SoTL projects employing interviews and focus groups 

Reflective writing 

The use of reflective writing is a rich way of gaining an in-depth understanding of the process of students’ learning and engagement. Reflective writing assignments and activities can be easily incorporated into courses in which students may be involved in an experiential learning component, a novel pedagogy or technology, or in courses that already have an active writing element to them.

If the use of reflective writing is solely for reflection on one’s practice there is no need to seek student’s consent. However, if the research moves beyond standard reflective practice (for example, dissemination: presentation and publication) measures need to be introduced to minimize the impact of a conflict of interest (e.g., use of a third party to be part of the consent process).

Explore SoTL projects employing reflective writing

Remember that the teaching team and students are often busy with regular course requirements. Introducing extra activities related to your research program could be challenging. Maximize the use of existing course activities. Be mindful of the rhythm of your course when introducing a new task (e.g. a survey is likely to be ignored if launched immediately before or during an exam period).

At ISoTL, we understand that evaluation is an iterative process that includes different stages. If you’re unsure of where to begin, consult with one of our team members about how best to evaluate your teaching practice(s). We’re here to help!

ETHICAL CONSIDERATIONS

Communicate to students the aims of your study and clarify that there will be no real or perceived influence toward students’ grade.

The consent letter or the introduction to the study (whether survey or interviews) must provide assurance to the students that their participation is voluntary and also no penalties (course grade) will result if they do not participate in the SoTL research. Transparency of the means and aims of the SoTL project.

Additional resources:

Chang, R.L., Gray, K. (2013). Ethics of research into learning and teaching with Web 2.0: reflections on eight case studies. J Comput High Educ 25, 147–165 https://doi.org/10.1007/s12528-013-9071-9

Hutchings, P. (2003). Competing Goods: Ethical Issues in the Scholarship of Teaching and Learning. Change, 35(5), 26-33. Retrieved July 20, 2020, from www.jstor.org/stable/40177229

Burman, M., & Kleinsasser, A. (2004). Ethical guidelines for use of student work: Moving from teaching’s invisibility to inquiry’s visibility in the Scholarship of Teaching and Learning. The Journal of General Education, 53(1), 59-79. Retrieved July 20, 2020, from www.jstor.org/stable/27797976

Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans – TCPS 2 (2018)

Additional Resources:

  • Hubball, H., & Clarke, A. (2010). Diverse methodological approaches and considerations for SoTL in higher education. The Canadian Journal for the Scholarship of Teaching and Learning, 1(1).
  • Pearson, M. L., Albon, S. P., & Hubball, H. (2015). Case study methodology: Flexibility, rigour, and ethical considerations for the scholarship of teaching and learning. Canadian Journal for the Scholarship of Teaching and Learning, 6(3), 12.